An image enhancement processing method for visual positioning

Abstract

The invention discloses an image enhancement processing method for visual positioning, comprising the following steps: S1 acquiring image data; acquiring image information of the surrounding environment through a visual sensor; S2 image enhancement; first extracting brightness information from a picture, then using a histogram equalization method to remove point data whose brightness exceeds a preset brightness range, and forming image information; S3 dividing the acquired image information into grids based on a YOLOv5 algorithm, and extracting feature information and label information from the grid information, wherein the label information is the position and category of a target object in annotated image, and then adjusting weight information of the feature information and label information through a Bayesian adaptive learning method, and then merging the feature information according to the adjusted weight information, and converting the merged image information into three-dimensional space information, so as to obtain accurate image information and facilitate subsequent accurate positioning.

Description

Technical Field

[0001] The invention belongs to an image processing method, and in particular relates to an image enhancement processing method for visual positioning. Background Art

[0002] Existing mobile robots are widely used in various industries due to their mobility. The positioning method in the mobile robot will affect the reliability of the mobile robot's movement, and is therefore favored by researchers from all walks of life.

[0003] In the prior art, visual sensors or laser radar devices are generally used to acquire images. However, the images acquired by the visual sensors need to be processed. If the processing is improper, the subsequent positioning will be unreliable.

[0004] In order to solve the above problems, a patent document with Chinese patent application number 201810481828.1 and publication date 2018.11.02 discloses a multi-sensor data fusion method and an AGV navigation and positioning device. The multi-sensor data fusion AGV navigation and positioning device includes an AGV body and sensors arranged on the AGV body. The sensors include a first infrared sensor and a second infrared sensor arranged on the left and right sides of the AGV body, and an ultrasonic sensor arranged on the front of the AGV body. There are 3 to 5 ultrasonic sensors. The extended Kalman filter algorithm is used to fuse the multi-sensor data information. This method can improve the system's fault tolerance and data credibility, reduce data acquisition costs, and use the extended Kalman filter algorithm to track and predict the system state measurement data in real time, thereby improving positioning accuracy.

[0005] This method only obtains image data through infrared sensors and ultrasonic sensors, and does not consider the existence of noise in the image data. There are many existing image processing algorithms. For example, the existing YOLOv5 algorithm needs to merge images of different sizes through the uncertain weight values ​​of the neural network to form a predicted image. However, this uncertain weight value will increase the uncertainty of the entire neural network, and also bring greater processing difficulty to image processing. Summary of the invention

[0006] The purpose of the present invention is to provide an image enhancement processing method for visual positioning, which can accelerate the image processing speed and make the image information obtained for visual positioning more accurate and reliable.

[0007] To achieve the above object, an image enhancement processing method for visual positioning includes the following steps: S1 obtains image data; obtains image information of the surrounding environment through visual sensors; S2 image enhancement: first extract brightness information from the image, then use the histogram equalization method to remove point data whose brightness exceeds the preset brightness range, and form image information; S3 divides the acquired image information into grids based on the YOLOv5 algorithm, and extracts feature information and label information from the grid information. The label information is the position and category of the target object in the annotated image. Then, the weight information of the feature information and label information is adjusted through the Bayesian adaptive learning method, and then the feature information is merged according to the adjusted weight information, and the merged image information is converted into three-dimensional space information.

[0008] The above setting, by acquiring the image collected by the visual sensor, and then using the histogram equalization method to eliminate the information of points with too high or too low brightness, thereby improving the effective recognition of the overall image, and then using the YOLOv5 algorithm to divide the image into multiple small units and extract feature information and label information, because the label information is used to mark the feature position and category in the image, and then the Bayesian adaptive learning method is used to determine the weight value according to the feature information and weight information, and finally the weight used for merging the graphics is determined according to the adjusted weight value. Since the existing YOLOv5 algorithm generally uses a complex distribution function, the weight parameters cannot be determined in the final merging process of the neural network, and the Bayesian adaptive learning method can determine the distribution function as a Gaussian distribution characteristic, because things in nature conform to the Gaussian distribution characteristics, so that this distribution can meet the requirements of most images, and at the same time, the distribution has a certain certainty, and can form a certain weight function, thereby ensuring the accuracy of the image and fast data processing.

[0009] Furthermore, the label information includes the category ID, bounding box coordinates and possible confidence of the target object.

[0010] In the above settings, the label information includes the category of the target object, the bounding box coordinates, and the confidence information, so that it is convenient to determine the loss situation and finally determine the prediction result during the subsequent prediction.

[0011] Furthermore, the S2 comprises the steps of: S21 extracts the brightness information in the color channel through the formula Gray = A* R + C * G +D * B, where R, G, and B represent the three colors of red, green, and blue respectively; S22 scans each pixel of the original grayscale image in sequence and calculates the grayscale histogram of the image; each pixel only contains the grayscale value; S23 by formula Calculate the cumulative histogram of the grayscale histogram, where q and p are the brightness information corresponding to the output image and the input image respectively; S23 by formula , calculate the mapping relationship between the input and the output. Here, H and W are the height and width dimensions of the image respectively, and hist is the histogram.

[0012] With the above settings, the grayscale conversion and histogram equalization processing mechanisms are applied to eliminate the pixels with too high or too low brightness in the image, making the detection target more recognizable.

[0013] Furthermore, the step S3 includes: S31 Extract features through the backbone network part, continuously reduce the feature map through the convolution module, and fuse the feature information into the channels; S32 The Neck neck module is responsible for fusing with the backbone network features; S33 Output multi-scale detection boxes through the head module, and the detection boxes contain label information.

[0014] S34 The Bayesian adaptive learning method includes given known feature information and label information on the premise that ; where f(k) is the Monte Carlo sampling function, which takes a finite number of random variable samples to approximate the complex integral; x(k) is the k-th new sample, y(k) is the k-th new label; q(w) is the Gaussian distribution function, w is the weight information, q(y(k)|x(k)) is the output feature information and label information, p(y(k)|f(k)) is the sampling information corresponding to the feature label information in the input image, and p(y(k)|x(k), w) is the feature information, label information in the input image, and the weight information under the corresponding Gaussian distribution.

[0015] With the above settings, the accuracy of identifying the target category is guaranteed, and a Bayesian-based network weight adaptive adjustment method is introduced to reduce the uncertainty in model detection.

[0016] Furthermore, after S34, there is also S35: Evaluate the Bayesian adaptive learning situation through the overall loss function, and the loss function includes confidence loss, classification loss, and bounding box regression loss.

[0017] With the above settings, the confidence loss can be mainly evaluated through the label information, the classification loss can be evaluated through the classification information in the target box, and the bounding box regression loss can be evaluated through the size of the bounding box, etc., so that the learning situation can be determined through the loss function to ensure the reliability of learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is the working process of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0019] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] like Figure 1 As shown, an image processing method for visual positioning is used in the visual positioning of a mobile robot, comprising the following steps: S1 obtains image data; obtains image information of the surrounding environment through visual sensors; S2 image enhancement: first extract the brightness information of the image, then use the histogram equalization method to remove the point data that exceeds the preset brightness range, and form image information; Step S2 includes S21-S23; S21 extracts the brightness information in the color channel through the formula Gray = A * R + C * G + D * B, where R, G, and B represent the three colors of red, green, and blue, respectively. In this embodiment, A=0.299; C=0.587; D=0.114.

[0021] S22 scans each pixel of the original grayscale image in sequence and calculates a grayscale histogram of the image; each pixel only contains a grayscale value.

[0022] S23 by formula Calculate the cumulative histogram of the grayscale histogram, where q and p are the brightness information corresponding to the output image and the input image respectively.

[0023] S23 by formula , calculate the mapping relationship between input and output, H and W are the height and width of the image respectively, and hist is the histogram.

[0024] S3 divides the acquired image information into grids based on the YOLOv5 algorithm, and extracts feature information and label information from the grid information. The label information is the location and category of the target object in the labeled image. Then, the weight information of the feature information and label information is adjusted through the Bayesian adaptive learning method. Then, the feature information is merged according to the adjusted weight information, and the merged image information is converted into three-dimensional coordinate information. S31 specifically includes S31-S35; S31 extracts features through the backbone network and continuously shrinks the feature map through the convolution module to integrate the feature information into the channel; S32 is responsible for integrating features with the backbone network through the Neck module; S33 outputs a multi-scale detection frame through the head module, and the detection frame contains label information.

[0025] S34 The Bayesian adaptive learning method includes given known feature information and label information on the premise that ; where f(k) is a Monte Carlo sampling function that takes a finite number of random variable samples to approximate a complex integral; x(k) is the k-th new sample, y(k) is the k-th new label; q(w) is a Gaussian distribution function, w is the weight information, q(y(k)|x(k)) is the output feature information and label information, p(y(k)|f(k)) is the sampling information corresponding to the feature label information in the input image, and p(y(k)|x(k), w) is the feature information, label information in the input image, and the weight information under the corresponding Gaussian distribution.

[0026] S35 evaluates the Bayesian adaptive learning situation through an overall loss function, which includes confidence loss, classification loss, and bounding box regression loss. In this embodiment, the confidence loss is mainly determined by the difference between the confidence information in the label information and the preset confidence information. The classification loss determines whether there is a loss based on the classification situation in the target box and the actual target box. The bounding box regression loss is determined by whether the learned bounding box matches the actual bounding box. If the matching degree exceeds a preset size, such as 80%, it is considered that there is no loss. If the evaluation shows that the loss function has no loss, the image information is determined according to the current weight. If the evaluation shows that the loss function has a loss, a prompt is given to adjust the weight information, which can be adjusted manually or in other ways.

[0027] S4 The mobile robot performs positioning and movement in three-dimensional space.

[0028] The working principle of the present invention: By acquiring the images collected by the vision sensor, and then using the histogram equalization method to eliminate the information of points with too high or too low brightness, thereby improving the overall effective recognition rate of the images. Then, through the YOLOv5 algorithm, the images are divided into multiple small units and the feature information and label information are extracted. Since the label information is used to annotate the feature positions and categories in the images, and then the Bayesian adaptive learning method is used to determine the weight value according to the feature information and the weight information. Finally, the weight used for combining the graphics is determined according to the adjusted weight value. Since the existing YOLOv5 algorithm generally uses complex distribution functions, the weight parameters cannot be determined during the final combination process of the neural network. However, using the Bayesian adaptive learning method can determine the distribution function as a Gaussian distribution characteristic because most things in nature conform to the Gaussian distribution characteristic. Therefore, using this kind of distribution can meet the requirements of most images, and at the same time, the distribution has a certain certainty, which can form a definite weight function, thus ensuring the accuracy of the formed images and fast data processing.

Claims

1. An image enhancement processing method for visual positioning, characterized in that: The following steps are involved: S1 obtains image data; obtains image information of the surrounding environment through visual sensors; S2 image enhancement: first extract brightness information from the image, then use the histogram equalization method to remove point data whose brightness exceeds the preset brightness range, and form image information; S3 divides the acquired image information into grids based on the YOLOv5 algorithm, and extracts feature information and label information from the grid information. The label information is the position and category of the target object in the annotated image. Then, the weight information of the feature information and label information is adjusted through the Bayesian adaptive learning method, and then the feature information is merged according to the adjusted weight information, and the merged image information is converted into three-dimensional space information.

2. The image enhancement processing method for visual positioning according to claim 1, characterized in that: The label information contains the category ID of the target object, the bounding box coordinates, and possible confidence.

3. The image enhancement processing method for visual positioning according to claim 1, characterized in that: The S2 comprises the steps of: S21 extracts the brightness information in the color channel through the formula Gray = A* R + C * G +D * B, where R, G, and B represent the three colors of red, green, and blue respectively; S22 scans each pixel of the original grayscale image in sequence and calculates the grayscale histogram of the image; Each pixel contains only grayscale values; S23 by formula Calculate the cumulative histogram of the grayscale histogram, where q and p are the brightness information corresponding to the output image and the input image respectively; S23 by formula , calculate the mapping relationship between input and output, H and W are the height and width of the image respectively, and hist is the histogram.

4. The positioning method based on multimodal data fusion according to claim 1, characterized in that: The step S3 comprises: S31 extracts features through the backbone network and continuously reduces the feature map through the convolution module, integrating the feature information into the channel; S32 is responsible for integrating features with the backbone network through the Neck module; S33 outputs a multi-scale detection frame through the head module, and the detection frame contains label information. S34 The Bayesian adaptive learning method includes given known feature information and label information On the premise of ; where f(k) is a Monte Carlo sampling function that takes a finite number of random variable samples to approximate a complex integral; x(k) is the k-th new sample, y(k) is the k-th new label; q(w) is a Gaussian distribution function, w is the weight information, q(y(k)|x(k)) is the output feature information and label information, p(y(k)|f(k)) is the sampling information corresponding to the feature label information in the input image, and p(y(k)|x(k), w) is the feature information, label information in the input image, and the weight information under the corresponding Gaussian distribution.

5. The positioning method based on multimodal data fusion according to claim 4, characterized in that: S34 also includes S35: evaluating the adaptive learning of Bayesian through the overall loss function, the loss function includes confidence loss, classification loss and bounding box regression loss.

Images