Image enhancement based target detection and localization method

Abstract

This invention relates to the field of target detection and localization, and particularly to an image enhancement-based target detection and localization method for mobile phone assembly scenarios. This method enhances the original image acquired by a visual sensor, specifically enhancing both the input image for target detection and the input image for target edge calculation. For target detection, overall brightness enhancement, histogram equalization, and Laplace sharpening are applied to enhance brightness and contrast. The enhanced image is then input into a deep learning network for target detection. For edge detection, the input RGB image is linearly enhanced using a single R-channel, the grayscale image is enhanced using the Sigmoid function, and finally, Sobel edge detection is performed, thereby achieving the target localization process.

Description

Technical Field

[0001] This invention belongs to the field of target detection and localization, and particularly relates to an image enhancement-based target detection and localization method for mobile phone assembly scenarios. Background Technology

[0002] Currently, due to the rapid development of artificial intelligence and deep learning technologies, a large number of robotic operations in industry utilize visual sensors. Efforts are underway to equip robots with human-like visual detection and localization capabilities. These capabilities are prerequisites for intelligent operations, such as intelligent assembly and intelligent defect detection. Target detection primarily involves detecting the position and bounding box of a target object in an image or video. Since target detection relies on visual sensors, changes in lighting significantly impact the success rate. Therefore, practical applications often require structured environments, which is detrimental to robots performing intelligent tasks. For example, in the intelligent assembly of mobile phone parts, where assembly precision requirements are typically at the micrometer level, changes in lighting during the process will affect the accuracy of target detection. Summary of the Invention

[0003] The main objective of this invention is to overcome the shortcomings of low target detection and localization accuracy under varying lighting conditions in existing technologies. It provides a target detection and localization method based on image enhancement. By performing a series of image enhancement methods, the influence of lighting on detection and localization can be effectively eliminated, thereby improving the accuracy and robustness of target detection and localization tasks. This method has good results in mobile phone assembly or other tasks that are easily affected by lighting conditions.

[0004] The technical solution for achieving the objective of this invention is as follows:

[0005] A target detection and localization method based on image enhancement acquires image data through a visual sensor, enhances the image before target detection, and enhances the image again during target edge detection to improve the accuracy of the task. The method includes the following steps:

[0006] Step (1) Increase the overall brightness of the image. For image information acquired in scenes with changing lighting, first increase the brightness of the image to reduce the impact of shadows on subsequent operations.

[0007] Step (2) improves the overall contrast of the image by histogram equalization. Based on the results of step (1), high-contrast image information is obtained to improve the brightness contrast of the image.

[0008] Step (3) uses the Laplace operator to sharpen the image, enhancing the contrast between light and dark at the edges of the image or where pixel values ​​change significantly, thus highlighting the subject;

[0009] Step (4) uses the YOLO v5 CNN convolutional neural network to detect the target, taking the result of step (3) as the input of the network, and finally outputting the target detection location and bounding box;

[0010] Step (5) uses the linear image enhancement method to linearly enhance the result of step (3), and uses a piecewise linear function to enhance the local information of the image;

[0011] Step (6) calculates the result of step (5) using the Sigmoid function to enhance the mobile phone assembly scenario and further improve the difference between mobile phone parts and the background.

[0012] Step (7) performs edge detection on the image using the Sobel edge operator. Using the pixel coordinates of the detection box in the result of step (4) and the result of step (6), the detection result part of the image is extracted. Then, this step is used to obtain the actual image boundary and then perform localization.

[0013] Furthermore, in step (1), the RGB image is brightened overall by increasing the pixel values ​​of the R, G, and B channels by 75 (this value is manually set), and then merging the three channels to improve the overall image brightness. The formula is as follows: ,in , , These are the three-channel pixel values ​​after the brightness is increased.

[0014] Further, step (2) processes the image generated in step (1) using histogram equalization, performing equalization on the R, G, and B channels of the RGB image respectively. First, the original image is sliced, with tileGridSize=64, where tileGridSize is the number of slices, which is the optimal number of slices for this method. Then, equalization is performed on each slice. The equalization process first records the pixel values ​​in the image. Number of times Where r is the value of each pixel in the image, then the cumulative distribution function (cdf) of each pixel value r is calculated, i.e., the formula... The CDF values ​​are sorted from smallest to largest. Pixels with CDF values ​​greater than the threshold clipLimit=6 are then evenly distributed among pixels with CDF values ​​less than the threshold. The CDF values ​​are recalculated until all CDF values ​​are less than the threshold. Finally, the minimum and maximum CDF values ​​are taken, and the result is calculated using the formula... Calculate the equalized pixel values After equalizing each image segment, the sliced ​​images are stitched together into a large image, thus completing the histogram equalization.

[0015] Furthermore, in step (3), the Laplace operator is used to sharpen the equalized image by selecting a convolution kernel. By performing a convolution operation on an image, a sharpened image can be obtained.

[0016] Furthermore, in step (4), the YOLO v5 deep learning network is used to perform target detection on the image enhanced in step (3). YOLO v5 is an open-source model that can detect targets and mark bounding boxes in the image.

[0017] Further, step (5) performs linear enhancement on the result of step (3). First, channel separation is performed on the three-channel image, mainly enhancing the R channel according to the formula. Where r is the pixel value in the R channel. The pixel values ​​are the result of linear enhancement. The three channels of the image are merged after individually enhancing the R channel.

[0018] Furthermore, step (6) further enhances the contrast of the sharpened image using the Sigmod function. First, the three-channel image is converted into a single-channel image, i.e., a grayscale image, and then the contrast is further enhanced according to the formula. , where r is the pixel value in the grayscale image. These are the pixel values ​​enhanced using the Sigmod function.

[0019] Further, in step (7), using the grayscale image saved in step (6) and the target bounding box position output in step (4), the target bounding box image is first extracted from the grayscale image, and then the Sobel edge operator is used to detect the edge of the target. First, convolution kernels are selected for the X and Y axes of the image respectively. and Then, two convolution kernels are used to perform convolution operations on the cropped images to obtain the results. and For the grayscale value of each pixel After the formula is used to calculate and output the final edge detection result, the target can be located.

[0020] This invention's method enhances image brightness, performs histogram equalization, and Laplace image enhancement during target detection, and further enhances the edge detection image during target localization. This includes enhancing the RGB three-channel image, performing linear enhancement on the R channel alone, and applying a Sigmoid function to the single-channel image, resulting in clearer target contours and facilitating target edge extraction using the Sobel operator. Specifically designed for mobile phone assembly scenarios, this invention reduces the impact of lighting variations on target detection and localization, thereby improving the accuracy of target detection and localization. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention;

[0022] Figure 2 This is a comparison image of the target detection results and edge detection results using the method of this invention with the original image;

[0023] Wherein, 2(a) is the target detection result of the original image, and 2(b) is the target detection result using the method of the present invention. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but this is not intended to limit the present invention.

[0025] Example

[0026] Image-enhanced target detection and localization methods, see [link to relevant documentation] Figure 1 By applying different enhancements to the target detection input image and the target localization input image, the accuracy of target detection and localization can be improved. Figure 1 This is a flowchart illustrating the specific process of the method of the present invention. This embodiment further illustrates the process through the assembly of a mobile phone flexible flat cable. The target detection and positioning method includes the following steps:

[0027] Step (1) Brighten the image obtained by the vision sensor to reduce the impact of lighting changes on subsequent operations;

[0028] Step (2) performs histogram equalization on the brightened image. Based on the results of step (1), high-contrast image information is obtained to improve the overall brightness and darkness contrast of the image.

[0029] Step (3) uses the Laplace operator to sharpen the image of the flexible flat cable captured by the tactile sensor, enhances the contrast between light and dark at the edge of the flexible flat cable, and highlights the main body of the flexible flat cable;

[0030] Step (4) uses YOLO v5's CNN convolutional neural network to detect the flat cable, where the result of step (3) is used as the network input, and the final output is the target detection location and bounding box. Finally, the result of this step is used in the target localization process.

[0031] Step (5) performs linear enhancement on the result of step (3) by using a piecewise linear function to enhance the local information of the RGB image;

[0032] Step (6) The previously enhanced image of the flexible flat cable is enhanced using the Sigmoid function to enhance the R channel image of the flexible flat cable, thereby further improving the difference between the mobile phone flexible flat cable and the background.

[0033] Step (7) performs edge detection of the soft flat cable using the Sobel operator. Using the pixel coordinates of the detection box in the result of step (4) and the result of step (6), the detection result part of the image is extracted. Then, the actual image boundary is obtained using this step, and then the location is performed.

[0034] In this embodiment, step (1) brightens the RGB image as a whole by increasing the pixel values ​​of the R, G, and B channels by 75 (this value is manually set), and then merging the three channels to improve the overall image brightness. The formula is as follows: ,in , , These are the three-channel pixel values ​​after the brightness is increased.

[0035] In this embodiment, the image generated in step (1) is processed by histogram equalization. First, the original RGB image of the flexible flat cable is sliced ​​into 64 small images. Then, equalization processing is performed on each image. The equalization process first records the pixel values ​​in the image. Number of times Where r is the value of each pixel in the image, then the cumulative distribution function (cdf) of each pixel value r is calculated, i.e., the formula... The CDF values ​​are sorted from smallest to largest. Pixels with CDF values ​​greater than the threshold clipLimit=6 are then evenly distributed among pixels with CDF values ​​less than the threshold. The CDF values ​​are recalculated until all CDF values ​​are less than the threshold. Finally, the minimum and maximum CDF values ​​are taken, and the result is calculated using the formula... Calculate the equalized pixel values After equalizing each image segment, the sliced ​​images are stitched together into a large image.

[0036] In this embodiment, the image after image equalization is sharpened using the Laplace operator. First, a convolution kernel is selected. Then, the image is convolved using this kernel to obtain a sharpened image. Finally, the sharpened image is input into the YOLO v5 deep learning network to perform object detection on the flat cable, outputting the detection results and bounding boxes of the flat cable.

[0037] In this embodiment, step (5) performs linear enhancement on the result of step (3). First, channel separation is performed on the three-channel image, mainly enhancing the R channel, according to the formula... Where r is the pixel value in the R channel. The pixel values ​​are the result of linear enhancement. The three channels of the image are merged after individually enhancing the R channel.

[0038] In this embodiment, for the result of linear enhancement, the three-channel image is first converted into a single-channel image, i.e., a grayscale image, and then the result is calculated according to the formula. Perform pixel value calculation, where r is the pixel value in the grayscale image. These are the pixel values ​​enhanced using the Sigmod function.

[0039] In this embodiment, using the grayscale image saved in step (6) and the target border position output in step (4), the target border image is first extracted from the grayscale image. Then, the Sobel operator is used to perform edge detection of the target. First, convolution kernels are selected for the X and Y axes of the image, respectively. and Then, two convolution kernels are used to perform convolution operations on the cropped images to obtain the results. and For the grayscale value of each pixel After calculation using this formula, the final edge detection result is output, which is used to locate the position of the flexible flat cable.

[0040] Comparative example:

[0041] Reference Figure 2 The positioning results of the mobile phone soft flat cable using the method of the present invention are compared with the target detection results of the original image. 2(a) is the target detection result of the original image. As can be seen from the figure, the original image does not perform well in the soft flat cable target detection. The detection box deviates from the soft flat cable, and the edge detection includes many irrelevant lines, which greatly interferes with the positioning of the edge detection results.

[0042] Figure 2(b) shows the target detection results using the method of the present invention. As can be seen from the figure, the method of the present invention has a better target detection effect on mobile phone flexible flat cables, a more accurate detection box, and a better edge detection effect on flexible flat cables, reducing the interference of irrelevant detection results on positioning.

[0043] pass Figure 2 Compared with the previous method, the method of the present invention is more accurate in detecting flexible flat cables, and the edge detection effect of the flexible flat cables is also better after image enhancement, thus making the positioning of the flexible flat cables more accurate.

Claims

1. A target detection and localization method based on image enhancement, characterized in that, The method involves performing image enhancement on images obtained from a visual sensor before target detection and target localization. The method includes the following steps: Step (1) Increase the overall brightness of the image. For image information acquired in scenes with changing lighting, first increase the brightness of the image to reduce the impact of shadows on subsequent operations. Step (2) improves the overall contrast of the image by histogram equalization. Based on the results of step (1), high-contrast image information is obtained to improve the brightness contrast of the image. Step (3) uses the Laplace operator to sharpen the image, enhancing the contrast between light and dark at the edges of the image or where pixel values ​​change significantly, thus highlighting the subject; Step (4) uses the YOLO v5 CNN convolutional neural network to detect the target, taking the result of step (3) as the input of the network, and finally outputting the target detection location and bounding box; Step (5) uses the linear image enhancement method to linearly enhance the result of step (3), and uses a piecewise linear function to enhance the local information of the image; Step (6) calculates the result of step (5) using the Sigmoid function; Step (7) performs edge detection on the image using the Sobel edge operator. Using the pixel coordinates of the detection box in the result of step (4) and the result of step (6), the detection result part of the image is extracted. Then, this step is used to obtain the actual image boundary and then perform localization. Specifically, using the grayscale image saved in step (6) and the target bounding box position output in step (4), the target bounding box image is first extracted from the grayscale image. Then, the Sobel edge operator is used to detect the edge of the target. First, convolution kernels are selected for the X and Y axes of the image respectively. and Then, two convolution kernels are used to perform convolution operations on the cropped images to obtain the results. and For the grayscale value of each pixel After calculation, the final edge detection result is output, which can then be used for target localization.

2. The target detection and localization method based on image enhancement as described in claim 1, characterized in that, Step (1) specifically involves: To brighten an RGB image overall, increase the pixel values ​​of the R, G, and B channels by 75 (this value is manually set), and then merge the three channels. The formula is as follows: ,in , , These are the three-channel pixel values ​​after the brightness is increased.

3. The target detection and localization method based on image enhancement as described in claim 1, characterized in that, Step (2) specifically involves: The image generated in step (1) is processed by histogram equalization. The three channels R, G and B in the RGB image are equalized respectively. First, the original image is sliced ​​and tileGridSize=64, where tileGridSize is the number of slices. Then, equalization is performed on each image block. The equalization process first records the pixel values ​​in the image. Number of times Where r is the value of each pixel in the image, then the cumulative distribution function (cdf) of each pixel value r is calculated, i.e., the formula... The CDF values ​​are sorted from smallest to largest. Pixels with CDF values ​​greater than the threshold clipLimit=6 are then evenly distributed among pixels with CDF values ​​less than the threshold. The CDF values ​​are recalculated until all CDF values ​​are less than the threshold. Finally, the minimum and maximum CDF values ​​are taken, and the result is calculated using the formula... Calculate the equalized pixel values After equalizing each image segment, the sliced ​​images are stitched together into a large image, thus completing the histogram equalization.

4. The target detection and localization method based on image enhancement as described in claim 1, characterized in that, Step (3) specifically involves: The Laplace operator is used to sharpen the equalized image, and the method involves selecting a convolution kernel. By performing a convolution operation on an image, a sharpened image can be obtained.

5. The target detection and localization method based on image enhancement as described in claim 1, characterized in that, Step (4) specifically involves: The YOLO v5 deep learning network is used to perform target detection on the image enhanced in step (3). YOLO v5 is an open source model that can detect targets and mark bounding boxes in the image.

6. The target detection and localization method based on image enhancement as described in claim 3, characterized in that, Step (5) specifically involves: Linear enhancement is performed on the result of step (3). First, channel separation is performed on the three channels, and the R channel is enhanced according to the formula. Where r is the pixel value in the R channel. The pixel values ​​are the result of linear enhancement. The three channels of the image are merged after individually enhancing the R channel.

7. The target detection and localization method based on image enhancement as described in claim 1, characterized in that, Step (6) specifically involves: The contrast of the sharpened image is further enhanced using the Sigmod function. First, the three-channel image is converted into a single-channel image, i.e., a grayscale image. Then, the formula is applied... , where r is the pixel value in the grayscale image. These are the pixel values ​​enhanced using the Sigmod function.

Images