Laser strip center extraction method

Abstract

The invention discloses a laser bar center extraction method, which comprises the following steps of: 1) training a segmentation module so that the segmentation module can segment a laser bar in a to-be-measured image, and outputting a Mask binary image; 2) inputting a to-be-tested image into a segmentation module, outputting a corresponding Mask binary image, screening foreground connected domains, and obtaining a reasoning Mask binary image; 3) performing morphological processing, extracting a connected domain, and obtaining a minimum bounding rectangle as an ROI region; 4) respectively acquiring F1, F2 and F3 by utilizing the ROI of the image to be detected, and then judging whether the ROI is a good area or a non-good area; and 5) for a good area, directly utilizing the ROI area corresponding to the image to be measured to perform light stripe center extraction, and for a non-good area, utilizing the ROI area corresponding to the reasoning Mask binary image to perform light stripe center extraction, and acquiring coordinates of a laser stripe central point. According to the method, the precision can be ensured, external interference can be effectively reduced, and the robustness is relatively high.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection, specifically to a method for extracting the center of a laser stripe. Background Technology

[0002] Multi-line structured light measurement is a 3D reconstruction technology based on active optics, which is widely used in industrial defect detection, face recognition, 3D scanning and other fields. Its core principle is to project specific multi-line structured light onto the surface of an object, capture the deformed structured light laser strips with a camera, and resolve the 3D shape of the object surface through methods such as triangulation. Among them, the extraction of the center pixel of the laser strip is the core link of the entire measurement system, and its accuracy will directly affect the measurement precision (pixel-level extraction error may be amplified into millimeter-level error). Therefore, accurate extraction of the center pixel of the structured light is of great significance.

[0003] To address the aforementioned issues, some inventions have mentioned methods for extracting the center of the laser stripe: CN 120446138 A first performs median filtering to remove image noise, then histogram equalization to increase image contrast, and finally uses the gray-scale centroid method or parabolic fitting algorithm to extract the center of the light stripe from the original image. This method has two problems: first, performing histogram equalization on the entire image leads to large pixel variations, severely damaging the edges of the laser stripe and affecting accuracy; second, it does not consider external interference such as ambient light and shadows, so it is only suitable for relatively ideal environments and not for complex industrial scenarios with high accuracy requirements. CN 120672856 A first uses Otsu's algorithm for threshold segmentation, and then uses the gray-scale centroid method to extract the center of the light stripe. This method also has a problem: using Otsu's algorithm globally for threshold segmentation is sensitive to external interference, and directly extracting the light stripe from the segmented image is too dependent on the segmentation result. When the segmentation result is not ideal, the system accuracy will naturally decrease. Therefore, this method is also only suitable for ideal scenarios with little external interference and relatively simple environments. CN120833350 A proposes a deep learning-based laser stripe extraction technique. First, the input image is preprocessed (filtering and denoising, histogram equalization). Then, a deep learning semantic segmentation model is used to perform threshold segmentation on the preprocessed image. Finally, the center line of the laser stripe is extracted using the gray-scale centroid method. This method, by introducing deep learning, has a certain ability to resist ambient light interference. However, it also has two problems: first, filtering and histogram equalization before inference change the pixel distribution, thus affecting the model's segmentation accuracy; second, extracting the laser stripe directly from the segmentation result binary image has lower accuracy than extracting from the original image (when the original image quality is good, i.e., with less external interference).

[0004] In summary, current laser stripe extraction algorithms have the following shortcomings:

[0005] (1) Based on traditional image processing algorithms, noise reduction and histogram equalization are performed on the whole image, which will lead to large changes in the pixel value of the laser stripe, thereby reducing the accuracy of light stripe extraction.

[0006] (2) Using traditional methods to perform threshold segmentation on the original image (such as Otsu's algorithm) is more sensitive to external interference. For example, areas with strong ambient light will be segmented into the foreground, resulting in low robustness.

[0007] (3) Using deep learning to segment the original image can effectively resist ambient light interference, but it relies too much on the segmentation results and the accuracy of laser strip extraction cannot be guaranteed. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a laser stripe center extraction method that can effectively reduce external interference while ensuring accuracy, and has high robustness.

[0009] The technical solution adopted in this invention is as follows:

[0010] A method for extracting the center of a laser stripe includes the following steps:

[0011] 1) Train the segmentation module to segment the laser stripe in the image to be tested and output a mask binary image;

[0012] 2) Input the image to be tested into the segmentation module obtained in step 1), output the Mask binary image corresponding to the image to be tested, filter the foreground connected components, exclude regions that do not meet the preset conditions, and obtain the inference Mask binary image.

[0013] 3) Perform morphological processing on the binary image of the inference Mask to merge any discontinuous light stripes and obtain the Mask. + Binary image, then extract the mask + Find the connected components and obtain their minimum bounding rectangle, which is used as the ROI region of the laser strip;

[0014] 4) Obtain F1, F2, and F3 from the ROI region of the map to be tested. If F1+F2+F3≥2, the ROI region is considered a good region; otherwise, it is a bad region.

[0015] The F1 score is obtained as follows: The normalized histogram of all pixels within the ROI region of the image to be tested is obtained, with gray levels ranging from 0 to 9. When 20% of the pixels have a gray level greater than threshold A, or the difference between the largest and smallest proportion of gray levels is less than threshold B, F1 = 0; otherwise, F1 = 1. The threshold A ranges from [value missing in original text]. The range of values ​​for threshold B is: ;

[0016] The method for obtaining F2 is as follows: calculate the pixel contrast C within the ROI region of the image to be tested, C = (I max -I min ) / (I min +I max ), where I max and I min Let C be the maximum and minimum pixel values, respectively. When C is less than the threshold P, F2 = 0; otherwise, F2 = 1. The threshold P ranges from [value missing]. ;

[0017] The method for obtaining F3 is as follows: calculate the standard deviation S of the gray level of the ROI region of the image to be tested using the following formula.

[0018]

[0019] Where N represents the total number of pixels within the ROI region of the image under test, u represents the average grayscale value of all pixels within the ROI region, and I i Let S represent the grayscale value of the i-th pixel. When S is less than the threshold Q, F3 = 0; otherwise, F3 = 1. The threshold Q ranges from...

[0020] 5) When the ROI region of the image to be tested is determined to be a good region, the center of the light stripe is extracted directly using the ROI region corresponding to the image to be tested, and the center point of the laser stripe in the current ROI region is obtained.

[0021] When the ROI region of the image to be tested is determined to be a non-good region, the center of the light stripe is extracted using the ROI region corresponding to the inference mask binary image to obtain the center point of the laser stripe within the current ROI region.

[0022] Further, in step 1), when training the segmentation module, collect no less than 1,000 laser strip images from different scenes, and perform pixel-level annotation based on the laser strips. Each image will have a corresponding Mask binary image (white for the foreground laser strip and black for the background) as a label.

[0023] Furthermore, in step 1), the training segmentation module is a deep learning network, and the main network architecture consists of an encoder and a decoder. Even further, the encoder is used to extract image features, obtaining feature maps downsampled by 8, 16, or 32 times; the decoder is used to restore the downsampled feature maps to the original image size.

[0024] Furthermore, the encoder is a transformer-based encoder or a CNN-based encoder.

[0025] Furthermore, in step 2), the preset condition for filtering the foreground connected components is: using an area threshold for filtering, the regions whose connected component area is greater than the area threshold are the regions that meet the condition.

[0026] Further, in step 2), the foreground connected components are filtered, and the case that meets the preset conditions is that the following two conditions are met simultaneously:

[0027] ① Set an area threshold; the area of ​​the connected components must be greater than the area threshold.

[0028] ②Pre-create a binary image of the laser strip template Mask, calculate the IoU between the binary image of the Mask to be tested and the image of the Mask, and ensure that the IoU is greater than the preset value.

[0029] Furthermore, in step 3), the morphological processing is a closing-then-opening operation.

[0030] Furthermore, the light stripe center extraction method used in step 5) is: gray-scale centroid method, Steger algorithm, or parabolic fitting method.

[0031] This invention provides a method for extracting the center of a laser stripe, which can effectively reduce external interference while ensuring accuracy and has high robustness. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the overall scheme of the laser stripe center extraction method for which protection is claimed in this invention.

[0033] Figure 2 Input the image to be tested;

[0034] Figure 3 for Figure 2 The binary image of the inference mask obtained in step 2);

[0035] Figure 4 for Figure 2 The mask obtained from the image in step 3) + Binary image;

[0036] Figure 5 for Figure 2 The image obtained in step 3) shows the region of interest (ROI) marked with laser bars.

[0037] Figure 6 for Figure 2 Results of light stripe center extraction in the image. Detailed Implementation

[0038] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0039] A method for extracting the center of a laser stripe includes the following steps:

[0040] 1) Train the segmentation module to segment the laser stripe in the test image and output a mask binary image. This segmentation module is a deep learning network. The main network architecture consists of an encoder and a decoder. The encoder extracts image features and obtains feature maps downsampled by 8, 16, or 32 times. Here, we take 32 times as an example. The decoder restores the downsampled feature map to the original image size. Taking a 512*512*3 input image as an example, after passing through the encoder, a 16*16*1024 feature map is obtained. Then, the decoder restores the feature map to a scale of 512*512*2. Channel 2 represents the pixel category. If the score of the 0th channel is high, the pixel is the background; if the score of the 1st channel is high, the pixel is the foreground.

[0041] In practice, the encoder can be either a transformer-based encoder or a CNN-based encoder. Transformer-based encoders have a large number of parameters, a more complex structure, a global receptive field, and a slower training convergence speed, but they have stronger feature extraction capabilities. When the scene is relatively simple and the task difficulty is low, a CNN-based encoder, such as ResNet or UNet, can be selected. These encoders have a simpler structure, fewer parameters, and are faster, but their feature extraction capabilities are weaker compared to transformers.

[0042] In the actual training of the segmentation module, at least 1,000 laser stripe images from different scenes are collected. Pixel-level annotations are performed based on the laser stripes, and each image is assigned a corresponding Mask binary image (white for the foreground laser stripe and black for the background) as a label. The training set, validation set, and test set are divided in an 8:1:1 ratio.

[0043] 2) The image to be tested (e.g.) Figure 2 (As shown) Input the segmentation module obtained in step 1), output the Mask binary map corresponding to the image to be tested, filter the foreground connected components, exclude regions that do not meet the preset conditions, and obtain the inference Mask binary map, as shown. Figure 3 As shown;

[0044] In practice, the input image is processed by the encoder to obtain a compressed feature map, while the decoder needs to restore the compressed feature map to the original image resolution size. The number of channels is 2 (to store the probability of each pixel belonging to the foreground and background). When the background probability is greater than the foreground probability, the pixel is judged as the background, and otherwise the pixel is the foreground. This results in a binary image with a width and height equal to the original image, which is the inference mask.

[0045] In practice, the preset condition for filtering the foreground connected components is: using an area threshold for filtering, the region whose area of ​​the connected component is greater than the area threshold is the region that meets the condition.

[0046] When the method provided by this invention is applied to scenes with high imaging repeatability, the screening of foreground connected components is considered to meet the preset conditions when the following two conditions are satisfied simultaneously:

[0047] ① Set an area threshold; the area of ​​the connected components must be greater than the area threshold.

[0048] ②Pre-create a binary image of the laser strip template Mask, calculate the IoU between the binary image of the Mask to be tested and the image of the Mask, and ensure that the IoU is greater than the preset value.

[0049] 3) Perform morphological processing on the binary image of the inference mask, specifically by performing a closing-then-opening process to merge any potentially discontinuous light stripes and obtain the mask. + Binary graphs, such as Figure 4 As shown, due to the opening and closing operations, the Mask at this time... + The connected components in the binary graph can no longer accurately describe the shape of the laser stripe, but can only describe the approximate area information of the laser stripe;

[0050] Extract Mask + Find the connected components and obtain their minimum bounding rectangle, which will be used as the Region of Interest (ROI) of the laser stripe, such as... Figure 5 As shown in the green box;

[0051] 4) Obtain F1, F2, and F3 from the ROI region of the map to be tested. If F1+F2+F3≥2, the ROI region is considered a good region; otherwise, it is a bad region.

[0052] The F1 score is obtained as follows: The normalized histogram of all pixels within the ROI region of the image to be tested is obtained, with gray levels ranging from 0 to 9. When 20% of the pixels have a gray level greater than threshold A, or the difference between the largest and smallest proportion of gray levels is less than threshold B, F1 = 0; otherwise, F1 = 1. The threshold A ranges from [value missing in original text]. The range of values ​​for threshold B is: ;

[0053] The method for obtaining F2 is as follows: calculate the pixel contrast C within the ROI region of the image to be tested, C = (I max -I min ) / (I min +I max ), where I max and I min Let C be the maximum and minimum pixel values, respectively. When C is less than the threshold P, F2 = 0; otherwise, F2 = 1. The threshold P ranges from [value missing]. ;

[0054] The method for obtaining F3 is as follows: calculate the standard deviation S of the gray level of the ROI region of the image to be tested using the following formula.

[0055]

[0056] Where N represents the total number of pixels within the ROI region of the image under test, u represents the average grayscale value of all pixels within the ROI region, and I i Let S represent the grayscale value of the i-th pixel. When S is less than the threshold Q, F3 = 0; otherwise, F3 = 1. The threshold Q ranges from...

[0057] 5) When the ROI region of the image to be tested is determined to be a good region, the center of the light stripe is extracted directly using the ROI region corresponding to the image to be tested, and the center point of the laser stripe in the current ROI region is obtained. In specific implementation, the light stripe center extraction method can be: gray-scale centroid method, Steger algorithm or parabolic fitting method.

[0058] When the ROI region of the image under test is determined to be a non-good region, the center of the light stripe is extracted using the ROI region corresponding to the inference mask binary image to obtain the center point of the laser stripe within the current ROI region, such as... Figure 6 As shown by the red line.

[0059] The average error of laser strip extraction is less than 1.5 pixels, indicating that the laser strip extraction is correct. Using the method provided by this invention, the accuracy of laser strip extraction in complex scenes is improved from less than 80% to more than 90%, which is a significant improvement.

[0060] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A laser stripe center extraction method, characterized by Comprising the following steps: 1) Train the segmentation module to enable it to segment the laser stripe in the image to be tested, and output a Mask binary image; 2) Input the image to be tested into the segmentation module obtained in step 1), output the Mask binary image corresponding to the image to be tested, filter the foreground connected domain, exclude the area not meeting the preset condition, and obtain an inference Mask binary image; 3) Morphological processing of the inference mask binary image to merge light bars that can have discontinuities + binary image, and extract the connected components of the mask + to obtain the minimum bounding rectangle of the connected components as the ROI region of the laser bar; 4) Obtain F1, F2 and F3 respectively using the ROI region of the image to be tested, if F1+F2+F3≥2, the ROI region is considered to be a good region, otherwise it is a non-good region; Wherein: the acquisition method of F1 is: obtaining the normalized histogram of all pixels in the ROI region of the to-be-tested image, taking the gray scale as 0~9, when 20% of the pixel points have a gray scale > threshold A, or the difference between the proportion of the largest gray scale and the proportion of the smallest gray scale is < threshold B, F1=0, otherwise, F1=1; the value range of threshold A is ; the value range of threshold B is ; The acquisition method of F2 is: calculating the pixel contrast C in the ROI region of the to-be-tested image, C=(I max -I min ) / (I min +I max ), wherein I max and I min are the maximum value and the minimum value of the pixel respectively, when C is less than a threshold value P, F2 = 0, otherwise, F2 = 1; the value range of the threshold value P is ; The method for obtaining F3 is to calculate the gray standard deviation S of the ROI region of the image to be tested using the following formula, Where N represents the total number of pixels in the ROI region of the image to be tested, u represents the average gray value of all pixels in the ROI region, I i represents the gray value of the i-th pixel, F3 = 0 when S is less than the threshold value Q, otherwise F3 = 1; the threshold value Q has a value range of 5) When the ROI region of the image to be tested is determined to be a good region, directly use the ROI region corresponding to the image to be tested to extract the light stripe center, and obtain the laser stripe center point in the current ROI region; When the ROI region of the image to be tested is determined to be a non-good region, use the ROI region corresponding to the inference Mask binary image to extract the light stripe center, and obtain the laser stripe center point in the current ROI region.

2. The method of claim 1, wherein: In step 1), when training the segmentation module, collect no less than 1000 laser stripe images of different scenes, label each image at the pixel level based on the laser stripe, and obtain a corresponding Mask binary image as a label.

3. The method of claim 1, wherein: In step 1), the trained segmentation module is a deep learning network, and the network main architecture is composed of an encoder and a decoder.

4. The method of claim 3, wherein: The encoder is used to extract image features to obtain a feature map reduced by 8 times, 16 times or 32 times; and the decoder is used to restore the reduced feature map to the original image size.

5. The method of claim 3, wherein: The encoder is a transformer structure-based encoder or a CNN structure-based encoder.

6. The method of claim 1, wherein: In step 2), the preset condition for filtering the foreground connected domain is to use an area threshold to filter, and the area of the connected domain greater than the area threshold is the area meeting the condition.

7. The method of claim 1, wherein: In step 2), the situation considered to meet the preset condition is to meet the following two conditions at the same time: ① Set an area threshold, and the area of the connected domain is greater than the area threshold; ② A laser stripe template Mask binary image is prepared in advance, the IoU between the Mask binary image of the image to be tested and the template is calculated, and the IoU is greater than a preset value.

8. The method of claim 1, wherein: In step 3), the morphological processing is a close-open operation.

9. The method of claim 1, wherein: In step 5), the light stripe center extraction method used is the gray gravity center method, the steger algorithm or the parabola fitting method.

Images