A light stripe recovery and cluster matching method based on restricted region growing

Abstract

This invention relates to a method for recovering and clustering optical stripes based on restricted region growing, belonging to the field of computer vision measurement technology. The method includes the following steps: Step 1: Obtaining stripe projection images of the object under test after segmentation of the region of interest (ROI) from two camera viewpoints; Step 2: Obtaining binary stripe images; Step 3: Extracting the stripe center lines from the binary stripe images through connected component analysis to obtain the seed point coordinates of each stripe; Step 4: Obtaining stripe point coordinate data; Step 5: Calculating the centroid coordinates of the stripes based on the stripe point coordinate data recovered in Step 4, and numbering the stripes in the left and right views; Step 6: Comparing the number of stripes in the left and right views. If they do not match, the radius of the region growing is increased, and Steps 4 and 5 are repeated until the number of stripes in the left and right views matches. This invention effectively addresses the problems of noise, breakage, and deformation in optical stripe images, avoiding the problem of mismatched multi-line laser stripes.

Description

Technical Field

[0001] This invention relates to a method for recovering and clustering light stripes based on growth of confined regions, belonging to the field of computer vision measurement technology. Background Technology

[0002] In multi-line laser-based 3D measurement systems, multiple laser lines are projected onto the surface of the object being measured. A camera captures the projected fringe image, and the 3D coordinates of the object's surface are calculated through steps such as centerline extraction, epipolar constraint, and stereo matching. However, the complex texture of the object's surface, variations in its shape, interference from external light sources, and image noise often severely affect the fringe extraction process, leading to fringe loss, intersection, breakage, or interference from noise points. These problems make fringe matching extremely difficult, causing incorrect fringe matching in the left and right images, ultimately affecting the accuracy of the object's 3D reconstruction.

[0003] To address these issues, existing technologies typically employ traditional image processing and clustering methods. However, these methods often perform poorly in stripe restoration and matching under complex backgrounds and lighting conditions, especially when stripes are broken or incomplete, resulting in low accuracy in stripe restoration and matching. Therefore, overcoming the difficulties in stripe extraction and matching through more efficient image restoration and matching algorithms remains an important research problem in this field. Summary of the Invention

[0004] To address the problem of inaccurate stripe clustering and matching in existing binocular vision-based multi-line laser 3D measurement systems when stripes are broken, blurred, or wide, this invention proposes a stripe restoration and clustering matching method based on restricted region growing. This method can restore and reconstruct damaged or missing multi-line stripes without the need for optical plane calibration, and complete the clustering and numbering matching of stripes between left and right views. This method utilizes the structural continuity of stripes and the physical consistency between left and right images, combined with region growing and clustering techniques. Even under complex interference and discontinuous stripe conditions, it can effectively restore and match corresponding stripes in left and right images, achieving stable stripe extraction and matching, and providing high-quality input data for subsequent 3D reconstruction.

[0005] The technical solution adopted in this invention is: a method for optical stripe recovery and clustering matching based on growth in confined regions, comprising the following steps:

[0006] Step 1: Construct a multi-line laser measurement model based on binocular vision to obtain fringe projection images of the region of interest of the measured object from the perspectives of two cameras;

[0007] Step 2: Blur, break, and enlarge the stripe projection image obtained in Step 1 to simulate the actual stripe deformation. Then, filter and binarize the processed stripe projection image to obtain a clear binary stripe image.

[0008] Step 3: Extract the stripe center lines from the binary stripe image in Step 2 through connected component analysis to obtain the seed point coordinates of each stripe, which will serve as the starting point for subsequent restoration.

[0009] Step 4: Based on the coordinates of each seed point, use the restricted region growing algorithm to recover the stripes and obtain the stripe point coordinate data;

[0010] Step 5: Based on the stripe point coordinate data recovered in Step 4, calculate the centroid coordinates of the stripes and number the stripes in the left and right views; the stripes with the same number in the left and right views are the corresponding matching stripes.

[0011] Step 6: Compare the number of stripes in the left and right views. If they do not match, increase the radius of the region growth and repeat steps 4 and 5 until the number of stripes in the left and right views matches.

[0012] Step 1 specifically includes the following steps:

[0013] When constructing a divergent multi-line laser measurement model based on binocular vision, the multi-line laser needs to be orthographically projected and the projected stripes need to be vertical or tilted in the view image.

[0014] During image acquisition, a solid-color background paper, distinct from the object being measured, is placed on the measurement platform. Two cameras first acquire an image of the object without laser stripe projection, without moving the object. Then, an image of the object with structured light stripe projection is acquired. By detecting and extracting edge features from the image without laser stripe projection, and performing image dilation on the extracted edge contours, accurate closed image contours are obtained. The closed contours are then filled to obtain a mask image of the ROI region of the object being measured. This mask image is then superimposed on the image of the object with stripe projection to obtain the stripe projection image after region of interest segmentation.

[0015] Specifically, the binarization process in step 2 refers to processing the stripes in the image into clear white and the background into black, and using OpenCV's thresholding method to convert the wide, blurred, broken and filtered image into a binary image.

[0016] The specific steps for step 3 are as follows:

[0017] 1) Connected component analysis: The binary stripe image obtained in step 2 is processed by the connected component analysis method to identify the independent region of each stripe. The pixel coordinates of each stripe are obtained by using the cv2.connectedComponents() method in OpenCV.

[0018] 2) Stripe Centerline Extraction: Analyze each stripe and calculate its centerline position. Specifically, calculate the average pixel value of each stripe region to obtain the centroid or center point coordinates (x, y, y) of that region. s ,y s );

[0019] 3) Saving seed point coordinates: Save the coordinates of the centroid or center point (x... s ,y s These points are saved as seed points, and their coordinates will serve as the starting points for subsequent restricted region growing algorithms.

[0020] The restricted region growing algorithm in step 4 specifically includes:

[0021] (1) Calculate the gradient of the input image I(x,y) by using the Sobel operator in both the horizontal and vertical directions to calculate the gradient magnitude:

[0022]

[0023] The gradient magnitude G(x,y) can then be expressed as:

[0024]

[0025] Among them, G x G y G(x,y) represents the gradient of a stripe image in the x and y directions, respectively, and G(x,y) is the total gradient magnitude of the stripe in the image. This gradient information is used to control the growth of the stripe.

[0026] (2) Initialize region growth parameters:

[0027] Using each stripe seed point S = (x) obtained in step 3 s ,y s Centered on W, the maximum and minimum values ​​of the growth window are set to W. min and W max Real-time window size W can be dynamically adjusted within the following range:

[0028] W min ≤W≤W max W∈{2k+1|k∈N}

[0029] Among them W min The minimum allowable size of the growth window;

[0030] W max : The maximum allowable size of the growth window;

[0031] W: The size of the current growth window, and it must satisfy that W is an odd number 2k+1, where k is a natural number;

[0032] (3) Regional growth starting from the seed point:

[0033] Define the current pixel to be expanded as P. c =(x c ,y c ), with P c A growth window of size W×W is established centered on the growth window, and the gradient magnitude region within the growth window is defined as follows:

[0034]

[0035] R W (P c ): With point P c The set of all gradient magnitude points within a W×W window centered on the target;

[0036] G(i,j): The gradient magnitude of the image at coordinates (i,j);

[0037] x c ,y c : The horizontal and vertical coordinates of the center point of the current window;

[0038] Half the length of the window side, rounded down;

[0039] And calculate the variance of the gradient magnitude within the window:

[0040]

[0041] in Window R W (P c variance of gradient magnitude

[0042] μ G Window R W (P c The average value of the gradient magnitude within )

[0043] W 2 The total number of pixels contained in the window;

[0044] like When the gradient change within the region is gradual, the window can be appropriately enlarged.

[0045] W = min(W+2, W) max )

[0046] like If the gradient changes drastically within a region, then the window size should be reduced.

[0047] W = max(W-2, W min );

[0048] T g Gradient threshold: used to determine whether a pixel has sufficient gradient features;

[0049] (4) Pixel inclusion criteria:

[0050] For pixel P within the window n =(x n ,y n If the following conditions are met:

[0051] Pixel I(x) n ,y n The grayscale value of ) is non-zero and has not been processed;

[0052] Gradient magnitude G(x) n ,y n () is greater than the preset threshold;

[0053] After adding a new point, calculate the three points P where the fringe is locally continuous. i-2 ,P i-1 ,P i The resulting curvature C is calculated using the curvature calculation formula:

[0054]

[0055] If C≤T c Then P n Add the point to the region growth queue and mark it as processed; otherwise, discard the point to avoid abnormal distortion of the stripe boundary. (T) c Curvature threshold, used to control the smoothness of stripe shape;

[0056] (5) Iterative process of region growth:

[0057] Continuously update the window center point P c For each new point in the queue, perform the above iterative process (3) and (4) until the queue is empty, thus completing the restoration of the striped region.

[0058] The beneficial effects of this invention are as follows: The confined region growth and restoration method proposed in this invention can address problems such as breaks, blurring, intersections, and wide areas in laser stripe images in real-world environments. Through seed point-driven precise restoration and clustering, it avoids the problems of traditional methods, such as strong dependence on stripe continuity and susceptibility to noise interference. Furthermore, this method does not rely on optical plane calibration, has strong adaptability, and can achieve stable stripe restoration and matching on irregular surfaces or in complex measurement scenarios. Attached Figure Description

[0059] Figure 1 This is a system flowchart of the present invention;

[0060] Figure 2 This is a schematic diagram of the measurement system of the present invention;

[0061] Figure 3 This is a schematic diagram of the region segmentation of the present invention;

[0062] Figure 4 A schematic diagram illustrating the growth of the restricted region in this invention;

[0063] Figure 5 The blurred, broken, and widened light stripe projection image

[0064] Figure 6 This is a schematic diagram of the experimental results of the present invention. Detailed Implementation

[0065] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0066] Example 1: This invention provides a method for recovering and clustering light stripes using confined region growing. First, the object to be measured is segmented from the loading plane. Second, the extracted centerline stripes are clustered using confined region growing. Finally, based on sequential consistency constraints, the stripes after clustering in the left and right views are numbered sequentially using the stripe centroid coordinates. Stereo matching is then performed on stripes with the same number. The structural diagram is shown below. Figure 1 As shown.

[0067] like Figure 2 The diagram shows a measurement system based on a method for clustering and matching multi-line laser stripes using a custom window iteration. The system includes two industrial cameras (1), a divergent multi-line laser (2), the object being measured (3), a background plate (4), a support (5), and a worktable (6). The two cameras have identical parameters and are approximately parallel in the horizontal direction, symmetrically arranged on the support (5) with respect to the multi-line laser.

[0068] The method for clustering and matching multi-line laser stripes based on custom window iteration includes the following steps:

[0069] Step 1: Construct a multi-line laser measurement model based on binocular vision to obtain stripe projection images of the region of interest of the measured object from the perspectives of two cameras.

[0070] In multi-line laser orthographic projection, the projected stripes must be vertical or tilted in the view image. During image acquisition, to ensure accurate region segmentation, a solid-color background paper, distinct from the object being measured, needs to be placed on the measurement platform. This invention provides a background material for use on the measurement platform to effectively reduce the influence of background noise. This material can be clearly distinguished from the object being measured and can be of various colors and textures. Two cameras first acquire an image of the object being measured without laser stripe projection, without moving the object. Then, an image of the object being measured with structured light stripe projection is acquired. By detecting and extracting edge features from the image without laser stripe projection, and performing image dilation on the extracted edge contours, accurate closed image contours are obtained. These closed contours are then filled to obtain a mask image of the ROI region of the object being measured. This mask image is then superimposed on the image of the object being measured with stripe projection to obtain a stripe projection image after region of interest segmentation, as shown below. Figure 3 As shown. Figure 3 The image includes four sub-images. The top left and top right images are full projection images of the test object and the background with structured light stripes projected onto a black background, captured by a binocular camera. The bottom left and bottom right images are projection images of the test object after the top left and top right images have passed through a mask (removing the light stripes in the black background area) of the test object's ROI region.

[0071] Step 2: Blur, break, and enlarge the stripe projection image obtained in Step 1 to simulate the actual stripe deformation. Then, filter and binarize the processed stripe projection image to obtain a clear binary stripe image.

[0072] Furthermore, binarization refers to processing the stripes in the image into clear white against a black background. OpenCV's thresholding method is used to convert the widened, blurred, fragmented, and filtered image into a binary image. Specifically, the coordinates of each pixel are plotted with a pixel value of 255 on an image with the same format as the original image but all pixel values ​​are 0, resulting in a binary image of the center line stripes.

[0073] Step 3: Perform connected component extraction analysis on the binary image of the centerline generated in Step 2, extracting all connected regions (i.e., complete stripes or broken fragments) in the image. Calculate the centroid of the pixel coordinate set within each connected region. The centroid position is the seed point coordinate of that stripe. These points will serve as the starting points for subsequent region growth, such as... Figure 4The schematic diagram illustrates the principle of "stripes restoration in confined regions." The algorithm starts with a seed point P0 on the broken stripe and gradually grows the stripe structure forward by analyzing image gradient and curvature information within a local window. At each step, if the gradient change at the current point is gradual, the window is appropriately enlarged (e.g., W2, W3) to obtain more contextual information; if the gradient fluctuates drastically or the curvature is too large (e.g., C>Tc), the window is reduced to avoid incorrect stripe expansion or deviation from the original path. Finally, under the control of dynamically adjusting the window size, the algorithm achieves complete restoration of stripes in distorted regions such as broken or blurred areas, outputting a continuous and smooth "restored complete stripe" trajectory. Specifically:

[0074] 1) Connected component analysis: The binary stripe image obtained in step 2 is processed by the connected component analysis method to identify the independent region of each stripe. The pixel coordinates of each stripe are obtained by using the cv2.connectedComponents() method in OpenCV.

[0075] 2) Stripe Centerline Extraction: Analyze each stripe and calculate its centerline position. Specifically, calculate the average pixel value of each stripe region to obtain the centroid or center point coordinates (x, y, y) of that region. s ,y s );

[0076] 3) Saving seed point coordinates: Save the coordinates of the centroid or center point (x... s ,y s These points are saved as seed points, and their coordinates will serve as the starting points for subsequent restricted region growing algorithms.

[0077] Step 4: Based on the coordinates of each seed point, the stripes are recovered using the restricted region growing algorithm to obtain the stripe point coordinate data. The specific steps are as follows: using the local gradient information of the image, the size and shape of the growing window are dynamically adjusted to ensure that the stripe shape is recovered and to avoid the intersection and breakage between stripes.

[0078] The restricted region growing algorithm in step 4 specifically includes:

[0079] (1) Calculate the gradient of the input image I(x,y) by using the Sobel operator in both the horizontal and vertical directions to calculate the gradient magnitude:

[0080]

[0081] The gradient magnitude G(x,y) can then be expressed as:

[0082]

[0083] Among them, G x Gy These are the gradients of the image in the x and y directions, respectively, and the growth of the stripes is controlled using this gradient information.

[0084] (2) Initialize region growth parameters:

[0085] Using each stripe seed point S = (x) obtained in step 3 s ,y s Centered on ), set the initial growth window size to W. min and W max Define the gradient threshold as T g The curvature threshold is T c Real-time window size W can be dynamically adjusted within the following range:

[0086] W min ≤W≤W max W∈{2k+1|k∈N}

[0087] (3) Regional growth starting from the seed point:

[0088] Define the current pixel to be expanded as P. c =(x c ,y c ), with P c A growth window of size W×W is established centered on the growth window, and the gradient magnitude region within the growth window is defined as follows:

[0089]

[0090] And calculate the variance of the gradient magnitude within the window:

[0091]

[0092] like When the gradient change within the region is gradual, the window can be appropriately enlarged.

[0093] W = min(W+2, W) max )

[0094] like If the gradient changes drastically within a region, then the window size should be reduced.

[0095] W = max(W-2, W min )

[0096] (4) Pixel inclusion criteria:

[0097] For pixel P within the window n =(x n ,y n If the following conditions are met:

[0098] Pixel I(x)n ,y n The grayscale value of ) is non-zero and has not been processed;

[0099] Gradient magnitude G(x) n ,y n () is greater than the preset threshold;

[0100] After adding a new point, calculate the three points P where the fringe is locally continuous. i-2 ,P i-1 ,P i The resulting curvature C is calculated using the curvature calculation formula:

[0101]

[0102] If C≤T c Then P n Add the point to the region growth queue and mark it as processed; otherwise, discard the point to avoid abnormal distortion of the stripe boundary.

[0103] (5) Iterative process of region growth:

[0104] Continuously update the window center point P c For each new point in the queue, perform the above iterative process (3) and (4) until the queue is empty, thus completing the restoration of the striped region.

[0105] like Figure 5 As shown, Figure 5 This study demonstrates common distortion types in laser stripe images during real-world measurement scenarios, including stripe breaks, blurring, widening, missing stripes, curved blurring, and severe deflection due to surface deformation. These distortions reflect the multiple interference factors encountered by multi-line lasers during projection onto complex surfaces. These distortions severely interfere with the accurate extraction of stripe centerlines, affecting the accuracy of subsequent 3D reconstruction. Figure 6 Then it is Figure 5 The restored stripe structure after processing by the confined region growing algorithm proposed in this paper is shown in the figure. As can be seen, the originally broken, blurred, and missing stripes have been effectively repaired. The stripe center lines have been successfully extracted and marked with a solid red line. The green dashed box represents the adaptively adjusted growth window, and the green dots are the initial seed points for each stripe. This figure demonstrates the high robustness and fine-grained restoration capability of this method under complex distortion conditions, significantly improving the integrity and continuity of the stripes, and providing an accurate input basis for subsequent clustering numbering and dual-view matching.

[0106] Step 5: Based on the stripe point coordinate data recovered in Step 4, calculate the centroid coordinates of the stripes and number the stripes in the left and right views; the stripes with the same number in the left and right views are the corresponding matching stripes.

[0107] Centroid coordinates of each stripe (ave) x ,ave y The formulas for calculating the Euclidean distance *dis* from the centroid coordinates to the origin of the pixel coordinates are as follows:

[0108]

[0109] Among them, ave x ,ave y Let g be the x and y coordinates of the centroid of the fringe, and g be the value of each fringe after clustering within the container v. j The number of point coordinates contained in x l y l This represents the horizontal and vertical coordinates of a point in the stripe, where dis is the Euclidean distance from the centroid coordinates to the origin of the pixel coordinates.

[0110] Step 6: Compare the number of stripes in the left and right views. If they do not match, increase the radius of the region growth and repeat steps 4 and 5 until the number of stripes in the left and right views matches.

[0111] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, such as the model parameters of the divergent multi-line laser, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0112] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for recovering and clustering optical stripes based on growth in confined regions, characterized in that: Includes the following steps: Step 1: Construct a multi-line laser measurement model based on binocular vision to obtain fringe projection images of the region of interest of the measured object from the perspectives of two cameras; Step 2: Blur, break, and enlarge the stripe projection image obtained in Step 1 to simulate the actual stripe deformation. Then, filter and binarize the processed stripe projection image to obtain a clear binary stripe image. Step 3: Extract the center line of the stripes in the binary stripe image from Step 2 through connected component analysis, and obtain the seed point coordinates of each stripe as the starting point for subsequent restoration; Step 4: Based on the coordinates of each seed point, use the restricted region growing algorithm to recover the stripes and obtain the stripe point coordinate data; Step 5: Based on the stripe point coordinate data recovered in Step 4, calculate the centroid coordinates of the stripes and number the stripes in the left and right views; the stripes with the same number in the left and right views are the corresponding matching stripes. Step 6: Compare the number of stripes in the left and right views. If they do not match, increase the radius of the region growth and repeat steps 4 and 5 until the number of stripes in the left and right views matches. The restricted region growing algorithm in step 4 specifically includes: (1) Input image I(x,y) Gradient calculations are performed by applying the Sobel operator in both the horizontal and vertical directions to calculate the gradient magnitude of the image. ； Then gradient magnitude It can be represented as: ； in, , These represent the gradients of a given striped image in the x and y directions, respectively. The total gradient magnitude of the stripe on the image is used to control the growth of the stripe. (2) Initialize region growth parameters: Using each stripe seed point obtained in step 3 Centered on the growth window, set the maximum and minimum values ​​to [values]. Real-time window Size dynamic adjustment range: ； in The minimum allowable size of the growth window; : The maximum allowable size of the growth window; The current size of the growth window, and it must satisfy the following conditions: It is an odd number 2 +1, where It is a natural number; (3) Regional growth starting from the seed point: Define the current pixel to be expanded as =( , ),by Establish a size of centered × The growth window, within which the gradient magnitude region is defined as: ； : with point The set of all gradient magnitude points within a W×W window centered on the target; : Image in coordinates ( Gradient magnitude at () , : The horizontal and vertical coordinates of the center point of the current window; Half the side length of the window, rounded down; And calculate the variance of the gradient magnitude within the window: ； in :window variance of inner gradient magnitude :window Average value of internal gradient magnitude The total number of pixels contained in the window; like When the gradient change within the region is gradual, the window can be appropriately enlarged. ； like If the gradient changes drastically within a region, then the window size should be reduced. ； Gradient threshold: used to determine whether a pixel has sufficient gradient features; (4) Pixel inclusion criteria: For pixels within the window If the following conditions are met: Pixels The grayscale value is non-zero and has not been processed; gradient magnitude Greater than the preset threshold; After adding a new point, calculate the three points where the fringe is locally continuous. The resulting curvature C is calculated using the curvature calculation formula: ； like Then Add the point to the region growth queue and mark it as processed; otherwise, discard the point to avoid abnormal distortion of the fringe boundary. Curvature threshold, used to control the smoothness of stripe shape; (5) Iterative process of region growth: Continuously update the window center point For each new point in the queue, perform the above iterative process (3) and (4) until the queue is empty, thus completing the restoration of the striped region.

2. The method for optical fringe recovery and clustering matching based on confined region growth according to claim 1, characterized in that, Step 1 specifically includes the following steps: When constructing a divergent multi-line laser measurement model based on binocular vision, the multi-line laser needs to be orthographically projected and the projected stripes need to be vertical or tilted in the view image. During image acquisition, a solid-color background paper, distinct from the object being measured, is placed on the measurement platform. Two cameras first acquire an image of the object without laser stripe projection, without moving the object. Then, an image of the object with structured light stripe projection is acquired. By detecting and extracting edge features from the image without laser stripe projection, and performing image dilation on the extracted edge contours, accurate closed image contours are obtained. The closed contours are then filled to obtain a mask image of the ROI region of the object being measured. This mask image is then superimposed on the image of the object with stripe projection to obtain the stripe projection image after region of interest segmentation.

3. The method for optical fringe recovery and clustering matching based on confined region growth according to claim 1, characterized in that, The binarization process in step 2 refers to processing the stripes in the image into clear white and the background into black. OpenCV's thresholding method is used to convert the wide, blurred, broken and filtered image into a binary image.

4. The method for light stripe recovery and clustering matching based on confined region growth according to claim 1, characterized in that, The specific steps for step 3 are as follows: 1) Connected component analysis: The binary stripe image obtained in step 2 is processed by the connected component analysis method to identify the independent region of each stripe. The pixel coordinates of each stripe are obtained by the method in OpenCV. 2) Stripe centerline extraction: Analyze each stripe and calculate its centerline position. Specifically, calculate the centroid or center point coordinates of the region by averaging the pixels in each stripe area. ; 3) Saving seed point coordinates: Save the coordinates of the centroid or center point. These seed points are saved as seed points, and their coordinates will serve as the starting points for subsequent restricted region growing algorithms.

5. The method for optical stripe recovery and clustering matching based on confined region growth according to claim 4, characterized in that, In OpenCV, this refers to the cv2.connectedComponents() method.

Images