High-precision line structured light center extraction method

Abstract

The invention provides a high-precision line structured light center extraction method, which belongs to the technical field of three-dimensional measurement, and specifically comprises the following steps: carrying out adaptive threshold segmentation on a line structured light image, and extracting a light strip candidate region; constructing a multi-scale multi-direction filter bank, and screening stable edge points through a direction consistency index; a fusion confidence model is established by combining gradient amplitude, gray level energy and curvature, and false edge points are effectively eliminated; after the reserved points are connected into a curve, a fracture area is repaired through piecewise linearity evaluation and a self-adaptive smoothing strategy, and the curve continuity is improved; and for the final curve, realizing sub-pixel-level center positioning by adopting a combined method of gray gravity center and neighborhood matching, and establishing a center point continuity constraint by utilizing an improved Markov random field to obtain a smooth and stable center line. The method has higher robustness and extraction precision under the conditions of high reflection, low contrast and multi-curvature surfaces, and is suitable for industrial visual inspection and three-dimensional scene reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional measurement technology, specifically relating to a high-precision method for extracting the center of a line structured light. Background Technology

[0002] Line structured light 3D measurement systems acquire the surface contour of the object being measured by projecting laser stripes and utilizing camera imaging principles. It is a non-contact 3D reconstruction method widely used in industrial inspection, robot vision, and microstructure measurement. One of its key steps is the light stripe center extraction algorithm, the accuracy of which directly determines the spatial resolution and measurement stability of subsequent 3D coordinate reconstruction. Existing light stripe center extraction methods mainly include the following categories: the gray-scale centroid method obtains the center point by calculating the centroid position of the gray-scale distribution of the light stripe; this method is computationally simple but sensitive to noise and brightness unevenness; the Steger algorithm, based on the second derivative of the image and Hessian matrix theory, can achieve sub-pixel level extraction, but is easily affected by errors in low signal-to-noise ratio or discontinuous stripe regions; and the Gaussian fitting method estimates the center position through local curve fitting, but is computationally complex and depends on the stability of the brightness distribution. Therefore, light stripe extraction algorithms have emerged, such as those described in patents with application numbers 202310336917.8 and 202510627537.9. However, these algorithms also have drawbacks: the former introduces directional information to enhance edge detection, but does not consider geometric features such as curvature and grayscale energy, which limits its applicability on reflective metals or low-contrast surfaces; the latter designs compensation mechanisms for dynamic scenes, but has limited ability to suppress stripe breaks and noise in static high-precision measurements. Summary of the Invention

[0003] The purpose of this invention is to propose a high-precision method for extracting the center of a line structured light, which solves the problems of low extraction accuracy, poor anti-interference ability, and insufficient computational efficiency of existing technologies under complex surface conditions.

[0004] This invention is achieved through the following technical solution:

[0005] A high-precision method for extracting the center of a line structured light source includes the following steps:

[0006] Step S1: After performing grayscale processing on the line structured light image acquired by the industrial camera, dynamic window adaptive threshold segmentation is used. The threshold calculation formula is: Where μ(x,y) is the local mean of pixel (x,y), σ(x,y) is the local standard deviation, R is the gray-level dynamic range constant, k is an adjustment parameter with a value range of 0.1 to 0.5, and var(x,y) is the neighborhood gray-level variance of pixel (x,y). max The maximum grayscale variance of the entire image is given; the threshold T(x,y) binarizes the image, preserving candidate regions for light stripes.

[0007] Step S2, constructing a dynamic scale direction filter set, which contains 8 direction angles (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°), and the scale number is dynamically adjusted according to the texture complexity of the light bar candidate region: when the local texture complexity TC(x, y) < TC th , 2 scales (s = 1, 2) are selected; when TC(x, y) ≥ TC th , 4 scales (s = 1, 2, 3, 4) are selected, wherein TC(x, y) is calculated by local gradient entropy, and TC th is a texture complexity threshold; the multi-scale direction response is obtained by convolution of the filter set and the binary image, and the pixel direction consistency criterion is calculated: where θ i is the main direction response angle at the i-th scale, is the average value of the main direction at all scales, N is the dynamically selected scale number, and α is the direction penalty coefficient and takes a value of 0.8-1.2; when C ≥ 0.6, it is determined that the pixel is a candidate edge point;

[0008] Step S3, calculating the gradient amplitude of the candidate edge point and introducing a multi-feature fusion confidence model: where G x , G y are the x and y direction gradients, E is the local gray energy, which is calculated by the 3 × 3 neighborhood gray square sum, and K is the curve curvature and is defined as y', y" are the first and second order derivatives of the curve, G max , E max , K max are the maximum gradient amplitude, the maximum gray energy, and the maximum curvature absolute value of the whole image, respectively; ω1, ω2, ω3 are dynamic weights and satisfy ω1 + ω2 + ω3 = 1, wherein σ G , σ E , σ K are the global standard deviations of the gradient amplitude, the gray energy, and the curvature, respectively, only the candidate edge points with P ≥ 0.7 are retained;

[0009] Step S4, connecting the screened candidate edge points into candidate curves according to the 8-neighbor connectivity, fitting each candidate curve by using the piecewise least square method, and calculating the piecewise linearity index: where M is the curve segmentation number, res j is the j-th fitting residual, res th is the standard residual threshold, and β is the residual penalty coefficient and takes a value of 0.6-1.0; when L < 0.8, the smoothing weight is automatically adjusted: Where λ0 is the initial smoothing weight, with a value of 0.3 to 0.5. The adjusted λ is used to perform weighted Gaussian smoothing on the candidate curve to optimize the continuity of the curve.

[0010] Step S5: For the smoothed candidate curve points, use gray-level centroid-neighborhood matching joint localization: First, calculate the initial sub-pixel coordinates using the gray-level centroid method: I(x,y) represents the pixel gray value; then, a 5×5 neighborhood gray value matrix is ​​constructed, and the similarity of the gray value distribution within the neighborhood is calculated: Ω represents the neighborhood range, and Δx and Δy represent the coordinate offsets. The initial coordinates are corrected based on Δx and Δy when S is at its maximum to obtain the final sub-pixel level coordinates.

[0011] Step S6: Establish continuity constraints for adjacent center points based on the improved Markov random field (MRF): Define the energy function: E = Σ i，j [γ1·|Δd i,j -d0|+γ2·|Δθ i,j -θ0|], where Δd i,j Let Δd0 be the distance deviation between the i-th and j-th center points, Δθ be the ideal distance, and Δθ be the distance between the i-th and j-th center points. i,j Let Δθ0 be the directional angle deviation of the i-th and j-th center points, γ1 and γ2 be the ideal directional angle, and γ1 and γ2 be the weighting coefficients with values ​​of 0.6 to 0.8 and 0.2 to 0.4, respectively. A dynamic programming algorithm is used to solve for the path corresponding to the minimum energy, and a continuous and smooth light stripe centerline is output.

[0012] Furthermore, in step S1, the size of the dynamic window is adaptively adjusted according to the width of the light stripe: when the width of the light stripe ω ≤ 5 pixels, the window size is 5×5; when 5 < ω ≤ 10 pixels, the window size is 7×7; when ω > 10 pixels, the window size is 9×9. The width of the light stripe ω is obtained by counting the number of horizontal pixels of the light stripe in the binarized image.

[0013] Furthermore, in step S2, p(g) is the probability of grayscale value g appearing in the 3×3 neighborhood of pixel (x,y), TC th The TC(x,y) distribution of the entire image is adaptively determined using the Otsu algorithm, resulting in texture complexity.

[0014] The calculation method for TC(x,y) is as follows:

[0015] Furthermore, in step S3, the calculation process of the curve curvature K is as follows: a quadratic curve y = ax is fitted to the candidate edge point sequence using a 5-point sliding window. 2 +bx+c, calculate the first and second derivatives using y=2ax+b and y″=2a, and substitute them into the curvature formula to obtain K.

[0016] Further, in the step S4, the candidate curve segment rule is that when the distance between two adjacent candidate edge points is greater than 2 pixels or the direction angle difference is greater than 30°, the curve is segmented, the segment number M is adaptively adjusted according to the total length of the curve, and each segment contains 8-12 pixel points.

[0017] Further, in the step S5, the calculation range of the neighborhood gray scale similarity S is Δx∈[-0.5, 0.5], Δy∈[-0.5, 0.5], and the step is 0.1 pixel; when S≥0.9, the initial coordinates are corrected by using the corresponding Δx and Δy; and when S<0.9, the initial gray scale gravity center coordinates are retained.

[0018] Further, in the step S6, the ideal distance d0 is 1.2 times of the theoretical pixel spacing of the light bar, the ideal direction angle θ0 is the average of the fitting direction angle of the current segment curve, and the path search range of the dynamic programming is the 5*5 neighborhood of the current center point.

[0019] Further, the method is suitable for three-dimensional measurement scenes of reflective metal surfaces, wet medium surfaces or multi-curvature surfaces, and through the joint evaluation of dynamic scale filtering, multi-feature confidence and segment linearity, the automatic repair of the light bar broken area and the accurate removal of the false edge points are realized.

[0020] The present application has the following beneficial effects:

[0021] 1、The line structured light center extraction method provided by the present application has significant technical superiority and wide application prospect. The present application obtains light bar main direction information from the perspective of multi-scale and multi-direction, and removes false edge points caused by reflection, noise and other factors through a direction consistency index, so that the edge detection maintains more stable response characteristics under low contrast, high reflection and multi-curvature surfaces; a multi-feature fusion confidence model based on gradient amplitude, gray energy and curvature is proposed, and a segment linearity evaluation and adaptive smoothing mechanism are introduced, so that the algorithm can automatically adjust parameters according to the curve shape, repair the broken area, and reduce false detection caused by discontinuous stripes, thereby improving the integrity and robustness of the overall curve; a sub-pixel positioning method combining gray gravity center and neighborhood matching is used, and an improved Markov random field is used to establish a center point continuity constraint, so that the extracted center line has higher smoothness, continuity and positioning accuracy. The present application has significant improvement in extraction accuracy, noise resistance and real-time performance, has the advantages of simple structure, efficient calculation and strong robustness, and can be widely applied to the fields of industrial vision measurement, three-dimensional reconstruction and high-reflective surface topography detection. BRIEF DESCRIPTION OF DRAWINGS

[0022] The present application will be further described in detail below with reference to the drawings.

[0023] Figure 1 It is a schematic diagram of the overall process of the method of the present application.

[0024] Figure 2 Steger method result schematic diagram in center extraction method.

[0025] Figure 3 Gray center method result schematic diagram in center extraction method.

[0026] Figure 4 Direction template method result schematic diagram in center extraction method.

[0027] Figure 5 Method result schematic diagram of the application. DETAILED DESCRIPTION

[0028] The embodiment takes a hollow fiber membrane end face three-dimensional measurement system as an application background, and the hardware of the system mainly comprises a KYX635N5-X1685 type line laser emitter with a wavelength of 635 nm, a MER2-231-41U3M / C plane array industrial camera, a V2520-MPZ fixed focus lens and a precision horizontal displacement platform, and the line structure light image is collected and processed.

[0029] As shown in the overall flow schematic diagram of the method of the application, the specific implementation steps are as follows: Figure 1

[0030] Step one: image acquisition and pretreatment

[0031] The line laser is vertically projected to the surface of the measured object, in order to eliminate the interference of ambient light on the image quality, the shooting process is completed in a dark box, and after the industrial camera collects the line structure light image at a fixed angle, the image is first converted into an 8-bit gray scale image; then the number of transverse pixels of the light bar in the binary pretreatment image is counted to determine the light bar width ω, and the dynamic window size is adaptively selected according to the value of ω: when the light bar width ω is less than or equal to 5 pixels, the window size is 5*5; when 5<ω≤10 pixels, the window size is 7*7; when ω>10 pixels, the window size is 9*9; for each pixel (x, y), the local mean μ(x, y), the local standard deviation σ(x, y) and the neighborhood gray scale variance var(x, y) are calculated, and the maximum gray scale variance var of the whole image is obtained through global statistics. max The above parameters are substituted into the threshold calculation formula: Wherein R is 255, k is 0.2, when the pixel gray value I(x, y) is greater than or equal to T(x, y), the pixel is set to 1, otherwise it is set to 0, and finally a binary image retaining only the light bar candidate area is obtained.

[0032] Step two: dynamic scale direction filtering and direction consistency screening

[0033] ​For each pixel in the binary image, the occurrence probability p(g) of the gray value g in the 3x3 neighborhood is calculated and substituted into the formula: The texture complexity TC(x, y) is calculated, and the Otsu algorithm is used to adaptively determine the texture complexity threshold TC th from the distribution of TC(x, y) of the entire image; a dynamic scale direction filter set with eight direction angles (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°) is constructed, and the number of scales is selected according to the size relationship between TC(x, y) and TC th : when TC(x, y) < TC th , two scales (s = 1, 2) are selected; when TC(x, y) ≥ TC th , four scales (s = 1, 2, 3, 4) are selected, and the filter function is constructed based on the Gaussian kernel as h(x, y, θ, s) = G(x, y, s) · cos(θ - arctan2(x, y)), where G(x, y, s) is the Gaussian kernel function; the binary image is convolved with the filter of each direction and each scale to obtain a multi-scale direction response, and for each pixel, the direction with the maximum response under different scales is selected as the main direction θ i , which is substituted into the direction consistency criterion formula: where α is 0.1, is the average value of the main directions under all scales, and N is the number of dynamically selected scales. When C ≥ 0.6, it is determined that the pixel is a candidate edge point.

[0034] Step three: multi-feature fusion confidence screening

[0035] The x-direction gradient G x and the y-direction gradient G y of the candidate edge point are calculated using the Sobel operator, and the gradient amplitude G is obtained through The maximum gradient amplitude G max of the entire image is calculated; the local gray energy E is calculated through the 3x3 neighborhood gray square sum, that is, The maximum gray energy E max of the entire image is calculated; a five-point sliding window is used to fit a quadratic curve y = ax 2 + bx + c for the candidate edge point sequence, the first and second derivatives are calculated through y' = 2ax + b and y'' = 2a, the curve curvature K is obtained by substituting the curvature formula , and the maximum absolute value K max of the curvature of the entire image is calculated; the dynamic weight W is calculated where σ G , σ E , and σ KLet ω1, ω2, and ω3 be the global standard deviations of gradient magnitude, grayscale energy, and curvature, respectively, and satisfy ω1 + ω2 + ω3 = 1. Substitute the dynamic weights and the above feature parameters into the confidence model. Only candidate edge points with P≥0.7 are retained.

[0036] Step 4: Piecewise linearity evaluation and adaptive smoothing

[0037] The selected candidate edge points are connected into candidate curves based on 8-neighbor connectivity. When the distance between two adjacent candidate edge points is greater than 2 pixels or the directional angle difference is greater than 30°, the curve is segmented, with each segment containing 8 to 12 pixels. The number of segments M is adaptively adjusted according to the total length of the curve. Piecewise least squares fitting is used for each curve segment, and the fitting residual res of the j-th segment is calculated. j Set the standard residual threshold res th =0.5 pixels, substitute into the piecewise linearity index formula β is set to 0.8; when L < 0.8, the smoothing weights are automatically adjusted. Where λ0 is set to 0.4, the candidate curve is subjected to weighted Gaussian smoothing using the adjusted λ to optimize the continuity of the curve and repair the broken areas.

[0038] Step 5: Gray-scale centroid-neighborhood matching combined with sub-pixel localization

[0039] For the smoothed candidate curve points, extract their 3×3 neighborhoods and calculate the initial sub-pixel coordinates using the gray-level centroid method, as shown in the formula. Where I(x,y) is the pixel gray value; construct a 5×5 neighborhood gray value matrix, set the coordinate offset range to Δx∈[-0.5,0.5], Δy∈[-0.5,0.5], and the step size to 0.1 pixels; substitute into the neighborhood gray value distribution similarity formula: Where Ω is the 5x5 neighborhood range; when S≥0.9, the initial coordinates are corrected using the corresponding Δx and Δy; when S<0.9, the initial gray-scale centroid coordinates are retained to obtain the final sub-pixel level coordinates.

[0040] Step Six: Improve the continuity optimization of MRF and dynamic programming

[0041] The ideal distance d0 is set to 1.2 times the theoretical pixel spacing of the light stripe, and the ideal direction angle θ0 is the mean of the direction angle fitted to the current segment of the curve. An energy function is established based on the improved Markov random field (MRF): E = ∑ i，j [γ1·|Δd i,j -d0|+γ2·|Δθ i,j -θ0|], where Δd i,j Let Δd0 be the distance deviation between the i-th and j-th center points, Δθ be the ideal distance, and Δθ be the distance between the i-th and j-th center points. i,jThe direction angle deviation of the i, j center point is 0.7 and 0.3 respectively, γ1 and γ2 are set as 0.7 and 0.3 respectively; the path searching range of dynamic programming is set as 5*5 neighborhood of the current center point, the path corresponding to the minimum energy is solved, 3*3 neighborhood mean filtering is performed on the output center line, and finally the continuous and smooth light strip center line is obtained.

[0042] In order to verify the effectiveness of the present application, the present application method is compared with Steger algorithm, gray gravity method and direction template method in the traditional center extraction method under the same experimental environment, and the results are shown in Figs. Figure 2 、 3 , 4.

[0043] The light strip center line extracted by the present application method is the best in continuity, and there is no obvious fracture or deviation phenomenon. In terms of average root mean square error, it is reduced by 14.91% compared with Steger method, reduced by 10.30% compared with gray gravity method, and reduced by 20.97% compared with direction template method. In terms of calculation time, it is reduced by 3.08% compared with Steger method, and reduced by 26.89% compared with direction template method.

[0044] The above is only a preferred embodiment of the present application, and therefore cannot limit the scope of the present application. Equivalent changes and modifications made according to the scope of the present application patent and the content of the specification should still be within the scope of the present application patent.

Claims

1. A high-precision method for extracting the center of a line structured light source, characterized in that: Includes the following steps: Step S1: After performing grayscale processing on the line structured light image acquired by the industrial camera, dynamic window adaptive threshold segmentation is used. The threshold calculation formula is: Where μ(x,y) is the local mean of pixel (x,y), σ(x,y) is the local standard deviation, R is the gray-level dynamic range constant, k is an adjustment parameter with a value range of 0.1 to 0.5, and var(x,y) is the neighborhood gray-level variance of pixel (x,y). max The maximum grayscale variance of the entire image is given; the threshold T(x,y) binarizes the image, preserving candidate regions for light stripes. Step S2: Construct a dynamic scale orientation filter bank, which includes 8 orientation angles (0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, 157.5°), and the number of scales is dynamically adjusted according to the texture complexity of the light stripe candidate region: when the local texture complexity TC(x,y) < TC th When , two scales are selected (s = 1, 2); when TC(x,y) ≥ TC th At that time, four scales were selected (s = 1, 2, 3, 4), where TC(x,y) was calculated using local gradient entropy. th As a texture complexity threshold; multi-scale directional responses are obtained by convolving a filter bank with a binarized image, and pixel orientation consistency criteria are calculated: Where θ i Let be the principal direction response angle at the i-th scale. C is the average value of the principal direction across all scales, N is the number of dynamically selected scales, and α is the direction penalty coefficient ranging from 0.8 to 1.

2. When C ≥ 0.6, the pixel is determined to be a candidate edge point. Step S3: Calculate the gradient magnitude of the candidate edge points. And a multi-feature fusion confidence model is introduced: Among them G x G y Let x and y be the gradients, respectively; E be the local gray-level energy, calculated using the sum of squared gray levels in a 3×3 neighborhood; and K be the curve curvature, defined as... y′ and y″ are the first and second derivatives of the curve, respectively, and G max E max K max These represent the maximum gradient magnitude, maximum grayscale energy, and maximum absolute curvature of the entire image, respectively; ω1, ω2, and ω3 are dynamic weights satisfying ω1 + ω2 + ω3 = 1, where σ G σ E σ K These represent the global standard deviations of gradient magnitude, grayscale energy, and curvature, respectively. Only candidate edge points with P ≥ 0.7 are retained; Step S4: Connect the selected candidate edge points into candidate curves based on 8-neighbor connectivity. Fit each candidate curve using piecewise least squares method and calculate the piecewise linearity index. Where M is the number of curve segments, res j Let res be the fitting residual for the j-th segment. th The standard residual threshold is β, which is the residual penalty coefficient and ranges from 0.6 to 1.0; when L < 0.8, the smoothing weights are automatically adjusted. Where λ0 is the initial smoothing weight, with a value of 0.3 to 0.

5. The adjusted λ is used to perform weighted Gaussian smoothing on the candidate curve to optimize the continuity of the curve. Step S5: For the smoothed candidate curve points, use gray-level centroid-neighborhood matching joint localization: First, calculate the initial sub-pixel coordinates using the gray-level centroid method: I(x,y) represents the pixel gray value; then, a 5×5 neighborhood gray value matrix is ​​constructed, and the similarity of the gray value distribution within the neighborhood is calculated: Ω represents the neighborhood range, and Δx and Δy represent the coordinate offsets. The initial coordinates are corrected based on Δx and Δy when S is at its maximum to obtain the final sub-pixel level coordinates. Step S6: Establish continuity constraints for adjacent center points based on the improved Markov random field (MRF): Define the energy function: E = Σ i，j [γ1·|Δd i,j -d0|+γ2·|Δθ i,j -θ0|], where Δd i,j Let Δd0 be the distance deviation between the i-th and j-th center points, Δθ be the ideal distance, and Δθ be the distance between the i-th and j-th center points. i,j Let Δθ0 be the directional angle deviation of the i-th and j-th center points, γ1 and γ2 be the ideal directional angle, and γ1 and γ2 be the weighting coefficients with values ​​of 0.6 to 0.8 and 0.2 to 0.4, respectively. A dynamic programming algorithm is used to solve for the path corresponding to the minimum energy, and a continuous and smooth light stripe centerline is output.

2. The high-precision line structured light center extraction method according to claim 1, characterized in that: In step S1, the size of the dynamic window is adaptively adjusted according to the width of the light stripe: when the width of the light stripe ω≤5 pixels, the window size is 5×5; when 5<ω≤10 pixels, the window size is 7×7; when ω>10 pixels, the window size is 9×9. The width of the light stripe ω is obtained by counting the number of horizontal pixels of the light stripe in the binarized image.

3. The high-precision line structured light center extraction method according to claim 1, characterized in that: In step S2, p(g) is the probability of gray value g appearing in the 3×3 neighborhood of pixel (x,y), TC th The TC(x,y) distribution of the entire image is adaptively determined using the Otsu algorithm. The texture complexity TC(x,y) is calculated as follows:

4. The high-precision line structured light center extraction method according to claim 1, characterized in that: In step S3, the curve curvature K is calculated as follows: a quadratic curve y = ax is fitted to the candidate edge point sequence using a 5-point sliding window. 2 +bx+c, calculate the first and second derivatives using y′=2ax+b and y″=2a, and substitute them into the curvature formula to obtain K.

5. The high-precision line structured light center extraction method according to claim 1, characterized in that: In step S4, the candidate curve segmentation rule is as follows: when the distance between two adjacent candidate edge points is greater than 2 pixels or the directional angle difference is greater than 30°, the curve is segmented. The number of segments M is adaptively adjusted according to the total length of the curve, and each segment contains 8 to 12 pixels.

6. The high-precision line structured light center extraction method according to claim 1, characterized in that: In step S5, the calculation range of the neighborhood grayscale similarity S is: Δx∈[-0.5,0.5], Δy∈[-0.5,0.5], with a step size of 0.1 pixels; when S≥0.9, the corresponding Δx and Δy are used to correct the initial coordinates; when S<0.9, the initial grayscale centroid coordinates are retained.

7. The high-precision line structured light center extraction method according to claim 1, characterized in that: In step S6, the ideal distance d0 is 1.2 times the theoretical pixel spacing of the light stripe, the ideal direction angle θ0 is the mean of the direction angle fitted to the current segment curve, and the path search range of dynamic planning is the 5×5 neighborhood of the current center point.

8. A high-precision line structured light center extraction method according to claims 1-7, characterized in that: This method is applicable to three-dimensional measurement scenarios of reflective metal surfaces, wet medium surfaces, or surfaces with multiple curvatures. Through dynamic scale filtering, joint evaluation of multi-feature confidence and piecewise linearity, it can achieve automatic repair of light stripe breakage areas and accurate removal of false edge points.

Images