Line structure light stripe center line extraction method

Through Gaussian filtering, adaptive threshold segmentation and singular value decomposition, combined with grayscale center of gravity method and Taylor expansion, the rapid and accurate extraction of the center line of the linear structure light stripe is achieved, solving the problem of taking into account both accuracy and speed in the existing technology. , significantly improving the accuracy of three-dimensional reconstruction.

CN118982685APending Publication Date: 2024-11-19HENAN ALSONTECH INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411119603.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

In the prior art, the extraction method of the linear structure light stripe centerline cannot guarantee high accuracy and rapid extraction at the same time, affecting the three-dimensional reconstruction accuracy.

Method used

By using Gaussian filtering denoising processing, adaptive threshold segmentation, singular value decomposition and Taylor expansion, the center point of the linear structured light stripe is gradually extracted and accurately calculated, and combined with the grayscale center of gravity method and subpixel-level accuracy offset, we obtain The exact center point of the linear structured light stripes.

Benefits of technology

It effectively improves the accuracy and speed of centerline extraction, and significantly reduces standard error and program running time compared with traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982685A_ABST
    Figure CN118982685A_ABST
Patent Text Reader

Abstract

The invention discloses a line structure light stripe center line extraction method. The method comprises the following steps: carrying out denoising processing on an original image containing line structure light stripes; separating the light stripes from the image background; roughly extracting a light stripe center point, and taking an extraction result as an initial center point; taking a pixel point where the initial center point is located as a center to obtain an ROI, and constructing a data matrix; decomposing to obtain the normal direction of the data matrix, projecting the normal direction of the data matrix along a two-dimensional pixel plane, and taking the projection as the normal direction of an initial center point on the two-dimensional pixel plane; and calculating the sub-pixel-level precision offset of the initial center point, and obtaining the precise center point of the line structure light stripe. Compared with a traditional Steger method, the extraction time of the line structure light stripe center is shortened, and meanwhile, compared with a traditional gray gravity center method, the extraction speed of the line structure light stripe center is increased.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of non-contact vision measurement in the field of vision measurement, and in particular relates to a method for extracting the center line of a light stripe of a line structured light stripe projected onto a target object. Background Art

[0002] Visual measurement is an emerging technology that applies computer vision principles to accurately measure and locate the geometric dimensions of specific objects. Compared with traditional manual measurement technology, visual measurement has broad application prospects in the field of industrial inspection due to its unique advantages of non-contact, real-time, and high precision. At present, the most widely used types of optical non-contact three-dimensional information measurement methods are laser ranging and line structured light scanning. Among them, line structured light three-dimensional visual measurement has been widely used in the field of industrial inspection due to its high precision, real-time and applicability to industrial environments.

[0003] Since the center of the line structured light stripe contains the geometric features of the object to be measured, the extraction error of the center of the line structured light stripe will affect the subsequent 3D reconstruction accuracy. The current mainstream centerline extraction methods, such as the grayscale centroid method and the Steger method, have the problem of not being able to balance the extraction accuracy and extraction speed. Therefore, a method is needed to quickly and accurately extract the center of the laser stripe from the image. Summary of the invention

[0004] In view of the technical problems existing in the existing center line extraction methods, the present invention proposes a center line extraction method for line structured light stripes, which ensures the extraction accuracy while taking into account the extraction speed.

[0005] The present invention proposes a method for extracting the center line of a line structured light stripe, which specifically includes:

[0006] Step 1. De-noising the original image containing line structured light stripes;

[0007] Step 2. Separate the light streaks from the image background;

[0008] Step 3. Roughly extract the center point of the light stripe and use the extracted result as the initial center point;

[0009] Step 4. Obtain ROI with the pixel where the initial center point is located as the center and construct a data matrix;

[0010] Step 5. Decompose and obtain the normal direction of the data matrix, project the normal direction of the data matrix along the two-dimensional pixel plane, and use the projection as the normal direction of the initial center point on the two-dimensional pixel plane;

[0011] Step 6. Calculate the sub-pixel precision offset of the initial center point to obtain the precise center point of the line structured light stripe.

[0012] Furthermore, in step 1, a camera is used to collect an original image containing line structured light fringes, and a Gaussian filter denoising process is performed on the original image after grayscale processing.

[0013] Furthermore, in step 2, considering that the brightness of the line structured light stripes is higher than that of the projected object, an adaptive threshold segmentation method is used to separate the light stripes from the image background.

[0014] Furthermore, in step 3, the sub-pixel center point of the light streak is roughly extracted using the grayscale centroid method, and the sub-pixel center point is used as the initial center point of the light streak.

[0015] Furthermore, in step 4, a 5×5 pixel area is selected as the ROI with the pixel at the initial center point as the center, and a 25×3 matrix is ​​constructed with the horizontal and vertical coordinates of each pixel in the ROI and the corresponding grayscale value as elements, and the matrix is ​​used as the data matrix.

[0016] Furthermore, in step 5, the process of decomposing and obtaining the normal direction of the data matrix is ​​as follows:

[0017] The data matrix is ​​decomposed by singular value decomposition to obtain a 3×3 dimensional eigenvector matrix, a 25×25 dimensional eigenvector matrix and a 25×3 dimensional eigenvalue matrix. The principal element direction in the 3×3 dimensional eigenvector matrix corresponding to the minimum eigenvalue in the eigenvalue matrix is ​​used as the normal direction of the data matrix.

[0018] Furthermore, in step 6, the grayscale distribution function of the initial center point is Taylor expanded along the normal direction of the initial center point, and the sub-pixel precision offset of the initial center point is calculated based on the characteristic that the light intensity of the light stripe satisfies the Gaussian distribution.

[0019] Furthermore, in step 6, the sub-pixel precision offset is accumulated to the initial center point, and finally the precise center point of the line structured light stripe is obtained.

[0020] By means of the above technical solution, the method for extracting the center line of a line structured light stripe of the present invention has the following advantages:

[0021] (1) Comparative analysis of the standard errors of the residual values ​​calculated by the Bessel formula shows that compared with the traditional grayscale centroid method, the present invention can effectively improve the centerline extraction accuracy.

[0022] (2) Comparative analysis of program running time shows that compared with the traditional Steger method, the present invention can effectively improve the centerline extraction speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1The present invention is a flowchart of a method for extracting the center line of a linear structured light stripe according to an embodiment of the present invention.

[0024] Figure 2 This is an original image with linear structured light stripes in one embodiment of the present invention.

[0025] Figure 3 This is an image after Gaussian filtering and denoising according to an embodiment of the present invention.

[0026] Figure 4 This is a light streak region image after adaptive threshold segmentation according to an embodiment of the present invention.

[0027] Figure 5 This is an effect diagram of the initial center point of the line structured light stripes obtained by coarse extraction using the grayscale centroid method in one embodiment of the present invention.

[0028] Figure 6 This is a rendering of the precise center point of the line structured light stripes obtained according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The specific embodiments of the present invention will be described in detail below in conjunction with the schematic diagram to clearly introduce the advantages and features of the present invention. It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those of ordinary skill in the art to which the present application belongs. The accompanying drawings are only used to facilitate and clearly assist in explaining the purpose of the embodiments of the present invention.

[0030] See also Figure 1 , is an embodiment of a method for extracting the center line of a line structured light stripe according to the present invention. This embodiment specifically includes the following implementation steps:

[0031] Step 1: After graying the original image collected by the camera and containing line structured light fringes, perform Gaussian filtering and denoising. The generation formula of the 3×3 Gaussian filter operator required for denoising is shown in formula (1):

[0032]

[0033] In formula (1), u and v are the pixel coordinates of the original image, σ 2 is the variance, the original image is Figure 2 As shown in Figure 2, the image after Gaussian filtering and denoising is as follows: Figure 3 shown.

[0034] Step 2: Separate the light streaks from the image background. This embodiment uses an adaptive threshold segmentation method to separate the light streaks from the image background. The specific steps are as follows:

[0035] Assume the segmentation threshold is TH, the pixel area less than TH is C1, the pixel area greater than TH is C2, and the pixel means in the two areas C1 and C2 are m1 and m2 respectively, and the global mean of the image is m G Assume that the probability of a pixel being classified into C1 and C2 is p1 and p2 respectively, thus we can get formula (2) and formula (3):

[0036] p1m1+p2m2=m G (2)

[0037] p1+p2=1 (3)

[0038] According to the concept of variance, the expression of between-class variance can be obtained, as shown in formula (4):

[0039] σ 2 =p1(m1-m G ) 2 +p2(m2-m G ) 2 (4)

[0040] Substituting formula (2) into formula (4), we can get formula (5):

[0041] σ 2 =p1p2(m1-m2) 2 (5)

[0042] Finally, let σ 2 The maximum TH is used as the image adaptive segmentation threshold to separate the line structured light stripes from the image background. The separated light stripes are as follows: Figure 4 shown.

[0043] Step 3: roughly extract the center point of the light stripe, and use the extracted result as the initial center point. This embodiment uses the grayscale centroid method to extract the initial center line of the light stripe, and the corresponding expression is as follows:

[0044]

[0045] Since the line structured light fringes in this embodiment are collected parallel to the imaging plane, in formula (6), j is the vertical axis coordinate of each column of pixels, T is the set of pixels participating in the calculation in each column, g(*) is the grayscale value of each pixel in each column, and y is Center To obtain the sub-pixel precision vertical coordinate.

[0046] The horizontal axis coordinate is combined with the obtained sub-pixel precision vertical axis coordinate, and finally the sub-pixel center point extracted by grayscale centroid method is obtained, which is used as the initial center point of the light streak. The extraction effect of the initial center point is as follows: Figure 5 shown.

[0047] Step 4: Get the ROI with the pixel where the initial center point is located as the center and construct the data matrix. The construction process is as follows:

[0048] With the pixel point where the initial center point extracted in step 3 is located as the center, a 5×5 pixel area is delineated as the ROI, and a 25×3-dimensional matrix is ​​constructed with the horizontal and vertical coordinates and corresponding grayscale values ​​of each pixel point in the ROI as elements, and this matrix is ​​used as the data matrix.

[0049] Step 5: Decompose the data matrix, obtain the normal direction of the data matrix (the normal direction is a space vector), project the normal direction of the data matrix along the two-dimensional pixel plane, and use the projection as the normal direction of the initial center point on the two-dimensional pixel plane. This embodiment uses the singular value decomposition method to obtain the normal direction of the data matrix (the normal direction is a space vector), and the specific steps are as follows:

[0050] Assume that the data matrix obtained in step 4 is P, and the calculation process is as follows:

[0051] The calculation formula of singular value decomposition is shown in formula (7):

[0052] P=UΣV T (7)

[0053] First, multiply the respective transposed matrices on both sides, as shown in formula (8):

[0054] P T P=(UΣV T ) T UΣV T =VΣU T UΣV T (8)

[0055] Eliminating the orthogonal matrix U, we can obtain formula (9):

[0056] P T P=VΣΣV T (9)

[0057] Let L = ΣΣ, formula (9) can be simplified to formula (10):

[0058] P T P=VLV T (10)

[0059] Multiplying both sides of the equation by the transpose of the orthogonal matrix V, we can get formula (11):

[0060] P T PV=VL (11)

[0061] Repeat the above steps and multiply the respective transposed matrices on the right to obtain formula (12):

[0062] PP T U=UL (12)

[0063] According to the concepts of eigenvalue and eigenvector, Among them, λ, P T The eigenvalues ​​and eigenvectors of the P matrix. Finally, we get P T P and PP T The eigenvector group V, U and eigenvalue matrix L of. Among them, The eigenvector group V is a 3×3 matrix, and its corresponding three column vectors are the three principal component directions of the new coordinate system. The principal component direction corresponding to the minimum eigenvalue is used as the normal direction of the data matrix.

[0064] Step 6: Calculate the sub-pixel precision offset of the initial center point to obtain the precise center point of the light streak. The calculation process is as follows:

[0065] The grayscale distribution function of the initial center point is subjected to a second-order Taylor expansion along the normal direction of the initial center point, as shown in formula (13):

[0066]

[0067] Where t is the offset of the initial center point along the normal direction, n is the unit normal vector of the initial center point, is the unit vector in the normal direction of each initial center point, I x , I y , I xx , I xy , I yx , I yy They are the first-order and second-order partial derivatives of the image calculated using the Sobel operator, and they must satisfy

[0068]

[0069] According to the characteristic that the intensity of the light stripes satisfies the Gaussian distribution, we can get From this, we can derive formula (14):

[0070]

[0071] Assume that the initial center point of the line structured light stripe is (x c ,y c ), and finally, using the obtained offset (i.e., sub-pixel precision offset), the precise center point of the line structured light stripe can be obtained as (x c +tηx ,y c +tη y ), the extraction effect of the precise center point is as follows Figure 6 shown.

[0072] Then, the Bessel formula is used to compare the accuracy of the center point calculated by this method with that calculated by the Steger method and the traditional grayscale centroid method, and the program running time is used to compare the time difference between the center point calculated by this method and that calculated by the Steger method and the traditional grayscale centroid method. The calculation method of the Bessel formula is shown in formula (15):

[0073]

[0074] In the formula, Represents the residual of the center point along the x-axis or y-axis, n represents the exact number of center points in the x-axis direction of the image, and s represents the standard error along the x-axis or y-axis direction.

[0075] The feasibility of this method is verified by a specific example. A host computer with an Intel Core i7-11800H 2.3GHz CPU, 16GB memory, and an NVIDIA GeForce RTX 3060 graphics card is used to perform a computational analysis using Matlab 2016a. Figure 2 The original image with light streaks with a resolution of 1440×1080 shown in the figure is respectively extracted by the Steger method, the grayscale centroid method and the present method. The standard error comparison along the x-axis and y-axis directions and the program running time comparison results are shown in Tables 1 and 2:

[0076] Table 1 Standard error of each method

[0077]

[0078] It can be seen from the deviation comparison of this embodiment that the standard error of this method is lower than that of the grayscale centroid method, and has a trend of approaching the Steger method with higher accuracy.

[0079] Table 2 Program running time

[0080]

[0081] It can be seen from the running time comparison of this embodiment that compared with the high computational complexity of the Steger method, the processing speed of a single frame image of this method is significantly improved.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting the center line of a line structured light stripe, characterized in that include: Step 1. De-noise the original image containing line structured light stripes; Step 2. Separate the light streaks from the image background; Step 3. Roughly extract the center point of the light streak and use the extracted result as the initial center point; Step 4. Obtain the ROI with the pixel where the initial center point is located as the center and construct a data matrix; Step 5. Decompose and obtain the normal direction of the data matrix, project the normal direction of the data matrix along the two-dimensional pixel plane, and use the projection as the normal direction of the initial center point on the two-dimensional pixel plane; Step 6. Calculate the sub-pixel precision offset of the initial center point to obtain the precise center point of the line structured light streak.

2. A method for extracting the center line of a line structured light stripe according to claim 1, characterized in that In the step 1, a camera is used to capture an original image containing line structured light fringes, and the original image is gray-scaled and then subjected to Gaussian filtering and denoising.

3. The method for extracting the center line of a line structured light stripe according to claim 1, characterized in that In step 2, considering that the brightness of the line structured light stripes is higher than that of the projected object, an adaptive threshold segmentation method is used to separate the light stripes from the image background.

4. The method for extracting the center line of a line structured light stripe according to claim 1, characterized in that In step 3, the sub-pixel center point of the light streak is roughly extracted using the grayscale centroid method, and the sub-pixel center point is used as the initial center point of the light streak.

5. The method for extracting the center line of a line structured light stripe according to claim 1, characterized in that In step 4, a 5×5 pixel area is selected as the ROI with the pixel where the initial center point is located as the center, and a 25×3 matrix is ​​constructed with the horizontal and vertical coordinates and corresponding grayscale values ​​of each pixel in the ROI as elements, and the matrix is ​​used as the data matrix.

6. The method for extracting the center line of a line structured light stripe according to claim 1, characterized in that In step 5, the process of decomposing and obtaining the normal direction of the data matrix is ​​as follows: The data matrix is ​​decomposed using the singular value decomposition method to obtain a 3×3 dimensional eigenvector matrix, a 25×25 dimensional eigenvector matrix and a 25×3 dimensional eigenvalue matrix. The principal element direction in the 3×3 dimensional eigenvector matrix corresponding to the minimum eigenvalue in the eigenvalue matrix is ​​used as the normal direction of the data matrix.

7. The method for extracting the center line of a line structured light stripe according to claim 1, characterized in that In step 6, the grayscale distribution function of the initial center point is Taylor expanded along the normal direction of the initial center point, and the sub-pixel precision offset of the initial center point is calculated based on the characteristic that the light intensity of the light stripe satisfies the Gaussian distribution.

8. A method for extracting the center line of a line structured light stripe according to claim 1, characterized in that In step 6, the sub-pixel precision offset is accumulated to the initial center point to finally obtain the precise center point of the line structured light stripe.