A Robust Method for Extracting the Center Line of Line Structured Light

Abstract

The present invention discloses a method for extracting the center line of a robust line structured light, and the method includes: constructing a database, and storing the scene feature information of different laser lines into the database; obtaining the feature information of a target laser line, comparing the feature information with the scene feature information in the database to obtain the scene information of the target laser line; matching a corresponding extraction algorithm based on the scene information, optimizing the parameters of the extraction algorithm, and using the optimized center line extraction algorithm to extract the center line of the structured light image. The present invention reversely deduces the environment where the sensor is located according to the light intensity distribution characteristics of the laser line, adaptively adjusts the parameter information of the center line extraction algorithm, and finally realizes the extraction of the center line of the line structured light image.

Description

Technical Field

[0001] The present invention relates to the technical field of line structured light, and particularly to a method for extracting the center line of robust line structured light. Background Art

[0002] Structured light three-dimensional measurement is an active three-dimensional optical measurement method. A structured light three-dimensional measurement system mainly includes a CCD camera and a laser. According to the different shapes of the laser projected onto the surface of the object to be measured, structured light can be divided into point structured light, line structured light, and multi-line structured light. Structured light sensors, with their characteristics of non-contact, high precision, strong real-time performance, and active control, are increasingly widely used in fields such as reverse engineering, industrial inspection, and medical diagnosis, as Figure 1 shown. Among them, a line structured light sensor projects a line light stripe onto the surface of the object to be measured, and obtains the two-dimensional contour of the measured surface by analyzing the deformed light stripe image obtained through modulation, and is mainly applied to fields such as industrial measurement, road disease detection, and computer vision.

[0003] Currently, line structured light is mainly applied to scenarios with stable ambient light to ensure obtaining a laser line image with uniform and stable light quality. For complex scenarios such as outdoor detection and underwater detection, the laser line in the image is interfered by the external ambient light, and characteristics such as the thickness and brightness of the laser line in the image will change. Traditional methods in the prior art usually require the laser line to meet specific brightness characteristics, and extract the laser center line through corresponding calculation models. This mode can usually only solve one or several of these situations. For example, the extreme value method is mainly applicable to line structured light with a light stripe conforming to an ideal Gaussian distribution and cannot be applied to line structured light with saturated brightness; the Gaussian curve fitting method is proposed based on the phenomenon that the normal gray value of the line structured light stripe approximately conforms to a Gaussian distribution and cannot adapt to the situation of uneven light stripe width; the Steger method based on the Hessian matrix has high extraction accuracy, but has high requirements for both the shape and continuity of the light stripe.

[0004] Traditional algorithms for extracting the center line of line structured light in the prior art mainly target a single scenario. When the structured light detection covers multiple different types of lighting environments, it is impossible to ensure robustness, and it is necessary to continuously adjust the parameters of the algorithm, and even replace different center line extraction algorithms.

[0005] Therefore, the prior art still needs to be improved and developed. Summary of the Invention

[0006] The main purpose of the present invention is to provide a method for extracting the center line of robust line structured light, aiming to solve the problem in the prior art that when the environment changes, a single algorithm cannot ensure robustness, and it is necessary to repeatedly adjust and calibrate the parameters of the algorithm, and even replace different center line extraction methods.

[0007] To achieve the above object, the present invention provides a method for extracting the center line of a robust line structured light, and the method for extracting the center line of the robust line structured light includes the following steps:

[0008] Construct a database, and store the scene feature information of different laser lines into the database;

[0009] Obtain the feature information of the target laser line, compare the feature information with the scene feature information in the database, and obtain the scene information of the target laser line;

[0010] Based on the scene information, match the corresponding extraction algorithm, optimize the parameters of the extraction algorithm, and use the optimized center line extraction algorithm to extract the center line of the structured light image.

[0011] Optionally, for the method for extracting the center line of the robust line structured light, wherein the constructing a database and storing the scene feature information of different laser lines into the database is specifically:

[0012] Pre-construct a database, where the database includes an extraction algorithm, a scene type, and scene feature parameters, and store different extraction algorithms, scene types, and scene feature parameters into the database.

[0013] Optionally, for the method for extracting the center line of the robust line structured light, wherein the obtaining the feature information of the target laser line, comparing the feature information with the scene feature information in the database, and obtaining the scene information of the target laser line specifically includes:

[0014] Obtain a set number of consecutive frame images of the target laser line, and determine the area of the laser stripe in each image;

[0015] Randomly generate a certain number of seed points within the area of the laser stripe, obtain sub-images according to the seed points, and calculate the sub-image feature parameters, where the sub-image feature parameters include a relative brightness feature parameter and a width feature parameter of the target laser line;

[0016] Compare the distance between the sub-image feature parameters and the feature parameters in the database, and obtain the scene information of the laser line after the comparison is completed.

[0017] Optionally, for the method for extracting the center line of the robust line structured light, wherein the obtaining a set number of consecutive frame images of the target laser line and determining the area of the laser stripe in each image further includes:

[0018] Take the pixels at the maximum gradient positions of each column of pixels in the continuous frame images as the upper and lower boundaries of the laser stripe. Among them, the maximum value of the neighborhood cumulative difference is taken as the maximum gradient, and the neighborhood cumulative difference values upward and downward of each pixel are calculated using the following formulas:

[0019] s down = [f(i, j) - f(i + 1, j)] + [f(i, j) - f(i + 2, j)] + … + [f(i, j) - f(i + m - 1, j)]

[0020] s up = [f(i, j) - f(i - 1, j)] + [f(i, j) - f(i - 2, j)] + … + [f(i, j) - f(i - m + 1, j)];

[0021] Among them, s down represents the neighborhood cumulative difference value downward of the pixel, and S up represents the neighborhood cumulative difference value upward of the pixel. i represents the row coordinate, j represents the column coordinate, f(i, j) represents the gray value of the pixel at the i-th row and j-th column, and m represents the number of rows of the long strip window used in the neighborhood cumulative difference process.

[0022] Optionally, for the robust line structured light centerline extraction method, the formula for calculating the relative brightness feature is as follows:

[0023]

[0024] Among them, C represents the relative brightness feature, μ T represents the average gray value of the pixels in the laser line region, and μ B represents the average gray value of the pixels in the background region.

[0025] Optionally, for the robust line structured light centerline extraction method, compare the distance between the sub-image feature parameters and the feature parameters in the database. The formula for calculating the distance is as follows:

[0026]

[0027] Among them, d represents the distance between the sub-image feature parameters and the feature parameters in the database, i represents the number of feature values, f i represents the i -th feature value of the sub-image, and g i represents the i-th feature value in a certain scenario in the method-scene-feature parameter library.

[0028] Optionally, for the robust line structured light centerline extraction method, based on the scene information, the parameters of the centerline extraction algorithm are optimized to extract the centerline of the line structured light image, specifically including:

[0029] Perform sub-pixel processing on the collected image data in integer form to obtain sample data with super-resolution increment;

[0030] Combine the sample data with the manually marked data to construct a scene sample set;

[0031] Based on the scene sample set, randomly initialize the centerline extraction algorithm and extract the centerline result of the manually marked data;

[0032] Subtract the centerline result from the ground truth to obtain the difference between corresponding points, and calculate the average error between the centerline and the ground truth. If the average error exceeds the set pixel value, use the gradient descent method for optimization to complete the extraction of the final centerline result.

[0033] In the present invention, a database is constructed, and the scene feature information of different laser lines is stored in the database; the feature information of the target laser line is obtained, and the feature information is compared with the scene feature information in the database to obtain the scene information of the target laser line; based on the scene information, the corresponding extraction algorithm is matched, the parameters of the extraction algorithm are optimized, and the optimized centerline extraction algorithm is used to extract the centerline of the structured light image. The present invention inversely deduces the environment where the sensor is located according to the light intensity distribution characteristics of the laser line, and adaptively adjusts the parameter information of the centerline extraction algorithm, and finally realizes the extraction of the centerline of the line structured light image. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic diagram of the application field of structured light;

[0035] Figure 2 is a schematic diagram of the principle of laser triangulation in the prior art;

[0036] Figure 3 is a schematic diagram of the three-dimensional measurement principle of line structured light in the prior art;

[0037] Figure 4 is a schematic diagram of the three-dimensional measurement of line structured light in the prior art;

[0038] Figure 5 is a schematic diagram of light stripe centerline extraction;

[0039] Figure 6 is a schematic diagram of the extreme value method in the centerline extraction method;

[0040] Figure 7It is a schematic diagram of the threshold method in the centerline extraction method;

[0041] Figure 8 It is a schematic diagram of the gray center of gravity method in the centerline extraction method;

[0042] Figure 9 It is a schematic diagram of the curve fitting method in the centerline extraction method;

[0043] Figure 10 It is the laser image of the laser line when the ambient light is strong;

[0044] Figure 11 It is the laser image of the laser line when the ambient light is weak;

[0045] Figure 12 It is the flowchart of the preferred embodiment of the robust line structured light centerline extraction method in the present invention;

[0046] Figure 13 It is the laser line diagram extracted by using the Steger algorithm when the ambient light is strong;

[0047] Figure 14 It is the laser line diagram extracted by using the gray center of gravity algorithm when the ambient light is weak;

[0048] Figure 15 It is the flowchart of step S20 in the preferred embodiment of the robust line structured light centerline extraction method in the present invention;

[0049] Figure 16 It is the flowchart of the scene inversion based on the laser line image features in the present invention;

[0050] Figure 17 It is a schematic diagram of the collected consecutive frame images;

[0051] Figure 18 It is the schematic diagram of the downward and upward neighborhood cumulative difference in the embodiment of the present invention;

[0052] Figure 19 It is the binary schematic diagram of the optical stripe in the present invention;

[0053] Figure 20 It is a schematic diagram of the sub-image randomly selected within the laser line region;

[0054] Figure 21 It is the flowchart of step S30 in the preferred embodiment of the robust line structured light centerline extraction method in the present invention;

[0055] Figure 22 It is the schematic diagram of the self-optimized centerline extraction method in the embodiment of the present invention;

[0056] Figure 23 It is the schematic diagram of the artificial marked data in the embodiment of the present invention;

[0057] Figure 24 It is a schematic diagram of the centerline result extracted by the gray - scale centroid method;

[0058] Figure 25 It is a schematic diagram of the change of error with the number of iterations;

[0059] Figure 26 It is a schematic diagram of the final centerline extraction result in the embodiment of the present invention;

[0060] Figure 27 It is a schematic diagram of the overall process of the robust line - structured light centerline extraction method in the present invention. Detailed implementation manners

[0061] To make the objectives, technical solutions and effects of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific examples described herein are only used to explain the present invention and are not used to limit the present invention.

[0062] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present invention, the directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0063] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first", "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0064] In the prior art, line - structured light three - dimensional measurement is based on the laser triangulation principle. The laser triangulation principle is as Figure 2 shown. A beam of laser emitted by a laser irradiates the surface of the object to be measured. After being reflected by the object, it forms an image on the CCD sensor. When the position of the object surface changes, the image formed on the image plane will also have a corresponding displacement, and there is a unique corresponding relationship between the two. Through the relationship formula between the image displacement and the actual displacement, the relative height between two points can be calculated. The relationship formula is: Among them, ΔH represents the distance of the point on the surface of the object to be measured relative to the reference plane, l represents the object distance, Δh represents the offset of the image formed on the image plane, α represents the angle between the laser line and the optical axis of the camera lens, β represents the angle between the CCD sensor and the optical axis of the camera, and d represents the image distance. Then, the three-dimensional coordinates of the light spot in the world coordinate system can be determined through the calculation of the relational expression.

[0065] As Figure 3 shown, in the line structured light detection system, the line laser projects a laser line onto the surface of the object to be measured. The laser line is modulated due to the change in the surface depth of the object and possible gaps, which is manifested as distortion or discontinuity of the laser line in an image such as Figure 4 . The degree of distortion is proportional to the depth, and the discontinuity shows the physical gap on the surface of the object. By using the laser line offset information captured by the CCD camera and calibrating the two-dimensional image and the stereo space transformation matrix of the camera, the corresponding three-dimensional space coordinates on the surface of the object can be determined through the center position of the light strip on the two-dimensional image. Each shot can obtain a measurement section. By generating a relative displacement between the line structured light and the object to be measured, multiple sections can be obtained, and then the three-dimensional information of the entire surface of the object to be measured can be obtained. As a key link in the entire measurement, the center line extraction algorithm is the key to the stability, real-time performance, and accuracy of the system.

[0066] In the line structured light image captured by the camera, the width of the light strip is usually several to dozens of pixels. When calculating the contour, only the pixel coordinates of the center of the light strip are required. Therefore, it is necessary to thin the light strip with a certain width. Figure 5 is a schematic diagram of the image plane coordinates of the laser line extracted; and the existing center line extraction algorithms include the extreme value method, the threshold method, the gray-level centroid method, the curve fitting method, the Steger method based on the Hessian matrix, etc. The extreme value method, as Figure 6 shown, the abscissa in the figure represents the position of a column of pixels, the ordinate represents the gray-level value, C represents the position where the center line should be located, and C′ represents the center line obtained by using the extreme value method. The extreme value method finds the maximum point of the brightness distribution on the local cross-section of the light strip as the center point. The algorithm is simple and has a fast operation speed. The threshold method, as Figure 7 shown, the abscissa in the figure represents the position of a column of pixels, the ordinate represents the gray-level value, A and B represent two points found from both sides of the light strip towards the middle, C represents the position where the center line should be located, and C′ represents the center line obtained by using the extreme value method. The threshold method finds the pixel points A and B that meet a certain threshold from both sides of the light strip towards the middle and takes the middle of the two pixels AB as the center position to obtain C′. The gray-level centroid method is as Figure 8As shown in the figure, the abscissa in the figure represents the positions of a column of pixels, the ordinate represents the grayscale value, points A and B represent two points on a fixed threshold line, and point C represents the point gmax (the point with the maximum grayscale value) with the maximum light intensity. The grayscale centroid method records the centroid (center of gravity) in the grayscale value distribution of the light stripe cross-section as the center of the light stripe. In a column of line lasers, the extreme value method is first used to find the point gmax (the point with the maximum grayscale value) with the maximum light intensity, and then a threshold K = gmax - Δg is determined (generally, the maximum value floats downward by 10 to 20 grayscale values, and Δg takes 10 - 20). Elements greater than the threshold K are judged on both sides of the threshold, and the position is weighted by the grayscale, and the calculated centroid position is used as the center of the light stripe; The Gaussian fitting method, as Figure 9 shown in the figure, the abscissa in the figure is the position of a column of pixels, the ordinate is the size of the grayscale, Raw Data is the original data (the grayscale size and distribution of a certain column of pixels), and Fitting Function is the curve fitted using the Gaussian function. The Gaussian fitting method fits the actual light intensity distribution of the light stripe cross-section using an ideal Gaussian function, uses a Gaussian binomial curve to replace the original light intensity distribution curve, and takes the extreme point as the center point; The light stripe center extraction algorithm based on the Hessian matrix first uses the Hessian matrix to solve the normal direction of the light stripe, and then uses the Taylor series expansion along the normal direction to find the point where the first derivative is zero and the second derivative has a maximum value as the center point.

[0067] In the same measurement section, the images of the laser lines obtained under different lighting environments are different. Figure 10 is the laser line image obtained under the condition of strong ambient light. Figure 11 is the laser line image obtained under the condition of weak ambient light.

[0068] The method for extracting the center line of robust line structured light according to the preferred embodiment of the present invention, as Figure 12 shown, the method for extracting the center line of robust line structured light includes the following steps:

[0069] Step S10: Construct a database and store the scene feature information of different laser lines in the database.

[0070] Specifically, first construct a database. The database includes extraction algorithms, scene types, and scene feature parameters, also known as the method-scene-feature parameter library. Store different extraction algorithms, scene types, and scene feature parameters in the method-scene-feature parameter library. In the embodiment of the present invention, the grayscale centroid method and the Steger method based on the Hessian matrix, two situations of brighter and weaker ambient light, and calibrated feature parameters are respectively pre-stored in the method-scene-feature parameter library; As Figure 13As shown, when the ambient light is bright, the Steger algorithm based on the Hessian matrix is used to extract the center line; as Figure 14 shown, when the ambient light is dim, the gray centroid method is used to extract the center line; the image scene feature parameters are manually calibrated to obtain the method-scene-feature parameter library list shown in the following table, where w is the light stripe width information and C is the relative brightness feature.

[0071]

[0072] Step S20: Obtain the feature information of the target laser line, compare the feature information with the scene feature information in the database, and obtain the scene information of the target laser line.

[0073] For the specific process, please refer to Figure 15 , which is the flowchart of step S20 in the robust line structured light center line extraction method provided by the present invention.

[0074] As Figure 15 shown, the step S20 includes:

[0075] Step S21: Obtain continuous frame images of a set number of the target laser lines, and determine the area of the laser stripe in each image;

[0076] Step S22: Randomly generate a certain number of seed points in the area of the laser stripe, obtain sub-images according to the seed points, and calculate the sub-image feature parameters, where the sub-image feature parameters include relative brightness feature parameters and the width feature parameters of the target laser line;

[0077] Step S23: Compare the distance between the sub-image feature parameters and the feature parameters in the database, and obtain the scene information of the laser line after the comparison is completed.

[0078] Furthermore, as Figure 16 shown, in the embodiment of the present invention, multiple groups of 33*33 pixel image samples are obtained in real time, and the image samples are subjected to integer-shaped sub-pixel expansion. After the feature information such as the width and relative brightness of the image laser line in the image passes through machine learning, the working scene of the laser (such as scene 1, scene 2 or scene 3) is automatically matched.

[0079] Specifically, obtain continuous frame images of a set number (for example, 10) of the target laser lines, and use the 1 / 8 sub-pixel subdivision technology in integer form for processing, and each pixel is expanded to a pixel size of 8*8. Figure 17It is an example diagram of consecutive frame images collected. To reduce the influence of noise on the light stripe, Gaussian filtering is performed on the images; then the region of the laser stripe in each image is determined, and the pixels at the position of the maximum gradient of each column of pixels are used as the upper and lower boundaries of the laser stripe. Among them, the maximum value of the neighborhood cumulative difference is used as the maximum gradient. As shown in Figure 18 shown, during the neighborhood cumulative difference process, a long bar-shaped window of m*1 slides on each column of pixels, and the neighborhood cumulative difference values of each pixel upward and downward are calculated respectively using the formula:

[0080] s down =[f(i,j)-f(i+1,j)]+[f(i,j)-f(i+2,j)]+…+[f(i,j)-f(i+m-1,j)]

[0081] s up =[f(i,j)-f(i-1,j)]+[f(i,j)-f(i-2,j)]+…+[f(i,j)-f(i-m+1,j)];

[0082] Among them, S down represents the neighborhood cumulative difference value of the pixel downward, S up represents the neighborhood cumulative difference value of the pixel upward, i represents the row coordinate, j represents the column coordinate, f(i, j) represents the gray value of the pixel at the i-th row and the j-th column, and m represents the number of rows of the long bar-shaped window used in the neighborhood cumulative difference process. For example, a long bar-shaped window of 3*1 slides on each column of pixels to obtain the neighborhood cumulative difference value at each pixel; the pixel at the position of the minimum value of the downward neighborhood cumulative difference is used as the upper boundary position of the light stripe, and the pixel at the position of the minimum value of the upward neighborhood cumulative difference is used as the lower boundary position of the light stripe. The region between the upper and lower boundaries is used as the width of the laser stripe; among them, the light stripe region is represented by 1, and the background region outside the light stripe is represented by 0, obtaining a binary image of the laser stripe as shown in Figure 19 ; A certain number (for example, 10) of seed points are generated within the region where the laser stripe is located. The coordinates of each seed point are (x i , y i ). Take 16 pixels in size above, below, left, and right of the seed point, exclude the seed points close to the boundary, and then randomly select 7 sub-images of 33*33 in size inside the laser stripe region of each image, as shown in Figure 20 shown.

[0083] Take the average width of the laser stripes in each column as the laser line width feature of each sub-region, calculate the relative brightness feature according to the gray-scale distribution of the background region and the target region, and define the relative brightness to measure the brightness difference between the laser line and the surrounding background environment. The brighter the laser line in the image compared to the background, the larger the relative brightness value, and vice versa. The formula for defining the relative brightness feature is as follows:

[0084] Among them, C represents the relative brightness feature, μ T represents the average gray value of the pixels in the laser line region, and μ B represents the average gray value of the pixels in the background region; the light stripe width and relative brightness feature information in each sub-image are shown in the following table:

[0085]

[0086] After that, compare the feature parameters of the sub-image with the distance of the feature information in the method-scene-feature parameter library, and take the scene with the smallest distance d as the current scene; assume that the features in a certain scene in the method-scene-feature parameter library are represented by the feature vector (g1, g2,..., g n ), and the features of the sub-image are represented by the feature vector (f1, f2,..., f n ), calculate the distance d between the two through the formula, and its formula is

[0087] Among them, d represents the distance between the feature parameters of the sub-image and the feature parameters in the method-scene-feature parameter library, i represents the number of feature values, f i represents the i-th feature value of the sub-image, and g i represents the i-th feature value in a certain scene in the method-scene-feature parameter library; when calculating the width distance, normalize the width, and take the ratio of the absolute value of the difference between the sub-image and the parameter library to the width in the parameter library as the distance size, that is, |w i -w| / w i , and the Euclidean distances of the feature parameters of each sub-image are shown in the following table:

[0088]

[0089] , the distances of the sample images from Scene 1 are all smaller than the distances from Scene 2. Therefore, it is determined that the sample image data is in a dark ambient light environment.

[0090] Step S30: Match the corresponding extraction algorithm based on the scene information, optimize the parameters of the extraction algorithm, and use the optimized centerline extraction algorithm to extract the centerline of the structured light image.

[0091] For the specific process, please refer to Figure 21, which is the flowchart of step S30 in the robust centerline extraction method of line structured light provided by the present invention.

[0092] As Figure 21 shown, the step S30 includes:

[0093] Step S31: Perform sub-pixel processing on the collected image data in integer form to obtain sample data with super-resolution increment;

[0094] Step S32: Combine the sample data with the manually marked data to construct a scene sample set;

[0095] Step S33: Based on the scene sample set, randomly initialize the centerline extraction algorithm and extract the centerline result of the manually marked data;

[0096] Step S34: Subtract the centerline result from the ground truth to obtain the difference between corresponding points, and calculate the average error between the centerline and the ground truth. If the average error exceeds the set pixel value, use the gradient descent method for optimization to complete the extraction of the final centerline result.

[0097] Further, as Figure 22 shown, construct a laser line scene sample set. The scene sample set is composed of manually marked data and the obtained sample data. The manually marked data is an image with a known scene type and the centerline ground truth. The scene type is the same as the sample data. Perform centerline extraction on the laser line scene sample set using the initialized centerline extraction algorithm, compare the centerline extraction result of the manually marked data with the ground truth, and judge whether the result of the manually marked data meets the requirements. If the result meets the requirements, store the result in the centerline extraction model; if the result does not meet the requirements, calculate the error of the manually marked data and update the parameters of the extraction algorithm. After the update, repeat the above process until the error between the result and the ground truth is within a reasonable range, and use the model with updated parameters as the final centerline extraction model.

[0098] Specifically, use integer-form sub-pixels for the collected image data to generate sample data with super-resolution increment, that is, each pixel expands to a pixel size of 8*8; combine the sample data with the manually marked data to construct a laser line scene sample set. The manually marked data is an image with a known scene type and the centerline ground truth. The scene type is the same as the sample data, and obtain Figure 23All are images in a darker environment; in the embodiments of the present invention, when the scene type of weaker ambient light is obtained, the gray center of gravity method is used to extract the center line. The gray center of gravity method is randomly initialized, and a random integer is generated between 50 and 100 using a random function as the threshold of the gray center of gravity method. Pixels higher than this threshold will participate in the calculation of the center line; the center line is extracted from the manually marked data according to the randomly obtained threshold, and the extraction result is as Figure 24 shown. The difference between the center line extraction result of the manually marked data and the ground truth is calculated. The ground truth data is in the format of the vertical coordinates of the center points of each column of pixels as shown in the following table:

[0099] 1 2 3 4 5 6 7 8 76.4434 76.5155 76.5875 76.6595 76.7315 76.8036 76.8756 76.9476

[0100] , and after the center line extraction algorithm with random initialization is used to extract the manually marked data, its format is also the center point coordinates in each column of pixels as shown in the following table:

[0101] 1 2 3 4 5 6 7 8 75.5351 75.9918 76.4676 76.9072 76.9704 77.4149 76.9714 76.5424

[0102] . After taking the absolute value of the difference between the extraction result and the ground truth column by column and calculating the average value, if the average error exceeds the set pixel (for example, 0.33 pixels), the gradient descent method is used to optimize the model parameters. The size of the gradient change each time is 1, and the step size is set to 1 to move the model error in a smaller direction; if the error is still large, the above process is repeated until the error between the extracted center line result and the ground truth is within a reasonable range. After multiple parameter improvements, a graph of the error change between the extraction result and the ground truth is obtained, as Figure 25 shown. The abscissa in the figure represents the number of iterations, and the ordinate represents the error value. When the iteration is 47 times, the error can reach a relatively small value. The center line result extracted using the optimized gray center of gravity method is as Figure 26 shown.

[0103] Furthermore, Figure 27 is the overall schematic diagram of the robust line-structured light center line extraction method in the present invention. A method-scene-feature parameter library is pre-constructed, and the extraction algorithms (algorithm 1, algorithm 2,..., algorithm n), scene categories (scene A, scene B,..., scene N), and scene features (f1, f2,...) are stored in the method-scene-feature parameter library. The feature information of the laser line is obtained, and the feature information is compared with the scene feature information in the database to match the scene features, infer the scene information of the laser line and the corresponding center line extraction algorithm, and optimize the algorithm in the center line extraction model to obtain the final center line extraction model. The center line extraction model can make an adaptive change when the illumination environment changes, making the center line extraction more robust and obtaining the final center line result.

[0104] The present invention provides a method for extracting the center line of a robust line structured light. The method provided by the present invention stores the scene feature information of different laser lines in a database by constructing the database; obtains the feature information of the target laser line, compares the feature information with the scene feature information in the database to obtain the scene information of the target laser line; matches the corresponding extraction algorithm based on the scene information, optimizes the parameters of the extraction algorithm, and uses the optimized center line extraction algorithm to extract the center line of the structured light image. The present invention inversely deduces the environment where the sensor is located according to the light intensity distribution characteristics of the laser line, and adaptively adjusts the parameter information of the center line extraction algorithm, and finally realizes the extraction of the center line of the line structured light image.

[0105] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including the element.

[0106] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by a computer program instructing related hardware (such as a processor, intelligent racing car, etc.). The program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the above method embodiments. The storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0107] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A robust line structured light centerline extraction method, characterized in that: The method for extracting the center line of the robust line structured light includes: Construct a database and store the scene feature information of different laser lines in the database; Obtain the feature information of the target laser line, compare the feature information with the scene feature information in the database, and obtain the scene information of the target laser line; The obtaining the feature information of the target laser line, comparing the feature information with the scene feature information in the database, and obtaining the scene information of the target laser line specifically includes: Obtain a set number of consecutive frame images of the target laser line and determine the area of the laser stripe in each image; Randomly generate a certain number of seed points within the area of the laser stripe, obtain sub-images according to the seed points, and calculate the sub-image feature parameters, where the sub-image feature parameters include the relative brightness feature parameter and the width feature parameter of the target laser line; Compare the distance between the sub-image feature parameters and the feature parameters in the database, and obtain the scene information of the laser line after the comparison is completed; Match the corresponding extraction algorithm based on the scene information, optimize the parameters of the extraction algorithm, and use the optimized center line extraction algorithm to extract the center line of the structured light image.

2. The robust line structured light centerline extraction method according to claim 1, characterized in that: The constructing the database and storing the scene feature information of different laser lines in the database specifically is: Pre-construct a database, where the database includes extraction algorithms, scene types, and scene feature parameters, and store different extraction algorithms, scene types, and scene feature parameters in the database.

3. The method for extracting the center line of the robust line structured light according to claim 1, wherein The obtaining a set number of consecutive frame images of the target laser line and determining the area of the laser stripe in each image further includes: Take the pixels at the maximum gradient position of each column of pixels in the consecutive frame images as the upper and lower boundaries of the laser stripe, where the maximum and minimum values of the neighborhood cumulative difference are used as the maximum gradient, and use the formula to calculate the neighborhood cumulative difference value upward and the neighborhood cumulative difference value downward for each pixel respectively, and the formula is: ； in, Indicates the cumulative difference value of the neighborhood below the pixel, Indicates the cumulative difference value of the neighborhood upward of the pixel, i represents the row coordinates, j represents the vertical coordinate, Indicates the pixel point i Row, No. j Grayscale value of the column, m Indicates the number of rows of the long strip window used in the neighborhood cumulative difference process.

4. The method for extracting the center line of the robust line structured light according to claim 1, characterized in that The formula for calculating the relative brightness feature is: ； Among them, represents the relative brightness feature, represents the average gray value of pixels in the laser line region, represents the average gray value of pixels in the background region.

5. The method for extracting the center line of the robust line structured light according to claim 1, wherein The formula for calculating the distance when comparing the distance between the sub-image feature parameters and the feature parameters in the database is: ； Among them, represents the distance between the sub-image feature parameter and the feature parameter in the database, i represents the number of eigenvalues, represents the i th eigenvalue of the sub-image, represents the i th eigenvalue in a certain scenario in the method-scenario-feature parameter library.

6. The method for extracting the center line of the robust line structured light according to claim 1, wherein The optimizing the parameters of the center line extraction algorithm based on the scene information to complete the extraction of the center line of the line structured light image specifically includes: Perform sub-pixel processing on the collected image data in integer form to obtain sample data with super-resolution increment; Combine the sample data with the manually marked data to construct a scene sample set; Based on the scene sample set, randomly initialize the center line extraction algorithm and extract the center line result of the manually marked data; Subtract the center line result from the ground truth to obtain the difference between the corresponding points, and calculate the average error between the center line and the ground truth. If the average error exceeds the set pixel, use the gradient descent method for optimization to complete the extraction of the final center line result.

Images