Line laser light strip extraction method and three-dimensional reconstruction method

Abstract

The invention discloses a light bar extraction method of line laser and a three-dimensional reconstruction method based on line laser scanning. When light stripe extraction is carried out on a light stripe image, for each column, a starting point and an end point of a light stripe on the column in the width direction are firstly positioned to define a light stripe area of each column, and then two types of points to be selected (a first point to be selected and a second point to be selected) are defined based on the light stripe area; according to the method, the two types of to-be-selected points are selected, and the position of one of the two types of to-be-selected points is selected as the position of the light stripe center point of the column based on the property of the light stripe region, so that the light stripe can be accurately extracted from the light stripe image when facing a non-opaque object surface (such as a semitransparent object surface).

Description

Technical Field

[0001] The invention relates to the field of machine vision, and in particular to a light strip extraction method of a line laser and a three-dimensional reconstruction method based on line laser scanning. Background Art

[0002] 3D reconstruction technology is one of the main research areas of machine vision. It can efficiently measure the 3D information and 3D posture of an object or environment by restoring its surface 3D structure. 3D reconstruction technology is widely used, including product measurement, detection, positioning and identification.

[0003] The purpose of 3D reconstruction is to obtain surface information of an object. According to the type of light source, it can be divided into passive light source and active light source. Among them, structured light is an active light source, and structured light measurement is an important direction in non-contact measurement. Structured light includes point structured light, line structured light, and coded structured light. Please refer to Figure 1 , which is a schematic diagram of line structured light measurement; the principle and process of measurement based on laser triangulation is as follows: fix the relative position of the camera and the laser transmitter, project a line laser line onto the surface of the object to be measured through the line laser sensor, and then use the camera to obtain the reflected light. The shape change of the object surface will be reflected in the light strip image received by the camera (the image of the reflected light on the image is called the light strip), causing the position of the light strip to shift, and the magnitude of this shift corresponds to the depth change of the object surface. Therefore, the offset of the light strip on the image can be calculated, and then the depth information of the object surface can be indirectly obtained - that is: when the line laser is projected onto the surface of the object, the bumps and depressions on the surface of the object to be measured will cause the light strip obtained by the camera to undergo corresponding displacements. Through this corresponding displacement, the bumps and depressions on the surface of the object can be calculated - for example Figure 1 When the surface of the object to be measured is at point D1, the corresponding image on the imaging plane is F1, and when the surface of the object to be measured is at point D2, the corresponding image on the imaging plane is F2. The light transmission path is a triangle in space, so it is called triangulation.

[0004] In actual industrial production environments, due to the complexity of the material of the object to be measured, problems may arise in measurements based on laser triangulation. That is, the laser triangulation-based measurement has poor adaptability to materials. For example, when a line laser is projected onto the surface of a translucent object such as glass or a highly reflective object (to be precise, a material with a certain degree of transparency that causes the reflected light of the line laser to become divergent, and the brightness distribution of the light strip does not conform to the Gaussian distribution), scattering is likely to occur. This creates difficulties in extracting the light strips from the light strip image acquired by the camera, often resulting in incorrect light strip extraction, which in turn causes incorrect information on the object surface calculated, resulting in a large deviation between the 3D reconstruction result and the actual result. Summary of the invention

[0005] In order to solve the above problems, the present invention provides a light stripe extraction method of a line laser and a three-dimensional reconstruction method based on line laser scanning, which are described in detail below.

[0006] According to the first aspect, an embodiment provides a

[0007] Acquire a light stripe image of a line laser;

[0008] The light stripe image is processed column by column, and for any column of the light stripe image:

[0009] Calculate a first gradient parameter of each pixel point in the column of the light stripe image;

[0010] According to the first gradient parameter of each pixel point in the column, a starting point and an end point are selected from the column to define a light bar area of ​​the column;

[0011] The point with the largest second gradient parameter in the light strip area of ​​the column, the starting point or the end point is taken as the first candidate point;

[0012] Selecting a pixel point in the light strip area of ​​the column as a second candidate point by a preset method;

[0013] For any column of the light bar image, at least according to the width of the light bar region of the column, a position of one of the first candidate point and the second candidate point of the column is selected as the position of the center point of the light bar of the column.

[0014] In one embodiment, selecting a starting point and an end point from the column to define a light bar region of the column according to the first gradient parameter of each pixel point in the column includes:

[0015] Traverse the pixel points in the direction of columns, take the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and continue to traverse to find the first pixel point whose gray value is less than or equal to the starting point as the end point; or,

[0016] The pixel point whose first gradient parameter is greater than or equal to the first threshold is taken as the starting point, and the pixel point whose first gradient parameter is greater than or equal to the second threshold is taken as the end point.

[0017] In one embodiment, the step of calculating the first gradient parameter of each pixel point in the column of the light stripe image includes:

[0018] Extracting the first-order gradient of each pixel point in the column as the first gradient parameter by a one-dimensional gradient operator; or,

[0019] Calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in the column; for each pixel point in the column, fit a quadratic function based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and solve the local maximum of the quadratic function as the first gradient parameter of the pixel point.

[0020] In one embodiment, the second gradient parameter is calculated by:

[0021] Extracting the first-order gradient of each pixel point in the column as the first gradient parameter by a one-dimensional gradient operator; or,

[0022] Calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in the column; for each pixel point in the column, fit a quadratic function based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and solve the local maximum of the quadratic function as the second gradient parameter of the pixel point.

[0023] In one embodiment, the step of selecting a pixel point in the light bar area of ​​the column as the second candidate point by a preset method includes:

[0024] Calculating the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column; fitting a quadratic function based on the grayscale value, the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column as parameters of the quadratic function, and solving the local maximum value and the corresponding position of the quadratic function of the pixel point; comparing the local maximum values ​​of the quadratic function of each pixel point in the column to obtain the largest local maximum value among these local maximum values, and taking the position corresponding to the largest local maximum value as the position of the second candidate point;

[0025] or,

[0026] Obtaining the grayscale value and weight of each pixel point in the light strip area of ​​the column, and calculating a weighted average grayscale value, wherein the weight obeys a Gaussian distribution; taking the pixel point in the light strip area of ​​the column whose grayscale value is greater than the weighted average grayscale value as the second candidate point;

[0027] or,

[0028] Obtain the grayscale values ​​of the pixel with the largest grayscale value in the light strip area of ​​the column and two pixels in its neighborhood, and fit a quadratic function of the grayscale value with respect to the position based on these three pixels, obtain the position corresponding to the local maximum of the quadratic function, and use this position as an offset plus the position of the pixel with the largest grayscale value to calculate the position obtained as the position of the second candidate point.

[0029] In one embodiment, the step of selecting one of the positions of the first candidate point and the second candidate point of the column as the position of the center point of the light bar of the column at least according to the width of the light bar region of the column includes:

[0030] determining whether the light strip area in the column belongs to the first type area or the second type area at least according to the width of the light strip area in the column;

[0031] If it belongs to the first type of area, the position of the first candidate point in the column is selected as the position of the center point of the light bar in the column;

[0032] If it belongs to the second type area, the position of the second candidate point in the column is selected as the position of the center point of the light bar in the column.

[0033] In one embodiment, determining whether the light bar area in the column belongs to the first type area or the second type area at least according to the width of the light bar area in the column includes:

[0034] Whether the light bar area of ​​the column belongs to the first type area or the second type area is determined according to the width of the light bar area of ​​the column and the width of the light bar area of ​​the columns in the neighborhood of the column.

[0035] In one embodiment, when the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or,

[0036] When the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or,

[0037] When the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or,

[0038] When the width of the light bar area in the row is less than the first width threshold, and the number of rows in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area in the row belongs to the second type of area.

[0039] In one embodiment, the step of selecting one of the positions of the first candidate point and the second candidate point of the column as the position of the center point of the light bar of the column at least according to the width of the light bar region of the column includes:

[0040] According to the width of the light bar area of ​​the column and the width of the light bar area of ​​the columns in the neighborhood of the column, one of the positions of the first candidate point and the second candidate point of the column is selected as the position of the center point of the light bar of the column.

[0041] In one embodiment, when the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of the column is selected as the position of the center point of the light bar of the column;

[0042] When the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of the column is selected as the position of the center point of the light bar of the column;

[0043] When the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, the position of the first candidate point of the column is selected as the position of the center point of the light bar of the column;

[0044] When the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, the position of the second candidate point of the column is selected as the position of the center point of the light bar of the column.

[0045] According to a second aspect, an embodiment provides a three-dimensional reconstruction method based on line laser scanning, comprising:

[0046] Extracting the light stripes by the light stripe extraction method described in any embodiment of this document;

[0047] Three-dimensional reconstruction is performed based on the extracted light strips.

[0048] According to a third aspect, an embodiment provides a computer-readable storage medium, comprising a program, wherein the program can be executed by a processor to implement a method as described in any embodiment herein.

[0049] According to the light stripe extraction method of line laser, the three-dimensional reconstruction method based on line laser scanning and the computer-readable storage medium of the above-mentioned embodiments, when extracting light stripes from a light stripe image, for each column, first locate the starting point and the end point of the light strip on the column in its width direction to define the light stripe area of ​​each column, and then define two types of candidate points (first candidate points and second candidate points) based on the light stripe area, and select the position of one of the two types of candidate points as the position of the center point of the light strip of the column based on the properties of the light stripe area. Through the scheme of the present application, when facing a non-opaque object surface (such as a translucent object surface), the light stripes can be extracted from the light stripe image more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the principle of measurement based on laser triangulation;

[0051] Figure 2 is an example of a light bar image;

[0052] Figure 3 A flowchart of a method for extracting light stripes from a line laser according to an embodiment;

[0053] Figure 4 A schematic diagram of how to perform row-by-row processing in a light stripe extraction method in an embodiment to obtain a light stripe region, a first point (position) to be selected, and a second point (position) to be selected in each column of a light stripe image;

[0054] Figure 5 A flowchart of a three-dimensional reconstruction method based on line laser scanning according to an embodiment;

[0055] Figure 6 The figure is a schematic structural diagram of a three-dimensional reconstruction system based on line laser scanning according to an embodiment. DETAILED DESCRIPTION

[0056] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0057] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0058] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0059] In actual industrial production environments, due to the complexity of the object to be measured, when measuring based on laser triangulation, conventional algorithms based on grayscale centers and the like often distort or even fail to extract the center of the light strip, and this situation is particularly difficult to handle when the surface of the object has both translucent and opaque materials. In some schemes of the present application, opaque areas and non-opaque areas (such as translucent areas) are identified on the light strip image, and the opaque areas and non-opaque areas (such as translucent areas) are distinguished, and different methods are used to extract the light strips, so that the coordinates of the laser line imaged on the image sensor (i.e., the coordinates of the light strip on the image) can be stably extracted in a complex environment. Some schemes of the present application can solve the problem of imaging accuracy of the laser profiler, effectively suppress the noise of the object, and in the reconstruction process, focus on the continuity assumption of the object, that is, the illumination change (grayscale change) caused by the reflection of the object is a continuous change, and the abnormal noise is removed based on this assumption.

[0060] When measuring based on laser triangulation, the applicant found that when imaging the line laser, due to the reflective characteristics of the non-opaque area (such as the semi-transparent area), the reflected light spot of the semi-transparent area often has a long trail - for example Figure 2 This is an example, which makes it very difficult to extract the center of the light strip, often resulting in inaccurate and distorted extraction of the center of the light strip. In some solutions, the applicant conceives that by selecting the upper edge or upper point of the spot area as the extraction result, this is closer to the actual position and has better stability.

[0061] Based on the above considerations, the applicant has conceived that: on the light stripe image, the brightness distribution of the laser line should be consistent with the high-period distribution, with bright center and dark sides; the extraction of the laser center point is column by column, and the light strip center point is extracted in the column direction of each column; for each column, the applicant considers first locating the starting point and end point of the light strip on the column in the width direction, and the starting point can be determined by the pixel point with a large grayscale change. At the same time, on the light stripe image, the brightness distribution of the laser line should be consistent with the high-period distribution, so the grayscale distribution has a mirror-like property, and the pixel point with a grayscale value less than the starting point can be used as the end point, thereby defining the light stripe area in each column, that is, the area of ​​the width of the light strip in the column direction. Then, for each column, the starting point, the end point or the point with the largest gradient correlation in the light bar area of ​​the column is selected as a non-opaque position to be selected (for example, a semi-transparent position to be selected) - that is, when it is determined that the light bar area of ​​the column corresponds to the non-opaque position (for example, a semi-transparent position to be selected) of the object to be measured, the non-opaque position to be selected (for example, a semi-transparent position to be selected) in the light bar area is used as the final center point of the light bar on the column. In addition, the pixel points that meet the set conditions in the light bar area of ​​each column are selected as opaque positions to be selected - that is, when it is determined that the light bar area of ​​the column corresponds to the opaque position of the object to be measured, The opaque position in the light bar area is selected as the final center point of the light bar on the column; it can be seen that in this process, non-opaque positions (such as translucent positions) and opaque positions are selected for the light bar area of ​​each column, and the next task is to determine what type of area the light bar area of ​​each specific column belongs to, whether it is an opaque area (that is, the area corresponding to the opaque position of the object to be measured) or a non-opaque area (that is, the area corresponding to the non-opaque position of the object to be measured). The non-opaque area can be, for example, a translucent area (that is, the area corresponding to the translucent position of the object to be measured).

[0062] It is understandable that the above is only an example of measurement based on laser triangulation. Similar problems may exist in some other laser measurement schemes, and the corresponding problems can also be solved by this application.

[0063] The following is further described through several embodiments.

[0064] Please refer to Figure 3 , the light stripe extraction method of the line laser proposed in some embodiments comprises the following steps:

[0065] Step 100: Acquire a light stripe image of a line laser.

[0066] The line laser sensor projects a line laser onto the surface of the object to be measured, and the reflected light of the line laser is formed by the reflection of the surface of the object to be measured. The camera images the reflected light to obtain a light strip image. Figure 2 is an example of a light bar image.

[0067] The light stripe image obtained in step 100 may be an original light stripe image or a light stripe image obtained by preprocessing the original light stripe image. A high-quality light stripe image may be obtained by preprocessing, and interference of external factors on the light stripe image may be reduced; the preprocessing method includes image filtering and / or image segmentation.

[0068] Image filtering can be performed by using mean filtering, Gaussian filtering, bilateral filtering or median filtering. For example, taking Gaussian filtering as an example, smoothing can be performed by Gaussian filtering to improve the stability of calculation. Specifically, the light strip image to be processed can be convolved with a Gaussian convolution kernel. The Gaussian convolution kernel can be a one-dimensional 1*3 convolution kernel [0.25, 0.5, 0.25] T , which can be repeated multiple times.

[0069] Image segmentation can reduce the image area that needs to be processed, retain the light bar and its surrounding images, and segment out useless information in the image.

[0070] Step 110: Process the light bar image column by column.

[0071] It can be understood that a two-dimensional image includes rows and columns, and rows and columns are relative on a two-dimensional plane: if the horizontal direction is the row direction, then the vertical direction is the column direction; if the horizontal direction is the column direction, then the vertical direction is the row direction. In order to make it easier to process, the direction roughly parallel to the light strip direction can be defined as the row direction, and the direction roughly perpendicular to the light strip direction can be defined as the column direction. For example, in the above text Figure 2 For example, the X direction in the figure is the row direction, and the Y direction in the figure is the column direction.

[0072] The following is a detailed description of the column-by-column processing of the light strip image in step 110. Figure 4 In some embodiments, for any column of the light bar image (for the convenience of discussion, let's call it C[x]), the following steps are performed:

[0073] Step 111: Calculate the first gradient parameter of each pixel in the light stripe image sequence C[x].

[0074] There are many ways to calculate the first gradient parameter of a pixel point, and several of them are listed below.

[0075] In some embodiments, step 111 may extract the first-order gradient of each pixel in column C[x] as the first gradient parameter of each pixel by using a one-dimensional gradient operator.

[0076] The one-dimensional gradient operator may be, for example, a Sobel operator, a Roberts operator, or a Prewitt operator. The gradient perpendicular to the light strip direction, that is, the column direction, is extracted by the one-dimensional gradient operator, which can be expressed by the following formula:

[0077] I' x,y =f(I x,y );

[0078] Among them I' x,y Represents pixel I x,y The first-order gradient of ; x and y represent rows and columns respectively, and f represents a convolution operation, where the convolution kernel used can be, for example, a one-dimensional Y-direction Sobel operator:

[0079]

[0080] In some embodiments, step 111 can also calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in column C[x]; for each pixel point in column C[x], a quadratic function is fit based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and the local maximum of the quadratic function is solved as the first gradient parameter of the pixel point.

[0081] The first-order gradient, second-order gradient and third-order gradient of the pixel point can be calculated by using the existing known or future methods; then the quadratic function is fitted based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as the parameters of the quadratic function. An example can be the following formula:

[0082]

[0083] Among them I' x,y , I” x,y ,I”' x，y Represents pixel I x,y The first-order gradient, second-order gradient, and third-order gradient of ; x and y represent rows and columns respectively; and then solve the local maximum of the quadratic function of the above pixel point as the first gradient parameter of the pixel point.

[0084] Based on the first-order gradient, second-order gradient and third-order gradient of the pixel point, a quadratic function is fitted and the first gradient parameter of the pixel point is finally calculated. This method calculates the gradient with high accuracy and good robustness.

[0085] Step 113: According to the first gradient parameter of each pixel point in column C[x], a starting point and an end point are selected from column C[x] to define a light bar region of column C[x].

[0086] In some embodiments, step 113 determines the starting point through the first gradient parameter, and determines the end point in combination with the grayscale information of the light bar image; for example, step 113 traverses the pixel points in the column direction, takes the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and continues to traverse to find the first pixel point whose grayscale value is less than or equal to the starting point as the end point.

[0087] In some embodiments, step 113 may also determine the starting point and the end point through the first gradient parameter, for example, taking the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and taking the pixel point whose first gradient parameter is greater than or equal to the second threshold as the end point.

[0088] Through the above steps, the light bar area of ​​column C[x] is defined, that is, the width range of the light bar on column C[x].

[0089] Step 115: The point with the largest second gradient parameter, the starting point or the end point in the light bar area of ​​column C[x] is selected as the first candidate point.

[0090] In some embodiments, step 115 may extract the first-order gradient of each pixel in column C[x] as the second gradient parameter of each pixel by using a one-dimensional gradient operator.

[0091] The one-dimensional gradient operator may be, for example, a Sobel operator, a Roberts operator, or a Prewitt operator. The gradient perpendicular to the light strip direction, that is, the column direction, is extracted by the one-dimensional gradient operator, which can be expressed by the following formula:

[0092] I' x,y =f(I x,y );

[0093] Among them I' x,y Represents pixel I x,y The first-order gradient of ; x and y represent rows and columns respectively, and f represents a convolution operation, where the convolution kernel used can be, for example, a one-dimensional Y-direction Sobel operator:

[0094]

[0095] In some embodiments, step 115 can calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in column C[x]; for each pixel point in column C[x], a quadratic function is fit based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and the local maximum of the quadratic function is solved as the second gradient parameter of the pixel point.

[0096] The first-order gradient, second-order gradient and third-order gradient of the pixel point can be calculated by using the existing known or future methods; then the quadratic function is fitted based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as the parameters of the quadratic function. An example can be the following formula:

[0097]

[0098] Among them I' x,y , I” x,y ,I”' x,y Represents pixel I x,y The first-order gradient, second-order gradient, and third-order gradient of ; x and y represent rows and columns respectively; and then solve the local maximum of the quadratic function of the above pixel point as the first gradient parameter of the pixel point.

[0099] Based on the first-order gradient, second-order gradient and third-order gradient of the pixel point, a quadratic function is fitted and the second gradient parameter of the pixel point is finally calculated. This method calculates the gradient with high accuracy and good robustness.

[0100] Step 117: Select a pixel point in the light bar area of ​​column C[x] as the second candidate point by a preset method.

[0101] In some embodiments, step 117 calculates the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​column C[x]; for each pixel point in the light strip area of ​​column C[x], a quadratic function is fitted based on the grayscale value, the first-order gradient and the second-order gradient of the pixel point as parameters of the quadratic function, and the local maximum and the corresponding position of the quadratic function of the pixel point are solved; the local maximum values ​​of the above-mentioned quadratic function of each pixel point in column C[x] are compared to obtain the largest local maximum value among these local maximum values, and the position corresponding to the largest local maximum value is used as the position of the second candidate point.

[0102] For each pixel in the light strip area of ​​column C[x], a quadratic function is fitted based on the grayscale value, first-order gradient and second-order gradient of the pixel as parameters of the quadratic function. An example may be the following formula:

[0103]

[0104] Among them I x,y , I' x,y , I” x，y Represents pixel I x,y The gray value, first-order gradient, and second-order gradient of the pixel are respectively represented by x and y. Then the point corresponding to the local maximum value of the quadratic function of the above pixel point is solved as the second candidate point, and its position is correspondingly used as the position of the second candidate point.

[0105] The second candidate point obtained by this method has very high accuracy, and the algorithm has a faster processing speed.

[0106] In some embodiments, step 117 obtains the grayscale value and weight of each pixel point in the light strip area of ​​column C[x], and calculates a weighted average grayscale value, wherein the weight obeys a Gaussian distribution; and the pixel point in the light strip area of ​​column C[x] whose grayscale value is greater than the weighted average grayscale value is taken as the second candidate point.

[0107] The second candidate point obtained by this method has very high accuracy, and the algorithm has a faster processing speed.

[0108] In some embodiments, step 117 obtains the grayscale values ​​of the pixel with the largest grayscale value in the light strip area of ​​column C[x] and the grayscale values ​​of two pixels in its neighborhood (it can be understood that the neighborhood here refers to the neighborhood in the column direction, for example, for the above Figure 2 Generally speaking, the two pixel points in the neighborhood of the pixel point in step 117 refer to two adjacent pixel points in the upper and lower directions of the pixel point, that is, in the column direction), and based on these three pixel points, a quadratic function of the grayscale value with respect to the position is fitted to obtain the position corresponding to the local maximum of the above quadratic function, and the position is used as the offset plus the position of the pixel point with the largest grayscale value to calculate the position obtained as the position of the second candidate point.

[0109] The above scheme is essentially based on local grayscale information estimation, using quadratic functions for fitting, taking the grayscale values ​​of the point with the highest grayscale value in the area and the two points in its neighborhood, fitting the quadratic function of the pixel grayscale value with respect to the coordinate position, taking the highest value of the quadratic function as the peak grayscale value, and saving it with the position information as the brightness of the light point, thereby also obtaining the position of the light point with the highest brightness (the point with the highest grayscale value in the area, whose coordinates plus the offset are equal to the position of the light point with the highest brightness), which improves the accuracy of grayscale estimation. It can be seen that in the process of extracting the center point, not only the grayscale distribution of the light strip is taken into account, but also the brightness is determined, taking into account the brightness changes of the laser, so as to be more conducive to solving imaging problems such as translucent or highly reflective objects.

[0110] For example, the following is an example:

[0111] In an example, let I0 represent the point with the highest grayscale value in the above-mentioned area, and the two points in its neighborhood are I -1 and I1; through these three pixels I -1 , I0 and I1 gray value and position fit a quadratic function I g =ax 2 +bx+c, and get the values ​​of a, b, and c; these three pixels I -1 The grayscale values ​​of I0, I1 are substituted into the Ig The corresponding x value can be solved to obtain the three pixel points I -1 The relative coordinates of I0 and I1 are a-b+c, c, a+b+c respectively; therefore, the offset obtained by further calculation is:

[0112]

[0113] The position of pixel I0 in the column direction (i.e., row coordinate, for example Figure 2 The X-coordinate in the ) plus the offset is equal to the position of the light point with the highest brightness (that is, the row coordinate, for example Figure 2 The X-coordinate in .

[0114] Through the above steps 111 to 117, the processing of a certain column in the light stripe image is completed. Steps 111 to 117 can be performed on each column one by one, so that the light stripe area, the first candidate point (position) and the second candidate point (position) of each column in the light stripe image can be obtained.

[0115] Step 120: For any column of the light bar image (for the sake of convenience, let's call it C[x]), select one of the first and second candidate points of the column as the position of the center point of the light bar of the column based on at least the width of the light bar area of ​​the column.

[0116] In some embodiments, there are multiple implementations of how step 120 selects one of the first candidate point and the second candidate point in column C[x] as the center point of the light bar of column C[x] (or selects the position of one as the position of the center point of the light bar of column C[x]), and a few of them are listed below.

[0117] In some embodiments, step 120 determines whether the light bar region of column C[x] belongs to the first type of region or the second type of region based on at least the width of the light bar region of column C[x]; if it belongs to the first type of region, step 120 selects the position of the first candidate point of column C[x] as the position of the center point of the light bar of column C[x]; if it belongs to the second type of region, step 120 selects the position of the second candidate point of column C[x] as the position of the center point of the light bar of column C[x]. This process involves how to determine whether the light bar region of column C[x] belongs to the first type of region or the second type of region. In some embodiments, step 120 determines whether the light bar region of column C[x] belongs to the first type of region or the second type of region based on the width of the light bar region of column C[x] and the width of the light bar region of the columns in the neighborhood of column C[x], for example:

[0118] When the width of the light bar area of ​​column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​column C[x] belongs to the first type of area;

[0119] When the width of the light bar region of column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose widths are greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar region of column C[x] belongs to the first type of region;

[0120] When the width of the light bar region of column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar region of column C[x] belongs to the first type of region;

[0121] When the width of the light bar region of column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar region of column C[x] belongs to the second type region.

[0122] It can be understood that the width of the light strip area refers to the length of the light strip area in the column direction, that is, the distance between the starting point and the end point of the light strip area.

[0123] Based on experiments and theoretical conclusions, the applicant concluded that for any column C[x], if the width of its own light stripe area is relatively large, and there are a large number of columns with relatively large light stripe area widths in its neighborhood (referring to some adjacent columns of column C[x], where the columns N columns away from column C[x] can be specifically defined as the columns in its neighborhood, and N can be taken according to actual conditions, such as 1, 2, 3, etc.), then it means that column C[x] corresponds to the area of ​​the non-opaque position (such as the semi-transparent position) of the object to be measured; if the width of the light stripe area of ​​column C[x] itself is relatively small, but there are a large number of columns with relatively large light stripe area widths in its neighborhood, then it means that column C[x] corresponds to the area of ​​the non-opaque position (such as the semi-transparent position) of the object to be measured. The calculation of the light bar area itself may be affected by problems, therefore, column C[x] is still regarded as the area corresponding to the non-opaque position (e.g., semi-transparent position) of the object to be measured, which helps to eliminate the influence of interference factors; if the width of the light bar area of ​​column C[x] itself is large, although the number of columns with large light bar area widths in its neighborhood is small, column C[x] should still be regarded as the area corresponding to the non-opaque position (e.g., semi-transparent position) of the object to be measured; if the width of the light bar area of ​​column C[x] itself is small, and the number of columns with large light bar area widths in its neighborhood is small, it means that column C[x] corresponds to the area of ​​the opaque position of the object to be measured.

[0124] In some other embodiments, step 120 selects one of the first candidate point and the second candidate point of column C[x] as the position of the center point of the light bar of column C[x] according to the width of the light bar area of ​​column C[x] and the width of the light bar area of ​​columns in the neighborhood of column C[x]. For example:

[0125] When the width of the light bar area of ​​column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of column C[x] is selected as the position of the center point of the light bar of column C[x];

[0126] When the width of the light bar area of ​​column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of column C[x] is selected as the position of the center point of the light bar of column C[x];

[0127] When the width of the light bar area of ​​column C[x] is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, the position of the first candidate point of column C[x] is selected as the position of the center point of the light bar of column C[x];

[0128] When the width of the light bar area of ​​column C[x] is less than the first width threshold, and the number of columns in its neighborhood whose widths are greater than or equal to the second width threshold is less than the threshold, the position of the second candidate point of column C[x] is selected as the position of the center point of the light bar of the column.

[0129] Through the above method, the center point of the light stripes in each column can be extracted, and the center points of the light strips in each column constitute the extracted light strips.

[0130] The light stripe extraction method of the present application can extract light stripes from a light stripe image more accurately when facing a non-opaque object surface (such as a translucent object surface); some schemes use gradients, which are suitable for using common hardware acceleration methods such as FPGA, GPU, etc.; in addition, a first candidate point and a second candidate point are determined from the light stripe area, and the position of one of them is finally selected as the position of the center point of the light stripe of the column, which takes into account both opaque positions and non-opaque positions (such as translucent positions), which makes the use environment of the scheme of the present application more universal and eliminates the complicated process of manually distinguishing regional features.

[0131] Please refer to Figure 5 In some embodiments of the present application, a three-dimensional reconstruction method based on line laser scanning is also provided, which comprises the following steps:

[0132] Step 200: extracting light stripes based on the light stripe image; for example, extracting light stripes from the light stripe image using the light stripe extraction method of any embodiment of the present invention.

[0133] Step 210: Perform three-dimensional reconstruction based on the extracted light strips.

[0134] Please refer to Figure 6 In some embodiments of the present application, a three-dimensional reconstruction system based on line laser scanning is also provided. The three-dimensional reconstruction system may include a line laser emitter 10, a moving component 20, an imaging component 30 and a controller 40; wherein A in the figure represents a schematic diagram of an object to be measured.

[0135] The line laser transmitter 10 is used to transmit a line laser to the object to be measured, and the imaging component 30 is used to receive the light obtained by the line laser reflected by the object to be measured to convert the light signal into an electrical signal to obtain a light strip image.

[0136] In some embodiments, the moving assembly 20 is used to carry and move the object to be measured, so that the line laser emitted by the line laser emitter 10 can complete the scanning of the object to be measured - in such an embodiment, the line laser emitter 10 and the imaging assembly 30 can be fixed.

[0137] In some embodiments, the mobile component 20 can also carry the line laser emitter 10 and the imaging component 30, so that the line laser emitted by the line laser emitter 10 can complete the scanning of the object to be measured, and the imaging component 30 can continuously perform imaging during the scanning process - in such an embodiment, the line laser emitter 10 and the imaging component 30 are mobile, but the relative position between the two is fixed, and at the same time, the position of the object to be measured is also fixed.

[0138] The controller 40 can control each component to work and process the light stripe image, for example, extract the light stripe and perform three-dimensional reconstruction based on the extracted light stripe.

[0139] Of course, during the specific three-dimensional reconstruction, it is necessary to calibrate the internal and external parameters of the imaging component 30, calibrate the line laser plane, and even calibrate the mobile component 20, and then control the mobile component 20 to work, so as to collect the light strip image. One light strip can be extracted from each light strip image, and based on all the extracted light strips; establish a world coordinate system (for example, in the example where the mobile component 20 carries the object to be measured and moves the object to be measured, the world coordinate system can be established based on the initial position of the mobile component 20), and solve the world coordinates corresponding to the light strip coordinates in each light strip image based on the laser triangulation method and the line-surface model; finally, the three-dimensional surface of the object to be measured can be reconstructed based on the obtained point cloud data.

[0140] The controller 40 may extract the light stripes from the light stripe image by using the light stripe extraction method of any embodiment of the present invention.

[0141] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, various operating steps and components for performing the operating steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0142] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. In addition, as understood by those skilled in the art, the principles of this article can be reflected in a computer program product on a computer-readable storage medium, which is pre-installed with a computer-readable program code. Any tangible, non-temporary computer-readable storage medium can be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD to ROM, DVD, Blu Ray disks, etc.), flash memory and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer or other programmable data processing device to form a machine, so that these instructions executed on a computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions can also be stored in a computer-readable memory, which can instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory can form a manufactured product, including an implementation device that implements a specified function. Computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on a computer or other programmable device to generate a computer-implemented process, so that the instructions executed on a computer or other programmable device can provide steps for implementing a specified function.

[0143] Although the principles of this invention have been shown in various embodiments, many modifications of structures, arrangements, proportions, elements, materials and components particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments will be included in the scope of this invention.

[0144] The foregoing specific description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure. Therefore, the consideration of the present disclosure will be illustrative rather than restrictive, and all these modifications will be included in its scope. Similarly, the advantages, other advantages and solutions to the problems of various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more clear, should not be interpreted as critical, necessary or necessary. The term "include" and any other variants used in this article are all non-exclusive inclusions, so that the process, method, article or device including the list of elements not only includes these elements, but also includes other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0145] Those skilled in the art will appreciate that many changes may be made to the details of the above-described embodiments without departing from the basic principles of the invention. Therefore, the scope of the present invention should be determined solely by the claims.

Claims

1. A method for extracting light stripes from a line laser, characterized in that: include: Acquire a light stripe image of a line laser; The light stripe image is processed column by column, and for any column of the light stripe image: Calculate a first gradient parameter of each pixel point in the column of the light stripe image; According to the first gradient parameter of each pixel point in the column, a starting point and an end point are selected from the column to define a light bar area of ​​the column; The point with the largest second gradient parameter within the light strip area of ​​the column, the starting point or the end point is taken as the first candidate point; Selecting a pixel point in the light strip area of ​​the column as a second candidate point by a preset method; For any column of the light bar image, at least according to the width of the light bar region of the column, a position of one of the first candidate point and the second candidate point of the column is selected as the position of the center point of the light bar of the column.

2. The method according to claim 1, characterized in that The step of selecting a starting point and an end point from the column to define a light bar region of the column according to the first gradient parameter of each pixel point in the column comprises: Traverse the pixel points in the direction of columns, take the pixel point whose first gradient parameter is greater than or equal to the first threshold as the starting point, and continue to traverse to find the first pixel point whose gray value is less than or equal to the starting point as the end point; or, The pixel point whose first gradient parameter is greater than or equal to the first threshold is taken as the starting point, and the pixel point whose first gradient parameter is greater than or equal to the second threshold is taken as the end point.

3. The method according to claim 1 or 2, characterized in that The calculating the first gradient parameter of each pixel point in the column of the light strip image includes: Extracting the first-order gradient of each pixel point in the column as the first gradient parameter by a one-dimensional gradient operator; or, Calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in the column; for each pixel point in the column, fit a quadratic function based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and solve the local maximum of the quadratic function as the first gradient parameter of the pixel point.

4. The method according to claim 1, characterized in that The second gradient parameter is calculated by: Extracting the first-order gradient of each pixel point in the column as the second gradient parameter by a one-dimensional gradient operator; or, Calculate the first-order gradient, second-order gradient and third-order gradient of each pixel point in the column; for each pixel point in the column, fit a quadratic function based on the first-order gradient, second-order gradient and third-order gradient of the pixel point as parameters of the quadratic function, and solve the local maximum of the quadratic function as the second gradient parameter of the pixel point.

5. The method according to claim 1, characterized in that The step of selecting a pixel point in the light bar area of ​​the column as the second point to be selected by a preset method includes: Calculating the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column; fitting a quadratic function based on the grayscale value, the first-order gradient and the second-order gradient of each pixel point in the light strip area of ​​the column as parameters of the quadratic function, and solving the local maximum value and the corresponding position of the quadratic function of the pixel point; comparing the local maximum values ​​of the quadratic function of each pixel point in the column to obtain the largest local maximum value among these local maximum values, and taking the position corresponding to the largest local maximum value as the position of the second candidate point; or, Obtaining the grayscale value and weight of each pixel point in the light strip area of ​​the column, and calculating a weighted average grayscale value, wherein the weight obeys a Gaussian distribution; taking the pixel point in the light strip area of ​​the column whose grayscale value is greater than the weighted average grayscale value as the second candidate point; or, Obtain the grayscale values ​​of the pixel with the largest grayscale value in the light strip area of ​​the column and two pixels in its neighborhood, and fit a quadratic function of the grayscale value with respect to the position based on these three pixels, obtain the position corresponding to the local maximum of the quadratic function, and use this position as an offset plus the position of the pixel with the largest grayscale value to calculate the position obtained as the position of the second candidate point.

6. The method according to claim 1, characterized in that The step of selecting one of the positions of the first candidate point and the second candidate point of the column as the position of the center point of the light bar of the column at least according to the width of the light bar area of ​​the column comprises: determining whether the light strip area in the column belongs to the first type area or the second type area at least according to the width of the light strip area in the column; If it belongs to the first type of area, the position of the first candidate point in the column is selected as the position of the center point of the light bar in the column; If it belongs to the second type area, the position of the second candidate point in the column is selected as the position of the center point of the light bar in the column.

7. The method according to claim 6, characterized in that The step of determining whether the light bar area in the column belongs to the first type area or the second type area at least according to the width of the light bar area in the column comprises: Whether the light bar area of ​​the column belongs to the first type area or the second type area is determined according to the width of the light bar area of ​​the column and the width of the light bar area of ​​the columns in the neighborhood of the column.

8. The method according to claim 7, characterized in that When the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or, When the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or, When the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area of ​​the column belongs to the first type of area; or, When the width of the light bar area in the row is less than the first width threshold, and the number of rows in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, it is determined that the light bar area in the row belongs to the second type of area.

9. The method according to claim 1, characterized in that The step of selecting one of the positions of the first candidate point and the second candidate point of the column as the position of the center point of the light bar of the column at least according to the width of the light bar area of ​​the column comprises: According to the width of the light bar area of ​​the column and the width of the light bar area of ​​the columns in the neighborhood of the column, one of the positions of the first candidate point and the second candidate point of the column is selected as the position of the center point of the light bar of the column.

10. The method according to claim 9, characterized in that When the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of the column is selected as the position of the center point of the light bar of the column; When the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is greater than or equal to the threshold, the position of the first candidate point of the column is selected as the position of the center point of the light bar of the column; When the width of the light bar area of ​​the column is greater than or equal to the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, the position of the first candidate point of the column is selected as the position of the center point of the light bar of the column; When the width of the light bar area of ​​the column is less than the first width threshold, and the number of columns in its neighborhood whose width is greater than or equal to the second width threshold is less than the threshold, the position of the second candidate point of the column is selected as the position of the center point of the light bar of the column.

11. A three-dimensional reconstruction method based on line laser scanning, characterized in that: include: Extracting the light strips by the method according to any one of claims 1 to 10; Three-dimensional reconstruction is performed based on the extracted light strips.

12. A computer-readable storage medium, characterized in that: The method comprises a program which can be executed by a processor to implement the method according to any one of claims 1 to 11.

Images