Method and device for extracting center point of light strip of line structured light, electronic equipment and medium

Abstract

The present disclosure relates to a line structured light light strip center point extraction method, device, electronic equipment and storage medium. The line structured light light strip center point extraction method comprises: preliminarily extracting the center point of the line structured light light strip; for each center point, determining a gray extreme point based on the gray scale feature of the line structured light light strip along the center point normal; updating the center point of the line structured light light strip based on the gray extreme point. The embodiment of the present disclosure can improve the accuracy and efficiency of the line structured light light strip center point extraction.

Description

Technical Field

[0001] This disclosure relates to the field of three-dimensional measurement technology, and in particular to a method, apparatus, electronic device, and storage medium for extracting the center point of a line structured light stripe. Background Technology

[0002] Line laser reconstruction is a technique that reconstructs the three-dimensional information of an object's surface by projecting a transmitted laser line onto it and capturing the laser line image using a camera. It boasts high efficiency and accuracy, and is widely used in various fields of 3D information reconstruction. The accuracy and time required for laser line center point extraction directly impact the final reconstruction's precision and efficiency; therefore, laser line center point extraction is a crucial step in line laser reconstruction.

[0003] Existing methods for extracting the center point of a laser line include: thresholding, gray-scale centroid method, extremum method, and Steger algorithm. The thresholding method is simple and suitable for rough calculation of the center line position, but its accuracy cannot meet general industrial scanning standards. The gray-scale centroid method can reduce errors caused by uneven gray-scale distribution in the laser line and is highly efficient, but it is easily affected by noise. The extremum method is suitable for extracting laser lines with a Gaussian gray-scale distribution, offering good extraction results and high efficiency, but it is also easily affected by noise. The Steger algorithm has high accuracy and robustness, but its computational load is enormous and its efficiency is low. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides a method, apparatus, electronic device, and storage medium for extracting the center point of a line structured light stripe, thereby improving the accuracy and efficiency of extracting the center point of a line structured light stripe.

[0005] In a first aspect, this disclosure provides a method for extracting the center point of a line structured light stripe, including:

[0006] Preliminary extraction of the center point of the structured light stripe;

[0007] For each center point, the gray-level extreme point is determined based on the gray-level characteristics of the line structure light stripe along the normal direction of the center point;

[0008] The center point of the linear structured light stripe is updated based on the gray-level extreme points.

[0009] Secondly, this disclosure provides a device for extracting the center point of a line structured light stripe, comprising:

[0010] The center point coarse extraction module is used to initially extract the center point of the line structure light stripe;

[0011] The gray-scale extreme point determination module is used to determine the gray-scale extreme point for each center point based on the gray-scale characteristics of the line structure light stripe along the normal direction of the center point;

[0012] The center point update module is used to update the center point of the line structured light stripe based on the grayscale extreme points.

[0013] Thirdly, this disclosure provides an electronic device, including:

[0014] processor;

[0015] Memory, used to store executable instructions;

[0016] The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the line structured light stripe center point extraction method described in the first aspect above.

[0017] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor implements the line structured light stripe center point extraction method described in the first aspect.

[0018] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0019] The technical solution provided in this disclosure first extracts the center point of the line structured light stripe, i.e., performs efficient coarse extraction of the center point of the line structured light stripe. Then, based on the gray-level features of the line structured light stripe along the normal direction of the center point, it determines the gray-level extreme points. Finally, it updates the center point of the line structured light stripe based on the gray-level extreme points, thereby optimizing the position of the initially extracted center point and improving the accuracy of center point extraction. Furthermore, this technical solution uses the normal direction of the center point instead of the Hessian matrix used in the Steger method to calculate the direction of maximum gradient, which simplifies the calculation and improves the efficiency of center point extraction. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a method for extracting the center point of a line structured light stripe, provided in an embodiment of this disclosure;

[0023] Figure 2A schematic diagram of a scene for acquiring a light stripe image provided in an embodiment of this disclosure;

[0024] Figure 3 A schematic diagram illustrating the direction of the ten-point fitting center point provided in an embodiment of this disclosure;

[0025] Figure 4 A schematic diagram illustrating the determination of each level of hierarchy line along the center point normal direction provided in this embodiment of the disclosure;

[0026] Figure 5 This is a schematic diagram illustrating the use of bilinear interpolation to interpolate subpixel grayscale values, provided in an embodiment of this disclosure.

[0027] Figure 6 This is a schematic diagram illustrating the search for neighboring center points provided in an embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram illustrating the search for the center point and supplementary points of the second target provided in an embodiment of this disclosure;

[0029] Figure 8 A structural block diagram of the method for extracting the center point of a line structured light stripe provided in this embodiment of the disclosure;

[0030] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0031] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0032] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0033] In this embodiment of the disclosure, the projector of the 3D scanner projects structured light onto the surface of the object being measured, which is then modulated by the surface and acquired by a camera. The structured light stripes involved in this disclosure are the light stripes in the light stripe image acquired by the camera, and may include laser lines or grating stripes, etc. For ease of understanding of the technical solution of this disclosure, each embodiment is described using a laser line as an example of structured light stripes.

[0034] Figure 1This is a flowchart illustrating a method for extracting the center point of a line structured light stripe according to an embodiment of this disclosure. This method is applicable to situations requiring secondary optimization of line structured light stripes. This method can be executed by a line structured light stripe center point extraction device, which can be implemented using software and / or hardware. Figure 1 As shown, this method includes the following steps:

[0035] S110, Preliminary extraction of the center point of the structured light stripe.

[0036] In this embodiment of the disclosure, the preliminary extraction of the center point of the line structured light stripe refers to the efficient extraction of the center point of the line structured light stripe. This can be achieved using unidirectional line extraction methods such as the gray-scale centroid method or the extreme value method, or other line extraction methods that have problems with line extraction integrity. Specifically, a light stripe image can be acquired first, the line structured light stripe can be identified from the light stripe image, and then the center point of the line structured light stripe can be preliminarily extracted. (Reference) Figure 2 light stripe image ( Figure 2 The illustration schematically shows light stripe images 41 and 42 (captured by the first camera 31 and the second camera 32 respectively) as images of the surface of object 2 after the projector 1 projects line structured light onto the surface of object 2. These images, captured by the cameras (first camera 31 and second camera 32), can contain three-dimensional contour information of the object 2 surface and form the basis for three-dimensional reconstruction of the object 2 surface. Furthermore, the line structured light projected by the projector 1 is a thin light plane that forms light stripes of a certain width on the object surface. The light stripes captured by the cameras are the line structured light stripes involved in this disclosure. For example, light stripe image 41 includes line structured light stripe 411, and light stripe image 42 includes line structured light stripe 421. In this embodiment, after the camera captures the light stripe image, the image can be filtered and denoised before the center point of the line structured light stripe in the image is extracted. In this embodiment, the projector can be a laser, and correspondingly, the line structured light is a line laser, and the line structured light stripe is a laser line. It is understood that... Figure 2 Only one projector 1 is shown schematically. This disclosure can also be applied to projection systems with two or more projectors to project line structured light onto object 2 from different directions. Furthermore, in order for the camera to capture the line structured light stripes projected onto the object surface by different projectors, each projector alternates in projecting the line structured light.

[0037] Currently, for ease of operation, both the gray-scale centroid method and the extremum method process all laser images using a fixed single direction. However, for some laser lines modulated to have a small angle with the lifting direction, there is a problem of missing some lifting points. To address this technical problem and reduce the number of points to be added in the present invention, and to improve the efficiency of point addition, in some embodiments, the initial extraction of the center point of the line structured light stripe includes: extracting the center point of the line structured light stripe using a single-direction lifting method based on the target lifting direction. The angle between the target lifting direction and the reference normal of the line structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the line structured light stripe, and the size of the preset angle limits the degree to which the target lifting direction is approximately perpendicular to the line structured light stripe. This allows for the extraction of the center point of the line structured light stripe closer to the direction of maximum gray-scale gradient, improving the accuracy and completeness of the extracted center point. The preset angle is used to ensure the accuracy and completeness of center point extraction. Taking a structured light strip as a laser line as an example, if the angle between the preset lifting direction and the normal direction of the laser line is less than the preset angle, center point extraction along this preset lifting direction can accurately and completely extract the center point of the laser line. Furthermore, the size of the preset angle can, as a whole, limit the degree to which the target lifting direction selected from multiple preset lifting directions is approximately perpendicular to the laser line. In other words, generally speaking, the smaller the preset angle, the closer the target lifting direction is to the laser line, and thus the higher the accuracy of the laser line center point extracted along the target lifting direction.

[0038] In one example, the center point of the line structured light stripe is extracted using a single-direction lifting method based on the target lifting direction. This includes: selecting at least one target lifting direction from multiple preset lifting directions in the pixel coordinate system, where the angle between the target lifting direction and the reference normal of the line structured light stripe is less than a preset angle; acquiring the light stripe features of the line structured light stripe along one of the target lifting directions; and extracting the center point of the line structured light stripe based on the light stripe features. It is understood that the more preset lifting directions designed in different directions, the better it is to ensure that a target lifting direction with an angle less than a preset angle to the normal of the laser line can be selected for laser lines of various directions. Furthermore, it can make the target lifting direction more perpendicular to the laser line, improving the extraction accuracy of the laser line center point. Additionally, the light stripe features are the imaging features of the line laser obtained when extracting the laser line center point using a single-direction lifting method in related technologies. In one embodiment, the light stripe features include grayscale features or gradient features. Specifically, for the acquisition of gradient features, gradient calculation can be performed along the target lifting direction using a one-dimensional difference method or a two-dimensional Gaussian convolution method.

[0039] Optionally, the preset lifting directions include a first lifting direction, a second lifting direction, a third lifting direction, and a fourth lifting direction. The first lifting direction is the u-axis direction of the pixel coordinate system, the second lifting direction is the v-axis direction of the pixel coordinate system, the third lifting direction has an angle of 45 degrees with the u-axis direction, the fourth lifting direction has an angle of 45 degrees with the v-axis direction, and the u-axis direction is perpendicular to the v-axis direction.

[0040] In a specific example, the intersection equation of the laser plane and the camera image plane is determined in the camera coordinate system. Based on the intersection equation and the camera's intrinsic parameter matrix, the angle between the laser line and the u-axis in the pixel coordinate system is determined. Based on the angles between the preset lifting directions and the u-axis, as well as the angle between the laser line and the u-axis, all preset lifting directions with an angle less than a preset angle to the normal of the laser line are determined, and these preset lifting directions are taken as target lifting directions. Then, the angles between each target lifting direction and the normal of the laser line are compared, and the target lifting direction with the smallest angle to the normal of the laser line is selected, thereby obtaining the light stripe features of the laser line along this target lifting direction. In this way, by obtaining the light stripe features of the laser line along the target lifting direction that is closest to being perpendicular to the laser line, the accuracy of laser line center point extraction is maximized. Finally, the center point of the laser line is extracted based on the light stripe features of the laser line. In this example, after calibrating the laser plane and camera respectively, the plane equation of the projected laser plane in the camera coordinate system can be determined through the camera calibration and laser plane calibration. Then, based on this plane equation, the intersection equation of the laser plane and the camera image plane is determined, and the direction of the intersection line corresponding to this equation is the direction of the laser line. Next, by multiplying the intersection equation by the camera intrinsic parameter matrix, the angle between the intersection line and the u-axis direction in the pixel coordinate system can be determined, i.e., the angle between the laser line and the u-axis direction. Then, for each preset lifting direction, the difference (positive value) between the preset lifting direction and the u-axis direction and the angle between the laser line and the u-axis direction is calculated to obtain the angle between the preset lifting direction and the laser line. Finally, the difference (positive value) between the angle between the preset lifting direction and the laser line and 90 degrees is calculated to obtain the normal angle between the preset lifting direction and the laser line.

[0041] S120. For each center point, determine the gray-level extreme points based on the gray-level characteristics of the line structured light stripe along the normal direction of the center point.

[0042] In one embodiment, determining grayscale extreme points based on the grayscale characteristics of the line structured light stripe along the normal direction of the center point includes: determining multiple grade lines along the normal direction of the center point of the line structured light stripe, wherein the grade lines reflect the distance from the sub-pixel point to the corresponding center point along the normal direction of the center point of the line structured light stripe; interpolating the sub-pixel grayscale value of the grade point based on the coordinates of the grade point on the grade line, wherein the grade point is located in the normal direction of the center point; and determining the grayscale extreme points based on the sub-pixel grayscale value.

[0043] In one example, multiple graded lines are identified along the normal direction of the center point of the structured light stripe, including:

[0044] Step A: Determine the normal of each center point based on the center point and fit the first-order level line.

[0045] For example, for any center point of a linear structured light stripe, at least one adjacent center point is determined on one or both sides of that center point; based on that center point and at least one adjacent center point, the normal of that center point is fitted. Figure 3 As shown, any center point of the structured light stripe is designated as center point ct. Five center points p1, p2, p3, p4, and p5 are obtained to the left of center point ct, and five center points p6, p7, p8, p9, and p10 are obtained to the right. A curve is fitted from these center points to obtain the tangent line passing through center point ct, and then the normal vector perpendicular to this tangent line and passing through center point ct is obtained. In this way, the normal vectors of each center point of the structured light stripe can be traversed. Furthermore, the first-order level lines are fitted based on the center points.

[0046] Step B: Using the positions of the center points of each line structured light strip as the initial positions, along the positive and negative directions of the center point normal, respectively, obtain the second-level level points, the third-level level points, and the fourth-level level line points on the line structured light strip according to the preset step size.

[0047] Step C: Fit the second-level, third-level, and fourth-level grade lines based on the sub-pixel points of the second-level, third-level, and fourth-level grade lines, respectively.

[0048] refer to Figure 4First, the first-level grade lines are determined based on the center point. Then, a preset step size is set for each center point. Moving according to the preset step size allows the grade points of all grade lines to be found. Thus, the grade points of different grade lines along each normal direction are traversed. Specifically, for each center point, a multi-point (center point) fitting method is used to fit the center point normal. Taking the center point as the initial position, along the center point normal, the second-level, third-level, and fourth-level grade points (corresponding to the sub-pixel points schematically shown in the figure) are obtained on both sides of the center point according to the set step size. In this invention application, the second-level grade point is one step away from the center point, the third-level grade point is twice the step away, and the fourth-level grade point is three times the step away. The unit step size is used to fit multiple level points of the corresponding level to obtain the level lines of the second, third, and fourth levels (the dashed lines in the figure represent the level lines of each level). That is, the level line of the second level is fitted by multiple level points of the second level located on the same side of the level line of the first level; the level line of the third level is fitted by multiple level points of the third level located on the same side of the level line of the first level; and the level line of the fourth level is fitted by multiple level points of the fourth level located on the same side of the level line of the first level. It should be noted that the set step size for different center points can be the same or different, depending on the actual situation.

[0049] Then, based on the coordinates of the level points on the level line, the sub-pixel grayscale values ​​of the level points are interpolated. In this embodiment, when initially extracting the center point of the line structured light stripe, the coordinates of each center point can be obtained. Based on the coordinates of the center points and a preset step size for moving along the normal, the coordinates of the level points on the level line can be obtained. Then, based on the coordinates of the level points on the level line, the sub-pixel grayscale values ​​of the sub-pixel points can be interpolated using bilinear interpolation. For example, see... Figure 5 The subpixel coordinates of the current level point are (x, y). Based on these subpixel coordinates, the coordinates of the four integer pixels surrounding the current level point can be determined as (int(x), int(y)), (int(x)+1, int(y)), (int(x), int(y)+1), and (int(x)+1, int(y)+1), where int() represents rounding down. This allows us to determine the corresponding integer grayscale values ​​of the four integer pixels. Thus, the subpixel grayscale values ​​corresponding to each level point can be determined using bilinear interpolation via the following formula:

[0050] G = G x +(x-int(x))(G x+1 -G x );

[0051] in:

[0052] G x =I(int(x),int(y))+(y-int(y))(I(int(x),int(y)+1)-I(int(x),int(y))),

[0053] G x+1 =I(int(x)+1,int(y))+(y-int(y))(I(int(x)+1,int(y)+1)-I(int(x)+1,int(y))),

[0054] I() represents the grayscale value of an integer pixel.

[0055] Finally, gray-level extrema are determined based on sub-pixel gray values. It can be understood that these gray-level extrema can be represented by the zero value of the gray-level gradient.

[0056] S130, Update the center point of the structured light stripe based on the gray-scale extreme points.

[0057] Update the grayscale extreme points to the center point.

[0058] The method for extracting the center point of a line structured light stripe provided in this disclosure first performs a preliminary extraction of the center point of the line structured light stripe, i.e., a highly efficient coarse extraction of the center point. Then, it determines the gray-level extreme points based on the gray-level features of the line structured light stripe along the normal direction of the center point. Finally, it updates the center point of the line structured light stripe based on the gray-level extreme points. This optimizes the position of the initially extracted center point and improves the accuracy of center point extraction. Furthermore, this technical solution uses the normal direction of the center point instead of the Hessian matrix used in the Steger method to calculate the direction of maximum gradient, simplifying the calculation and thus improving the efficiency of center point extraction.

[0059] Based on the above technical solution, in one embodiment, the method further includes:

[0060] Based on the updated center point of the line structured light stripe, the gray-scale extreme points are determined again based on the gray-scale characteristics of the line structured light stripe along the normal direction of the center point. The center point of the line structured light stripe is then updated based on the gray-scale extreme points until the distance between each updated center point and the corresponding center point of the previous update is less than a preset distance threshold.

[0061] In this embodiment, when the distance between each updated center point and its corresponding corresponding center point from the previous update is less than a preset distance threshold, the updated center points can be considered to be at their most accurate positions and do not require further updates. Thus, by continuously updating and iterating the center points, the accuracy of center point extraction is further improved. Preferably, after each update of the center points of the structured light stripe, a target center point whose distance from the previously updated center point is greater than or equal to a preset distance threshold is determined; the target center point is then updated again. In this way, after each update of the center points of the structured light stripe, there is no need to update center points whose distance from the previously updated center point is less than the preset distance threshold, reducing the amount of data to be processed and improving data processing efficiency.

[0062] Currently, in image processing, Gaussian filtering with a small window is used to reduce noise in the image. However, for light stripe images with specific grayscale characteristics, after Gaussian filtering, the grayscale values ​​of the middle and edge sub-pixels within the window will affect each other, thereby damaging the grayscale characteristics of the light stripe image, reducing the accuracy of image feature extraction, and causing center point shift. To address this, this disclosure also provides a scheme for anisotropic filtering of the grayscale characteristics of line structured light stripes to avoid mutual interference between grayscale values ​​of different orders, protect the grayscale characteristics of the line structured light stripes, and prevent center point shift caused by filtering. Specifically, in this embodiment, after initially extracting the center point of the line structured light stripe, the method further includes: obtaining multi-level grade lines based on the center point and the center point normal of the line structured light stripe, wherein the grade lines reflect the distance from the sub-pixel point to the corresponding center point along the center point normal; performing gray-level Gaussian filtering within each grade line based on the gray levels on the line structured light stripe; extracting the third target center point of the line structured light stripe after gray-level Gaussian filtering; and updating the third target center point as the center point of the line structured light stripe. Specifically, the determination of multiple grade lines on the line structured light stripe is described in the above embodiment. Similarly, based on the sub-pixel coordinates of the grade points on the grade lines, bilinear interpolation can be used to interpolate the sub-pixel gray values ​​corresponding to the grade points, and gray-level Gaussian filtering can be performed within the grade lines based on the sub-pixel gray values ​​corresponding to the grade points.

[0063] Considering that unidirectional line extraction methods all suffer from the problem of missing center points, resulting in some sparse center points, if these sparse center points cannot find adjacent center points based on existing topological search rules, or cannot be clustered into long line segments with a number greater than the threshold of small line segment points, they will be considered as stray points or miscellaneous line segments and mistakenly deleted, further reducing the completeness of center point extraction. Therefore, in one embodiment, after initially extracting the center points of the line structured light stripe, the method further includes: supplementing the sparse center points of the line structured light stripe. Specifically, the process of filling in the sparse center points of the line structured light stripe includes: determining a first target center point based on the center point of the line structured light stripe, wherein the first target center point is either the endpoint of a plurality of consecutive center points constituting a first topological line segment, or a first independent center point without adjacent center points, and the endpoint of the plurality of consecutive center points constituting the first topological line segment is the first point and / or the end point among the plurality of consecutive center points; searching for a second target center point closest to the first target center point according to a preset search rule, wherein the second target center point is either the endpoint of a plurality of consecutive center points constituting a second topological line segment, or a second independent center point without adjacent center points; and if a second target center point is found, then filling in the gap between the first target center point and the second target center point.

[0064] In theory, all the center points of a line structured light stripe will connect to form a continuous line segment, i.e., the centerline. In reality, due to the existence of center point extraction bias, the extracted center points can be clustered into multiple line segments (each line segment is a topological line segment). Since the centerline should be continuous, it is necessary to fill in the gaps between the endpoints of one topological line segment and the endpoints of another topological line segment, or between an independent center point without adjacent center points and the endpoints of one topological line segment or another independent center point. Based on this, this embodiment first determines a first target center point based on the center points. The first target center point is either the endpoint of multiple continuous center points constituting the first topological line segment, or a first independent center point without adjacent center points.

[0065] In some embodiments, determining a first target center point based on a center point includes: establishing a topological relationship between the center points, wherein the topological relationship characterizes the adjacency relationship of the center points; determining a center point for which no topological point exists on at least one side based on the topological relationship, and determining the center point as the first target center point.

[0066] Specifically, the center point's position is stored in pixel coordinates. For example, if the sub-pixel coordinates of the center point are (1.1, 1.2), then the stored position of the center point is pixel coordinates (1, 1). (Reference) Figure 6 For each center point (taking center point p as an example in the diagram), search for the center points adjacent to that center point according to the existing search rules, that is, sequentially according to... Figure 6The search process sequentially searches for center points among pixels A, B, C, and D, while simultaneously traversing all the stored pixel coordinates. If the pixel coordinates of any pixel among A, B, C, and D are the same as the stored pixel coordinates, then the center point is determined to have a topological relationship; otherwise, the center point does not have a topological relationship. After determining the topological relationships of each center point, the center point where only one side lacks a topological point can be directly determined based on the topological relationship; this center point is the first target center point. Specifically, assuming the laser line direction is from the upper left to the lower right, if the topological relationship determines that there is no topological point to the left of the center point but a topological point to the right, then the center point is the starting point of the first topological line segment; if the topological relationship determines that there is no topological point to the right of the center point but a topological point to the left, then the center point is the ending point of the first topological line segment. Furthermore, in the above technical solution, the center points with topological relationships can be determined based on the topological relationships, and thus the center points without topological relationships (i.e., no topological points on either side) can be obtained, thereby determining this center point as the first independent center point. It is understood that the mention of one or both sides of the center point in this disclosure refers to one or both sides of the center point in the direction of the laser line.

[0067] To ensure the continuity of the extracted centerline, a second target center point corresponding to the first target center point needs to be found. In this embodiment, the center point closest to the first target center point is first searched according to a preset search rule, and then the second target center point is determined based on the above-mentioned topological relationship. The first target center point and the second target center point satisfy any of the following relationships: when the first target center point is the starting point of the first topological line segment, the second target center point is the ending point of the second topological line segment or a second independent center point; when the first target center point is the ending point of the first topological line segment, the second target center point is the starting point of the second topological line segment or a second independent center point; when the first target center point is the first independent center point, the second target center point is the endpoint of the second topological line segment or a second independent center point.

[0068] In addition, the preset search rules include: determining the first pixel (such as pixels A, B, C and D above) used to search for adjacent center points of the first target center point based on the position of the first target center point, determining the second and third pixels adjacent to the first pixel, and the fourth and fifth pixels adjacent to the second and third pixels, and searching the second pixel, third pixel, fourth pixel and fifth pixel in sequence.

[0069] For example, see Figure 7The first target center point is center point p. Based on the position of center point p, the first pixel used to search for the center points adjacent to center point p is determined, including pixels A, B, C and D as shown in the figure. Pixels E, F, G, and H adjacent to pixel B in the vertical direction, F, G, and H in the horizontal direction are searched sequentially. If a center point is found, it is determined whether the center point is the first point of the second topological line segment or the second independent center point. If the center point is the first point of the second topological line segment or the second independent center point, then the center point is the second target center point.

[0070] In the above scheme, the first and second target center points can be searched along the direction from the beginning to the end of the structured light stripe, or along the direction from the end to the beginning of the structured light stripe. For example, if the first and second target center points are searched along the direction from the beginning to the end of the structured light stripe, in one example, the first target center point is the end point and the second target center point is the beginning point. That is, the first and second target center points are obtained by searching along a single direction of the structured light stripe, avoiding the problem of large computational load caused by repeated searches.

[0071] If a second target center point is found, it means that there is a missing center point between the first and second target center points, and it is necessary to fill in the missing point between the first and second target center points.

[0072] In some embodiments, interpolation between the first target center point and the second target center point includes: performing linear interpolation between the first target center point and the second target center point to obtain multiple interpolation points. Specifically, refer to... Figure 7 Connect the first target center point p and the second target center point pn with a straight line. On the line connecting the first target center point p and the second target center point pn, select a point whose vertical axis coordinate is the pixel vertical coordinate (in this embodiment, the pixel coordinate is located at the pixel center point) to obtain the interpolation point ps.

[0073] The above technical solution improves the completeness of center point extraction by supplementing the center points. However, there is a possibility of multiple center points being supplemented or some noise points being mistakenly identified as the second target center point. To address this technical problem, in some embodiments, after supplementing the center points between the first and second target center points if a second target center point is found, the method further includes: determining the validity of the supplemented center points; if the supplemented center points are valid, they are retained; if the supplemented center points are invalid, they are deleted. Thus, by determining the validity of the supplemented center points, erroneous supplemented center points can be deleted, ensuring the completeness of center point extraction while avoiding the introduction of noise points.

[0074] Specifically, determining the validity of the interpolation point obtained by interpolation includes: determining whether the gray value at the interpolation point is greater than the gray value threshold of the center point (usually set to 5); if the gray value at the interpolation point is greater than the gray value threshold of the center point, then determining whether there is a gray extreme point within a preset distance from the interpolation point along the normal direction of the interpolation point; if there is a gray extreme point, the interpolation point is determined to be valid; if the gray value at the interpolation point is less than or equal to the gray value threshold of the center point, or if there is no gray extreme point, the interpolation point is determined to be invalid.

[0075] If an interpolation point is valid, it must be located near or at the center point. First, it can be determined whether the grayscale value at the interpolation point is greater than the grayscale threshold at the center point. If the grayscale value at the interpolation point is greater than the center point's grayscale threshold, then it's further determined whether there are grayscale extrema near the interpolation point; otherwise, the interpolation point is directly deemed invalid. Taking a Gaussian distribution of laser line grayscale as an example, if the grayscale value at the interpolation point is greater than the center point's grayscale threshold, and there are grayscale maxima near the interpolation point, then the interpolation point is valid. If the grayscale value at the interpolation point is less than or equal to the center point's grayscale threshold, or if the grayscale value at the interpolation point is greater than the center point's grayscale threshold, but there are no grayscale maxima near the interpolation point, then the interpolation point is invalid. It can be understood that the aforementioned grayscale extrema can be represented by the 0 value of the grayscale gradient.

[0076] Based on the above embodiments, in one specific embodiment, a method for extracting the center point of a line structured light stripe is provided, specifically including:

[0077] Step a: Initially extract the center point of the line structured light stripe;

[0078] Step b: Obtain multi-level lines based on the center point and center point normal of the line structured light stripe; perform gray-level Gaussian filtering on each level line based on the gray level of the line structured light stripe, and extract the center point of the line structured light stripe after gray-level Gaussian filtering.

[0079] Step c: Fill in the center points of the extracted gray-scale Gaussian filtered line structured light stripes;

[0080] Step d: Optimize the center point after the supplementary points are added to smooth the center line.

[0081] Based on the above steps, accurate and complete center points of the line structured light stripe can be extracted. Therefore, when using the center points obtained from the above steps for 3D reconstruction, the accuracy and completeness of the 3D reconstruction can be improved. Detailed descriptions of the above steps can be found in the embodiments of this disclosure. Step d can be implemented using the line structured light stripe center point extraction method provided by this disclosure, and will not be repeated here. Furthermore, it should be noted that in other embodiments, any one or any two of steps b, c, and d can be executed after step a. It should be noted that the center points processed in steps c and d should be replaced with the center points obtained in the previous step. For example, in one embodiment including steps a and c, step c should be replaced by supplementing the center points of the initially extracted line structured light stripe. In another embodiment including steps a, b, and d, step d should be replaced by optimizing the center points of the extracted grayscale Gaussian filtered line structured light stripe to smooth the center line. Other cases can be replaced accordingly, and will not be listed here.

[0082] This disclosure also provides a line structured light stripe center point extraction device for implementing the above-described line structured light stripe center point extraction method. The following is in conjunction with... Figure 8 Please provide an explanation. Figure 8 This is a structural block diagram of a device for extracting the center point of a line structured light stripe, provided in an embodiment of this disclosure. Figure 8 As shown, the line structured light stripe center point extraction device includes a center point coarse extraction module 21, a grayscale extreme value point determination module 22, and a center point update module 23.

[0083] Among them, the center point coarse extraction module 21 is used to initially extract the center point of the line structure light stripe;

[0084] The gray-scale extreme point determination module 22 is used to determine the gray-scale extreme point for each center point based on the gray-scale characteristics of the line structure light stripe along the normal direction of the center point;

[0085] The center point update module 23 is used to update the center point of the structured light stripe based on the grayscale extreme points.

[0086] In some embodiments, the grayscale extreme point determination module 22 is used for:

[0087] Determine multiple graded lines along the normal direction of the center point of the line structured light stripe, wherein the graded lines reflect the distance from the sub-pixel point to the corresponding center point along the normal direction of the center point of the line structured light stripe;

[0088] Based on the coordinates of the level points on the level line, the sub-pixel grayscale values ​​of the level points are interpolated, wherein the level points are located on the normal direction of the center point;

[0089] The grayscale extreme points are determined based on the sub-pixel grayscale values.

[0090] In some embodiments, the grayscale extreme point determination module 22 is specifically used for:

[0091] Based on the center point, determine the normal of each center point and fit the first-order level line;

[0092] Taking the position of each center point of the line structured light strip as the initial position, the second-level level point, the third-level level point, and the fourth-level level point are obtained on the line structured light strip according to the positive and negative directions of the center point normal, respectively, and according to the preset step size.

[0093] Based on the level points of the second, third, and fourth levels respectively, level lines of the second, third, and fourth levels are fitted.

[0094] In some embodiments, the center point update module 23 is further configured to:

[0095] Based on the updated center point of the line structured light stripe, gray-scale extreme points are determined again based on the gray-scale characteristics of the line structured light stripe along the normal direction of the center point, and the center point of the line structured light stripe is updated based on the gray-scale extreme points until the distance between each updated center point and the corresponding center point of the previous update is less than a preset distance threshold.

[0096] In some embodiments, the center point update module 23 is further configured to:

[0097] After each update of the center point of the line structured light stripe, a target center point is determined whose distance from the previously updated center point is greater than or equal to a preset distance threshold.

[0098] Update the target center point again.

[0099] In some embodiments, the center point coarse extraction module 21 is used to: extract the center point of the line structure light stripe based on the target line extraction direction using a unidirectional line extraction method, wherein the angle between the target line extraction direction and the reference normal of the line structure light stripe is less than a preset angle, the reference normal characterizes the coarse direction of the normal of the center point of the line structure light stripe, and the size of the preset angle limits the degree to which the target line extraction direction is approximately perpendicular to the line structure light stripe.

[0100] In some embodiments, the center point coarse extraction module 21 is specifically used for:

[0101] Select at least one target lifting direction from multiple preset lifting directions in the pixel coordinate system, whose angle with the reference normal of the line structure light stripe is less than a preset angle.

[0102] Acquire the light stripe features of the line structure light stripe along one of the target lifting directions;

[0103] The center point of the structured light stripe is extracted based on the light stripe feature.

[0104] In some embodiments, a filtering module is further included, for:

[0105] After initially extracting the center point of the line structured light stripe

[0106] Multi-level lines are obtained based on the center point and center point normal of the line structure light stripe. The level lines reflect the distance from the sub-pixel point to the corresponding center point in the center point normal direction.

[0107] Based on the grayscale values ​​on the linear structured light stripe levels, Gaussian grayscale filtering is performed within each level line.

[0108] Extract the center point of the line structured light stripe after grayscale Gaussian filtering;

[0109] The center point of the line structured light stripe after grayscale Gaussian filtering is updated to the center point of the line structured light stripe.

[0110] In some embodiments, a point-filling module is also included, for:

[0111] After initially extracting the center points of the line structured light stripes, the sparse center points of the line structured light stripes are supplemented.

[0112] In some embodiments, the point-filling module is specifically used for:

[0113] The first target center point is determined based on the center point of the line structure light stripe, wherein the first target center point is either the endpoint of multiple consecutive center points constituting the first topological line segment, or the first independent center point without adjacent center points.

[0114] The search algorithm searches for the second target center point that is closest to the first target center point according to the preset search rules. The second target center point is either the endpoint of multiple consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points.

[0115] If a second target center point is found, then a supplementary point is added between the first target center point and the second target center point.

[0116] It should be noted that, Figure 8 The line structure light stripe center point extraction device shown can perform... Figure 1 The various steps in the method embodiment shown are implemented. Figure 1 The processes and effects in the method embodiments shown are not described in detail here.

[0117] Figure 9A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown.

[0118] like Figure 9 As shown, the electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0119] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0120] Memory 302 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway device. In a particular embodiment, memory 302 is a non-volatile solid-state memory. In a particular embodiment, memory 302 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0121] The processor 301 reads and executes the computer program instructions stored in the memory 302 to perform the steps of the line structured light stripe center point extraction method provided in this embodiment of the present disclosure.

[0122] In one example, the electronic device may also include a transceiver 303 and a bus 304. Wherein, as... Figure 9 As shown, the processor 301, memory 302 and transceiver 303 are connected via bus 304 and communicate with each other.

[0123] Bus 304 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 304 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0124] The following are embodiments of a computer-readable storage medium provided in this disclosure. This computer-readable storage medium and the line structured light stripe center point extraction method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the computer-readable storage medium, please refer to the embodiments of the line structured light stripe center point extraction method described above.

[0125] This embodiment provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a method for extracting the center point of a line structured light stripe. The method includes:

[0126] Preliminary extraction of the center point of the structured light stripe;

[0127] For each center point, the gray-level extreme points are determined based on the gray-level characteristics of the line structured light stripe along the normal direction of the center point;

[0128] The center point of the structured light stripe is updated based on the gray-level extreme points.

[0129] Of course, the computer-executable instructions provided in the embodiments of this disclosure are not limited to the above-described method operations, but can also perform related operations in the line structured light stripe center point extraction method provided in any embodiment of this disclosure.

[0130] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this disclosure can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer cloud platform (which may be a personal computer, server, or network cloud platform, etc.) to execute the line structured light stripe center point extraction method provided in the various embodiments of this disclosure.

[0131] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0132] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for extracting the center point of a line structured light stripe, characterized in that, include: Preliminary extraction of the center point of the structured light stripe; For each center point, the gray-level extreme point is determined based on the gray-level characteristics of the line structure light stripe along the normal direction of the center point; The center point of the linear structure light stripe is updated based on the gray-level extreme points; The step of determining the gray-level extreme points based on the gray-level characteristics of the line structure light stripe along the normal direction of the center point includes: Determine multiple graded lines along the normal direction of the center point of the line structured light stripe, wherein the graded lines reflect the distance from the sub-pixel point to the corresponding center point along the normal direction of the center point of the line structured light stripe; Based on the coordinates of the level points on the level line, the sub-pixel grayscale values ​​of the level points are interpolated, wherein the level points are located on the normal direction of the center point; The grayscale extreme points are determined based on the sub-pixel grayscale values.

2. The method according to claim 1, characterized in that, The determination of the multiple graded lines of the linear structured light stripe along the normal direction of the center point includes: Based on the center point, determine the normal of each center point and fit the first-order level line; Taking the position of each center point of the line structured light strip as the initial position, the second-level level point, the third-level level point, and the fourth-level level point are obtained on the line structured light strip according to the positive and negative directions of the center point normal, respectively, and according to the preset step size. Based on the level points of the second, third, and fourth levels respectively, level lines of the second, third, and fourth levels are fitted.

3. The method according to claim 1, characterized in that, The method further includes: Based on the updated center point of the line structured light stripe, gray-scale extreme points are determined again based on the gray-scale characteristics of the line structured light stripe along the normal direction of the center point, and the center point of the line structured light stripe is updated based on the gray-scale extreme points until the distance between each updated center point and the corresponding center point of the previous update is less than a preset distance threshold.

4. The method according to claim 3, characterized in that, The method further includes: After each update of the center point of the line structure light stripe, a target center point is determined whose distance from the previously updated center point is greater than or equal to the preset distance threshold. The target center point is updated again.

5. The method according to claim 1, characterized in that, The initial extraction of the center point of the linear structured light stripe includes: Based on the target lifting direction, the center point of the linear structured light stripe is extracted using a unidirectional lifting method. The angle between the target lifting direction and the reference normal of the linear structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the linear structured light stripe. The size of the preset angle limits the degree to which the target lifting direction and the linear structured light stripe are approximately perpendicular.

6. The method according to claim 5, characterized in that, The step of extracting the center point of the line structured light stripe based on the target lifting direction using a unidirectional lifting method includes: Select at least one target lifting direction from a plurality of preset lifting directions in the pixel coordinate system, wherein the angle between the target direction and the reference normal of the line structure light stripe is less than a preset angle. Acquire the light stripe features of the line structure light stripe along one of the target lifting directions; The center point of the linear structure light stripe is extracted based on the light stripe features.

7. The method according to claim 1, characterized in that, After the initial extraction of the center point of the line structure light stripe, the process further includes: Based on the center point and center point normal of the line structure light stripe, a multi-level hierarchy line is obtained, wherein the hierarchy line reflects the distance from the sub-pixel point to the corresponding center point in the center point normal direction; Based on the grayscale values ​​on the line structured light stripe level lines, grayscale Gaussian filtering is performed in each level of the level line. Extract the center point of the line structured light stripe after grayscale Gaussian filtering; The center point of the line structured light stripe after grayscale Gaussian filtering is updated to the center point of the line structured light stripe.

8. The method according to claim 1, characterized in that, After the initial extraction of the center point of the line structure light stripe, the process further includes: The sparse center points of the line structure light stripe are supplemented.

9. The method according to claim 8, characterized in that, The process of filling in the sparse center points of the linear structured light stripe includes: The first target center point is determined based on the center point of the line structure light stripe, wherein the first target center point is the endpoint of a plurality of consecutive center points constituting the first topological line segment, or is the first independent center point without adjacent center points. The search algorithm searches for the second target center point that is closest to the first target center point according to a preset search rule. The second target center point is either the endpoint of a plurality of consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points. If the second target center point is found, then a supplementary point is added between the first target center point and the second target center point.

10. A device for extracting the center point of a linear optical stripe, characterized in that, include: The center point coarse extraction module is used to initially extract the center point of the line structure light stripe; The gray-scale extreme point determination module is used to determine the gray-scale extreme point for each center point based on the gray-scale characteristics of the line structure light stripe along the normal direction of the center point; The center point update module is used to update the center point of the line structure light stripe based on the gray-scale extreme point; The step of determining the gray-level extreme points based on the gray-level characteristics of the line structure light stripe along the normal direction of the center point includes: Determine multiple graded lines along the normal direction of the center point of the line structured light stripe, wherein the graded lines reflect the distance from the sub-pixel point to the corresponding center point along the normal direction of the center point of the line structured light stripe; Based on the coordinates of the level points on the level line, the sub-pixel grayscale values ​​of the level points are interpolated, wherein the level points are located on the normal direction of the center point; The grayscale extreme points are determined based on the sub-pixel grayscale values.

11. An electronic device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the line structured light stripe center point extraction method according to any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the method for extracting the center point of a line structured light stripe as described in any one of claims 1-9.

Images