Method and device for filling in center points of light strips of line structured light, electronic equipment and medium

Abstract

This disclosure relates to a method, apparatus, electronic device, and storage medium for supplementing the center point of a line structured light stripe. The method includes: initially extracting the center point of the line structured light stripe; determining a first target center point based on the center point, wherein the first target center point is either the endpoint of a plurality of consecutive center points constituting a first topological line segment, or a first independent center point without adjacent center points; searching for a second target center point closest to the first target center point according to a preset search rule, wherein the second target center point is either the endpoint of a plurality of consecutive center points constituting a second topological line segment, or a second independent center point without adjacent center points; if the second target center point is found, supplementing the center point between the first and second target center points. This disclosure improves the completeness of center point extraction.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision measurement technology, and in particular to a method, apparatus, electronic device, and storage medium for supplementing the center point of a line structured light stripe. Background Technology

[0002] Line laser reconstruction is a technique that reconstructs the three-dimensional information of an object's surface by projecting a transmitted laser line onto it and capturing the laser line image using a camera. It boasts high efficiency and accuracy, and is widely used in various fields of 3D information reconstruction. The accuracy and time required for laser line center point extraction directly impact the final reconstruction's precision and efficiency; therefore, laser line center point extraction is a crucial step in line laser reconstruction.

[0003] Existing methods for extracting the center point of laser lines include thresholding, gray-scale centroid methods, extremum methods, and the Steger algorithm. Since the quality of current laser images is generally acceptable, to ensure efficient reconstruction, simpler methods such as extremum methods or gray-scale centroid methods are typically used for center point extraction.

[0004] However, both the extreme value method and the gray-scale centroid method use a unidirectional (such as the u-axis or v-axis in the pixel coordinate system) line extraction approach, which can easily lead to the omission of center points, resulting in some sparse center points. If these sparse center points cannot find adjacent center points based on existing topological search rules, or cannot be clustered into long line segments larger than the threshold of the number of small line segments, they will be considered as stray points or miscellaneous line segments and mistakenly deleted, reducing the completeness of center point extraction. Summary of the Invention

[0005] To address the aforementioned technical problems, this disclosure provides a method, apparatus, electronic device, and storage medium for supplementing the center point of a line structured light stripe, thereby improving the completeness of center point extraction.

[0006] Firstly, this disclosure provides a method for supplementing the center point of a line structured light stripe, including:

[0007] Preliminary extraction of the center point of the structured light stripe;

[0008] A first target center point is determined based on the center point, wherein the first target center point is either the endpoint of a plurality of consecutive center points constituting a first topological line segment, or a first independent center point without adjacent center points.

[0009] The search algorithm searches for the second target center point that is closest to the first target center point according to a preset search rule. The second target center point is either the endpoint of a plurality of consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points.

[0010] If the second target center point is found, then a supplementary point is added between the first target center point and the second target center point.

[0011] Secondly, this disclosure provides a device for supplementing the center point of a line structured light stripe, comprising:

[0012] The center point coarse extraction module is used to initially extract the center point of the line structure light stripe;

[0013] The first target center point determination module is used to determine a first target center point based on the center point, wherein the first target center point is the endpoint of a plurality of consecutive center points constituting a first topological line segment, or is a first independent center point without adjacent center points.

[0014] The second target center point search module is used to search for the second target center point closest to the first target center point according to a preset search rule. The second target center point is either the endpoint of a plurality of consecutive center points constituting the second topological line segment, or a second independent center point without adjacent center points.

[0015] The point-filling module is used to fill in the gap between the first target center point and the second target center point if the second target center point is found.

[0016] Thirdly, this disclosure provides an electronic device, including:

[0017] processor;

[0018] Memory, used to store executable instructions;

[0019] The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the method for supplementing the center point of the line structure light stripe described in the first aspect above.

[0020] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor implements the method for supplementing the center point of the line structured light stripe described in the first aspect.

[0021] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0022] The technical solution provided in this disclosure, based on the preliminary extraction of the center point of the line structure light stripe, determines the endpoint of multiple consecutive center points constituting the first topological line segment, or the first independent center point without adjacent center points, i.e., the first target center point, based on the center point. In response to finding the second target center point closest to the first target center point, wherein the second target center point is either the endpoint of multiple consecutive center points constituting the second topological line segment, or the second independent center point without adjacent center points, a point is inserted between the first target center point and the second target center point. This allows for the insertion of center points between sparse center points and between sparse center points and other center points, so that sparse center points and other center points form consecutive adjacent center points, which can be clustered into long line segments with a point count greater than the threshold of small line segments. This solves the problem of sparse center points being mistakenly deleted and improves the completeness of center point extraction. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0024] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a method for supplementing the center point of a line structured light stripe according to an embodiment of this disclosure;

[0026] Figure 2 A schematic diagram of a scene for acquiring a light stripe image provided in an embodiment of this disclosure;

[0027] Figure 3 This is a schematic diagram illustrating the search for neighboring center points provided in an embodiment of the present disclosure;

[0028] Figure 4 A schematic diagram illustrating the search for the center point and supplementary points of the second target provided in an embodiment of this disclosure;

[0029] Figure 5 A schematic diagram illustrating the determination of each level of hierarchy line along the center point normal direction provided in this embodiment of the disclosure;

[0030] Figure 6 A structural block diagram of the device for supplementing the center point of a line structure light stripe provided in this embodiment of the disclosure;

[0031] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0032] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0033] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0034] In this embodiment of the disclosure, the projector of the 3D scanner projects structured light onto the surface of the object being measured, which is then modulated by the surface and acquired by a camera. The structured light stripes involved in this disclosure are the light stripes in the light stripe image acquired by the camera, and may include laser lines or grating stripes, etc. For ease of understanding of the technical solution of this disclosure, each embodiment is described using a laser line as an example of structured light stripes.

[0035] Figure 1 This is a flowchart illustrating a method for supplementing the center point of a line structured light stripe according to an embodiment of this disclosure. This method is applicable to supplementing the center point of a line structured light stripe extracted in a single direction, thereby improving the completeness of the center point extraction. This method can be executed by a device for supplementing the center point of a line structured light stripe, which can be implemented in software and / or hardware. Figure 1 As shown, this method includes the following steps:

[0036] S110, Preliminary extraction of the center point of the line structured light stripe.

[0037] In this embodiment of the disclosure, the preliminary extraction of the center point of the line structured light stripe refers to the efficient extraction of the center point of the line structured light stripe. This can be achieved using unidirectional line extraction methods such as the gray-scale centroid method or the extreme value method, or other line extraction methods that have problems with line extraction integrity. Specifically, a light stripe image can be acquired first, the line structured light stripe can be identified from the light stripe image, and then the center point of the line structured light stripe can be preliminarily extracted. (Reference) Figure 2 light stripe image ( Figure 2The illustration schematically shows light stripe images 41 and 42 (captured by the first camera 31 and the second camera 32 respectively) as images of the surface of object 2 after the projector 1 projects line structured light onto the surface of object 2. These images, captured by the cameras (first camera 31 and second camera 32), can contain three-dimensional contour information of the object 2 surface and form the basis for three-dimensional reconstruction of the object 2 surface. Furthermore, the line structured light projected by the projector 1 is a thin light plane that forms light stripes of a certain width on the object surface. The light stripes captured by the cameras are the line structured light stripes involved in this disclosure. For example, light stripe image 41 includes line structured light stripe 411, and light stripe image 42 includes line structured light stripe 421. In this embodiment, after the camera captures the light stripe image, the image can be filtered and denoised before the center point of the line structured light stripe in the image is extracted. In this embodiment, the projector can be a laser, and correspondingly, the line structured light is a line laser, and the line structured light stripe is a laser line. It is understood that... Figure 2 Only one projector 1 is shown schematically. This disclosure can also be applied to projection systems with two or more projectors to project line structured light onto object 2 from different directions. Furthermore, in order for the camera to capture the line structured light stripes projected onto the object surface by different projectors, each projector alternates in projecting the line structured light.

[0038] Currently, for ease of operation, both the grayscale centroid method and the extremum method use a fixed unidirectional direction to process the line structured light stripe. However, for some line structured light stripes modulated to have a small angle with the lifting direction, there is a problem of some lifting points being missed. To address this technical problem and reduce the number of points to be added in the present invention, and to improve the efficiency of point addition, in some embodiments, the initial extraction of the center point of the line structured light stripe includes: extracting the center point of the line structured light stripe using a unidirectional lifting method based on the target lifting direction. The angle between the target lifting direction and the reference normal of the line structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the line structured light stripe, and the size of the preset angle limits the degree to which the target lifting direction is approximately perpendicular to the line structured light stripe. In this way, the center point of the line structured light stripe can be extracted closer to the direction of maximum grayscale gradient, improving the accuracy and completeness of the extracted center point. The preset angle is used to ensure the accuracy and completeness of center point extraction. Taking a structured light strip as a laser line as an example, if the angle between the preset lifting direction and the normal direction of the laser line is less than the preset angle, center point extraction along this preset lifting direction can accurately and completely extract the center point of the laser line. Furthermore, the size of the preset angle can, as a whole, limit the degree to which the target lifting direction selected from multiple preset lifting directions is approximately perpendicular to the laser line. In other words, generally speaking, the smaller the preset angle, the closer the target lifting direction is to the laser line, and thus the higher the accuracy of the laser line center point extracted along the target lifting direction.

[0039] In one example, the center point of the line structured light stripe is extracted using a single-direction lifting method based on the target lifting direction. This includes: selecting at least one target lifting direction from multiple preset lifting directions in the pixel coordinate system, where the angle between the target lifting direction and the reference normal of the line structured light stripe is less than a preset angle; acquiring the light stripe features of the line structured light stripe along one of the target lifting directions; and extracting the center point of the line structured light stripe based on the light stripe features. It is understood that the more preset lifting directions designed in different directions, the better it is to ensure that a target lifting direction with an angle less than a preset angle to the normal of the laser line can be selected for laser lines of various directions. Furthermore, it can make the target lifting direction more perpendicular to the laser line, improving the extraction accuracy of the laser line center point. Additionally, the light stripe features are the imaging features of the line laser obtained when extracting the laser line center point using a single-direction lifting method in related technologies. In one embodiment, the light stripe features include grayscale features or gradient features. Specifically, for the acquisition of gradient features, gradient calculation can be performed along the target lifting direction using a one-dimensional difference method or a two-dimensional Gaussian convolution method.

[0040] Optionally, the preset lifting directions include a first lifting direction, a second lifting direction, a third lifting direction, and a fourth lifting direction. The first lifting direction is the u-axis direction of the pixel coordinate system, the second lifting direction is the v-axis direction of the pixel coordinate system, the third lifting direction has an angle of 45 degrees with the u-axis direction, the fourth lifting direction has an angle of 45 degrees with the v-axis direction, and the u-axis direction is perpendicular to the v-axis direction.

[0041] In a specific example, the intersection equation of the laser plane and the camera image plane is determined in the camera coordinate system. Based on the intersection equation and the camera's intrinsic parameter matrix, the angle between the laser line and the u-axis in the pixel coordinate system is determined. Based on the angles between the preset lifting directions and the u-axis, as well as the angle between the laser line and the u-axis, all preset lifting directions with an angle less than a preset angle to the normal of the laser line are determined, and these preset lifting directions are taken as target lifting directions. Then, the angles between each target lifting direction and the normal of the laser line are compared, and the target lifting direction with the smallest angle to the normal of the laser line is selected, thereby obtaining the light stripe features of the laser line along this target lifting direction. In this way, by obtaining the light stripe features of the laser line along the target lifting direction that is closest to being perpendicular to the laser line, the accuracy of laser line center point extraction is maximized. Finally, the center point of the laser line is extracted based on the light stripe features of the laser line. In this example, after calibrating the laser plane and camera respectively, the plane equation of the projected laser plane in the camera coordinate system can be determined through the camera calibration and laser plane calibration. Then, based on this plane equation, the intersection equation of the laser plane and the camera image plane is determined, and the direction of the intersection line corresponding to this equation is the direction of the laser line. Next, by multiplying the intersection equation by the camera intrinsic parameter matrix, the angle between the intersection line and the u-axis direction in the pixel coordinate system can be determined, i.e., the angle between the laser line and the u-axis direction. Then, for each preset lifting direction, the difference (positive value) between the preset lifting direction and the u-axis direction and the angle between the laser line and the u-axis direction is calculated to obtain the angle between the preset lifting direction and the laser line. Finally, the difference (positive value) between the angle between the preset lifting direction and the laser line and 90 degrees is calculated to obtain the normal angle between the preset lifting direction and the laser line.

[0042] S120. Determine the first target center point based on the center point.

[0043] The first target center point is either the endpoint of a plurality of consecutive center points constituting the first topological line segment, or the first independent center point without adjacent center points. The endpoints of the plurality of consecutive center points constituting the first topological line segment are the first and / or last points among the plurality of consecutive center points.

[0044] In theory, all the center points of a line structured light stripe will connect to form a continuous line segment, i.e., the centerline. In reality, due to the existence of center point extraction bias, the extracted center points can be clustered into multiple line segments (each line segment is a topological line segment). Since the centerline should be continuous, it is necessary to fill in the gaps between the endpoints of one topological line segment and the endpoints of another topological line segment, or between an independent center point without adjacent center points and the endpoints of one topological line segment or another independent center point. Based on this, this embodiment first determines a first target center point based on the center points. The first target center point is either the endpoint of multiple continuous center points constituting the first topological line segment, or a first independent center point without adjacent center points.

[0045] In some embodiments, determining a first target center point based on a center point includes: establishing a topological relationship between the center points, wherein the topological relationship characterizes the adjacency relationship of the center points; determining a center point for which no topological point exists on at least one side based on the topological relationship, and determining the center point as the first target center point.

[0046] Specifically, the center point's position is stored in pixel coordinates. For example, if the sub-pixel coordinates of the center point are (1.1, 1.2), then the stored position of the center point is pixel coordinates (1, 1). (Reference) Figure 3 For each center point (taking center point p as an example in the diagram), search for the center points adjacent to that center point according to the existing search rules, that is, sequentially according to... Figure 3 The search process sequentially searches for center points among pixels A, B, C, and D, while simultaneously traversing all the stored pixel coordinates. If the pixel coordinates of any pixel among A, B, C, and D are the same as the stored pixel coordinates, then the center point is determined to have a topological relationship; otherwise, the center point does not have a topological relationship. After determining the topological relationships of each center point, the center point where only one side lacks a topological point can be directly determined based on the topological relationship; this center point is the first target center point. Specifically, assuming the laser line direction is from the upper left to the lower right, if the topological relationship determines that there is no topological point to the left of the center point but a topological point to the right, then the center point is the starting point of the first topological line segment; if the topological relationship determines that there is no topological point to the right of the center point but a topological point to the left, then the center point is the ending point of the first topological line segment. Furthermore, in the above technical solution, the center points with topological relationships can be determined based on the topological relationships, and thus the center points without topological relationships (i.e., no topological points on either side) can be obtained, thereby determining this center point as the first independent center point. It is understood that the mention of one or both sides of the center point in this disclosure refers to one or both sides of the center point in the direction of the laser line.

[0047] S130. Search for the second target center point that is closest to the first target center point according to the preset search rules.

[0048] The second target center point is either the endpoint of multiple consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points.

[0049] As can be seen from S120, in order to achieve the continuity of the extracted centerline, a second target center point corresponding to the first target center point needs to be found. In this embodiment of the present disclosure, the center point closest to the first target center point is first searched according to a preset search rule, and then the second target center point is determined based on the above-mentioned topological relationship. The first target center point and the second target center point satisfy any of the following relationships: when the first target center point is the starting point of the first topological line segment, the second target center point is the ending point of the second topological line segment or a second independent center point; when the first target center point is the ending point of the first topological line segment, the second target center point is the starting point of the second topological line segment or a second independent center point; when the first target center point is the first independent center point, the second target center point is the endpoint of the second topological line segment or a second independent center point.

[0050] In addition, the preset search rules include: determining the first pixel (such as pixels A, B, C and D above) used to search for adjacent center points of the first target center point based on the position of the first target center point, determining the second and third pixels adjacent to the first pixel, and the fourth and fifth pixels adjacent to the second and third pixels, and searching the second pixel, third pixel, fourth pixel and fifth pixel in sequence.

[0051] For example, see Figure 4 The first target center point is center point p. Based on the position of center point p, the first pixel used to search for the center points adjacent to center point p is determined, including pixels A, B, C and D as shown in the figure. Pixels E, F, G, and H that are adjacent to pixel B vertically, F horizontally, G vertically, and H horizontally are searched in sequence. If a center point is found, it is determined whether the center point is the first point of the second topological line segment or the second independent center point. If the center point is the first point of the second topological line segment or the second independent center point, then the center point is the second target center point.

[0052] In the above scheme, the first and second target center points can be searched along the direction from the beginning to the end of the structured light stripe, or along the direction from the end to the beginning of the structured light stripe. For example, if the first and second target center points are searched along the direction from the beginning to the end of the structured light stripe, in one example, the first target center point is the end point and the second target center point is the beginning point. That is, the first and second target center points are obtained by searching along a single direction of the structured light stripe, avoiding the problem of large computational load caused by repeated searches.

[0053] S140. If a second target center point is found, then fill in the gap between the first target center point and the second target center point.

[0054] If a second target center point is found, it means that there is a missing center point between the first and second target center points, and it is necessary to fill in the missing point between the first and second target center points.

[0055] In some embodiments, interpolation between the first target center point and the second target center point includes: performing linear interpolation between the first target center point and the second target center point to obtain multiple interpolation points. Specifically, refer to... Figure 4 Connect the first target center point p and the second target center point pn with a straight line. On the line connecting the first target center point p and the second target center point pn, select a point whose vertical axis coordinate is the pixel vertical coordinate (in this embodiment, the pixel coordinate is located at the pixel center point) to obtain the interpolation point ps.

[0056] The method for filling in the center points of line structured light stripes provided in this disclosure, based on the initial extraction of the center points of the line structured light stripes, determines the endpoints of multiple consecutive center points constituting a first topological line segment, or a first independent center point without adjacent center points, i.e., a first target center point. In response to finding a second target center point closest to the first target center point, wherein the second target center point is either an endpoint of multiple consecutive center points constituting a second topological line segment, or a second independent center point without adjacent center points, filling in the center points between the first and second target center points. This allows for the interpolation of center points between sparse center points and between sparse center points and other center points, enabling sparse center points to form consecutive adjacent center points, which can be clustered into long line segments with a point count greater than the threshold for small line segments. This solves the problem of erroneous deletion of sparse center points and improves the completeness of center point extraction. In particular, when applied to line laser reconstruction technology, this technical solution can improve the completeness of three-dimensional reconstruction.

[0057] The above technical solution improves the completeness of center point extraction by supplementing the center points. However, there is a possibility of multiple center points being supplemented or some noise points being mistakenly identified as the second target center point. To address this technical problem, in some embodiments, after supplementing the center points between the first and second target center points if a second target center point is found, the method further includes: determining the validity of the supplemented center points; if the supplemented center points are valid, they are retained; if the supplemented center points are invalid, they are deleted. Thus, by determining the validity of the supplemented center points, erroneous supplemented center points can be deleted, ensuring the completeness of center point extraction while avoiding the introduction of noise points.

[0058] Specifically, determining the validity of the interpolation point obtained by interpolation includes: determining whether the gray value at the interpolation point is greater than the gray value threshold of the center point (usually set to 5); if the gray value at the interpolation point is greater than the gray value threshold of the center point, then determining whether there is a gray extreme point within a preset distance from the interpolation point along the normal direction of the interpolation point; if there is a gray extreme point, the interpolation point is determined to be valid; if the gray value at the interpolation point is less than or equal to the gray value threshold of the center point, or if there is no gray extreme point, the interpolation point is determined to be invalid.

[0059] If an interpolation point is valid, it must be located near or at the center point. First, it can be determined whether the grayscale value at the interpolation point is greater than the grayscale threshold at the center point. If the grayscale value at the interpolation point is greater than the center point's grayscale threshold, then it's further determined whether there are grayscale extrema near the interpolation point; otherwise, the interpolation point is directly deemed invalid. Taking a Gaussian distribution of laser line grayscale as an example, if the grayscale value at the interpolation point is greater than the center point's grayscale threshold, and there are grayscale maxima near the interpolation point, then the interpolation point is valid. If the grayscale value at the interpolation point is less than or equal to the center point's grayscale threshold, or if the grayscale value at the interpolation point is greater than the center point's grayscale threshold, but there are no grayscale maxima near the interpolation point, then the interpolation point is invalid. It can be understood that the aforementioned grayscale extrema can be represented by the 0 value of the grayscale gradient.

[0060] Currently, in image processing, Gaussian filtering with a small window is used to reduce noise in the image. However, for light stripe images with specific grayscale characteristics, after Gaussian filtering, the grayscale values ​​of the middle and edge sub-pixels within the window will affect each other, thereby damaging the grayscale characteristics of the light stripe image, reducing the accuracy of image feature extraction, and causing center point shift. To address this, this disclosure also provides a scheme for anisotropic filtering of the grayscale characteristics of line structured light stripes to avoid mutual interference between grayscale values ​​of different orders, protect the grayscale characteristics of the line structured light stripes, and prevent center point shift caused by filtering. Specifically, in this embodiment, after initially extracting the center point of the line structured light stripe, the method further includes: obtaining multi-level grade lines based on the center point and the center point normal of the line structured light stripe, wherein the grade lines reflect the distance from the sub-pixel point to the corresponding center point along the center point normal; performing gray-level Gaussian filtering on each grade line based on the gray level of the line structured light stripe; extracting the third target center point of the line structured light stripe after gray-level Gaussian filtering; and updating the third target center point as the center point of the line structured light stripe.

[0061] Based on the above technical solution, in a specific example, after initially extracting the center points of the line structured light stripes, for each center point, a multi-point (center point) fitting method is used to fit the normal of each center point. (Reference) Figure 5First, based on each center point, a first-order level line is fitted. Then, with the center point as the initial position, second-order level points, third-order level points, and fourth-order level points (corresponding to sub-pixel points schematically shown in the figure) are obtained on both sides of the center point along the center point normal according to a set step size. In this invention application, the second-order level point is one step away from the center point, the third-order level point is twice the step away from the center point, and the fourth-order level point is three times the step away from the center point. Then, multiple level points of the corresponding order are fitted to obtain the second-order level line, the third-order level line, and the fourth-order level line (the dashed lines in the figure represent the level lines of each order). That is, the second-order level line is fitted by multiple second-order level points located on the same side of the first-order level line, the third-order level line is fitted by multiple third-order level points located on the same side of the first-order level line, and the fourth-order level line is fitted by multiple fourth-order level points located on the same side of the first-order level line. Based on the sub-pixel coordinates of the level points, bilinear interpolation is used to interpolate the corresponding sub-pixel grayscale values. Based on the sub-pixel grayscale values ​​corresponding to the level lines, a Gaussian grayscale filter is applied within the level lines. Then, based on the grayscale values ​​of the level lines after the Gaussian grayscale filter, the new center point of the line structured light stripe is extracted again. It should be noted that the set step size for different center points can be the same or different, depending on the actual situation. The extraction of the new center point of the line structured light stripe based on the grayscale values ​​of the level lines after the Gaussian filter can be achieved using extreme value methods or grayscale centroid methods, or it can be achieved using the single-direction line extraction method based on the target line extraction direction provided in this disclosure. This disclosure does not impose any restrictions on this method.

[0062] Furthermore, since the above-mentioned technical solutions directly use linear interpolation to obtain interpolation points, the extraction of the corresponding center points usually has deviations. Therefore, this embodiment of the present disclosure performs secondary optimization on the center points obtained after interpolation to improve the deviation problem. Therefore, if a second target center point is found, after interpolating between the first and second target center points, the method further includes: optimizing all the center points obtained after interpolation to smooth the center line. Specifically, the interpolation points obtained after interpolation are updated as the center points of the line structured light stripe; for each center point, gray-level extreme points are determined based on the gray-level characteristics of the line structured light stripe along the normal direction of the center point; and the center points of the line structured light stripe are updated based on the gray-level extreme points.

[0063] In one example, the sub-pixel grayscale values ​​on each level line can be interpolated using the method described in the above embodiment. The grayscale features of the line structured light stripe along the normal direction of the center point are the grayscale distribution of the level line along the normal direction of the center point of the line structured light stripe. Based on this grayscale distribution, the grayscale extreme points are determined and the grayscale extreme points are updated to the center point.

[0064] Based on the above embodiments, in one specific embodiment, a method for extracting the center point of a line structured light stripe is provided, specifically including:

[0065] Step a: Initially extract the center point of the line structured light stripe;

[0066] Step b: Obtain multi-level lines based on the center point and center point normal of the line structured light stripe; perform gray-level Gaussian filtering on each level line based on the gray level of the line structured light stripe, and extract the center point of the line structured light stripe after gray-level Gaussian filtering.

[0067] Step c: Fill in the center points of the extracted gray-scale Gaussian filtered line structured light stripes;

[0068] Step d: Optimize the center point after the supplementary points are added to smooth the center line.

[0069] Based on the above steps, accurate and complete center points of the line structured light stripe can be extracted. Therefore, when using the center points obtained from the above steps for 3D reconstruction, the accuracy and completeness of the 3D reconstruction can be improved. Detailed descriptions of the above steps can be found in the embodiments of this disclosure. Step c can be implemented using the line structured light stripe center point supplementation method provided in this disclosure, which will not be repeated here. Furthermore, it should be noted that in other embodiments, any one or any two of steps b, c, and d can be executed after step a. It should be noted that the center points processed in steps c and d should be replaced with the center points obtained in the previous step. For example, in one embodiment including steps a and c, step c should be replaced by supplementing the center points of the initially extracted line structured light stripe. In another embodiment including steps a, b, and d, step d should be replaced by optimizing the center points of the extracted grayscale Gaussian filtered line structured light stripe to smooth the center line. Other cases can be replaced accordingly, and will not be listed here.

[0070] This disclosure also provides a device for supplementing the center point of a line structured light stripe to implement the above-described method for supplementing the center point of a line structured light stripe. The following is a description of this device in conjunction with... Figure 6 Please provide an explanation. Figure 6 This is a structural block diagram of a device for supplementing the center point of a linear structured light stripe, provided in an embodiment of this disclosure. (See diagram below.) Figure 6 As shown, the device for supplementing the center point of the structured light stripe includes a center point coarse extraction module 21, a first target center point determination module 22, a second target center point search module 23, and a supplementing module 24.

[0071] Among them, the center point coarse extraction module 21 is used to initially extract the center point of the line structure light stripe;

[0072] The first target center point determination module 22 is used to determine the first target center point based on the center point, wherein the first target center point is the endpoint of a plurality of consecutive center points constituting the first topological line segment, or is the first independent center point without adjacent center points;

[0073] The second target center point search module 23 is used to search for the second target center point that is closest to the first target center point according to a preset search rule. The second target center point is either the endpoint of a plurality of consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points.

[0074] The point-filling module 24 is used to fill in the gap between the first target center point and the second target center point if a second target center point is found.

[0075] In some embodiments, the apparatus further includes a point-filling processing module, for:

[0076] If a second target center point is found, after filling in the gap between the first target center point and the second target center point, the validity of the interpolated point obtained by filling in the gap is determined.

[0077] If the interpolation point is valid, then retain the interpolation point;

[0078] If the interpolation point is invalid, delete the interpolation point.

[0079] In some embodiments, the point-filling processing module is specifically used for:

[0080] Determine whether the gray value at the interpolation point is greater than the gray value threshold at the center point;

[0081] If the gray value at the interpolation point is greater than the gray value threshold at the center point, then determine whether there is a gray extreme point within a preset distance from the interpolation point along the normal direction of the interpolation point.

[0082] If there are extreme gray values, the interpolation point is considered valid;

[0083] If the gray value at the interpolation point is less than or equal to the gray value threshold of the center point, or if there is no gray extreme point, the interpolation point is deemed invalid.

[0084] In some embodiments, the first target center point determination module 22 is specifically used for:

[0085] The topological relationships of the center points are established based on the center points, and the topological relationships represent the adjacency relationships of the center points.

[0086] Based on the topological relationship, determine the center point where there is no topological point on at least one side, and determine this center point as the first target center point.

[0087] In some embodiments, the point-filling module 24 is specifically used for:

[0088] Linear interpolation is performed between the first target center point and the second target center point to obtain multiple interpolation points.

[0089] In some embodiments, the center point coarse extraction module 21 is used to: extract the center point of the line structure light stripe based on the target line extraction direction using a unidirectional line extraction method, wherein the angle between the target line extraction direction and the reference normal of the line structure light stripe is less than a preset angle, the reference normal characterizes the coarse direction of the normal of the center point of the line structure light stripe, and the size of the preset angle limits the degree to which the target line extraction direction is approximately perpendicular to the line structure light stripe.

[0090] In some embodiments, the center point coarse extraction module 21 is specifically used for:

[0091] Select at least one target lifting direction from multiple preset lifting directions in the pixel coordinate system, whose angle with the reference normal of the line structure light stripe is less than a preset angle.

[0092] Acquire the light stripe features of the line structure light stripe along one of the target lifting directions;

[0093] The center point of the structured light stripe is extracted based on the light stripe feature.

[0094] In some embodiments, a filtering module is further included, for:

[0095] After initially extracting the center point of the line structured light stripe, multi-level grade lines are obtained based on the center point and the center point normal. The grade lines reflect the distance from the sub-pixel point to the corresponding center point in the center point normal direction.

[0096] Based on the grayscale values ​​on the linear structured light stripe levels, Gaussian grayscale filtering is performed within each level line.

[0097] Extract the third target center point of the line structured light stripe after grayscale Gaussian filtering;

[0098] Update the center point of the third target to the center point of the line structured light stripe.

[0099] In some embodiments, a center point optimization module is further included, for:

[0100] If a second target center point is found, then after filling in the gaps between the first and second target center points, all the center points obtained after filling in the gaps are optimized to smooth the center line.

[0101] In some embodiments, the center point optimization module is specifically used for:

[0102] Update the interpolation points obtained from the point addition to the center points of the line structure light stripe;

[0103] For each center point, the gray-level extreme points are determined based on the gray-level characteristics of the line structured light stripe along the normal direction of the center point;

[0104] The center point of the structured light stripe is updated based on the gray-level extreme points.

[0105] It should be noted that, Figure 6 The device for supplementing the center point of the line structure light stripe shown can perform... Figure 1 The various steps in the method embodiment shown are implemented. Figure 1 The processes and effects in the method embodiments shown are not described in detail here.

[0106] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown.

[0107] like Figure 7 As shown, the electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0108] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0109] Memory 302 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway device. In a particular embodiment, memory 302 is a non-volatile solid-state memory. In a particular embodiment, memory 302 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0110] The processor 301 reads and executes the computer program instructions stored in the memory 302 to perform the steps of the method for supplementing the center point of the line structured light stripe provided in the embodiments of this disclosure.

[0111] In one example, the electronic device may also include a transceiver 303 and a bus 304. Wherein, as... Figure 7 As shown, the processor 301, memory 302 and transceiver 303 are connected via bus 304 and communicate with each other.

[0112] Bus 304 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 304 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0113] The following are embodiments of a computer-readable storage medium provided in this disclosure. This computer-readable storage medium and the method for supplementing the center point of a line structured light stripe in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the computer-readable storage medium, please refer to the embodiments of the method for supplementing the center point of a line structured light stripe described above.

[0114] This embodiment provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a method for supplementing the center point of a line structured light stripe. The method includes:

[0115] Preliminary extraction of the center point of the structured light stripe;

[0116] The first target center point is determined based on the center point, wherein the first target center point is either the endpoint of a plurality of consecutive center points constituting the first topological line segment, or the first independent center point without adjacent center points.

[0117] The search algorithm searches for the second target center point that is closest to the first target center point according to the preset search rules. The second target center point is either the endpoint of multiple consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points.

[0118] If a second target center point is found, then a supplementary point is added between the first target center point and the second target center point.

[0119] Of course, the computer-executable instructions provided in the embodiments of this disclosure are not limited to the above-described method operations, but can also perform related operations in the method for supplementing the center point of a line structured light stripe provided in any embodiment of this disclosure.

[0120] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this disclosure can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer cloud platform (which may be a personal computer, server, or network cloud platform, etc.) to execute the line structured light stripe center point supplementation method provided in the various embodiments of this disclosure.

[0121] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0122] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for supplementing the center point of a linear structured light stripe, characterized in that, include: Preliminary extraction of the center point of the structured light stripe; A first target center point is determined based on the center point, wherein the first target center point is either the endpoint of a plurality of consecutive center points constituting a first topological line segment, or a first independent center point without adjacent center points. The search algorithm searches for the second target center point that is closest to the first target center point according to a preset search rule. The second target center point is either the endpoint of a plurality of consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points. Wherein, the first target center point and the second target center point satisfy one of the following conditions: when the first target center point is the starting point of the first topological line segment, the second target center point is the ending point of the second topological line segment or a second independent center point; when the first target center point is the ending point of the first topological line segment, the second target center point is the starting point of the second topological line segment or a second independent center point; when the first target center point is the first independent center point, the second target center point is the endpoint of the second topological line segment or a second independent center point. If the second target center point is found, then a supplementary point is added between the first target center point and the second target center point.

2. The method according to claim 1, characterized in that, The search can be performed by moving the first target center point and the second target center point in the direction from the beginning to the end of the line structured light strip, or by moving the first target center point and the second target center point in the direction from the end to the beginning of the line structured light strip.

3. The method according to claim 1, characterized in that, After the step of filling in the gap between the first and second target center points if the second target center point is found, the method further includes: Determine the validity of the interpolated points obtained by the interpolation point addition; If the interpolation point is valid, then retain the interpolation point; If the interpolation point is invalid, then delete the interpolation point.

4. The method according to claim 3, characterized in that, The determination of the validity of the interpolation point obtained by the interpolation point includes: Determine whether the gray value at the interpolation point is greater than the gray value threshold at the center point; If the gray value at the interpolation point is greater than the gray value threshold at the center point, then it is determined whether there is a gray extreme point within a preset distance from the interpolation point along the normal direction of the interpolation point. If there are extreme gray values, the interpolation point is considered valid. If the gray value at the interpolation point is less than or equal to the gray value threshold of the center point, or if there is no gray extreme point, then the interpolation point is determined to be invalid.

5. The method according to claim 1, characterized in that, Determining the first target center point based on the center point includes: A topological relationship is established based on the central point, and the topological relationship represents the adjacency relationship of the central point; Based on the topological relationship, a center point is determined where no topological point exists on at least one side, and this center point is determined as the first target center point.

6. The method according to claim 1, characterized in that, The step of filling in the gap between the first target center point and the second target center point includes: Linear interpolation is performed between the first target center point and the second target center point to obtain multiple interpolation points.

7. The method according to claim 1, characterized in that, The initial extraction of the center point of the linear structured light stripe includes: Based on the target lifting direction, the center point of the linear structured light stripe is extracted using a unidirectional lifting method. The angle between the target lifting direction and the reference normal of the linear structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the linear structured light stripe. The size of the preset angle limits the degree to which the target lifting direction and the linear structured light stripe are approximately perpendicular.

8. The method according to claim 7, characterized in that, The step of extracting the center point of the line structured light stripe based on the target lifting direction using a unidirectional lifting method includes: Select at least one target lifting direction from a plurality of preset lifting directions in the pixel coordinate system, whose angle with the reference normal of the line structure light stripe is less than a preset angle. Acquire the light stripe features of the line structure light stripe along one of the target lifting directions; The center point of the linear structure light stripe is extracted based on the light stripe features.

9. The method according to claim 1, characterized in that, After the initial extraction of the center point of the line structure light stripe, the process further includes: Based on the center point and center point normal of the line structure light stripe, a multi-level hierarchy line is obtained, wherein the hierarchy line reflects the distance from the sub-pixel point to the corresponding center point in the center point normal direction; Based on the grayscale values ​​on the line structured light stripe level lines, grayscale Gaussian filtering is performed in each level of the level line. Extract the third target center point of the line structured light stripe after grayscale Gaussian filtering; Update the center point of the third target to the center point of the line structure light stripe.

10. The method according to any one of claims 1-9, characterized in that, After the step of filling in the gap between the first target center point and the second target center point if the second target center point is found, the method further includes: The center points obtained after the addition of points are optimized to smooth the center line.

11. The method according to claim 10, characterized in that, The optimization of all center points obtained after the point supplementation to smooth the center line includes: Update the interpolation point obtained by the interpolation point to the center point of the line structure light stripe; For each center point, the gray-level extreme point is determined based on the gray-level characteristics of the line structure light stripe along the normal direction of the center point; The center point of the linear structured light stripe is updated based on the gray-level extreme points.

12. A device for supplementing the center point of a linear optical stripe, characterized in that, include: The center point coarse extraction module is used to initially extract the center point of the line structure light stripe; The first target center point determination module is used to determine a first target center point based on the center point, wherein the first target center point is the endpoint of a plurality of consecutive center points constituting a first topological line segment, or is a first independent center point without adjacent center points. The second target center point search module is used to search for the second target center point closest to the first target center point according to a preset search rule. The second target center point is either the endpoint of a plurality of consecutive center points constituting the second topological line segment, or a second independent center point without adjacent center points. Wherein, the first target center point and the second target center point satisfy one of the following conditions: when the first target center point is the starting point of the first topological line segment, the second target center point is the ending point of the second topological line segment or a second independent center point; when the first target center point is the ending point of the first topological line segment, the second target center point is the starting point of the second topological line segment or a second independent center point; when the first target center point is the first independent center point, the second target center point is the endpoint of the second topological line segment or a second independent center point. The point-filling module is used to fill in the gap between the first target center point and the second target center point if the second target center point is found.

13. An electronic device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the method for supplementing the center point of the line structure light stripe as described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, The storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the method for supplementing the center point of the line structured light stripe as described in any one of claims 1-11.

Images