Method, device and equipment for extracting center point of light strip of line structured light and storage medium

Abstract

The present disclosure relates to a line structured light light strip center point extraction method, device, electronic equipment and storage medium. The line structured light light strip center point extraction method comprises: acquiring a light strip image, the light strip image comprising a line structured light light strip; based on a target line extraction direction, a single direction line extraction method is used to extract the center point of the line structured light light strip, wherein the included angle between the target line extraction direction and the reference normal of the line structured light light strip is less than a preset angle, the reference normal represents the rough direction of the normal of the line structured light light strip center point, and the size of the preset angle limits the degree of perpendicularity between the target line extraction direction and the line structured light light strip. The technical scheme of the present disclosure can improve the center point extraction error and missing extraction problem caused by single direction line extraction, improve the accuracy and integrity of laser line center point extraction, and at the same time ensure the extraction efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of three-dimensional measurement technology, and in particular to a method, apparatus, electronic device, and storage medium for extracting the center point of a line structured light stripe. Background Technology

[0002] Line laser reconstruction is a technique that reconstructs the three-dimensional information of an object's surface by projecting a transmitted laser line onto it and capturing the laser line image using a camera. It boasts high efficiency and accuracy, and is widely used in various fields of 3D information reconstruction. The accuracy and time required for laser line center point extraction directly impact the final reconstruction's precision and efficiency; therefore, laser line center point extraction is a crucial step in line laser reconstruction.

[0003] Existing methods for extracting the center point of laser lines include: thresholding, gray-scale centroid method, extremum method, and Steger algorithm. The thresholding method is simple and suitable for coarse calculation of the center line position, but its accuracy cannot meet general industrial scanning standards. The gray-scale centroid method can reduce errors caused by uneven gray levels in the laser line and is highly efficient, but it is easily affected by noise. The extremum method is suitable for extracting laser lines with a Gaussian gray-scale distribution, with good extraction results and high efficiency, but it is also easily affected by noise. The Steger algorithm has high accuracy and robustness, but its computational load is huge, its efficiency is low, and improper selection of the Gaussian kernel can lead to image information distortion. Since the quality of laser images is currently acceptable, to ensure efficient reconstruction, the relatively simple extremum method or gray-scale centroid method is usually used for center point extraction.

[0004] Both the gray-scale centroid method and the extremum method require selecting a direction to analyze and calculate the gray-scale distribution characteristics of the laser line within that directional cross-section. The choice of this direction affects the accuracy and completeness of laser line center extraction. For ease of operation, many devices currently use a single direction (u or v) to process all laser images. The advantage of using a single direction for line extraction is that the single direction is a whole pixel, simplifying calculations. However, for some laser lines modulated to have a small angle with the extraction direction, there may be issues with large extraction errors or missed extractions. Summary of the Invention

[0005] To address the aforementioned technical problems, this disclosure provides a method, apparatus, electronic device, and storage medium for extracting the center point of a line structured light stripe, thereby improving the accuracy and completeness of center point extraction and ensuring extraction efficiency.

[0006] In a first aspect, this disclosure provides a method for extracting the center point of a line structured light stripe, including:

[0007] Acquire a light stripe image, the light stripe image including line-structured light stripes;

[0008] Based on the target lifting direction, the center point of the linear structured light stripe is extracted using a unidirectional lifting method. The angle between the target lifting direction and the reference normal of the linear structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the linear structured light stripe. The size of the preset angle limits the degree to which the target lifting direction and the linear structured light stripe are approximately perpendicular.

[0009] Secondly, this disclosure provides a device for extracting the center point of a line structured light stripe, comprising:

[0010] An image acquisition module is used to acquire a light bar image, wherein the light bar image includes line-structured light bars;

[0011] The center point extraction module is used to extract the center point of the line structured light stripe based on the target lifting direction using a unidirectional lifting method. The angle between the target lifting direction and the reference normal of the line structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the line structured light stripe. The size of the preset angle limits the degree to which the target lifting direction is approximately perpendicular to the line structured light stripe.

[0012] Thirdly, this disclosure provides an electronic device, including:

[0013] processor;

[0014] Memory, used to store executable instructions;

[0015] The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the line structured light stripe center point extraction method described in the first aspect above.

[0016] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor implements the line structured light stripe center point extraction method described in the first aspect.

[0017] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0018] The technical solution provided in this disclosure obtains the light stripe features of the line structure light stripe along the target lifting direction, and extracts the center point of the line structure light stripe based on the light stripe features. The angle between the target lifting direction and the reference normal of the line structure light stripe is less than a preset angle. The reference normal characterizes the approximate direction of the normal of the center point of the line structure light stripe. The size of the preset angle limits the degree to which the target lifting direction and the line structure light stripe are nearly perpendicular. Thus, compared to existing fixed u-axis or v-axis unidirectional line extraction, the technical solution of this disclosure can adaptively select a target line extraction direction with an angle smaller than a preset angle with the reference normal, based on the actual line structured light stripe. Because the angle between the target line extraction direction and the reference normal is smaller than the preset angle, the target line extraction direction is nearly perpendicular (or even perpendicular) to the line structured light stripe. Therefore, the center point can be extracted along the direction close to the maximum grayscale gradient of the line structured light stripe, thereby preventing the angle between the line extraction direction and the line structured light stripe from being too small. This effectively improves the center point extraction errors and omissions that may be caused by unidirectional line extraction, improving the accuracy and completeness of the line structured light stripe center point extraction. Furthermore, the method of extracting the center point along the target line extraction direction is more efficient than the Steger algorithm and can meet the requirements of high-speed real-time scanning of laser scanning equipment. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for extracting the center point of a line structured light stripe, provided in an embodiment of this disclosure;

[0022] Figure 2 A schematic diagram of a scene for acquiring a light stripe image provided in an embodiment of this disclosure;

[0023] Figure 3 A schematic diagram showing the distribution of preset lifting directions provided in the embodiments of this disclosure;

[0024] Figure 4 A schematic diagram illustrating the determination of each level of hierarchy line along the center point normal direction provided in this embodiment of the disclosure;

[0025] Figure 5 This is a schematic diagram illustrating the search for neighboring center points provided in an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram illustrating the search for the center point and supplementary points of the second target provided in an embodiment of this disclosure;

[0027] Figure 7 A structural block diagram of a line structure light stripe center point extraction device provided in this embodiment of the present disclosure;

[0028] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0029] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0030] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0031] Figure 1 This is a flowchart illustrating a method for extracting the center point of a line structured light stripe according to an embodiment of this disclosure. This method is applicable to extracting the center point of a line structured light stripe along the lifting direction. This method can be executed by a line structured light stripe center point extraction device, which can be implemented in software and / or hardware. Figure 1 As shown, this method includes the following steps:

[0032] S110. Acquire the light bar image.

[0033] The light stripe image includes line-structured light stripes. In this embodiment of the disclosure, reference is made to... Figure 2 light stripe image ( Figure 2The illustration schematically shows light stripe images 41 and 42 (captured by the first camera 31 and the second camera 32 respectively) as images of the surface of object 2 after the projector 1 projects line structured light onto the surface of object 2. These images, captured by the cameras (first camera 31 and second camera 32), can contain three-dimensional contour information of the object 2 surface and form the basis for three-dimensional reconstruction of the object 2 surface. Furthermore, the line structured light projected by the projector 1 is a thin light plane that forms light stripes of a certain width on the object surface. The light stripes captured by the cameras are the line structured light stripes involved in this disclosure. For example, light stripe image 41 includes line structured light stripe 411, and light stripe image 42 includes line structured light stripe 421. In this embodiment, after the camera captures the light stripe image, the image can be filtered and denoised before the center point of the line structured light stripe in the image is extracted. In this embodiment, the projector can be a laser, and correspondingly, the line structured light is a line laser, and the line structured light stripe is a laser line. It is understood that... Figure 2 Only one projector 1 is shown schematically. This disclosure can also be applied to projection systems with two or more projectors to project line structured light onto object 2 from different directions. Furthermore, in order for the camera to capture the line structured light stripes projected onto the object surface by different projectors, each projector alternates in projecting the line structured light.

[0034] S120. Based on the target lifting direction, the center point of the structured light stripe is extracted using a unidirectional lifting method.

[0035] Specifically, the angle between the target lifting direction and the reference normal of the structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal to the center point of the structured light stripe, and the preset angle limits the degree to which the target lifting direction is nearly perpendicular to the structured light stripe. This allows for the extraction of the center point of the original light stripe more closely aligned with the direction of the maximum grayscale gradient, improving the accuracy and completeness of the center point extraction.

[0036] In one example, based on the target lifting direction, a unidirectional lifting method is used to extract the center point of the line structured light stripe, including:

[0037] Step A: Select at least one target lifting direction from multiple preset lifting directions in the pixel coordinate system, whose angle with the reference normal is less than a preset angle.

[0038] In this embodiment, the preset angle is used to ensure the accuracy and completeness of center point extraction. Specifically, if the angle between the preset lifting direction and the reference normal (normal direction) is less than the preset angle, center point extraction along this preset lifting direction can accurately and completely extract the center point on the line structured light stripe. Furthermore, the size of the preset angle can, as a whole, limit the degree to which the target lifting direction selected from multiple preset lifting directions is approximately perpendicular to the line structured light stripe. In other words, generally speaking, the smaller the preset angle, the closer the target lifting direction is to being perpendicular to the line structured light stripe, thus increasing the accuracy of the center point extracted along the target lifting direction. It is understood that designing more preset lifting directions in different directions ensures that, on the one hand, a target lifting direction with an angle less than the preset angle to the reference normal can be selected for line structured light stripes in various directions; on the other hand, it makes the target lifting direction more perpendicular to the line structured light stripe, improving the extraction accuracy of the center point of the line structured light stripe.

[0039] In one embodiment, see Figure 3 The preset lifting directions include a first lifting direction, a second lifting direction, a third lifting direction, and a fourth lifting direction. The first lifting direction is the u-axis direction of the pixel coordinate system, the second lifting direction is the v-axis direction of the pixel coordinate system, the third lifting direction and the u-axis direction make an angle of 45 degrees, the fourth lifting direction and the v-axis direction make an angle of 45 degrees, and the u-axis direction is perpendicular to the v-axis direction. The third lifting direction is also perpendicular to the fourth lifting direction. Therefore, by setting the above four preset lifting directions, this embodiment can select the target lifting direction from these four preset lifting directions for most current laser scanning devices. This reduces the computational load while improving the accuracy and completeness of center point extraction. It should be noted that the preset lifting directions in this embodiment are not limited to the above four types; the number and angle of the preset lifting directions can be customized according to the actual hardware structure, camera calibration, and optical plane calibration.

[0040] In one embodiment, selecting at least one target lifting direction from a plurality of preset lifting directions in the pixel coordinate system, wherein the angle between the target lifting direction and the reference normal is less than a preset angle, includes: determining the intersection equation of the light plane and the camera image plane in the camera coordinate system; determining the angle between the line structured light stripe and the u-axis direction in the pixel coordinate system based on the intersection equation and the camera's intrinsic parameter matrix; determining all preset lifting directions whose angle between the target lifting direction and the u-axis direction is less than a preset angle based on the angle between the preset lifting direction and the u-axis direction and the angle between the line structured light stripe and the u-axis direction, and taking all preset lifting directions whose angle between the target lifting direction and the reference normal is less than a preset angle as the target lifting direction.

[0041] Specifically, after determining the projector's direction, position, and the angle between the projector and the camera, the plane equation of the projected light plane in the camera coordinate system can be determined through camera calibration and light plane calibration. Then, based on this plane equation, the intersection equation between the light plane and the camera image plane is determined. The direction of the intersection line corresponding to this equation is the direction of the line structured light stripe. Next, by multiplying the intersection equation by the camera intrinsic parameter matrix, the angle between the intersection line and the u-axis direction in the pixel coordinate system can be determined, i.e., the angle between the line structured light stripe and the u-axis direction. Then, for each preset lifting direction, the difference (positive value) between the preset lifting direction and the u-axis direction and the angle between the line structured light stripe and the u-axis direction is calculated to obtain the angle between the preset lifting direction and the line structured light stripe. Finally, the difference (positive value) between the angle between the preset lifting direction and the line structured light stripe and 90 degrees is calculated to obtain the angle between the preset lifting direction and the reference normal. Finally, the angle between each preset lifting direction and the reference normal is compared with a preset angle, and all preset lifting directions whose angle with the reference normal is less than the preset angle are taken as target lifting directions.

[0042] Step B: Obtain the light stripe features of the line structure light stripe along one of the target lifting directions.

[0043] Based on the above steps, it is known that the angle between the target lifting direction and the reference normal is less than a preset angle. Therefore, as long as the center point of the line structured light stripe is extracted along the target lifting direction, the accuracy and completeness of the line structured light stripe center point extraction can be guaranteed. This embodiment can acquire the light stripe features of the line structured light stripe along any target lifting direction. In a preferred embodiment, the light stripe features of the line structured light stripe are acquired along the target lifting direction with the smallest angle to the reference normal. Specifically, the angles between each target lifting direction and the reference normal are compared, and the target lifting direction with the smallest angle to the reference normal is selected, thereby acquiring the light stripe features of the line structured light stripe along this target lifting direction. Thus, by acquiring the light stripe features of the line structured light stripe along the target lifting direction that is closest to perpendicular to the line structured light stripe, the accuracy of the line structured light stripe center point extraction is maximized.

[0044] Furthermore, the light stripe feature refers to the imaging feature of the line structured light stripe obtained when the center point of the line structured light stripe is extracted using a unidirectional line lifting method in related technologies. In one embodiment, the light stripe feature includes grayscale distribution features or gradient distribution features. In particular, for the acquisition of gradient distribution features, gradient calculation can be performed along the target line lifting direction using a one-dimensional difference method or a two-dimensional Gaussian convolution method.

[0045] Step C: Extract the center point of the structured light stripe based on the light stripe feature.

[0046] Taking a linear structured light stripe as an example, since the grayscale of the laser line follows a Gaussian distribution—that is, the grayscale is maximum at its center and gradually decreases towards both sides—the center point of the laser line can be extracted based on its grayscale distribution characteristics. Furthermore, the gradient of a laser line refers to the change in its grayscale; therefore, the gradient can be considered as the first derivative of the grayscale. Based on the gradient distribution characteristics, the gradient zero point, or gradient extremum point, can be obtained, and the location corresponding to this gradient zero point is the center point of the laser line.

[0047] The method for extracting the center point of a line structured light stripe provided in this disclosure, by setting multiple different preset lifting directions, can adaptively select a target lifting direction with an angle smaller than a preset angle to the reference normal based on the line structured light stripe in the acquired light stripe image. Then, it acquires the light stripe features of the line structured light stripe along the target lifting direction and extracts the center point of the line structured light stripe based on these features. Thus, compared to existing fixed single-direction lifting, this technical solution can adaptively select a lifting direction (i.e., the target lifting direction) with an angle smaller than a preset angle to the reference normal according to the actual line structured light stripe. Because the angle between the target lifting direction and the reference normal is smaller than the preset angle, the target lifting direction is nearly perpendicular (or even perpendicular) to the line structured light stripe. Therefore, it can extract the center point along the direction close to the maximum grayscale gradient of the line structured light stripe, thereby preventing the lifting direction from having a small angle with the line structured light stripe. This effectively improves the problem of center point extraction errors and omissions that may be caused by single-direction lifting, and improves the accuracy and completeness of line structured light stripe center point extraction. Meanwhile, the method of extracting the center point along the target lifting direction is more efficient than the Steger algorithm and can meet the requirements of high-speed real-time scanning of laser scanning equipment.

[0048] In some embodiments, before selecting at least one target lifting direction whose angle with the reference normal is less than a preset angle from a plurality of preset lifting directions in the pixel coordinate system, the method further includes: obtaining the direction of the line structured light stripe; and filtering out candidate lifting directions from the preset lifting directions based on the direction of the line structured light stripe.

[0049] In the aforementioned embodiments, during the process of selecting at least one target lifting direction from multiple preset lifting directions whose angle with the reference normal is less than a preset angle, it is necessary to compare the angle between each preset lifting direction and the reference normal with a preset angle. This involves a large amount of computation, especially when the number of preset lifting directions is large, which significantly reduces the efficiency of extracting the center point of the line structured light stripe. Considering that most of the preset lifting directions may obviously not satisfy the relationship that the angle with the reference normal is less than a preset angle, the comparison calculation for these preset lifting directions is invalid. Therefore, in some embodiments, the preset lifting directions most likely to satisfy the relationship that the angle with the reference normal is less than a preset angle can be initially screened based on the direction of the line structured light stripe, i.e., the candidate lifting directions. For example, combined with Figure 3 If the direction of the line structured light stripe is from the first quadrant to the third quadrant, the third lifting direction can be clearly eliminated (the angle between the third lifting direction and the line structured light stripe is necessarily small). Since the specific direction of the line structured light stripe is uncertain at this point, the first, second, and fourth lifting directions can all be considered as candidate lifting directions. It is understood that the above example only uses four preset lifting directions for illustrative purposes. The more preset lifting directions there are, the more preset lifting directions can be eliminated, and the more significant the reduction in computational load. Correspondingly, at least one target lifting direction with an angle smaller than a preset angle to the reference normal is selected from the candidate lifting directions in the pixel coordinate system. Thus, by filtering candidate lifting directions from the preset lifting directions based on the direction of the line structured light stripe, the computational load of comparing the angle between the preset lifting direction and the reference normal with the preset angle can be reduced, thereby improving the overall efficiency of extracting the center point of the line structured light stripe.

[0050] In 3D reconstruction, there are typically two modes: system calibration mode and scanning mode. This embodiment can acquire the direction of the line structured light stripe in system calibration mode or in scanning mode.

[0051] In one example, under system calibration mode, based on the light plane calibration and camera calibration results, the intersection direction of the light plane and the camera image plane is determined, and this intersection direction is used as the direction of the line structured light stripe. Specifically, after the projector's direction, position, and the angle between the projector and the camera are determined, the direction of the line structured light stripe is fixed. By calibrating the light plane and the camera, the intersection direction of the light plane and the camera image plane can be obtained, i.e., the direction of the line structured light stripe. Then, based on the direction of the line structured light stripe, candidate line lifting directions can be selected from preset lifting directions.

[0052] In another example, in scanning mode, line structured light stripes are extracted from the light stripe image based on a thresholding method, and the direction of the line structured light stripes is determined.

[0053] Specifically, the threshold method can quickly and roughly obtain the direction of the line structured light stripe, and then, based on the direction of the line structured light stripe, candidate lifting directions can be selected from the preset lifting directions. Correspondingly, at least one target lifting direction with an angle smaller than a preset angle to the reference normal is selected from the candidate lifting directions in the pixel coordinate system. Similarly, this embodiment can reduce the computational load of comparing the angle between the preset lifting direction and the reference normal with the preset angle, thereby improving the overall efficiency of line structured light stripe center point extraction. It should be noted that, in scanning mode, line structured light bars can be extracted from the first frame of light bar image to obtain the direction of the line structured light bars, and this direction of the line structured light bars can be used as the direction of the line structured light bars in each subsequent frame of light bar image; alternatively, line structured light bars can be extracted from each frame of light bar image to obtain the direction of the line structured light bars in each frame of light bar image; alternatively, a frame of light bar image can be acquired at regular intervals, and line structured light bars can be extracted from that frame of light bar image to obtain the direction of the line structured light bars in that frame of light bar image, and this direction of the line structured light bars can be used as the direction of the line structured light bars in each subsequent frame of light bar image within the interval period.

[0054] The above embodiments can be applied to scenarios with line structured light in one projection direction, or to scenarios with line structured light in different projection directions. In some embodiments, the light stripe image includes a first light stripe image and / or a second light stripe image; the extraction of the center point of the line structured light stripe based on the target lifting direction using a unidirectional lifting method includes: extracting the center point of the first line structured light stripe in the first light stripe image based on the first target lifting direction; and / or, extracting the center point of the second line structured light stripe in the second light stripe image based on the second target lifting direction, wherein the direction of the second line structured light stripe is different from the direction of the first line structured light stripe, and the second target lifting direction is different from the first target lifting direction. For example, taking a scenario with line structured light projected in different directions, assume there are two projectors, namely a first projector and a second projector. The first and second projectors alternately project line structured light onto the object being measured from different directions. The first projector projects a first line structured light, and the second projector projects a second line structured light. The first projector projects the first line structured light onto the surface of the object being measured, and after modulation by the surface, a first light stripe image is captured by a camera. The second projector projects the second line structured light onto the surface of the object being measured, and after modulation by the surface, a second light stripe image is captured by a camera. Then, the first line structured light stripe is identified from the first light stripe image, and the second line structured light stripe is identified from the second light stripe image. Since the projection directions of the first and second projectors are different, the directions of the first and second line structured light stripes are also different. Therefore, based on the different first and second target lifting directions, the center point of the first line structured light stripe in the first light stripe image and the center point of the second line structured light stripe in the second light stripe image are extracted.

[0055] Currently, in image processing, Gaussian filtering with a small window is used to reduce noise in the image. However, for light stripe images with specific grayscale characteristics, after Gaussian filtering, the grayscale values ​​of the middle and edge sub-pixels within the window will affect each other, thereby damaging the grayscale characteristics of the light stripe image, reducing the accuracy of image feature extraction, and causing center point shift. To address this, this disclosure also provides a scheme for anisotropic filtering of the grayscale characteristics of line structured light stripes to avoid mutual interference between grayscale values ​​of different orders, protect the grayscale characteristics of the line structured light stripes, and prevent center point shift caused by filtering. Specifically, in this embodiment, after extracting the center point of the line structured light stripe using a unidirectional line extraction method based on the target extraction direction, the method further includes: obtaining multi-level grade lines based on the center point of the line structured light stripe and the center point normal, wherein the grade lines reflect the distance from the sub-pixel point to the corresponding center point along the center point normal; performing gray-level Gaussian filtering on each grade line based on the gray level of the line structured light stripe; extracting the third target center point of the line structured light stripe after gray-level Gaussian filtering; and updating the third target center point as the center point of the line structured light stripe.

[0056] Based on the above technical solution, in a specific example, after initially extracting the center points of the line structured light stripes, for each center point, a multi-point (center point) fitting method is used to fit the normal of each center point. (Reference) Figure 4First, based on each center point, a first-order level line is fitted. Then, with the center point as the initial position, second-order level points, third-order level points, and fourth-order level points (corresponding to sub-pixel points schematically shown in the figure) are obtained on both sides of the center point along the center point normal according to a set step size. In this invention application, the second-order level point is one step away from the center point, the third-order level point is twice the step away from the center point, and the fourth-order level point is three times the step away from the center point. Then, multiple level points of the corresponding order are fitted to obtain the second-order level line, the third-order level line, and the fourth-order level line (the dashed lines in the figure represent the level lines of each order). That is, the second-order level line is fitted by multiple second-order level points located on the same side of the first-order level line, the third-order level line is fitted by multiple third-order level points located on the same side of the first-order level line, and the fourth-order level line is fitted by multiple fourth-order level points located on the same side of the first-order level line. Based on the sub-pixel coordinates of the level points, bilinear interpolation is used to interpolate the corresponding sub-pixel grayscale values. Based on the sub-pixel grayscale values ​​corresponding to the level lines, a Gaussian grayscale filter is applied within the level lines. Then, based on the grayscale values ​​of the level lines after the Gaussian grayscale filter, the new center point of the line structured light stripe is extracted again. It should be noted that the set step size for different center points can be the same or different, depending on the actual situation. The extraction of the new center point of the line structured light stripe based on the grayscale values ​​of the level lines after the Gaussian filter can be achieved using extreme value methods or grayscale centroid methods, or it can be achieved using the single-direction line extraction method based on the target line extraction direction provided in this disclosure. This disclosure does not impose any restrictions on this method.

[0057] Considering that unidirectional line extraction methods all suffer from the problem of missing center points, resulting in some sparse center points, if these sparse center points cannot find adjacent center points based on existing topological search rules, or cannot be clustered into long line segments with a number greater than the threshold of small line segment points, they will be considered as stray points or miscellaneous line segments and mistakenly deleted, further reducing the completeness of center point extraction. To address this, in one embodiment, after extracting the center points of the line structured light stripe using a unidirectional line extraction method based on the target line extraction direction, the method further includes: supplementing the sparse center points of the line structured light stripe. Specifically, the process of filling in the sparse center points of the line structured light stripe includes: determining a first target center point based on the center point of the line structured light stripe, wherein the first target center point is either the endpoint of a plurality of consecutive center points constituting a first topological line segment, or a first independent center point without adjacent center points, and the endpoint of the plurality of consecutive center points constituting the first topological line segment is the first point and / or the end point among the plurality of consecutive center points; searching for a second target center point closest to the first target center point according to a preset search rule, wherein the second target center point is either the endpoint of a plurality of consecutive center points constituting a second topological line segment, or a second independent center point without adjacent center points; and if a second target center point is found, then filling in the gap between the first target center point and the second target center point.

[0058] In theory, all the center points of a line structured light stripe will connect to form a continuous line segment, i.e., the centerline. In reality, due to the existence of center point extraction bias, the extracted center points can be clustered into multiple line segments (each line segment is a topological line segment). Since the centerline should be continuous, it is necessary to fill in the gaps between the endpoints of one topological line segment and the endpoints of another topological line segment, or between an independent center point without adjacent center points and the endpoints of one topological line segment or another independent center point. Based on this, this embodiment first determines a first target center point based on the center points. The first target center point is either the endpoint of multiple continuous center points constituting the first topological line segment, or a first independent center point without adjacent center points.

[0059] In some embodiments, determining a first target center point based on a center point includes: establishing a topological relationship between the center points, wherein the topological relationship characterizes the adjacency relationship of the center points; determining a center point for which no topological point exists on at least one side based on the topological relationship, and determining the center point as the first target center point.

[0060] Specifically, the center point's position is stored in pixel coordinates. For example, if the sub-pixel coordinates of the center point are (1.1, 1.2), then the stored position of the center point is pixel coordinates (1, 1). (Reference) Figure 5 For each center point (taking center point p as an example in the diagram), search for the center points adjacent to that center point according to the existing search rules, that is, sequentially according to... Figure 5The search process sequentially searches for center points among pixels A, B, C, and D, while simultaneously traversing all the stored pixel coordinates. If the pixel coordinates of any pixel among A, B, C, and D are the same as the stored pixel coordinates, then the center point is determined to have a topological relationship; otherwise, the center point does not have a topological relationship. After determining the topological relationships of each center point, a center point where only one side lacks a topological point can be directly determined based on the topological relationship; this center point is the first target center point. Specifically, assuming the direction of the line structured light strip is from the upper left to the lower right, if the topological relationship determines that there is no topological point to the left of the center point but a topological point to the right, then this center point is the starting point of the first topological line segment; if the topological relationship determines that there is no topological point to the right of the center point but a topological point to the left, then this center point is the ending point of the first topological line segment. Furthermore, in the above technical solution, it is also possible to determine center points with topological relationships based on the topological relationships, and then obtain center points without topological relationships (i.e., no topological points on either side), thereby determining this center point as the first independent center point. It is understood that the "one side or both sides" of the center point mentioned in this disclosure refers to one side or both sides of the center point in the direction of the structured light stripe.

[0061] To ensure the continuity of the extracted centerline, a second target center point corresponding to the first target center point needs to be found. In this embodiment, the center point closest to the first target center point is first searched according to a preset search rule, and then the second target center point is determined based on the above-mentioned topological relationship. The first target center point and the second target center point satisfy any of the following relationships: when the first target center point is the starting point of the first topological line segment, the second target center point is the ending point of the second topological line segment or a second independent center point; when the first target center point is the ending point of the first topological line segment, the second target center point is the starting point of the second topological line segment or a second independent center point; when the first target center point is the first independent center point, the second target center point is the endpoint of the second topological line segment or a second independent center point.

[0062] In addition, the preset search rules include: determining the first pixel (such as pixels A, B, C and D above) used to search for adjacent center points of the first target center point based on the position of the first target center point, determining the second and third pixels adjacent to the first pixel, and the fourth and fifth pixels adjacent to the second and third pixels, and searching the second pixel, third pixel, fourth pixel and fifth pixel in sequence.

[0063] For example, see Figure 6The first target center point is center point p. Based on the position of center point p, the first pixel used to search for the center points adjacent to center point p is determined, including pixels A, B, C and D as shown in the figure. Pixels E, F, G, and H that are adjacent to pixel B vertically, F horizontally, G vertically, and H horizontally are searched in sequence. If a center point is found, it is determined whether the center point is the first point of the second topological line segment or the second independent center point. If the center point is the first point of the second topological line segment or the second independent center point, then the center point is the second target center point.

[0064] In the above scheme, the first and second target center points can be searched along the direction from the beginning to the end of the structured light stripe, or along the direction from the end to the beginning of the structured light stripe. For example, if the first and second target center points are searched along the direction from the beginning to the end of the structured light stripe, in one example, the first target center point is the end point and the second target center point is the beginning point. That is, the first and second target center points are obtained by searching along a single direction of the structured light stripe, avoiding the problem of large computational load caused by repeated searches.

[0065] If a second target center point is found, it means that there is a missing center point between the first and second target center points, and it is necessary to fill in the missing point between the first and second target center points.

[0066] In some embodiments, interpolation between the first target center point and the second target center point includes: performing linear interpolation between the first target center point and the second target center point to obtain multiple interpolation points. Specifically, refer to... Figure 6 Connect the first target center point p and the second target center point pn with a straight line. On the line connecting the first target center point p and the second target center point pn, select a point whose vertical axis coordinate is the pixel vertical coordinate (in this embodiment, the pixel coordinate is located at the pixel center point) to obtain the interpolation point ps.

[0067] The above technical solution improves the completeness of center point extraction by supplementing the center points. However, there is a possibility of multiple center points being supplemented or some noise points being mistakenly identified as the second target center point. To address this technical problem, in some embodiments, after supplementing the center points between the first and second target center points if a second target center point is found, the method further includes: determining the validity of the supplemented center points; if the supplemented center points are valid, they are retained; if the supplemented center points are invalid, they are deleted. Thus, by determining the validity of the supplemented center points, erroneous supplemented center points can be deleted, ensuring the completeness of center point extraction while avoiding the introduction of noise points.

[0068] Specifically, determining the validity of the interpolation point obtained by interpolation includes: determining whether the gray value at the interpolation point is greater than the gray value threshold of the center point (usually set to 5); if the gray value at the interpolation point is greater than the gray value threshold of the center point, then determining whether there is a gray extreme point within a preset distance from the interpolation point along the normal direction of the interpolation point; if there is a gray extreme point, the interpolation point is determined to be valid; if the gray value at the interpolation point is less than or equal to the gray value threshold of the center point, or if there is no gray extreme point, the interpolation point is determined to be invalid.

[0069] If an interpolation point is valid, it must be located near or at the center point. First, it can be determined whether the grayscale value at the interpolation point is greater than the grayscale threshold of the center point. If the grayscale value at the interpolation point is greater than the center point's grayscale threshold, then it is further determined whether there are grayscale extreme points near the interpolation point; otherwise, the interpolation point is directly determined to be invalid. Taking the Gaussian distribution of grayscale values ​​in a structured light stripe as an example, if the grayscale value at the interpolation point is greater than the center point's grayscale threshold, and there are grayscale extreme points near the interpolation point, then the interpolation point is valid. If the grayscale value at the interpolation point is less than or equal to the center point's grayscale threshold, or if the grayscale value at the interpolation point is greater than the center point's grayscale threshold, but there are no grayscale extreme points near the interpolation point, then the interpolation point is invalid. It can be understood that the aforementioned grayscale extreme points can be represented by the 0 value of the grayscale gradient.

[0070] Furthermore, since the above-mentioned technical solutions directly use linear interpolation to obtain the interpolation points, the extraction of the corresponding center points usually has deviations. Therefore, this embodiment of the present disclosure performs secondary optimization on the center points obtained after interpolation to improve the deviation problem. Therefore, after extracting the center points of the structured light stripe based on the target lifting direction using a unidirectional lifting method, the method further includes: optimizing the center points to smooth the center lines. Specifically, for each center point, based on the grayscale characteristics of the structured light stripe along the normal direction of the center point, a grayscale extreme point is determined; the center point of the structured light stripe is updated based on the grayscale extreme points.

[0071] In one example, the sub-pixel grayscale values ​​on each level line can be interpolated using the method described in the above embodiment. The grayscale features of the line structured light stripe along the normal direction of the center point are the grayscale distribution of the level line along the normal direction of the center point of the line structured light stripe. Based on this grayscale distribution, the grayscale extreme points are determined and the grayscale extreme points are updated to the center point.

[0072] Based on the above embodiments, in one specific embodiment, a method for extracting the center point of a line structured light stripe is provided, specifically including:

[0073] Step a: Initially extract the center point of the line structured light stripe;

[0074] Step b: Obtain multi-level lines based on the center point and center point normal of the line structured light stripe; perform gray-level Gaussian filtering on each level line based on the gray level of the line structured light stripe, and extract the center point of the line structured light stripe after gray-level Gaussian filtering.

[0075] Step c: Fill in the center points of the extracted gray-scale Gaussian filtered line structured light stripes;

[0076] Step d: Optimize the center point after the supplementary points are added to smooth the center line.

[0077] Based on the above steps, accurate and complete center points of the line structured light stripe can be extracted. Therefore, when using the center points obtained from the above steps for 3D reconstruction, the accuracy and completeness of the 3D reconstruction can be improved. Detailed descriptions of the above steps can be found in the embodiments of this disclosure. Step a can be implemented using the line structured light stripe center point extraction method provided in this disclosure, and will not be repeated here. Furthermore, it should be noted that in other embodiments, any one or any two of steps b, c, and d can be executed after step a. It should be noted that the center points processed in steps c and d should be replaced with the center points obtained in the previous step. For example, in one embodiment including steps a and c, step c should be replaced by supplementing the center points of the initially extracted line structured light stripe. In another embodiment including steps a, b, and d, step d should be replaced by optimizing the center points of the extracted grayscale Gaussian filtered line structured light stripe to smooth the center line. Other cases can be replaced accordingly, and will not be listed here.

[0078] This disclosure also provides a line structured light stripe center point extraction device for implementing the above-described line structured light stripe center point extraction method. The following is in conjunction with... Figure 7 Please provide an explanation. Figure 7 This is a structural block diagram of a device for extracting the center point of a line structured light stripe, provided in an embodiment of this disclosure. Figure 7 As shown, the line structured light stripe center point extraction device includes an image acquisition module 21 and a center point extraction module 22.

[0079] The image acquisition module 21 is used to acquire light stripe images, which include line-structured light stripes.

[0080] The center point extraction module 22 is used to extract the center point of the line structured light stripe based on the target lifting direction and using a unidirectional lifting method. The angle between the target lifting direction and the reference normal of the line structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the line structured light stripe. The size of the preset angle limits the degree to which the target lifting direction and the line structured light stripe are nearly perpendicular.

[0081] It should be noted that, Figure 7 The line structure light stripe center point extraction device shown can perform... Figure 1 The various steps in the method embodiment shown are implemented. Figure 1 The processes and effects in the method embodiments shown are not described in detail here.

[0082] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown.

[0083] like Figure 8 As shown, the electronic device may include a processor 301 and a memory 302 storing computer program instructions.

[0084] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0085] Memory 302 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway device. In a particular embodiment, memory 302 is a non-volatile solid-state memory. In a particular embodiment, memory 302 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0086] The processor 301 reads and executes the computer program instructions stored in the memory 302 to perform the steps of the line structured light stripe center point extraction method provided in this embodiment of the present disclosure.

[0087] In one example, the electronic device may also include a transceiver 303 and a bus 304. Wherein, as... Figure 8 As shown, the processor 301, memory 302 and transceiver 303 are connected via bus 304 and communicate with each other.

[0088] Bus 304 may include hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 304 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0089] The following are embodiments of a computer-readable storage medium provided in this disclosure. This computer-readable storage medium and the line structured light stripe center point extraction method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the computer-readable storage medium, please refer to the embodiments of the line structured light stripe center point extraction method described above.

[0090] This embodiment provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to perform a method for extracting the center point of a line structured light stripe. The method includes:

[0091] Acquire a light stripe image, which includes line-structured light stripes;

[0092] Based on the target lifting direction, the center point of the line structured light stripe is extracted using a unidirectional lifting method. The angle between the target lifting direction and the reference normal of the line structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the normal of the center point of the line structured light stripe. The size of the preset angle limits the degree to which the target lifting direction and the line structured light stripe are approximately perpendicular.

[0093] Of course, the computer-executable instructions provided in the embodiments of this disclosure are not limited to the above-described method operations, but can also perform related operations in the line structured light stripe center point extraction method provided in any embodiment of this disclosure.

[0094] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this disclosure can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer cloud platform (which may be a personal computer, server, or network cloud platform, etc.) to execute the line structured light stripe center point extraction method provided in the various embodiments of this disclosure.

[0095] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0096] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for extracting the center point of a line structured light stripe, characterized in that, include: Acquire a light stripe image, the light stripe image including line-structured light stripes; Based on the target lifting direction, the center point of the linear structured light stripe is extracted using a single-direction lifting method. The angle between the target lifting direction and the reference normal of the linear structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the center point normal of the linear structured light stripe. The preset angle limits the degree to which the target lifting direction and the linear structured light stripe are approximately perpendicular. The center point normal is obtained by fitting multiple center points based on the center point of the linear structured light stripe. After extracting the center point of the line structured light stripe using a unidirectional lifting method based on the target lifting direction, the method further includes: Based on the center point and center point normal of the line structure light stripe, a multi-level hierarchy line is obtained, wherein the hierarchy line reflects the distance from the sub-pixel point to the corresponding center point in the center point normal direction; Based on the grayscale values ​​on the line structured light stripe level lines, grayscale Gaussian filtering is performed in each level of the level line. Extract the third target center point of the line structured light stripe after grayscale Gaussian filtering; The center point of the line structured light stripe after grayscale Gaussian filtering is updated to the center point of the line structured light stripe.

2. The method according to claim 1, characterized in that, The step of extracting the center point of the line structured light stripe based on the target lifting direction using a unidirectional lifting method includes: Select at least one target lifting direction from a plurality of preset lifting directions in the pixel coordinate system, wherein the angle between the target direction and the reference normal is less than a preset angle. Acquire the light stripe features of the line structure light stripe along one of the target lifting directions; The center point of the linear structure light stripe is extracted based on the light stripe features.

3. The method according to claim 2, characterized in that, The preset lifting directions include a first lifting direction, a second lifting direction, a third lifting direction, and a fourth lifting direction. The first lifting direction is the u-axis direction of the pixel coordinate system, the second lifting direction is the v-axis direction of the pixel coordinate system, the third lifting direction and the u-axis direction form an angle of 45 degrees, the fourth lifting direction and the v-axis direction form an angle of 45 degrees, and the u-axis direction is perpendicular to the v-axis direction.

4. The method according to claim 2, characterized in that, Selecting at least one target lifting direction from a plurality of preset lifting directions in the pixel coordinate system, wherein the angle between the target direction and the reference normal is less than a preset angle, includes: Determine the equation of the intersection line between the light plane and the camera image plane in the camera coordinate system; Based on the intersection equation and the camera's intrinsic parameter matrix, the angle between the line structured light stripe and the u-axis direction in the pixel coordinate system is determined. Based on the angle between the preset lifting direction and the u-axis direction, and the angle between the line structure light strip and the u-axis direction, all preset lifting directions whose angle with the reference normal is less than a preset angle are determined, and all preset lifting directions whose angle with the reference normal is less than a preset angle are taken as the target lifting direction.

5. The method according to claim 2, characterized in that, The step of acquiring the light stripe features of the line structure light stripe along one of the target lifting directions includes: The light stripe features of the line structure light stripe are obtained along the target lifting direction with the smallest angle to the reference normal.

6. The method according to claim 2, characterized in that, Before selecting at least one target lifting direction from a plurality of preset lifting directions in the pixel coordinate system, wherein the angle between the target direction and the reference normal is less than a preset angle, the method further includes: Obtain the direction of the line structured light stripe; Based on the direction of the line structure light strip, a candidate lifting direction is selected from the preset lifting directions.

7. The method according to claim 6, characterized in that, The direction of acquiring the linear structured light stripe includes: In system calibration mode, based on the light plane calibration and camera calibration results, the intersection direction of the light plane and the camera image plane is determined, and the intersection direction is used as the direction of the line structure light stripe; Alternatively, in scanning mode, line structured light stripes are extracted from the light stripe image based on a thresholding method, and the direction of the line structured light stripes is determined.

8. The method according to claim 1, characterized in that, The light stripe image includes a first light stripe image and / or a second light stripe image; the extraction of the center point of the line structure light stripe based on the target lifting direction using a unidirectional lifting method includes: The center point of the first line structured light stripe in the first light stripe image is extracted based on the first target lifting direction. And / or, extract the center point of the second line structured light stripe in the second light stripe image based on the second target lifting direction, wherein the direction of the second line structured light stripe is different from the direction of the first line structured light stripe, and the second target lifting direction is different from the first target lifting direction.

9. The method according to claim 1, characterized in that, After extracting the center point of the line structured light stripe using a unidirectional lifting method based on the target lifting direction, the method further includes: The sparse center points of the line structure light stripe are supplemented.

10. The method according to claim 9, characterized in that, The process of filling in the sparse center points of the linear structured light stripe includes: The first target center point is determined based on the center point of the line structure light stripe, wherein the first target center point is the endpoint of a plurality of consecutive center points constituting the first topological line segment, or is the first independent center point without adjacent center points; The search algorithm searches for the second target center point that is closest to the first target center point according to a preset search rule. The second target center point is either the endpoint of a plurality of consecutive center points that constitute the second topological line segment, or a second independent center point without adjacent center points. If the second target center point is found, then a supplementary point is added between the first target center point and the second target center point.

11. The method according to claim 1, characterized in that, After extracting the center point of the line structured light stripe using a unidirectional lifting method based on the target lifting direction, the method further includes: The center point is optimized to smooth the center line.

12. The method according to claim 11, characterized in that, The optimization of the center point to smooth the center line includes: For each center point, the gray-level extreme point is determined based on the gray-level characteristics of the line structure light stripe along the normal direction of the center point; The center point of the linear structured light stripe is updated based on the gray-level extreme points.

13. A device for extracting the center point of a linear optical stripe, characterized in that, include: An image acquisition module is used to acquire a light bar image, wherein the light bar image includes line-structured light bars; The center point extraction module is used to extract the center point of the linear structured light stripe based on the target lifting direction using a unidirectional lifting method. The angle between the target lifting direction and the reference normal of the linear structured light stripe is less than a preset angle. The reference normal represents the approximate direction of the center point normal of the linear structured light stripe. The preset angle limits the degree to which the target lifting direction is approximately perpendicular to the linear structured light stripe. The center point normal is obtained by fitting multiple center points based on the center point of the linear structured light stripe. After extracting the center point of the line structured light stripe using a unidirectional lifting method based on the target lifting direction, the method further includes: Based on the center point and center point normal of the line structure light stripe, a multi-level hierarchy line is obtained, wherein the hierarchy line reflects the distance from the sub-pixel point to the corresponding center point in the center point normal direction; Based on the grayscale values ​​on the line structured light stripe level lines, grayscale Gaussian filtering is performed in each level of the level line. Extract the third target center point of the line structured light stripe after grayscale Gaussian filtering; The center point of the line structured light stripe after grayscale Gaussian filtering is updated to the center point of the line structured light stripe.

14. An electronic device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the line structured light stripe center point extraction method according to any one of claims 1-12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, The storage medium stores a computer program, which, when executed by a processor, causes the processor to implement the method for extracting the center point of a line structured light stripe as described in any one of claims 1-12.

Images