A method for acquiring three-dimensional point clouds and a structured light scanning device

Abstract

The present application provides a three-dimensional point cloud acquisition method and a structured light scanning device, which relate to the field of three-dimensional technology. The method includes: controlling a line light source emitter to project light in multiple directions to the surface of an object to be scanned at multiple moments, and obtaining multiple sets of stereo images acquired by two depth cameras at multiple moments, respectively performing pose estimation on the multiple sets of stereo images, and obtaining the pose information of the structured light scanning device at multiple moments, and then performing line stereo matching on a set of stereo images at each moment according to the pose information at each moment, and obtaining a three-dimensional line set at each moment, and finally performing point cloud screening on the three-dimensional line set at adjacent moments according to the three-dimensional line set at adjacent moments in the multiple moments, and obtaining a three-dimensional point cloud image of the object to be scanned. Combined with the consideration of the three-dimensional line set at adjacent moments in the multiple moments, the three-dimensional line set at adjacent moments is screened for point clouds, and the erroneous point clouds are effectively eliminated to obtain a three-dimensional point cloud image of the object to be scanned.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional technology, and in particular to a three-dimensional point cloud acquisition method and a structured light scanning device. Background Art

[0002] Scanners based on line structured light are widely used in the field of industrial measurement due to their high reconstruction accuracy. As users' demand for production efficiency increases, binocular structured light scanners that project multiple lines have become the mainstream of the industry. Compared with traditional single-line structured light scanners, the measurement efficiency has increased several times or even dozens of times. However, the increase in the number of lines also brings challenges to the stereo matching of lines. In some cases, there will be a one-to-many situation. For example, a line from the left camera intersects with different lines from the right camera on different structured light surfaces, which may lead to mismatching and generate some false surface data, affecting the reliability of the measurement results. Summary of the invention

[0003] The purpose of the present invention is to provide a three-dimensional point cloud acquisition method and a structured light scanning device to address the deficiencies in the above-mentioned prior art, so as to combine and consider the three-dimensional line sets at adjacent moments in multiple moments, perform point cloud screening on the three-dimensional line sets at adjacent moments, and effectively eliminate the false point clouds generated by erroneous line stereo matching, so as to obtain a reliable point cloud and a three-dimensional point cloud image of the object to be scanned.

[0004] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0005] In a first aspect, an embodiment of the present application provides a three-dimensional point cloud acquisition method, which is applied to a processor of a structured light scanning device, wherein the structured light scanning device further includes: two depth cameras and a line light source transmitter, and the method includes:

[0006] Controlling the line light source emitter to project light in multiple directions to the surface of the object to be scanned at multiple times, and acquiring multiple sets of stereo images captured by the two depth cameras at the multiple times;

[0007] Performing pose estimation on the multiple groups of stereo images respectively to obtain pose information of the structured light scanning device at the multiple moments;

[0008] According to the posture information at each moment, performing line stereo matching on a set of stereo images at each moment to obtain a three-dimensional line set at each moment;

[0009] According to the three-dimensional line sets at adjacent moments among the multiple moments, point cloud screening is performed on the three-dimensional line sets at adjacent moments to obtain a three-dimensional point cloud image of the object to be scanned.

[0010] In an optional implementation, the performing pose estimation on the multiple groups of stereo images respectively to obtain pose information of the structured light scanning device at the multiple moments includes:

[0011] Acquire a preset marker point set of the object to be scanned, the preset marker point set comprising: first three-dimensional coordinates of a plurality of preset marker points in a preset world coordinate system, wherein the number of non-collinear marker points in the plurality of preset marker points that are within a common field of view of the two depth cameras is greater than or equal to 3;

[0012] Based on the set of stereo images at each moment, obtaining a set of acquisition marker points in the current field of view at each moment, wherein the set of acquisition marker points includes: second three-dimensional coordinates of a plurality of acquisition marker points in a preset device coordinate system;

[0013] Determine the same-named marker point pairs at each moment according to the first three-dimensional coordinates of the plurality of preset marker points and the second three-dimensional coordinates of the plurality of collected marker points, wherein the same-named marker point pairs include: the first three-dimensional coordinates and the second three-dimensional coordinates of the plurality of same-named marker points;

[0014] According to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple markers with the same name, the rotation matrix and the translation vector of the structured light scanning device at each moment in the preset world coordinate system are obtained, and the posture information at each moment includes: the rotation matrix and the translation vector.

[0015] In an optional embodiment, obtaining the rotation matrix and translation vector of the structured light scanning device at each moment in the preset world coordinate system according to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple marker points with the same name includes:

[0016] Obtaining, according to the first three-dimensional coordinates of the plurality of marker points with the same name, the first central three-dimensional coordinates of the marker points with the same name in the preset marker point set;

[0017] According to the second three-dimensional coordinates of the plurality of marker points with the same name, obtaining the second central three-dimensional coordinates of the marker points with the same name in the collection marker point set;

[0018] Calculating the rotation matrix according to the first three-dimensional coordinates of the plurality of marker points with the same name, the first three-dimensional coordinates of the center, the second three-dimensional coordinates of the plurality of marker points with the same name, and the second three-dimensional coordinates of the center;

[0019] The translation vector is calculated according to the rotation matrix, the first central three-dimensional coordinates and the second central three-dimensional coordinates.

[0020] In an optional implementation, performing line stereo matching on a set of stereo images at each moment according to the posture information at each moment to obtain a three-dimensional line set at each moment includes:

[0021] Determining structured light surfaces at the multiple moments according to the light in the multiple directions;

[0022] According to the structured light surface at each of the multiple moments, line matching is performed on a set of stereo images at each moment to obtain multiple pairs of matching lines at each moment;

[0023] According to the position and posture information at each moment, coordinate transformation is performed on the multiple pairs of matching lines at each moment to obtain a three-dimensional line set at each moment.

[0024] In an optional implementation, performing line matching on a set of stereo images at each moment according to the structured light surface at each moment in a plurality of moments to obtain a plurality of pairs of matching lines at each moment includes:

[0025] Performing line matching on a set of stereo images at each moment to obtain multiple groups of candidate matching points for each line pair in the multiple line pairs at each moment, wherein each line pair includes: two lines of different images in a set of stereo images, and the multiple groups of candidate matching points include: two matching points of different lines in the two lines;

[0026] According to the multiple groups of candidate matching points, a plurality of three-dimensional points corresponding to each line are obtained, and the distance between the multiple three-dimensional points and the structured light surface at each moment is acquired;

[0027] Obtaining a distance standard deviation corresponding to each line pair according to the distances between the multiple three-dimensional points and the structured light surface at each moment;

[0028] If the distance standard deviations corresponding to the plurality of target line pairs among the plurality of line pairs are smaller than a preset threshold, the plurality of target line pairs are determined to be the plurality of matching line pairs at each moment.

[0029] In an optional embodiment, the performing point cloud screening on the three-dimensional line sets at adjacent moments among the multiple moments to obtain the three-dimensional point cloud image of the object to be scanned includes:

[0030] Performing connected domain division on the three-dimensional line set at adjacent moments to obtain at least one connected domain at the adjacent moments, each connected domain comprising: at least two connected three-dimensional lines in the three-dimensional line set at the adjacent moments;

[0031] If the number of connected domains is at least two, sorting the at least two connected domains according to the number of three-dimensional lines and the number of three-dimensional intersections of each of the at least two connected domains;

[0032] Conflict detection is performed on the at least two sorted connected domains one by one, and a three-dimensional point cloud image of the object to be scanned is generated according to the detection result.

[0033] In an optional implementation, the sorting of the at least two connected domains according to the number of three-dimensional lines and the number of three-dimensional intersections of each of the at least two connected domains includes:

[0034] Determine a first quantity value and a second quantity value of the at least two connected domains according to the quantity of three-dimensional lines and the quantity of three-dimensional intersections of the at least two connected domains;

[0035] Arranging the at least two connected domains in descending order according to the first quantity value;

[0036] If the first quantity values ​​of the at least two connected domains are the same, the at least two connected domains are arranged in descending order according to the second quantity value.

[0037] In an optional implementation, the performing conflict detection on the at least two sorted connected domains one by one, and generating a three-dimensional point cloud image of the object to be scanned according to the detection result, includes:

[0038] Perform conflict detection between a sorted connected domain and the next connected domain;

[0039] If there is a line conflict between the one connected domain and the next connected domain, the next connected domain is deleted, and the conflict detection between the one connected domain and the subsequent connected domains is continued until the at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned;

[0040] If there is no line conflict between the one connected domain and the next connected domain, the one connected domain and the next connected domain are merged to obtain a new connected domain, and the conflict detection of the new connected domain and the subsequent connected domains is continued until the at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0041] In an optional embodiment, the step of performing point cloud screening on the three-dimensional line sets at adjacent moments among the multiple moments to obtain the three-dimensional point cloud image of the object to be scanned further includes:

[0042] If the number of connected domains is one, the three-dimensional point cloud image is generated according to all three-dimensional lines in one connected domain.

[0043] In a second aspect, an embodiment of the present application further provides a three-dimensional point cloud acquisition device, which is applied to a processor of a structured light scanning device, wherein the structured light scanning device further includes: two depth cameras and a line light source transmitter, and the device includes:

[0044] An acquisition module, used to control the line light source emitter to project light in multiple directions to the surface of the object to be scanned at multiple times, and to acquire multiple sets of stereo images captured by the two depth cameras at the multiple times;

[0045] An estimation module, used to perform pose estimation on the multiple groups of stereo images respectively, to obtain pose information of the structured light scanning device at the multiple moments;

[0046] A matching module, configured to perform line stereo matching on a set of stereo images at each moment according to the posture information at each moment, so as to obtain a three-dimensional line set at each moment;

[0047] The screening module is used to perform point cloud screening on the three-dimensional line sets at adjacent moments according to the three-dimensional line sets at adjacent moments among the multiple moments, so as to obtain a three-dimensional point cloud image of the object to be scanned.

[0048] In a third aspect, an embodiment of the present application further provides a structured light scanning device, comprising: two depth cameras, a line light source emitter and a processor, wherein the two depth cameras and the line light source emitter are rigidly connected, and the processor is respectively connected to the two depth cameras and the line light source emitter, and the processor executes the steps of the three-dimensional point cloud acquisition method as described in any one of the first aspects.

[0049] The beneficial effects of this application are:

[0050] The embodiment of the present application provides a three-dimensional point cloud acquisition method and a structured light scanning device, the method comprising: controlling a line light source emitter to project light in multiple directions to the surface of an object to be scanned at multiple moments, and acquiring multiple sets of stereo images acquired by two depth cameras at multiple moments, respectively performing pose estimation on the multiple sets of stereo images, obtaining pose information of the structured light scanning device at multiple moments, and then performing line stereo matching on a set of stereo images at each moment according to the pose information at each moment, obtaining a three-dimensional line set at each moment, and finally performing point cloud screening on the three-dimensional line set at adjacent moments according to the three-dimensional line set at adjacent moments in the multiple moments, to obtain a three-dimensional point cloud image of the object to be scanned. The method of the present application combines the three-dimensional line set at adjacent moments in multiple moments, performs point cloud screening on the three-dimensional line set at adjacent moments, effectively eliminates the false point cloud generated by the erroneous line stereo matching, thereby obtaining a reliable point cloud, and obtaining a three-dimensional point cloud image of the object to be scanned, reducing false surfaces, improving the accuracy of line stereo matching, and also improving the reliability of obtaining the three-dimensional point cloud image of the object to be scanned. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 One of the flow diagrams of a three-dimensional point cloud acquisition method provided in an embodiment of the present application;

[0053] Figure 2 A second flow chart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application;

[0054] Figure 3 A third flow chart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application;

[0055] Figure 4 A fourth flowchart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application;

[0056] Figure 5 A fifth flow chart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application;

[0057] Figure 6 A schematic diagram of determining a three-dimensional line provided in an embodiment of the present application;

[0058] Figure 7A sixth flow chart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application;

[0059] Figure 8 A flowchart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application is shown in FIG7;

[0060] Fig. 9 FIG8 is a flow chart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application;

[0061] Fig.10 A schematic diagram of point cloud screening provided in an embodiment of the present application;

[0062] Fig.11 A schematic diagram of the functional modules of a three-dimensional point cloud acquisition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0064] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0065] In the description of the present application, it should be noted that if the terms "upper", "lower", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or is the orientation or position relationship in which the product of the application is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0066] In addition, the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0067] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0068] An embodiment of the present application provides a structured light scanning device, which includes: two depth cameras, a line light source emitter and a processor. The structured light scanning device can also be called a binocular line structured light scanning device, wherein the two depth cameras and the line light source emitter are rigidly connected, and the processor can control the line light source emitter to project multiple light rays in different directions to the surface of the object to be scanned, and can project multiple light rays in two or three directions to the surface of the object to be scanned. The line light source emitter can be a laser emitter; the two depth cameras are used to simultaneously capture stereo images of the surface of the object to be scanned in multiple directions.

[0069] Among them, the two depth cameras are depth cameras set in different directions, for example, the two depth cameras are a depth camera set on the left side of the object to be scanned, and a depth camera set on the right side of the object to be scanned, or the two depth cameras are a depth camera set above the object to be scanned, and a depth camera set below the object to be scanned. There is no restriction on the setting positions of the two depth cameras.

[0070] Since there are multiple lines in the stereo images captured by the two depth cameras, when stereo matching is performed on the multiple lines, a line captured by one depth camera may intersect with different lines captured by another depth camera on different structured light surfaces, resulting in erroneous matching and generating some false surface data. In order to avoid generating false surface data and improve the accuracy of line stereo matching, an embodiment of the present application further provides a three-dimensional point cloud acquisition method, which is applied to a processor of a structured light scanning device. The method controls a line light source emitter to project light in multiple directions to the surface of an object to be scanned at multiple times, and obtains multiple groups of stereo images captured by two depth cameras at multiple times. According to the multiple groups of stereo images, pose estimation is performed to obtain pose information of the structured light scanning device at multiple times. According to the pose information at each time, line stereo matching is performed on a group of stereo images at each time to obtain a three-dimensional line set at each time. Finally, according to the three-dimensional line sets at adjacent times in the multiple times, point cloud screening is performed on the three-dimensional line sets at adjacent times to obtain a three-dimensional point cloud image of the object to be scanned, and false point clouds generated by erroneous matching are effectively eliminated, thereby obtaining a reliable point cloud and a three-dimensional point cloud image of the object to be scanned.

[0071] The following is a detailed explanation of a 3D point cloud acquisition method provided by an embodiment of the present application through specific examples in conjunction with the accompanying drawings. A 3D point cloud acquisition method provided by an embodiment of the present application can be implemented by running an algorithm or software by a processor pre-installed with: a detection algorithm of a preset 3D point cloud acquisition method. Figure 1 One of the flow diagrams of a three-dimensional point cloud acquisition method provided in an embodiment of the present application; Figure 1 As shown, the method includes:

[0072] S101, controlling a line light source emitter to project light in multiple directions to a surface of an object to be scanned at multiple times, and acquiring multiple sets of stereo images captured by two depth cameras at multiple times.

[0073] In this embodiment, the processor controls the line light source emitter to project light in one direction to the surface of the object to be scanned at one moment, and the two depth cameras synchronously capture the stereo image containing the lines at this moment to obtain a set of stereo images at this moment. Then the processor controls the line light source emitter to project light in another direction to the surface of the object to be scanned at the next moment, and the two depth cameras synchronously capture the stereo image containing the lines at the next moment to obtain a set of stereo images at the next moment, thereby obtaining multiple sets of stereo images captured by the two depth cameras at multiple moments.

[0074] If the line light source emitter only projects light in two directions, the two depth cameras also only collect stereo images in two directions, and the two groups of stereo images in the two directions collected at two adjacent moments are recorded as a cross-line image group; if the line light source emitter projects light in three directions, the two depth cameras also collect stereo images in three directions, and the three groups of stereo images in the three directions collected at three adjacent moments are recorded as a cross-line image group.

[0075] If the two depth cameras are a depth camera disposed on the left side of the object to be scanned and a depth camera disposed on the right side of the object to be scanned, a set of stereo images includes: a left stereo image and a right stereo image.

[0076] It should be noted that, in order to facilitate subsequent stereo matching, multiple sets of stereo images collected by two depth cameras at multiple times are subjected to epipolar resampling so that the points with the same name in each set of stereo images are located in the same row of the stereo images.

[0077] S102 , respectively estimate the poses of the multiple sets of stereo images to obtain the pose information of the structured light scanning device at multiple moments.

[0078] Specifically, a pose estimation method is used to estimate the poses of multiple sets of stereo images to obtain the pose information of the structured light scanning device at multiple times. The pose estimation method can adopt a landmark point estimation method, a tracker estimation method, a robotic arm estimation method, etc.

[0079] Among them, the landmark point estimation method determines the pose information of the structured light scanning device at multiple times by converting the coordinates of the landmark points collected from multiple sets of stereo images with the preset landmark points; the tracker estimation method uses deep learning technology to track and estimate the pose of objects in the real world in real time; the robotic arm estimation method is achieved by calculating the position and direction of the object to be scanned, including the calculation and use of the rotation matrix, so as to obtain the pose information of the structured light scanning device at multiple times.

[0080] S103 . Perform line stereo matching on a set of stereo images at each moment according to the position and posture information at each moment, to obtain a three-dimensional line set at each moment.

[0081] Specifically, line stereo matching is performed on two stereo images in a group of stereo images at each moment to determine a three-dimensional line obtained by the front intersection of every two lines in the two stereo images. Therefore, for the front intersection of multiple lines in the two stereo images, multiple three-dimensional lines can be obtained, and the multiple three-dimensional lines constitute a three-dimensional line set of a group of stereo images at each moment.

[0082] S104 , performing point cloud screening on the three-dimensional line sets at adjacent moments according to the three-dimensional line sets at adjacent moments among the multiple moments, to obtain a three-dimensional point cloud image of the object to be scanned.

[0083] Specifically, if the line light source emitter only projects light in two directions, then two groups of stereo images collected at two adjacent moments in multiple moments are recorded as a cross-line image group; if the line light source emitter projects light in three directions, then three groups of stereo images collected at three adjacent moments in multiple moments are recorded as a cross-line image group. Therefore, based on the three-dimensional line sets at multiple moments, three-dimensional line sets of multiple cross-line image groups can be divided.

[0084] The three-dimensional line set of each cross-line image group is screened by point cloud, multiple target three-dimensional lines of each cross-line image group are determined, and reliable point clouds of the multiple target three-dimensional lines are obtained, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0085] In summary, the embodiment of the present application provides a three-dimensional point cloud acquisition method, including: controlling a line light source emitter to project light in multiple directions to the surface of an object to be scanned at multiple moments, and obtaining multiple sets of stereo images acquired by two depth cameras at multiple moments, respectively performing pose estimation on the multiple sets of stereo images, obtaining pose information of the structured light scanning device at multiple moments, and then performing line stereo matching on a set of stereo images at each moment according to the pose information at each moment, obtaining a three-dimensional line set at each moment, and finally performing point cloud screening on the three-dimensional line set at adjacent moments according to the three-dimensional line set at adjacent moments in the multiple moments, to obtain a three-dimensional point cloud image of the object to be scanned. The method of the present application combines the three-dimensional line set at adjacent moments in multiple moments, performs point cloud screening on the three-dimensional line set at adjacent moments, effectively eliminates the false point cloud generated by the erroneous line stereo matching, thereby obtaining a reliable point cloud, and obtaining a three-dimensional point cloud image of the object to be scanned, reducing false surfaces, improving the accuracy of line stereo matching, and also improving the reliability of obtaining the three-dimensional point cloud image of the object to be scanned.

[0086] Based on the three-dimensional point cloud acquisition method provided in the above embodiment, the embodiment of the present application further provides another possible implementation of the three-dimensional point cloud acquisition method. Figure 2 The second flow chart of a three-dimensional point cloud acquisition method provided in the embodiment of the present application. Figure 2 As shown, pose estimation is performed on multiple sets of stereo images to obtain pose information of the structured light scanning device at multiple moments, including:

[0087] S201: Obtain a set of preset landmark points of an object to be scanned.

[0088] The preset marker point set includes: first three-dimensional coordinates of multiple preset marker points in a preset world coordinate system, and the number of non-collinear marker points in the multiple preset marker points that are within the common field of view of the two depth cameras is greater than or equal to 3.

[0089] In this embodiment, a landmark estimation method is used to perform pose estimation on multiple groups of stereo images. Specifically, multiple preset landmarks are pre-arranged on the surface or around the object to be scanned, so that the common area of ​​the field of view of the two depth cameras simultaneously covers 3 or more non-collinear preset landmarks. The first three-dimensional coordinates of multiple preset landmarks in a preset world coordinate system are pre-acquired by methods such as photogrammetry to obtain a set of preset landmarks, which can be expressed as M={M1,M2,…,M n}, where n represents the number of preset landmark points.

[0090] S202: Based on a set of stereo images at each moment, a set of acquisition marker points in the current field of view at each moment is obtained.

[0091] The collection mark point set includes: second three-dimensional coordinates of multiple collection mark points in a preset device coordinate system.

[0092] When two depth cameras collect a set of stereo images at each moment, they simultaneously obtain the information of the collection mark points from a set of stereo images, and calculate the second three-dimensional coordinates of multiple collection mark points in the preset device coordinate system under the current field of view to obtain a collection mark point set, which can be expressed as M t ={M1 t ,M2 t ,…,M k t}, where t represents the time when the stereo image is collected, and k represents the number of collected landmarks in the current field of view.

[0093] S203: Determine a pair of marker points with the same name at each moment according to the first three-dimensional coordinates of the plurality of preset marker points and the second three-dimensional coordinates of the plurality of collected marker points.

[0094] The pairs of marker points with the same name include: first three-dimensional coordinates and second three-dimensional coordinates of multiple marker points with the same name.

[0095] Based on the length invariance feature of the rigid body, if the preset marker point M1 and the collected marker point M1 t For the same landmark, preset landmark M2 and collected landmark M2 t If they are the same landmarks, then the landmarks with the same name can be determined. , the same-named landmark pairs can be determined at each moment, and the same-named landmark pairs may include: (M 1t , M1 t )、(M 2t , M2 t )、…、(M kt , M k t ).

[0096] S204, obtaining a rotation matrix and a translation vector of the structured light scanning device at each moment in a preset world coordinate system according to the first three-dimensional coordinates and the second three-dimensional coordinates of the plurality of marker points with the same name.

[0097] The posture information at each moment includes: rotation matrix and translation vector.

[0098] Specifically, according to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple markers with the same name at each moment, the rotation matrix R of the structured light scanning device at each moment in the preset world coordinate system is obtained. t and the translation vector T t , thereby obtaining the position information of the structured light scanning device at multiple times, which is used for coordinate conversion, that is, converting the three-dimensional coordinates in the preset device coordinate system into the three-dimensional coordinates in the preset world coordinate system.

[0099] In the method provided in the embodiment of the present application, a preset marker point set of the object to be scanned is obtained, and the preset marker point set includes: the first three-dimensional coordinates of multiple preset marker points in a preset world coordinate system, and the number of non-collinear marker points in the multiple preset marker points that are in the common field of view of two depth cameras is greater than or equal to 3; based on a set of stereo images at each moment, a collection marker point set in the current field of view at each moment is obtained, and the collection marker point set includes: the second three-dimensional coordinates of multiple collection marker points in a preset device coordinate system; according to the first three-dimensional coordinates of the multiple preset marker points and the second three-dimensional coordinates of the multiple collection marker points, the same-named marker point pairs at each moment are determined, and the same-named marker point pairs include: multiple same-named markers The first three-dimensional coordinates and the second three-dimensional coordinates of the point; according to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple marker points with the same name, the rotation matrix and the translation vector of the structured light scanning device in the preset world coordinate system at each moment are obtained, and the posture information at each moment includes: the rotation matrix and the translation vector, by determining the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple marker points with the same name, and then determining the rotation matrix and the translation vector of the structured light scanning device in the preset world coordinate system at each moment, the three-dimensional coordinates in the preset device coordinate system at each moment can be converted into the three-dimensional coordinates in the preset world coordinate system through the rotation matrix and the translation vector of the structured light scanning device in the preset world coordinate system at each moment, thereby realizing the coordinate conversion.

[0100] The present application embodiment also provides another possible implementation of the three-dimensional point cloud acquisition method. Figure 3 The third flow chart of a three-dimensional point cloud acquisition method provided in the embodiment of the present application. Figure 3 As shown, according to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple markers with the same name, the rotation matrix and the translation vector of the structured light scanning device at each moment in the preset world coordinate system are obtained, including:

[0101] S301. Obtain, according to the first three-dimensional coordinates of a plurality of marker points with the same name, the first central three-dimensional coordinates of the marker points with the same name in a preset marker point set.

[0102] S302. Obtain the second central three-dimensional coordinates of the marker points with the same name in the collection marker point set according to the second three-dimensional coordinates of the multiple marker points with the same name.

[0103] In this embodiment, the first three-dimensional coordinates of multiple marker points with the same name are summed and then averaged to determine the first central three-dimensional coordinates C of the marker points with the same name in the preset marker point set M.

[0104] Similarly, the second three-dimensional coordinates of multiple landmarks with the same name are summed and averaged to determine the collection landmark point set M t The second center three-dimensional coordinates C of the same-name landmark point t .

[0105] S303, calculating a rotation matrix according to the first three-dimensional coordinates of the plurality of marker points with the same name, the first three-dimensional coordinates of the center, the second three-dimensional coordinates of the plurality of marker points with the same name, and the second three-dimensional coordinates of the center.

[0106] S304, calculating a translation vector according to the rotation matrix, the first central three-dimensional coordinates and the second central three-dimensional coordinates.

[0107] Among them, the same-name landmark pairs may include: (M 1t ,M1 t )、(M 2t ,M2 t )、…、(M kt ,M k t ), according to the first three-dimensional coordinates of the multiple markers with the same name, the first central three-dimensional coordinates C, the second three-dimensional coordinates of the multiple markers with the same name, and the second central three-dimensional coordinates C t , construct the following expression:

[0108] M 1t -C=R t (M1 t -C t )

[0109] M 2t -C=R t (M2 t -C t )

[0110] …

[0111] M kt -C=R t (M k t -C t )

[0112] T t =CR t C t

[0113] When the number of landmarks with the same name is greater than or equal to 3 and they are not completely collinear, the covariance matrix is ​​calculated, and the expression is as follows:

[0114]

[0115] Then perform singular value decomposition (SVD) on the covariance matrix to obtain: .

[0116] Finally determine the rotation matrix , the translation vector .

[0117] In the method provided in an embodiment of the present application, based on the first three-dimensional coordinates of multiple marker points with the same name, the first central three-dimensional coordinates of the marker points with the same name in a preset marker point set are obtained, and based on the second three-dimensional coordinates of the multiple marker points with the same name, the second central three-dimensional coordinates of the marker points with the same name in the collected marker point set are obtained; based on the first three-dimensional coordinates of the multiple marker points with the same name, the first central three-dimensional coordinates, the second three-dimensional coordinates of the multiple marker points with the same name, and the second central three-dimensional coordinates, a rotation matrix is ​​calculated; based on the rotation matrix, the first central three-dimensional coordinates and the second central three-dimensional coordinates, a translation vector is calculated, thereby obtaining the position and posture information of the structured light scanning device at multiple times.

[0118] The present application embodiment also provides another possible implementation of the three-dimensional point cloud acquisition method. Figure 4 The fourth flow chart of a three-dimensional point cloud acquisition method provided in the embodiment of the present application. Figure 4 As shown, according to the posture information at each moment, a set of stereo images at each moment is matched by lines to obtain a set of three-dimensional lines at each moment, including:

[0119] S401, determining structured light surfaces at multiple moments according to light rays in multiple directions.

[0120] In this embodiment, the line light source emitter projects light in multiple directions to the surface of the object to be scanned at multiple times, forming projections at multiple times, thereby obtaining structured light surfaces at multiple times.

[0121] S402 , performing line matching on a group of stereo images at each moment according to the structured light surface at each moment among the multiple moments, to obtain multiple pairs of matching lines at each moment.

[0122] S403. According to the position information at each moment, coordinate transformation is performed on multiple pairs of matching lines at each moment to obtain a three-dimensional line set at each moment.

[0123] Specifically, line matching is performed on a plurality of lines in a set of stereo images at each moment to determine a plurality of pairs of matching lines of the structured light surface at that moment.

[0124] Then, the same-name matching points on multiple pairs of matching lines intersect in front to obtain multiple three-dimensional lines. According to the posture information at each moment, the coordinates of the multiple three-dimensional lines at each moment are transformed to obtain a set of three-dimensional lines at each moment, which is expressed as: PC={LINE1,LINE2,…}, where LINE1 represents the first three-dimensional line.

[0125] In the method provided in the embodiment of the present application, structured light surfaces at multiple moments are determined based on light in multiple directions, and line matching is performed on a group of stereo images at each moment based on the structured light surfaces at each of the multiple moments to obtain multiple pairs of matching lines at each moment. Based on the posture information at each moment, coordinate transformation is performed on the multiple pairs of matching lines at each moment to obtain a three-dimensional line set at each moment, and a three-dimensional line set for each group of stereo images is also obtained.

[0126] Based on the three-dimensional point cloud acquisition method provided in the above embodiment, the embodiment of the present application further provides another possible implementation of the three-dimensional point cloud acquisition method. Figure 5 The fifth flowchart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application. Figure 6 A schematic diagram of determining a three-dimensional line provided in an embodiment of the present application. Figure 5 As shown, according to the structured light surface at each of the multiple moments, line matching is performed on a set of stereo images at each moment to obtain multiple pairs of matching lines at each moment, including:

[0127] S501 , performing line matching on a set of stereo images at each moment, and obtaining multiple sets of candidate matching points for each line pair among multiple line pairs at each moment.

[0128] Each line pair includes: two lines of different images in a set of stereo images, and the multiple groups of candidate matching points include: two matching points of different lines in the two lines.

[0129] In this embodiment, both images in a set of stereoscopic images have multiple lines. If the two images in a set of stereoscopic images are a left stereoscopic image and a right stereoscopic image, and the row coordinates of a point pl(xl,yl) on a line line_l of the left stereoscopic image are the same as the row coordinates of a point pr(xr,yr) on a line line_r of the right stereoscopic image, that is, yr=yl, then the line line_l and the line line_r are considered to be a line pair, and pl and pr are a set of candidate matching points for the line pair. Therefore, the multiple points on the multiple lines of the left stereoscopic image are matched with the multiple points on the multiple lines of the right stereoscopic image to obtain multiple sets of candidate matching points for each line pair in the multiple line pairs for a set of stereoscopic images at that moment, thereby obtaining multiple sets of candidate matching points for each line pair in the multiple line pairs for multiple sets of stereoscopic images at multiple moments.

[0130] S502 . Obtain multiple three-dimensional points corresponding to each line according to multiple groups of candidate matching points, and obtain distances between the multiple three-dimensional points and the structured light surface at each moment.

[0131] S503 , obtaining a distance standard deviation corresponding to each line pair according to the distances between the multiple three-dimensional points and the structured light surface at each moment.

[0132] S504: If the distance standard deviations corresponding to the multiple target line pairs in the multiple line pairs are less than a preset threshold, the multiple target line pairs are determined to be multiple pairs of matching lines at each moment.

[0133] Specifically, a set of candidate matching points pl and pr of a line pair line_l and line_r can be interacted in front to calculate a three-dimensional point P, and then the distance from the three-dimensional point P to the structured light surface at each moment is calculated. Then, multiple sets of candidate matching points of a line pair line_l and line_r can be interacted in front to calculate multiple three-dimensional points, and the distances from the multiple three-dimensional points to the structured light surface at each moment are calculated to obtain the distance standard deviation corresponding to the line pair line_l and line_r.

[0134] If the standard deviation of the distances from multiple three-dimensional points calculated from the line pair line_l and line_r to the structured light surface S is less than a preset threshold, the line pair line_l and line_r are determined to be a target line pair, and the line pair line_l and line_r are a pair of matching lines at the corresponding moment of the structured light surface S, and a set of candidate matching points pl and pr are a pair of matching points with the same name at the corresponding moment of the structured light surface S.

[0135] Then the above steps are adopted for each line pair to determine multiple target line pairs and multiple pairs of matching lines at each moment.

[0136] For example, Figure 6As shown, two images in a set of stereo images are a left stereo image and a right stereo image. The left stereo image includes three lines, namely line_l1, line_l2 and line_l3, and the right stereo image includes three lines, namely line_r1, line_r2 and line_r3, wherein line_l1 and line_r1 intersect at structured light surface 1, are a pair of matching lines of structured light surface 1, and the corresponding three-dimensional line is LINE1; line_l2 and line_r2 intersect at structured light surface 2, are a pair of matching lines of structured light surface 2, and the corresponding three-dimensional line is LINE1. The line is LINE2; line_l3 and line_r3 intersect at structured light surface 3, which is a pair of matching lines of structured light surface 3, and the corresponding 3D line is LINE3; in addition, line_l1 and line_r3 also intersect at structured light surface 2, which is a pair of matching lines of structured light surface 2, and the corresponding 3D line is LINE5, where LINE5 is an incorrect matching line; line_l3 and line_r1 also intersect at structured light surface 2, which is a pair of matching lines of structured light surface 2, and the corresponding 3D line is LINE4; where LINE4 is also an incorrect matching line. A set of 3D lines of a stereo image is obtained, PC={LINE1,LINE2,LINE3,LINE4,LINE5}.

[0137] In the method provided by the embodiment of the present application, line matching is performed on a group of stereo images at each moment to obtain multiple groups of candidate matching points for each line pair in multiple line pairs at each moment, each line pair includes: two lines of different images in a group of stereo images, and the multiple groups of candidate matching points respectively include: two matching points of different lines in the two lines; based on the multiple groups of candidate matching points, multiple three-dimensional points corresponding to each line are obtained, and the distances between the multiple three-dimensional points and the structured light surface at each moment are obtained; based on the distances between the multiple three-dimensional points and the structured light surface at each moment, the standard deviation of the distance corresponding to each line pair is obtained; if the standard deviation of the distances corresponding to multiple target line pairs in the multiple line pairs is less than a preset given threshold, the multiple target line pairs are determined to be multiple pairs of matching lines at each moment, thereby obtaining a three-dimensional line set at each moment, in which there are three-dimensional lines with correct matching, and there may also be three-dimensional lines with incorrect matching.

[0138] The present application embodiment also provides another possible implementation of the three-dimensional point cloud acquisition method. Figure 7 The sixth flow chart of a three-dimensional point cloud acquisition method provided in the embodiment of the present application. Figure 7 As shown, according to the three-dimensional line sets at adjacent moments in multiple moments, point cloud screening is performed on the three-dimensional line sets at adjacent moments to obtain a three-dimensional point cloud image of the object to be scanned, including:

[0139] S601: Perform connected domain division on a three-dimensional line set at adjacent moments to obtain at least one connected domain at adjacent moments.

[0140] Each connected domain includes: at least two connected three-dimensional lines in a set of three-dimensional lines at adjacent moments.

[0141] S602: If the number of connected domains is at least two, sort the at least two connected domains according to the number of three-dimensional lines and the number of three-dimensional intersections of each of the at least two connected domains.

[0142] S603: Perform conflict detection on the at least two sorted connected domains one by one, and generate a three-dimensional point cloud image of the object to be scanned according to the detection result.

[0143] In this embodiment, according to the above steps S401-S504, the three-dimensional line set at each moment is determined. If the line light source emitter only projects light in two directions, two groups of stereoscopic images collected at two adjacent moments in multiple moments are recorded as a cross line image group, and a three-dimensional line set of a cross line image group is obtained. If the line light source emitter projects light in three directions, three groups of stereoscopic images collected at three adjacent moments in multiple moments are recorded as a cross line image group, and a three-dimensional line set of a cross line image group is obtained.

[0144] The connected domain is divided for the three-dimensional line set of each cross-line image group to obtain at least one connected domain of each cross-line image group. For example, if the three-dimensional line set of a cross-line image group includes PC t ={LINE1 t ,LINE2 t ,…} and PC t+1 ={LINE1 t+1 ,LINE2 t+1 ,…}, there exists a point P u t ∈LINE i t and P v t+1 ∈LINE j t+1 So that point P u t and point P v t+1 If the distance is the smallest and is less than a given threshold, the three-dimensional line is called LINE i t and 3D line LINE j t+1 Connect, point P u t and point P v t+1 LINEi t and 3D line LINE j t+1 If the intersection of LINE i t and 3D line LINE j t+1 Connectivity, 3D Line LINE i t Also with 3D Line LINE j+1 t+1 connected, then the three-dimensional line LINE j t+1 and 3D line LINE j+1 t+1 Connected, all connected three-dimensional lines constitute a connected domain D = {LINE i t ,LINE j t+1 , LINE j+1 t+1 ,…}. Then the connected domains are divided for the three-dimensional line sets at adjacent moments to obtain at least one connected domain.

[0145] If the number of connected domains is at least two, the at least two connected domains are sorted according to the number of three-dimensional lines and the number of three-dimensional intersections in each of the at least two connected domains, conflict detection is performed one by one on the sorted at least two connected domains, and a three-dimensional point cloud image of the object to be scanned is generated based on the detection results.

[0146] Optionally, if the number of connected domains is one, a three-dimensional point cloud image is generated based on all three-dimensional lines in one connected domain. It is determined that the point clouds of all three-dimensional lines are reliable point clouds, and a three-dimensional point cloud image of the object to be scanned is obtained.

[0147] In the method provided in the embodiment of the present application, a connected domain is divided for a set of three-dimensional lines at adjacent moments to obtain at least one connected domain at adjacent moments, and each connected domain includes: at least two three-dimensional lines connected in the set of three-dimensional lines at adjacent moments; if the number of connected domains is at least two, the at least two connected domains are sorted according to the number of three-dimensional lines and the number of three-dimensional intersections in each of the at least two connected domains; conflict detection is performed on the at least two sorted connected domains one by one, and a three-dimensional point cloud image of the object to be scanned is generated based on the detection result. By performing conflict detection on the three-dimensional lines of at least two connected domains, false point clouds and mismatched three-dimensional lines can be determined, and by screening them, a reliable point cloud can be obtained, and a three-dimensional point cloud image of the object to be scanned can also be obtained.

[0148] The present application embodiment also provides another possible implementation of the three-dimensional point cloud acquisition method. Figure 8The seventh flowchart of a three-dimensional point cloud acquisition method provided in the embodiment of the present application. Figure 8 As shown, at least two connected domains are sorted according to the number of three-dimensional lines and the number of three-dimensional intersections of each of the at least two connected domains, including:

[0149] S701 : Determine first quantity values ​​and second quantity values ​​of at least two connected domains according to the quantity of three-dimensional lines and the quantity of three-dimensional intersections of at least two connected domains.

[0150] S702: Arrange at least two connected domains in descending order according to the first quantity value.

[0151] S703: If the first quantity values ​​of at least two connected domains are the same, arrange the at least two connected domains in descending order according to the second quantity values.

[0152] In this embodiment, the first quantity value of the at least two connected domains is determined according to the quantity of the three-dimensional lines of the at least two connected domains, and the second quantity value of the at least two connected domains is determined according to the quantity of the three-dimensional intersections of the at least two connected domains.

[0153] At least two connected domains are arranged in descending order according to the size of the first quantity value to obtain a connected domain sequence; if the first quantity values ​​of at least two connected domains are the same, they are arranged in descending order according to the size of the second quantity value to obtain a connected domain sequence.

[0154] The first quantity value of the first connected domain in the connected domain sequence is the largest, or, if the first quantity value of the first connected domain is the same as the first quantity value of the second connected domain, the second quantity value of the first connected domain is greater than the second quantity value of the second connected domain. The point cloud in the first connected domain is a relatively reliable point cloud.

[0155] The present application embodiment also provides another possible implementation of the three-dimensional point cloud acquisition method. Fig. 9 FIG8 is a flow chart of a three-dimensional point cloud acquisition method provided in an embodiment of the present application. Fig.10 A schematic diagram of point cloud screening provided in an embodiment of the present application. Fig. 9 As shown, performing conflict detection on at least two sorted connected domains one by one, and generating a three-dimensional point cloud image of the object to be scanned according to the detection result, including:

[0156] S801: Perform conflict detection on a sorted connected domain and the next connected domain.

[0157] S802: If there is a line conflict between a connected domain and the next connected domain, the next connected domain is deleted, and the conflict detection between the first connected domain and the subsequent connected domains is continued until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0158] S803. If there is no line conflict between a connected domain and the next connected domain, merge the connected domain and the next connected domain to obtain a new connected domain, and continue to perform conflict detection on the new connected domain and subsequent connected domains until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0159] In this embodiment, a connected domain after sorting is used as a reference to perform conflict detection with the next connected domain. If a 3D line in a connected domain is composed of line line_l1 t and line line_r1 t The next connected domain has a 3D line formed by line_l3. t and line line_r1 t Intersection, line line_r1 t If they intersect with two lines respectively, it is determined that there is a line conflict between a connected domain and the next connected domain, the next connected domain is deleted, and the conflict detection between the first connected domain and the subsequent connected domains is continued until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0160] If there is no line conflict between one connected domain and the next connected domain, the one connected domain and the next connected domain are merged to obtain a new connected domain, and the conflict detection between the new connected domain and the subsequent connected domains is continued until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0161] For example, Fig.10 As shown, the 3D line set corresponding to a cross-line image group includes: PC t ={LINE1 t ,LINE2 t , LINE3 t , LINE4 t , LINE5 t} and PC t+1 ={LINE1 t+1 ,LINE2 t+1 , LINE3 t+1 ,LINE4 t+1 , LINE5 t +1}, divide the connected domain of the three-dimensional line set, and get D1={LINE1 t ,LINE2t ,LINE3 t ,LINE1 t+1 ,LINE2 t+1 ,LINE3 t+1},D2={LINE4 t , LINE4 t+1},D3={LINE5 t ,LINE5 t+1}Three connected domains, among which D1 has a first quantity value of 6 and a second quantity value of 9; D2 has a first quantity value of 1 and a second quantity value of 1; D3 has a first quantity value of 1 and a second quantity value of 1. The three connected domains are arranged, and the sorting results are D1, D2, and D3. Among them, the three-dimensional line LINE4 in D2 t By line line_l3 t and line line_r1 t The intersection results in the three-dimensional line LINE1 in D1. t By line_l1 t and line line_r1 t If the intersection is obtained, it is determined that there is a conflict between D1 and D2, and D2 is deleted. Similarly, the three-dimensional line LINE5 in D3 t and LINE1 in D1 t Conflict, delete D3, and finally obtain a reliable matching point cloud D1, and obtain the three-dimensional point cloud image of the object to be scanned.

[0162] In the method provided in the embodiment of the present application, a connected domain after sorting and a next connected domain are subjected to conflict detection; if there is a line conflict between a connected domain and the next connected domain, the next connected domain is deleted, and the conflict detection between a connected domain and the subsequent connected domain is continued until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned; if there is no line conflict between a connected domain and the next connected domain, the connected domain and the next connected domain are merged to obtain a new connected domain, and the conflict detection between the new connected domain and the subsequent connected domain is continued until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned. By performing conflict detection on a connected domain after sorting and the next connected domain, false point clouds and mismatched lines can be determined from at least two connected domains, and they can be screened to obtain reliable point clouds, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0163] The following is a corresponding explanation of the three-dimensional point cloud acquisition device provided by any of the above embodiments of the present application. Its specific implementation process and the technical effects produced are the same as those of the corresponding method embodiments mentioned above. For the sake of brief description, the parts not mentioned in this embodiment can refer to the corresponding contents in the method embodiments.

[0164] Fig.11 This is a functional module diagram of a three-dimensional point cloud acquisition device provided in an embodiment of the present application. Fig.11 As shown, a processor applied to a structured light scanning device, the structured light scanning device also includes: two depth cameras, a line light source transmitter, and the three-dimensional point cloud acquisition device 100 includes:

[0165] An acquisition module 110, configured to control the line light source emitter to project light in multiple directions to the surface of the object to be scanned at multiple times, and to acquire multiple sets of stereo images captured by the two depth cameras at the multiple times;

[0166] An estimation module 120, configured to perform pose estimation on the plurality of sets of stereo images respectively, and obtain pose information of the structured light scanning device at the plurality of moments;

[0167] A matching module 130, configured to perform line stereo matching on a set of stereo images at each moment according to the posture information at each moment, to obtain a three-dimensional line set at each moment;

[0168] The screening module 140 is used to perform point cloud screening on the three-dimensional line sets at adjacent moments according to the three-dimensional line sets at adjacent moments among the multiple moments, so as to obtain a three-dimensional point cloud image of the object to be scanned.

[0169] Optionally, the estimation module 120 is also used to obtain a set of preset marker points of the object to be scanned, the preset marker point set including: the first three-dimensional coordinates of multiple preset marker points in a preset world coordinate system, the number of non-collinear marker points in the multiple preset marker points that are within the common field of view of the two depth cameras is greater than or equal to 3; based on a set of stereo images at each moment, obtain a set of acquisition marker points in the current field of view at each moment, the acquisition marker point set including: the second three-dimensional coordinates of multiple acquisition marker points in a preset device coordinate system; according to the first three-dimensional coordinates of the multiple preset marker points and the second three-dimensional coordinates of the multiple acquisition marker points, determine the same-name marker point pairs at each moment, the same-name marker point pairs including: the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple same-name marker points; according to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple same-name marker points, obtain the rotation matrix and the translation vector of the structured light scanning device at each moment in the preset world coordinate system, the posture information at each moment includes: the rotation matrix and the translation vector.

[0170] Optionally, the estimation module 120 is also used to obtain the first central three-dimensional coordinates of the marker points with the same name in the preset marker point set according to the first three-dimensional coordinates of the multiple marker points with the same name; obtain the second central three-dimensional coordinates of the marker points with the same name in the collected marker point set according to the second three-dimensional coordinates of the multiple marker points with the same name; calculate the rotation matrix according to the first three-dimensional coordinates of the multiple marker points with the same name, the first central three-dimensional coordinates, the second three-dimensional coordinates of the multiple marker points with the same name and the second central three-dimensional coordinates; calculate the translation vector according to the rotation matrix, the first central three-dimensional coordinates and the second central three-dimensional coordinates.

[0171] Optionally, the matching module 130 is further used to determine the structured light surfaces at the multiple moments based on the light in the multiple directions; perform line matching on a set of stereo images at each moment based on the structured light surfaces at each of the multiple moments to obtain multiple pairs of matching lines at each moment; perform coordinate transformation on the multiple pairs of matching lines at each moment based on the posture information at each moment to obtain a three-dimensional line set at each moment.

[0172] Optionally, the matching module 130 is further used to perform line matching on a group of stereo images at each moment to obtain multiple groups of candidate matching points for each line pair in the multiple line pairs at each moment, wherein each line pair includes: two lines of different images in a group of stereo images, and the multiple groups of candidate matching points respectively include: two matching points of different lines in the two lines; based on the multiple groups of candidate matching points, multiple three-dimensional points corresponding to each line are obtained, and the distances between the multiple three-dimensional points and the structured light surface at each moment are obtained; based on the distances between the multiple three-dimensional points and the structured light surface at each moment, a distance standard deviation corresponding to each line pair is obtained; if the distance standard deviation corresponding to multiple target line pairs in the multiple line pairs is less than a preset given threshold, the multiple target line pairs are determined to be the multiple pairs of matching lines at each moment.

[0173] Optionally, the screening module 140 is further used to divide the three-dimensional line set at the adjacent moments into connected domains to obtain at least one connected domain at the adjacent moments, each connected domain including: at least two three-dimensional lines connected in the three-dimensional line set at the adjacent moments; if the number of connected domains is at least two, sorting the at least two connected domains according to the number of three-dimensional lines and the number of three-dimensional intersections in each of the at least two connected domains; performing conflict detection on the at least two sorted connected domains one by one, and generating a three-dimensional point cloud image of the object to be scanned according to the detection results.

[0174] Optionally, the screening module 140 is further used to determine a first quantity value and a second quantity value of the at least two connected domains according to the number of three-dimensional lines and the number of three-dimensional intersections of the at least two connected domains; arrange the at least two connected domains in a descending order according to the first quantity value; if the first quantity values ​​of the at least two connected domains are the same, arrange the at least two connected domains in a descending order according to the second quantity value.

[0175] Optionally, the screening module 140 is further used to perform conflict detection on a connected domain and a next connected domain after sorting; if there is a line conflict between the connected domain and the next connected domain, the next connected domain is deleted, and the conflict detection is continued between the connected domain and the subsequent connected domains until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned; if there is no line conflict between the connected domain and the next connected domain, the connected domain and the next connected domain are merged to obtain a new connected domain, and the conflict detection is continued between the new connected domain and the subsequent connected domains until at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

[0176] Optionally, the screening module 140 is further configured to generate the three-dimensional point cloud image according to all three-dimensional lines in one connected domain if the number of the connected domains is one.

[0177] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0178] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0179] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A three-dimensional point cloud acquisition method, characterized in that: A processor applied to a structured light scanning device, wherein the structured light scanning device further comprises: two depth cameras and a line light source emitter, and the method comprises: Controlling the line light source emitter to project light in multiple directions to the surface of the object to be scanned at multiple times, and acquiring multiple sets of stereo images captured by the two depth cameras at the multiple times; Performing pose estimation on the multiple groups of stereo images respectively to obtain pose information of the structured light scanning device at the multiple moments; According to the posture information at each moment, performing line stereo matching on a set of stereo images at each moment to obtain a three-dimensional line set at each moment; According to the three-dimensional line sets at adjacent moments among the multiple moments, point cloud screening is performed on the three-dimensional line sets at adjacent moments to obtain a three-dimensional point cloud image of the object to be scanned; The step of performing point cloud screening on the three-dimensional line sets at adjacent moments according to the three-dimensional line sets at adjacent moments among the multiple moments to obtain the three-dimensional point cloud image of the object to be scanned includes: Performing connected domain division on the three-dimensional line set at adjacent moments to obtain at least one connected domain at the adjacent moments, each connected domain comprising: at least two connected three-dimensional lines in the three-dimensional line set at the adjacent moments; If the number of connected domains is at least two, sorting the at least two connected domains according to the number of three-dimensional lines and the number of three-dimensional intersections of each of the at least two connected domains; Conflict detection is performed on the at least two sorted connected domains one by one, and a three-dimensional point cloud image of the object to be scanned is generated according to the detection result.

2. The method according to claim 1, characterized in that The step of respectively estimating the poses of the plurality of sets of stereo images to obtain the pose information of the structured light scanning device at the plurality of moments includes: Acquire a preset marker point set of the object to be scanned, the preset marker point set comprising: first three-dimensional coordinates of a plurality of preset marker points in a preset world coordinate system, wherein the number of non-collinear marker points in the plurality of preset marker points that are within a common field of view of the two depth cameras is greater than or equal to 3; Based on the set of stereo images at each moment, obtaining a set of acquisition marker points in the current field of view at each moment, wherein the set of acquisition marker points includes: second three-dimensional coordinates of a plurality of acquisition marker points in a preset device coordinate system; Determine the same-named marker point pairs at each moment according to the first three-dimensional coordinates of the plurality of preset marker points and the second three-dimensional coordinates of the plurality of collected marker points, wherein the same-named marker point pairs include: the first three-dimensional coordinates and the second three-dimensional coordinates of the plurality of same-named marker points; According to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple markers with the same name, the rotation matrix and the translation vector of the structured light scanning device at each moment in the preset world coordinate system are obtained, and the posture information at each moment includes: the rotation matrix and the translation vector.

3. The method according to claim 2, characterized in that The step of obtaining the rotation matrix and the translation vector of the structured light scanning device at each moment in the preset world coordinate system according to the first three-dimensional coordinates and the second three-dimensional coordinates of the multiple marker points with the same name includes: Obtaining, according to the first three-dimensional coordinates of the plurality of marker points with the same name, the first central three-dimensional coordinates of the marker points with the same name in the preset marker point set; According to the second three-dimensional coordinates of the plurality of marker points with the same name, obtaining the second central three-dimensional coordinates of the marker points with the same name in the collection marker point set; Calculating the rotation matrix according to the first three-dimensional coordinates of the plurality of marker points with the same name, the first three-dimensional coordinates of the center, the second three-dimensional coordinates of the plurality of marker points with the same name, and the second three-dimensional coordinates of the center; The translation vector is calculated according to the rotation matrix, the first central three-dimensional coordinates and the second central three-dimensional coordinates.

4. The method according to claim 1, characterized in that: The method of performing line stereo matching on a set of stereo images at each moment according to the posture information at each moment to obtain a three-dimensional line set at each moment includes: Determining structured light surfaces at the multiple moments according to the light in the multiple directions; According to the structured light surface at each of the multiple moments, line matching is performed on a set of stereo images at each moment to obtain multiple pairs of matching lines at each moment; According to the position and posture information at each moment, coordinate transformation is performed on the multiple pairs of matching lines at each moment to obtain a three-dimensional line set at each moment.

5. The method according to claim 4, characterized in that The method of performing line matching on a set of stereo images at each moment according to the structured light surface at each moment in the multiple moments to obtain multiple pairs of matching lines at each moment includes: Performing line matching on a set of stereo images at each moment to obtain multiple groups of candidate matching points for each line pair in the multiple line pairs at each moment, wherein each line pair includes: two lines of different images in a set of stereo images, and the multiple groups of candidate matching points include: two matching points of different lines in the two lines; According to the multiple groups of candidate matching points, a plurality of three-dimensional points corresponding to each line are obtained, and the distance between the multiple three-dimensional points and the structured light surface at each moment is acquired; Obtaining a distance standard deviation corresponding to each line pair according to the distances between the multiple three-dimensional points and the structured light surface at each moment; If the distance standard deviations corresponding to the plurality of target line pairs among the plurality of line pairs are smaller than a preset threshold, the plurality of target line pairs are determined to be the plurality of matching line pairs at each moment.

6. The method according to claim 1, characterized in that The step of sorting the at least two connected domains according to the number of three-dimensional lines and the number of three-dimensional intersections of each of the at least two connected domains comprises: Determine a first quantity value and a second quantity value of the at least two connected domains according to the quantity of three-dimensional lines and the quantity of three-dimensional intersections of the at least two connected domains; Arranging the at least two connected domains in descending order according to the first quantity value; If the first quantity values ​​of the at least two connected domains are the same, the at least two connected domains are arranged in descending order according to the second quantity value.

7. The method according to claim 1, characterized in that The step of performing conflict detection on the at least two sorted connected domains one by one and generating a three-dimensional point cloud image of the object to be scanned according to the detection result includes: Perform conflict detection between a sorted connected domain and the next connected domain; If there is a line conflict between the one connected domain and the next connected domain, the next connected domain is deleted, and the conflict detection between the one connected domain and the subsequent connected domains is continued until the at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned; If there is no line conflict between the one connected domain and the next connected domain, the one connected domain and the next connected domain are merged to obtain a new connected domain, and the conflict detection of the new connected domain and the subsequent connected domains is continued until the at least two connected domains are detected, thereby obtaining a three-dimensional point cloud image of the object to be scanned.

8. The method according to claim 1, characterized in that: The step of performing point cloud screening on the three-dimensional line sets at adjacent moments according to the three-dimensional line sets at adjacent moments among the multiple moments to obtain the three-dimensional point cloud image of the object to be scanned further includes: If the number of connected domains is one, the three-dimensional point cloud image is generated according to all three-dimensional lines in one connected domain.

9. A structured light scanning device, characterized in that: The structured light scanning device includes: two depth cameras, a line light source emitter and a processor, the two depth cameras and the line light source emitter are rigidly connected, the processor is respectively connected to the two depth cameras and the line light source emitter, and the processor executes the steps of the three-dimensional point cloud acquisition method as described in any one of claims 1 to 8.

Images