Dynamic three-dimensional reconstruction method based on multi-line coding and tracking

Abstract

The invention discloses a dynamic three-dimensional reconstruction method based on multi-line coding and tracking. The dynamic three-dimensional reconstruction method comprises the following steps: (1) predefining serial numbers of lasers and corresponding laser lines; (2) calibrating the left camera and the right camera; (3) performing initialization scanning on the object, determining the position of each laser line, and processing to obtain coding images of the left image and the right image containing coding points at the position of each laser line; (4) scanning the dynamic object, traversing the coded images of the left image and the right image obtained in the previous step, and judging whether to update the laser line coded points at the corresponding positions; and (5) updating the latest coded images of the left image and the right image of the camera in real time according to the result of the previous step, and generating point cloud data of the dynamic object in real time. According to the dynamic three-dimensional reconstruction method, multi-line laser can be effectively and accurately extracted, and dynamic tracking of multi-line laser data is achieved. The method is simple in implementation steps, and has the advantages of high measurement precision and high measurement speed.

Description

Technical Field

[0001] The invention relates to the technical field of optical three-dimensional digital imaging, and in particular to a dynamic three-dimensional reconstruction method based on multi-line encoding and tracking. Background Art

[0002] In recent years, with the rapid development of 3D measurement technology, optical 3D measurement has been widely used in 3D reconstruction of objects of various sizes, complex shapes and materials with its non-contact, high efficiency and high precision. Currently, single-line laser scanning technology is time-consuming, and multi-line stereo vision measurement methods often use target and reverse reprojection methods, in which the target needs to collect target data of different postures, and the steps are cumbersome; and when the reverse reprojection multi-line laser is matched, there are some wrong matching point pairs, and the distance to the laser plane is also less than a certain threshold, resulting in redundant wrong matching.

[0003] The Chinese patent application number 202211621677.8 discloses a binocular multi-line 3D reconstruction method and system based on region coding. Although this method can improve the measurement accuracy, it uses a single-line laser and cannot be applied to moving objects. It can only be used to scan static objects. The method of using multiple lasers for dynamic 3D reconstruction is still the focus of research in this field. Summary of the invention

[0004] The purpose of the present invention is to provide a dynamic three-dimensional reconstruction method based on multi-line coding and tracking to address the deficiencies of the prior art, so as to solve the problems of the prior art such as cumbersome process and low precision.

[0005] In order to solve the above technical problems, a technical solution adopted by the present invention is to provide a dynamic three-dimensional reconstruction method based on multi-line encoding and tracking, the method comprising the following steps:

[0006] (1) Selecting appropriate positions and numbers of lasers according to the object to be scanned, and pre-defining the numbers of the selected lasers and the corresponding laser lines in combination with the position sequence of each laser;

[0007] (2) Calibrate the left and right cameras based on the triangulation principle;

[0008] (3) controlling the selected corresponding lasers and two cameras to perform initial scanning on the object, determining the position of each laser line, and processing to obtain coded images of the left and right images containing the coded points of the position of each laser line;

[0009] (4) The object starts to move, and the selected lasers work simultaneously to scan the dynamic object, and at the same time, traverse the coded images of the left and right images obtained in the previous step respectively, and determine whether to update the laser line coded points at the corresponding positions and whether to update the coded images of the left and right images;

[0010] (5) According to the result of the previous step, the latest coded images of the left and right images of the camera are updated in real time, each laser line is dynamically tracked, and point cloud data of the dynamic object is generated in real time.

[0011] In another embodiment of the present invention, in step (3), the process of initializing the scan of the object is as follows: according to the predefined laser number, the corresponding laser is controlled to turn on and off according to the preset coding sequence, and any laser that turns on triggers the two cameras to take pictures, and the initialization scan is completed when the lasers corresponding to the preset codes have all finished working.

[0012] In another embodiment of the present invention, in step (3), the preset coding sequence means that the selected laser operates according to a predetermined rule, and the number and sequence of the laser lines that light up each time will change.

[0013] In another embodiment of the present invention, in step (3), the method for determining the position of each laser line is as follows: when initializing the scan, two cameras take pictures to obtain multiple groups of images, and based on the corresponding laser lines and coding sequences captured in the multiple groups of images, the center point of the laser stripes is extracted to determine the position of each laser line.

[0014] In another embodiment of the present invention, in step (3), the position of each laser line is determined, that is, the laser lines in the left and right images of the camera are extracted, and then three-dimensional point cloud data of multiple laser lines are obtained according to a three-dimensional reconstruction method, and two line laser images with coded points are generated at the same time, that is, coded images of the left and right images are generated.

[0015] In another embodiment of the present invention, in step (4), the conditions for determining whether to update the laser line coding point at the corresponding position include at least: the coding point position in the coded image, whether there is a coding point in the local neighborhood, and whether the point recalculated in the latest acquired image satisfies the laser line feature.

[0016] In another embodiment of the present invention, in step (4), if the judgment result is that the laser line coding point at the corresponding position needs to be updated, the laser line data of the left and right images of the camera are extracted, and then the three-dimensional point cloud data of multiple laser lines are obtained according to the three-dimensional reconstruction method, and the latest coded images of the left and right images of the camera are updated at the same time.

[0017] In another embodiment of the present invention, when multiple lasers are working, they project a group of mutually parallel laser planes onto the surface of the object, and simultaneously trigger the left and right cameras to take pictures, thereby obtaining a multi-laser image of the two camera images.

[0018] The beneficial effects of the present invention are as follows: the dynamic three-dimensional reconstruction method based on multi-line coding and tracking can effectively and accurately extract multi-line lasers, realize dynamic tracking of multi-line laser data through multi-line laser coding, and generate three-dimensional point cloud data in real time. The method has simple implementation steps, can not only solve the problems of wrong matching and inability to scan dynamic objects in existing methods, but also has high measurement accuracy, fast measurement speed, strong adaptability to the environment, and can be widely used in various environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a structural diagram of a three-dimensional binocular vision sensor used in the dynamic three-dimensional reconstruction method of the present invention;

[0020] Figure 2 It is a flow chart of the dynamic three-dimensional reconstruction method based on multi-line coding and tracking of the present invention.

[0021] Numbers in the figure: 1. Left camera; 2. Right camera; 3. Laser; 4. Shell. DETAILED DESCRIPTION

[0022] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0023] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0024] like Figure 1 The figure shows the structure of the three-dimensional binocular vision sensor used in the dynamic three-dimensional reconstruction method of the present invention. As can be seen from the figure, the sensor includes a shell 4, in which a left camera 1, a right camera 2 and a group of lasers 3 are arranged. The number of lasers here needs to be configured according to actual conditions and is not limited to the number shown in the figure.

[0025] like Figure 2 FIG. 1 is a flow chart of a dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to the present invention. As can be seen from the figure, the method includes the following steps:

[0026] (1) Selecting appropriate positions and numbers of lasers according to the object to be scanned, and pre-defining the numbers of the selected lasers and the corresponding laser lines in combination with the position sequence of each laser;

[0027] (2) Calibrate the left and right cameras based on the triangulation principle;

[0028] (3) controlling the selected corresponding lasers and two cameras to perform initial scanning on the object, determining the position of each laser line, and processing to obtain coded images of the left and right images containing the coded points of the position of each laser line;

[0029] (4) The object starts to move, and the selected lasers work simultaneously to scan the dynamic object, and at the same time, traverse the coded images of the left and right images obtained in the previous step respectively, and determine whether to update the laser line coded points at the corresponding positions and whether to update the coded images of the left and right images;

[0030] (5) According to the result of the previous step, the latest coded images of the left and right images of the camera are updated in real time, each laser line is dynamically tracked, and point cloud data of the dynamic object is generated in real time.

[0031] Preferably, in step (1), it is necessary to determine which positions of the laser can be involved in the work and the specific number of lasers required based on the actual size of the object to be measured. The selection criteria here are to ensure that the laser line emitted by the laser 3 can cover 80% to 100% of the surface area of ​​the object to be measured.

[0032] In step (2), the camera is calibrated using the triangulation principle, which is a common technique in the art and will not be described in detail here.

[0033] Furthermore, in step (3), the process of initializing the scan of the object is as follows: according to the predefined laser number, the corresponding laser is controlled to turn on and off according to the preset coding sequence. When any laser is turned on, it triggers the two cameras to take pictures. When the lasers corresponding to the preset codes have all finished working, the initialization scan is completed.

[0034] Furthermore, in the step (3), the preset coding sequence means that the selected laser operates according to a predetermined rule, and the number and sequence of the laser lines that light up each time will change.

[0035] Furthermore, in step (3), the method for determining the position of each laser line is as follows: during the initialization scan, two cameras take pictures to obtain multiple groups of images, and based on the corresponding laser lines and coding sequences captured in the multiple groups of images, the center point of the laser stripes is extracted to determine the position of each laser line.

[0036] Furthermore, in step (3), the position of each laser line is determined, that is, the laser lines in the left and right images of the camera are extracted, and then according to the three-dimensional reconstruction method, the three-dimensional point cloud data of multiple laser lines are obtained, and two line laser images with coded points are generated at the same time, that is, coded images of the left and right images are generated.

[0037] Furthermore, in step (4), the conditions for determining whether to update the laser line coding point at the corresponding position include at least: the coding point position in the coded image, whether there is a coding point in the local neighborhood, and whether the point recalculated in the latest acquired image satisfies the laser line feature.

[0038] Furthermore, in step (4), if the judgment result is that the laser line coding point at the corresponding position needs to be updated, the laser line data of the left and right images of the camera are extracted, and then the three-dimensional point cloud data of multiple laser lines are obtained according to the three-dimensional reconstruction method, and the latest coded images of the left and right images of the camera are updated at the same time.

[0039] In this embodiment, multi-line laser is generated by installing multiple lasers. Specifically, when working, multiple lasers work simultaneously to project a group of parallel laser planes onto the surface of the object, and simultaneously trigger the left and right cameras to take pictures and collect images, thereby obtaining a multi-laser image of the two camera images.

[0040] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A dynamic three-dimensional reconstruction method based on multi-line coding and tracking, characterized in that: The method comprises the following steps: (1) Selecting appropriate positions and numbers of lasers according to the object to be scanned, and pre-defining the numbers of the selected lasers and the corresponding laser lines in combination with the position sequence of each laser; (2) Calibrate the left and right cameras based on the triangulation principle; (3) controlling the selected corresponding lasers and two cameras to perform initial scanning on the object, determining the position of each laser line, and processing to obtain coded images of the left and right images containing the coded points of the position of each laser line; (4) The object starts to move, and the selected lasers work simultaneously to scan the dynamic object, and at the same time, traverse the coded images of the left and right images obtained in the previous step respectively, and determine whether to update the laser line coded points at the corresponding positions and whether to update the coded images of the left and right images; (5) According to the result of the previous step, the latest coded images of the left and right images of the camera are updated in real time, each laser line is dynamically tracked, and point cloud data of the dynamic object is generated in real time.

2. The dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to claim 1, characterized in that: In step (3), the process of initializing the scan of the object is as follows: according to the pre-defined laser number, the corresponding laser is controlled to turn on and off according to the preset coding sequence. When any laser is turned on, it triggers the two cameras to take pictures. When the lasers corresponding to the preset codes have all finished working, the initialization scan is completed.

3. The dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to claim 2, characterized in that: In the step (3), the preset coding sequence means that the selected laser operates according to a predetermined rule, and the number and sequence of the laser lines that light up each time will change.

4. The dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to claim 3 is characterized in that: In step (3), the method for determining the position of each laser line is as follows: during the initialization scan, two cameras take pictures to obtain multiple sets of images, and based on the corresponding laser lines and coding sequences captured in the multiple sets of images, the center point of the laser stripes is extracted to determine the position of each laser line.

5. The dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to claim 4, characterized in that: In step (3), the position of each laser line is determined, that is, the laser lines in the left and right images of the camera are extracted, and then the three-dimensional point cloud data of multiple laser lines are obtained according to the three-dimensional reconstruction method, and two line laser images with coded points are generated at the same time, that is, coded images of the left and right images are generated.

6. The dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to claim 5, characterized in that: In step (4), the conditions for determining whether to update the laser line coding point at the corresponding position include at least: the coding point position in the coded image, whether there is a coding point in the local neighborhood, and whether the point recalculated in the latest acquired image meets the laser line characteristics.

7. The dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to claim 6, characterized in that: In step (4), if the judgment result is that the laser line coding point at the corresponding position needs to be updated, the laser line data of the left and right images of the camera are extracted, and then the three-dimensional point cloud data of multiple laser lines are obtained according to the three-dimensional reconstruction method, and the latest coded images of the left and right images of the camera are updated at the same time.

8. The dynamic three-dimensional reconstruction method based on multi-line coding and tracking according to claim 7, characterized in that: When multiple lasers are working, they project a set of parallel laser planes onto the surface of the object, and simultaneously trigger the left and right cameras to take pictures, thereby obtaining a multi-laser image of the two camera images.

Images