3D imaging method and system based on structured light

Abstract

The invention discloses a 3D imaging method and system based on structured light. The method comprises the following steps: providing a camera device and two projection devices; starting the two projection devices in sequence, and acquiring a first image and a second image of the current target object through the camera device; calculating the phase of each pixel point in the first image, and unwrapping the phases to obtain a first phase set; calculating the phase of each pixel point in the second image, and unwrapping the phases to obtain a second phase set; calculating the phase difference of corresponding pixel points in the first phase set and the second phase set to obtain a phase difference set; and calculating a 3D point cloud of the target object based on the phase difference set and calibration data of the camera device. Therefore, according to the method, through introduction of the phase difference set, the geometric advantages of the double-projection included angle are utilized, the new point cloud generated through the phase difference has higher resolution and precision, an existing hardware structure does not need to be greatly modified, and therefore the overall system cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of 3D imaging technology, and in particular to a 3D imaging method and system based on structured light. Background Art

[0002] Structured light 3D imaging technology is a non-contact measurement method based on the principle of triangulation. It projects a structured light pattern (such as streak light) onto the target object and captures the reflected light image with a camera to calculate the three-dimensional information of the object surface. Due to its high resolution and high precision, this technology is widely used in industrial inspection, medical imaging, reverse engineering and other fields. However, in the existing technology, the design and application of structured light 3D imaging solutions still have certain limitations.

[0003] The current structured light 3D imaging technology is mainly divided into two implementation schemes: main camera and main projection.

[0004] For the main camera solution, the camera is arranged facing the target object, while the projection device is tilted at a certain angle. The height information of the object is calculated by the angle between the camera and the projection device combined with triangulation. In order to overcome the sampling blind spots caused by reflective areas and shadow areas, two or more tilted projection devices are usually used to cover more target areas. Although the main camera solution has a great advantage in accuracy and adaptability to application scenarios, its measurement accuracy is still limited by many factors. Among them, the angle between the camera and the projection device is a key factor affecting accuracy. The larger the angle, the higher the measurement accuracy. However, too large an angle may be limited by the on-site installation space. In addition, the increase in the angle will affect the range of the 3D point cloud. In the prior art, the 3D point cloud is usually increased by designing the Scheimpflug angle, but this requires redesigning the lens, which significantly increases the hardware cost. Summary of the invention

[0005] The object of the present invention is to provide a 3D imaging method and system based on structured light that can effectively improve imaging accuracy without increasing equipment cost.

[0006] In order to achieve the above object, the present invention provides a 3D imaging method based on structured light, which comprises: A camera device and two projection devices are provided, wherein the camera device is located directly above a target object to be imaged, and the two projection devices are respectively located on both sides of the camera device, and the projection directions of the two projection devices are inclined to the central axis of the target object; Activating one of the two projection devices to project stripe light toward the target object, and acquiring a first image of the target object through the camera device; activating the other of the two projection devices to project stripe light toward the target object, and acquiring a second image of the target object through the camera device; Calculating the phase of each pixel in the first image, and unfolding the phase to obtain a first phase set; Calculating the phase of each pixel in the second image, and unfolding the phase to obtain a second phase set; Calculate the phase difference between the first phase set and the second phase set corresponding to the pixel point to obtain a phase difference set; A 3D point cloud of the target object is calculated based on the phase difference set and preset calibration data of the camera device.

[0007] Preferably, the angle α between the two projection devices satisfies the following condition: 5°≤α≤60°.

[0008] Preferably, only when the phase difference is less than or equal to a preset value, the phase difference is added to the phase difference set.

[0009] Preferably, the intensity of the stripe light projected by the projection device changes adaptively following the brightness of the environment where the target object is located.

[0010] The present invention also provides a 3D imaging system based on structured light, comprising: A camera device, the camera device is located directly above the target object to be imaged, and the camera device is used to capture an image of the target object; Two projection devices, the two projection devices are respectively located on both sides of the camera device, and the projection directions of the two projection devices are inclined to the central axis of the target object, and the projection devices are used to successively project stripe light to the target object respectively; An image processing device, the image processing device is connected to the camera device, and the image processing device is used to process the first image and the second image of the target object respectively captured by the camera device under the stripe light projected by the two projection devices to obtain a phase difference set including a plurality of phase differences; The phase difference is the phase difference between the corresponding pixels in the first image and the second image in the phase unwrapped state; A depth calculation module, the depth calculation module is connected to the image processing device, and the depth calculation module is used to calculate the 3D point cloud of the target object according to the phase difference set and preset calibration data of the camera device.

[0011] Preferably, the angle θ between any one of the projection devices and the central axis of the target object satisfies the following condition: 5°≤θ≤30°.

[0012] Preferably, only when the phase difference is less than or equal to a preset value, the phase difference is added to the phase difference set.

[0013] Preferably, the device further comprises an ambient light sensor disposed near the target object, and the intensity of the striped light projected by the projection device changes adaptively following the detection value of the ambient light sensor.

[0014] The present invention also provides a 3D imaging system, comprising: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for executing the 3D imaging method as described above.

[0015] The present invention also provides a computer-readable storage medium, which includes a computer program, and the computer program can be executed by a processor to implement the 3D imaging method as described above.

[0016] Compared with the prior art, the 3D imaging method provided by the above technical solution of the present invention obtains two images of the target object through the cooperation of the camera device and the two projection devices, and obtains the phase difference set of the two images, and finally calculates the 3D point cloud of the target object based on the phase difference set. It can be seen that compared with the traditional dual-projection solution, the accuracy of the 3D point cloud is determined by the single angle of the left and right projections. In the present invention, the geometric advantage of the dual-projection angle (about twice the single-projection angle) is achieved by introducing the phase difference set, so that the new point cloud generated by the phase difference has higher resolution and accuracy, and there is no need to significantly modify the existing hardware structure, thereby reducing the overall system cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A state diagram for 3D imaging implementation in the prior art.

[0018] Figure 2 This is a flow chart of the 3D imaging principle in the prior art.

[0019] Figure 3 This is a state diagram of a 3D imaging embodiment in an embodiment of the present invention.

[0020] Figure 4 This is a flow chart of the 3D imaging principle in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to explain the technical content, structural features, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the implementation methods and the accompanying drawings.

[0022] This embodiment discloses a 3D imaging method based on structured light, which is used to generate a 3D image of a target object. To facilitate the description of the principle of 3D imaging in this embodiment, the principle of 3D imaging in the prior art is first described.

[0023] like Figure 1 In the prior art, a 3D imaging system based on structured light includes a camera 1 and two projection devices 2, 2'. The camera 1 is located directly above a target 3, and the two projection devices 2, 2' project streaks of light from both sides of the target 3. Figure 2 , the algorithm principle of the imaging system is: 1. The left projection device 2 projects stripe light onto the target object 3, the camera device 1 takes a picture to obtain image A, calculates the unfolding phase P1 of image A, and then calculates the 3D point cloud H1 according to the unfolding phase P1, thereby obtaining a set of 3D point cloud data of the target object.

[0024] 2. The right projection device 2' projects stripe light onto the target object, the camera device 1 takes a picture to obtain image B, calculates the unwrapped phase P2 of image B, and then calculates the 3D point cloud H2 according to the unwrapped phase P2, thereby obtaining a set of 3D point cloud data of the target object.

[0025] 3. The 3D point cloud H1 and the 3D point cloud H2 are fused to obtain the target 3D point cloud H, thereby obtaining a new set of 3D point cloud data of the target object, and a 3D image of the target object can be generated based on the 3D point cloud data.

[0026] In this regard, the 3D image accuracy is related to the projection angle θ of the projection device 2 or 2 ′ (ie, the angle between the projection device 2 or 2 ′ and the central axis of the target object 3 ). The larger the angle θ, the higher the 3D image accuracy.

[0027] Therefore, in order to improve the accuracy of 3D images, the angle θ must be increased, but the increase in the angle θ is limited by the cost of the site and the corresponding lens. In practice, the angle θ is generally not greater than 30°.

[0028] More specifically, for traditional 3D imaging systems, two sets of 3D point clouds are generated through left and right projections respectively, and the accuracy of these point clouds depends on the camera-projection angle corresponding to each projection. However, the generation of left and right projection point clouds is independent, and the geometric relationship and information redundancy between the two projections are not fully utilized.

[0029] To solve this problem, a new 3D imaging method is provided in this embodiment. The hardware cost required by this method is the same as that of the prior art. Figure 3 , including a camera device 5 and two projection devices 6, 6', the camera device 5 is located directly above the target object 7 to be imaged, the two projection devices 6, 6' are respectively located on both sides of the camera device 5, and the projection directions of the two projection devices 6, 6' are inclined to the central axis of the target object 7.

[0030] Based on this, Figure 3 and Figure 4 The 3D imaging method in this embodiment includes the following steps: S1: Start the projection device 6 on the left to project stripe light toward the target object, and obtain a first image of the current target object 7 through the camera device 5.

[0031] S2: Start the projection device 6 ′ on the right side to project stripe light to the target object 7 , and obtain a second image of the current target object 7 through the camera device 5 .

[0032] S3: Calculate the phase of each pixel in the first image, and unfold the phase to obtain a first phase set.

[0033] S4: Calculate the phase of each pixel in the second image, and unfold the phase to obtain a second phase set.

[0034] S5: Calculate the phase difference between the corresponding pixel points in the first phase set and the second phase set to obtain a phase difference set.

[0035] S6: Calculate the 3D point cloud of the target object 7 based on the phase difference set and the preset calibration data of the camera device 5, so as to obtain the 3D point cloud of the target object 7, and generate a 3D imaging map of the target object 7 based on the 3D point cloud.

[0036] In this embodiment, the 3D point cloud is calculated based on the phase difference corresponding to each pixel point in the phase difference set. Therefore, the accuracy of the 3D image finally formed is related to the angle α between the two projection devices 6 and 6 ′.

[0037] More specifically, the 3D imaging method in this embodiment generates a new set of point clouds using phase difference based on two sets of point clouds independently generated by left and right projections. Since the phase difference corresponds to the measurement result of the double projection angle, the accuracy of the new point cloud is significantly higher than that of the point cloud generated by a single projection angle.

[0038] Therefore, the increase in the double projection angle in this embodiment effectively extends the baseline length of triangulation and improves the resolution of the measurement. In other words, a larger double projection angle means that the phase change on the projection for the same height difference is more significant, thereby being able to more accurately reflect the height characteristics of the object surface.

[0039] Assuming that the single projection angle is 30°, based on the traditional technical solution, the corresponding measurement accuracy is Δh. Based on the technical solution of the present invention, when the double projection angle is close to 60°, the measurement accuracy can be theoretically improved to Δh / 2.

[0040] In addition, the phase difference method not only utilizes the independent information of the left and right projections, but also further optimizes data processing through the geometric redundancy between the two projections, including: First, there are independent errors in the phase unwrapping of the left and right projections, which can be partially offset by the phase difference method, thereby improving the overall measurement accuracy.

[0041] Second, since the calculation of the phase difference is based on the combined information of the dual projections, the measurement result is more robust to local noise or systematic errors of a single projection.

[0042] In general, compared with the prior art, the 3D imaging method provided by the embodiment of the present invention, combined with the geometric advantages of dual projection and the phase difference method, significantly improves the measurement accuracy of 3D point clouds, and enhances noise resistance and reliability. By replacing hardware complexity with algorithm optimization, costs are reduced and the applicable scenarios of the technology are expanded.

[0043] Specifically, the angle α between the two projection devices 6 and 6 ′ satisfies the following condition: 5°≤α≤60°.

[0044] On the other hand, when any phase difference is greater than the preset value, the phase difference is abandoned to be added to the phase difference set, so that some points with relatively large differences can be eliminated, thus increasing the stability of the data.

[0045] On the other hand, the intensity of the stripe light projected by the projection devices 6, 6' changes adaptively with the brightness of the environment where the target object 7 is located, thereby ensuring the quality of the first image and the second image, and the obtained phase difference data is more accurate and more comprehensive.

[0046] In another preferred embodiment of the present invention, a 3D imaging system based on structured light is also disclosed. Figure 3 , which includes a camera device 5, two projection devices 6, 6', an image processing device and a depth calculation module.

[0047] The camera device 5 is located directly above the target object 7 to be imaged, and is used to capture an image of the target object 7 .

[0048] The two projection devices 6, 6' are respectively located on both sides of the camera device 5, and the projection directions of the two projection devices 6, 6' are inclined to the central axis of the target object 7. The projection devices 6, 6' are used to project stripe light, The image processing device is connected to the camera device 5 and is used to process the first image and the second image of the target object 7 captured by the camera device 5 under the stripe light projected by the two projection devices 6 and 6' to obtain a phase difference set including a plurality of phase differences.

[0049] The phase difference is the phase difference between corresponding pixels in the first image and the second image in the phase unwrapped state.

[0050] The depth calculation module is connected to the image processing device, and is used to calculate the 3D point cloud of the target object 7 according to the phase difference set and the preset calibration data of the camera device 5 .

[0051] On the other hand, the angle θ between any projection device 6 , 6 ′ and the central axis of the target object 7 satisfies the following condition: 5°≤θ≤60°.

[0052] On the other hand, when any phase difference is greater than a preset value, the phase difference is abandoned from being added to the phase difference set.

[0053] Furthermore, the 3D imaging system further comprises an ambient light sensor 8 disposed near the target object 7 , and the intensity of the stripe light projected by the projection devices 6 , 6 ′ changes adaptively following the detection value of the ambient light sensor 8 .

[0054] The working principle and method of the 3D imaging system in this embodiment are detailed in the 3D imaging method in the above embodiment, which will not be described again here.

[0055] The present invention also discloses another 3D imaging system, which includes one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the program includes instructions for executing the 3D imaging method as described above. The processor can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, which are used to execute relevant programs to implement the functions required to be executed by the modules in the 3D imaging system of the embodiment of the present application, or to execute the 3D imaging method of the method embodiment of the present application.

[0056] The present invention also discloses a computer-readable storage medium, which includes a computer program, and the computer program can be executed by a processor to complete the 3D imaging method as described above. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a read-only memory (ROM), or a random access memory (RAM), or a magnetic medium, such as a floppy disk, a hard disk, a tape, a magnetic disk, or an optical medium, such as a digital versatile disc (DVD), or a semiconductor medium, such as a solid state disk (SSD).

[0057] The embodiment of the present application also discloses a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above-mentioned 3D imaging method.

[0058] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope covered by the present invention.

Claims

1. A 3D imaging method based on structured light, characterized in that: include: A camera device and two projection devices are provided, wherein the camera device is located directly above a target object to be imaged, and the two projection devices are respectively located on both sides of the camera device, and the projection directions of the two projection devices are inclined to the central axis of the target object; Activating one of the two projection devices to project stripe light toward the target object, and acquiring a first image of the target object through the camera device; activating the other of the two projection devices to project stripe light toward the target object, and acquiring a second image of the target object through the camera device; Calculating the phase of each pixel in the first image, and unfolding the phase to obtain a first phase set; Calculating the phase of each pixel in the second image, and unfolding the phase to obtain a second phase set; Calculate the phase difference between the first phase set and the second phase set corresponding to the pixel point to obtain a phase difference set; A 3D point cloud of the target object is calculated based on the phase difference set and preset calibration data of the camera device.

2. The 3D imaging method based on structured light according to claim 1, characterized in that: The angle α between the two projection devices satisfies the following condition: 5°≤α≤60°.

3. The 3D imaging method based on structured light according to claim 1, characterized in that: Only when the phase difference is less than or equal to a preset value, the phase difference is added to the phase difference set.

4. The 3D imaging method based on structured light according to claim 1, characterized in that: The intensity of the stripe light projected by the projection device changes adaptively following the brightness of the environment where the target object is located.

5. A 3D imaging system based on structured light, characterized in that: include: A camera device, the camera device is located directly above the target object to be imaged, and the camera device is used to capture an image of the target object; Two projection devices, the two projection devices are respectively located on both sides of the camera device, and the projection directions of the two projection devices are inclined to the central axis of the target object, and the projection devices are used to successively project stripe light to the target object respectively; An image processing device, the image processing device is connected to the camera device, and the image processing device is used to process the first image and the second image of the target object respectively captured by the camera device under the stripe light projected by the two projection devices to obtain a phase difference set including a plurality of phase differences; The phase difference is the phase difference between the corresponding pixels in the first image and the second image in the phase unwrapped state; A depth calculation module, the depth calculation module is connected to the image processing device, and the depth calculation module is used to calculate the 3D point cloud of the target object according to the phase difference set and preset calibration data of the camera device.

6. The 3D imaging system based on structured light according to claim 5, characterized in that: The angle θ between any of the projection devices and the central axis of the target object satisfies the following condition: 5°≤θ≤30°.

7. The 3D imaging system based on structured light according to claim 5, characterized in that: Only when the phase difference is less than or equal to a preset value, the phase difference is added to the phase difference set.

8. The 3D imaging system based on structured light according to claim 5, characterized in that: It also includes an ambient light sensor disposed near the target object, and the intensity of the striped light projected by the projection device changes adaptively following the detection value of the ambient light sensor.

9. A 3D imaging system, characterized in that: include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the programs comprising instructions for executing the 3D imaging method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The invention comprises a computer program, wherein the computer program can be executed by a processor to implement the 3D imaging method according to any one of claims 1 to 4.

Images