Flying spot removal system based on phase shift and line shift fusion

Abstract

The invention provides a flying spot removal system based on phase shift and line shift fusion, and the system comprises a first phase value generation part which is used for obtaining a phase shift structured light pattern, and determining a first phase value of each pixel in a camera unit according to the phase shift structured light pattern; the second phase value generation part is used for acquiring a line-shift structured light pattern and determining a second phase value corresponding to each pixel in the camera unit according to the line-shift structured light pattern; and the phase fusion part is used for fusing the first phase value and the second phase value corresponding to each pixel to generate a target phase value of the pixel so as to generate a point cloud image of which the flying spot is removed. According to the method, the flying spots can be effectively removed, the imaging error of the depth camera is reduced, and the interference of the flying spots on point-to-point cloud processing and measurement is reduced.

Description

Background Art

[0002] Commonly used projector technologies include DLP (Digital Light Processing), line laser with MEMS galvanometer or mechanical galvanometer, etc.

[0003] Among them, with the maturity of LED light source and DLP technology, DLP projectors have developed rapidly and become a widely used projection method. In 1987, TI invented the DMD device, which enabled DLP digital light processing technology to be applied in the world and promoted the rise of DLP projectors. The DMD device is a binary pulse width modulated digital optical switch and is currently the most complex optical switch device in the world. Thousands of tiny square lenses are built on a hinge structure above the static random access memory to form the DMD. Each lens can turn on and off the light of a pixel. The hinge structure allows the lens to tilt between two states, +10 degrees for "on" and -10 degrees for "off".

[0004] In the depth map measured by the depth camera based on the DLP structured light principle, there are often a large number of erroneous depth measurement values ​​at the edge of the object. After the point cloud is generated, it visually appears as invalid points flying in the air, which is called flying point noise. Flying point noise makes it impossible for the depth camera to effectively obtain the depth information of the edge of the object, causing errors in depth imaging and interfering with point cloud processing and measurement. Summary of the invention

[0005] In view of the defects in the prior art, an object of the present invention is to provide a flying spot removal system based on phase shift and line shift fusion.

[0006] The flying spot removal system based on phase shift and line shift fusion provided by the present invention comprises:

[0007] a first phase value generating unit, configured to obtain a phase-shift structured light pattern and determine a first phase value of each pixel in the camera unit according to the phase-shift structured light pattern;

[0008] a second phase value generating unit, configured to obtain a line-shifted structured light pattern, and determine a second phase value corresponding to each pixel in the camera unit according to the line-shifted structured light pattern;

[0009] The phase fusion unit is used to fuse the first phase value and the second phase value corresponding to each pixel to generate a target phase value of the pixel, and then generate a point cloud image with flying spots removed.

[0010] Preferably, the first phase value generating unit includes the following units:

[0011] A first light projection control unit, used for controlling the projector to project phase-shifted structured light toward the target object;

[0012] A first camera control unit, used for controlling a camera unit to collect a first phase-shift structured light pattern formed by reflection of the first phase-shift structured light by a target object;

[0013] The first phase value determining unit is configured to determine a first phase value of each pixel in the camera unit according to the first phase-shifted structured light pattern.

[0014] Preferably, the second phase value generating unit includes the following units:

[0015] A second light projection control unit controls the projector to project line-shifted structured light toward the target object, wherein the line-shifted stripe light beams in the line-shifted structured light are encoded by Gray code;

[0016] a second camera control unit, controlling the camera unit to collect a line-shifted structured light pattern formed by reflection of the line-shifted structured light by the target object;

[0017] A second phase value determining unit is configured to determine a second phase value of each pixel in the camera unit according to the line-shifted structured light pattern.

[0018] Preferably, the phase fusion unit includes the following units:

[0019] an invalid value exclusion unit, configured to determine whether one of the first phase value and the second phase value is an invalid value, and when either the first phase value or the second phase value is an invalid value, determine that the target phase value corresponding to the pixel is an invalid value;

[0020] A phase difference value generating unit, configured to generate an absolute value of a difference between the first phase value and the second phase value;

[0021] The target phase value determination unit is used to determine whether the absolute value of the difference is less than a preset difference threshold. When the absolute value of the difference is less than the preset difference threshold, the target phase value corresponding to the pixel is determined to be a first phase value; otherwise, the phase value corresponding to the pixel is determined to be an invalid value.

[0022] Preferably, the phase fusion unit includes the following units:

[0023] an invalid value exclusion unit, configured to determine whether one of the first phase value and the second phase value is an invalid value, and when either the first phase value or the second phase value is an invalid value, determine that the target phase value corresponding to the pixel is an invalid value;

[0024] a phase difference value generating unit, configured to generate an absolute value of a difference between the first phase value and the second phase value, and determine whether the absolute value of the difference is less than a preset difference threshold;

[0025] The target phase value determining unit is used to determine that the target phase value corresponding to the pixel is the first phase value when the absolute value of the difference is less than a preset difference threshold, otherwise, the target phase value corresponding to the pixel is determined to be half of the sum of the first phase value and the second phase value.

[0026] Preferably, the stripe widths of the phase-shifted structured light pattern and the line-shifted structured light pattern in a single period are the same;

[0027] The stripe grayscale value of the phase-shifted structured light pattern changes sinusoidally periodically, and the proportion of light stripes in the line-shifted structured light pattern is 1 / n of the stripe width, where n is a natural number greater than 2.

[0028] Preferably, it also includes:

[0029] A pixel coordinate generating unit, used for calculating the corresponding pixel coordinate value on the projector through the target phase value and the calibration information;

[0030] A depth information generating unit, configured to generate depth information of each pixel according to two pixel coordinate values ​​matched between the camera unit and the projector;

[0031] The point cloud reconstruction unit is used to perform three-dimensional reconstruction according to the depth information of each pixel to generate the point cloud image.

[0032] Preferably, n is any natural number from 5 to 10.

[0033] Preferably, the calibration information is generated when the depth camera is pre-calibrated.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] The present invention projects a phase-shifted structured light pattern and a line-shifted structured light pattern through a projector to generate a first phase value and a second phase value, fuses the first phase value and the second phase value corresponding to each pixel to generate a target phase value of the pixel, and then performs depth reconstruction according to the target phase value to generate a point cloud image, which can effectively remove flying spots, reduce the error of depth camera imaging, and reduce the interference of flying spots on point cloud processing and measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0037] Figure 1 It is a module schematic diagram of a flying spot removal system based on phase shift and line shift fusion in an embodiment of the present invention;

[0038] Figure 2 Schematic diagram of a module of a first phase value generating unit in an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of a module of a second coordinate value generating unit in an embodiment of the present invention;

[0040] Figure 4 This is a module schematic diagram of a phase fusion unit in an embodiment of the present invention;

[0041] Figure 5 It is a module schematic diagram of a flying spot removal system based on phase shift and line shift fusion in a modified example of the present invention;

[0042] Figure 6 Schematic diagram of a phase-shifted structured light pattern and a line-shifted structured light pattern in an embodiment of the present invention;

[0043] Figure 7 It is a flowchart of the steps of the flying spot removal method based on phase shift and line shift fusion in an embodiment of the present invention;

[0044] Figure 8 is a structural schematic diagram of a flying spot removal device based on phase shift and line shift fusion in an embodiment of the present invention; and

[0045] Fig. 9 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0047] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0048] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0049] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0050] Figure 1 FIG. 1 is a module schematic diagram of a flying spot removal system based on phase shift and line shift fusion in an embodiment of the present invention. Figure 1 As shown, the flying spot removal system based on phase shift and line shift fusion provided by the present invention is used for a depth camera based on the structured light principle, wherein the depth camera includes a projector and a camera unit, including the following modules:

[0051] a first phase generating module, configured to obtain a phase-shift structured light pattern and determine a first phase value of each pixel in the camera unit according to the phase-shift structured light pattern;

[0052] a second phase generating module, configured to obtain a line-shifted structured light pattern, and determine a second phase value corresponding to each pixel in the camera unit according to the line-shifted structured light pattern;

[0053] The point cloud generation module is used to fuse the first phase value and the second phase value corresponding to each pixel to generate a target phase value of the pixel, and then generate a point cloud image with flying spots removed.

[0054] In an embodiment of the present invention, the projector is a DLP projector, the projector is used to project phase-shift structured light and line-shift structured light, and the camera unit is used to collect a phase-shift structured light pattern formed by the phase-shift structured light on the target object, and collect a line-shift structured light pattern formed by the line-shift structured light on the target object;

[0055] The processor module is used to control the synchronous switching of the projector and the camera unit, and to generate a point cloud image or a depth image of the target object according to the phase-shifted structured light pattern.

[0056] Figure 2 Schematic diagram of a module of a first phase value generating unit in an embodiment of the present invention. Figure 3 As shown, the first phase value generating unit includes the following units:

[0057] A first light projection control unit, used for controlling the projector to project phase-shifted structured light toward the target object;

[0058] A first camera control unit, used for controlling a camera unit to collect a first phase-shift structured light pattern formed by reflection of the first phase-shift structured light by a target object;

[0059] The first phase value determining unit is configured to determine a first phase value of each pixel in the camera unit according to the first phase-shifted structured light pattern.

[0060] In an embodiment of the present invention, a phase-shifted structured light is projected in a direction orthogonal to the baseline, and for each camera pixel, an absolute phase value is calculated and can be mapped to the X coordinate of the projector pixel. Based on the iterative line-plane intersection reconstruction method, the 3D space point corresponding to the camera pixel can be calculated.

[0061] In an embodiment of the present invention, when calculating the first coordinate value, it is necessary to obtain in advance the internal and external parameters of the projector and the internal and external parameters of the camera unit generated by calibrating the depth camera, and calculate and generate the first coordinate value based on the absolute phase value and the internal and external parameters of the projector and the camera unit.

[0062] Figure 3 FIG. 1 is a module schematic diagram of a flying spot removal system based on phase shift and line shift fusion in an embodiment of the present invention. Figure 3 As shown, the second phase value generating unit includes the following units:

[0063] A second light projection control unit controls the projector to project line-shifted structured light toward the target object, wherein the line-shifted stripe light beams in the line-shifted structured light are encoded by Gray code;

[0064] a second camera control unit, controlling the camera unit to collect a line-shifted structured light pattern formed by reflection of the line-shifted structured light by the target object;

[0065] A second phase value determining unit is configured to determine a second phase value of each pixel in the camera unit according to the line-shifted structured light pattern.

[0066] In the embodiment of the present invention, the first phase value and the second phase value are absolute phase values.

[0067] Figure 4 FIG. 1 is a schematic diagram of a module of a phase fusion unit in an embodiment of the present invention. Figure 4 As shown, the phase fusion unit includes the following units:

[0068] an invalid value exclusion unit, configured to determine whether one of the first phase value and the second phase value is an invalid value, and when either the first phase value or the second phase value is an invalid value, determine that the target phase value corresponding to the pixel is an invalid value;

[0069] A phase difference value generating unit, configured to generate an absolute value of a difference between the first phase value and the second phase value;

[0070] The target phase value determination unit is used to determine whether the absolute value of the difference is less than a preset difference threshold. When the absolute value of the difference is less than the preset difference threshold, the target phase value corresponding to the pixel is determined to be a first phase value; otherwise, the phase value corresponding to the pixel is determined to be an invalid value.

[0071] In a variation of the present invention, the phase fusion unit includes the following units:

[0072] an invalid value exclusion unit, configured to determine whether one of the first phase value and the second phase value is an invalid value, and when either the first phase value or the second phase value is an invalid value, determine that the target phase value corresponding to the pixel is an invalid value;

[0073] a phase difference value generating unit, configured to generate an absolute value of a difference between the first phase value and the second phase value, and determine whether the absolute value of the difference is less than a preset difference threshold;

[0074] The target phase value determining unit is used to determine that the target phase value corresponding to the pixel is the first phase value when the absolute value of the difference is less than a preset difference threshold, otherwise, the target phase value corresponding to the pixel is determined to be half of the sum of the first phase value and the second phase value.

[0075] Table 1

[0076]

[0077] As shown in Table 1, the fusion method in the embodiment and the variation of the present invention can effectively improve the imaging accuracy of the depth camera.

[0078] Figure 6Schematic diagram of a phase-shifted structured light pattern and a line-shifted structured light pattern in an embodiment of the present invention. Figure 6 As shown, (a) is a phase-shifted structured light pattern, and (b) is a line-shifted structured light pattern, and the stripe widths of the phase-shifted structured light pattern and the line-shifted structured light pattern in a single period are the same;

[0079] The stripe grayscale value of the phase-shifted structured light pattern changes sinusoidally periodically, and the proportion of light stripes in the line-shifted structured light pattern is 1 / n of the stripe width, where n is a natural number greater than 2.

[0080] In the embodiment of the present invention, n is any natural number from 5 to 10, preferably 8. The stripe width is the overall width of the black stripes and the white stripes in a single period.

[0081] Figure 5 FIG. 1 is a schematic diagram of a module of a flying spot removal system based on phase shift and line shift fusion in a modified example of the present invention. Figure 5 As shown, in the embodiment of the present invention, when generating a point cloud image, the processor module further includes:

[0082] A pixel coordinate generating unit, used for calculating the corresponding pixel coordinate value on the projector through the target phase value and the calibration information;

[0083] A depth information generating unit, configured to generate depth information of each pixel according to two pixel coordinate values ​​matched between the camera unit and the projector;

[0084] The point cloud reconstruction unit is used to perform three-dimensional reconstruction according to the depth information of each pixel to generate the point cloud image.

[0085] In an embodiment of the present invention, the calibration information is calculated and generated when calibrating the depth camera. When performing depth reconstruction, a triangulation method is used. The calibration information includes the camera's intrinsic parameters, extrinsic parameters, and distortion coefficients.

[0086] Figure 7 FIG. 1 is a flowchart of the steps of a flying spot removal method based on phase shift and line shift fusion in an embodiment of the present invention; Figure 7 As shown, the flying spot removal method based on phase shift and line shift fusion provided by the present invention is used for a depth camera based on the structured light principle, wherein the depth camera includes a projector and a camera unit, and includes the following steps:

[0087] Step S1: acquiring a phase-shift structured light pattern, and determining a first phase value of each pixel in a camera unit according to the phase-shift structured light pattern;

[0088] Step S2: acquiring a line-shifted structured light pattern, and determining a second phase value corresponding to each pixel in the camera unit according to the line-shifted structured light pattern;

[0089] Step S3: The first phase value and the second phase value corresponding to each pixel are fused to generate a target phase value of the pixel, thereby generating a point cloud image with flying spots removed.

[0090] The present invention also provides a flying spot removal device based on phase shift and line shift fusion, comprising a processor and a memory. The memory stores executable instructions of the processor. The processor is configured to execute the flying spot removal system steps based on phase shift and line shift fusion by executing the executable instructions.

[0091] As described above, in this embodiment, a phase-shifted structured light pattern and a line-shifted structured light pattern are projected respectively by a projector to generate a first phase value and a second phase value, and the first phase value and the second phase value corresponding to each pixel are fused to generate a target phase value of the pixel. Then, depth reconstruction is performed according to the target phase value to generate a point cloud image, which can effectively remove flying spots, reduce the error of depth camera imaging, and reduce the interference of flying spots on point cloud processing and measurement.

[0092] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0093] Figure 8 Schematic diagram of the structure of the flying spot removal device based on phase shift and line shift fusion in the embodiment of the present invention. Figure 8 The electronic device 600 according to this embodiment of the present invention is described. Figure 8 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0094] like Figure 8 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0095] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of various exemplary embodiments of the present invention described in the flying spot removal system based on phase shift and line shift fusion in the above description of the present invention. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0096] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0097] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0098] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0099] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, cameras, depth cameras, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Figure 8 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0100] In an embodiment of the present invention, a computer-readable storage medium is further provided for storing a program, and the steps of the flying spot removal system based on phase shift and line shift fusion are implemented when the program is executed. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the flying spot removal system based on phase shift and line shift fusion in the above-mentioned part of this specification.

[0101] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, a phase-shifted structured light pattern and a line-shifted structured light pattern are projected respectively through a projector to generate a first phase value and a second phase value, and the first phase value and the second phase value corresponding to each pixel are fused to generate a target phase value of the pixel, and then depth reconstruction is performed according to the target phase value to generate a point cloud image, which can effectively remove flying spots, reduce the error of depth camera imaging, and reduce the interference of flying spots on point cloud processing and measurement.

[0102] Fig. 9 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Fig. 9 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0103] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0104] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0105] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0106] In the embodiment of the present invention, a phase-shifted structured light pattern and a line-shifted structured light pattern are projected respectively by a projector to generate a first phase value and a second phase value, and the first phase value and the second phase value corresponding to each pixel are fused to generate a target phase value of the pixel. Then, depth reconstruction is performed according to the target phase value to generate a point cloud image, which can effectively remove flying spots, reduce the error of depth camera imaging, and reduce the interference of flying spots on point cloud processing and measurement.

[0107] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0108] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A flying spot removal system based on phase shift and line shift fusion, characterized in that: include: a first phase value generating unit, configured to obtain a phase-shift structured light pattern and determine a first phase value of each pixel in the camera unit according to the phase-shift structured light pattern; a second phase value generating unit, configured to obtain a line-shifted structured light pattern, and determine a second phase value corresponding to each pixel in the camera unit according to the line-shifted structured light pattern; The phase fusion unit is used to fuse the first phase value and the second phase value corresponding to each pixel to generate a target phase value of the pixel, and then generate a point cloud image with flying spots removed.

2. The flying spot removal system based on phase shift and line shift fusion according to claim 1, characterized in that: The first phase value generating unit includes the following units: A first light projection control unit, used for controlling the projector to project phase-shifted structured light toward the target object; A first camera control unit, used for controlling a camera unit to collect a first phase-shift structured light pattern formed by reflection of the first phase-shift structured light by a target object; The first phase value determining unit is configured to determine a first phase value of each pixel in the camera unit according to the first phase-shifted structured light pattern.

3. The flying spot removal system based on phase shift and line shift fusion according to claim 1, characterized in that: The second phase value generating unit includes the following units: A second light projection control unit controls the projector to project line-shifted structured light toward the target object, wherein the line-shifted stripe light beams in the line-shifted structured light are encoded by Gray code; a second camera control unit, controlling the camera unit to collect a line-shifted structured light pattern formed by reflection of the line-shifted structured light by the target object; A second phase value determining unit is configured to determine a second phase value of each pixel in the camera unit according to the line-shifted structured light pattern.

4. The flying spot removal system based on phase shift and line shift fusion according to claim 1, characterized in that: The phase fusion unit includes the following units: an invalid value exclusion unit, configured to determine whether one of the first phase value and the second phase value is an invalid value, and when either the first phase value or the second phase value is an invalid value, determine that the target phase value corresponding to the pixel is an invalid value; A phase difference value generating unit, configured to generate an absolute value of a difference between the first phase value and the second phase value; The target phase value determination unit is used to determine whether the absolute value of the difference is less than a preset difference threshold. When the absolute value of the difference is less than the preset difference threshold, the target phase value corresponding to the pixel is determined to be a first phase value; otherwise, the phase value corresponding to the pixel is determined to be an invalid value.

5. The flying spot removal system based on phase shift and line shift fusion according to claim 2, characterized in that: The phase fusion unit includes the following units: an invalid value exclusion unit, configured to determine whether one of the first phase value and the second phase value is an invalid value, and when either the first phase value or the second phase value is an invalid value, determine that the target phase value corresponding to the pixel is an invalid value; a phase difference value generating unit, configured to generate an absolute value of a difference between the first phase value and the second phase value, and determine whether the absolute value of the difference is less than a preset difference threshold; The target phase value determining unit is used to determine that the target phase value corresponding to the pixel is the first phase value when the absolute value of the difference is less than a preset difference threshold, otherwise, the target phase value corresponding to the pixel is determined to be half of the sum of the first phase value and the second phase value.

6. The flying spot removal system based on phase shift and line shift fusion according to claim 1, characterized in that: The stripe widths of the phase-shifted structured light pattern and the line-shifted structured light pattern in a single period are the same; The stripe grayscale value of the phase-shifted structured light pattern changes sinusoidally periodically, and the proportion of light stripes in the line-shifted structured light pattern is 1 / n of the stripe width, where n is a natural number greater than 2.

7. The flying spot removal system based on phase shift and line shift fusion according to claim 5, characterized in that: Also includes: A pixel coordinate generating unit, used for calculating the corresponding pixel coordinate value on the projector through the target phase value and the calibration information; A depth information generating unit, configured to generate depth information of each pixel according to two pixel coordinate values ​​matched between the camera unit and the projector; The point cloud reconstruction unit is used to perform three-dimensional reconstruction according to the depth information of each pixel to generate the point cloud image.

8. The flying spot removal system based on phase shift and line shift fusion according to claim 6, characterized in that: n is any natural number from 5 to 10.

9. The flying spot removal system based on phase shift and line shift fusion according to claim 7, characterized in that: The calibration information is generated when the depth camera is pre-calibrated.

Images