TOF camera anti-interference implementation method and device based on time division

Abstract

The invention relates to the technical field of distance measurement of a depth camera of a mobile robot, in particular to a TOF camera anti-interference implementation method and device based on time division, and the method comprises the steps: detecting whether there is external light or not through a light signal detection module when the TOF camera starts to expose or before the TOF camera starts to expose; if external light is detected, the TOF camera does not emit light; if no external light is detected, the TOF camera emits light; after light emitting, detecting whether interference occurs or not through an interference signal processing module, and repeating the execution until no interference exists; and outputting the image. According to the method, the interference of the interference light signal on the reflected light signal can be effectively suppressed, so that the accuracy of the distance measurement result of the depth camera is improved, and the application scene and the application range of multi-robot cooperation of the mobile robot are effectively expanded.

Description

Technical Field

[0001] The present invention relates to the technical field of depth camera ranging for mobile robots, and particularly to an anti-interference implementation method and device for a TOF camera based on time sharing. Background Art

[0002] In the prior art, there are many methods for depth camera ranging. This patent relates to the ranging principle of the ToF ranging method. The full name of ToF is Time-of—Flight, that is, the time of flight. The ToF ranging method is to emit light pulses (generally invisible light) onto the observed object, and then receive the light pulses reflected back from the object, and calculate the distance between the measured object and the camera by detecting the flight (round-trip) time of the light pulses. In the ToF technology, the technology that directly measures the light flight time is called dToF (direct-TOF); the technology that periodically modulates the emitted light signal, measures the phase delay of the reflected light signal relative to the emitted light signal, and then calculates the flight time from the phase delay is called iToF (Indirect-TOF) technology.

[0003] According to the different modulation and demodulation type methods, it can be divided into continuous wave (CW) modulation and demodulation methods and pulse (Pulse Modulated, PM) modulation and demodulation methods. In the field of mobile robots, depth cameras are used for identification, obstacle avoidance, navigation, etc., and have the advantages of good timeliness, high frame rate, and three-dimensional imaging. At the same time, there are also defects. In the field of mobile robots, there will inevitably be two or more machines working together, and there will be interference problems between cameras. Since the measurement device based on the iToF technology needs to emit light actively, when multiple iToF devices work simultaneously at a relatively short distance. The modulated light of other cameras is received by this camera and parsed to calculate an incorrect depth value, which will cause the robot to misjudge the command, thus posing a safety hazard. Among them, Zhu Liang et al. in "An Anti-Interference Distance Measurement Method and Depth Camera" segmented the light signal by changing the modulation frequency of the light signal and keeping the pulse interval unchanged, which can effectively suppress the interference of the interference light signal on the reflected light signal, and improve the accuracy of the depth camera detection result. However, this method uses a frequency division method and needs to constantly change the frequency. It is relatively complex in software implementation and will reduce the frame rate. Summary of the Invention

[0004] (1) Technical Problems to be Solved

[0005] In view of the above disadvantages and deficiencies of the prior art, the present invention provides an anti-interference implementation method and device for a time-division TOF camera, which solves the technical problem that when two or more mobile robots work together, there is interference between cameras, causing the mobile robot to misjudge instructions and posing a safety hazard.

[0006] (II) Technical Solution

[0007] To achieve the above object, the main technical solutions adopted by the present invention include:

[0008] In the first aspect, an embodiment of the present invention provides an anti-interference implementation method for a time-division TOF camera, including:

[0009] S1: When the TOF camera starts to expose or before starting to expose, detect whether there is external light through the light signal detection module;

[0010] S2: If external light is detected, the TOF camera of the present invention does not emit light;

[0011] If no external light is detected, the TOF camera of the present invention emits light;

[0012] S3: After the TOF camera of the present invention emits light, detect whether interference occurs through the interference signal processing module, and repeatedly execute steps S2 and S3 until there is no interference;

[0013] S4: Output an image.

[0014] The anti-interference implementation method for a time-division TOF camera proposed by the embodiment of the present invention senses external light through the light signal detection module, thereby separating it from its own light emission. It can effectively suppress the interference of interference light signals on reflected light signals, improve the accuracy of the detection results of the depth camera, and effectively expand the application scenarios and scope of application of multi-robot cooperation.

[0015] Optionally, in step S3, detecting whether interference occurs through the interference signal processing module includes the following steps:

[0016] The interference signal processing module detects whether multiple taps in a frame overlap. If overlap occurs, it is determined that there is interference;

[0017] If interference is detected again, the interference frame is retained, and then steps S2 and S3 are executed until multiple taps in a frame do not overlap, and it is determined that there is no interference.

[0018] Optionally, step S2 further includes: when external light is detected, detect whether there is external light again after delaying for a duration T until the preset detection times or detection duration are reached, and then execute step S3.

[0019] Optionally, step S4 further includes:

[0020] When the anti-multi-camera function of the interference signal processing module is enabled, the camera receiving end is configured in the trigger mode; when the anti-multi-camera function is not enabled, the camera receiving end is configured in the continuous mode;

[0021] When the camera receiving end is in the trigger mode: after the camera receives an external trigger command, it starts to expose for a predetermined duration, and outputs a frame of image after the exposure ends;

[0022] When the camera receiving end is in the continuous mode: the camera automatically and continuously exposes for a predetermined duration and continuously outputs images.

[0023] Optionally, in step S1, before starting the exposure, it includes: at a moment Tus time earlier than the moment of starting the exposure.

[0024] Optionally, the number of multiple Taps is 4; each exposure process includes four exposures, and outputting a frame of image also includes outputting a depth value; the TOF camera uses four exposures to calculate the depth value, and the calculation formula is as follows:

[0025]

[0026] Among them, Q1, Q2, Q3, and Q4 are the intensity values of the four exposures of the TOF camera, C is the speed of light, 3×10 8 m / s; F is the modulation frequency of the TOF camera.

[0027] Optionally, the calculation formula for the number N of cameras of the method tolerance is as follows:

[0028] The frame rate of the depth camera is x fps, the single exposure time is Texp, the exposure interval time is Ta, and the interfering cameras are controlled within Tb, then the calculation formula for Tb is as follows:

[0029]

[0030] Then the value range of N:

[0031]

[0032] Among them, N is a positive integer of 0 or 1, 2, 3... n.

[0033] In a second aspect, an embodiment of the present invention provides a time-sharing-based TOF camera anti-interference implementation device, including:

[0034] An optical signal detection module, configured to detect whether there is external light when the TOF camera starts to expose or before starting to expose;

[0035] The camera transmitting end is used for not emitting light when the light signal detection module detects that there is external light; and emitting light when the light signal detection module detects that there is no external light;

[0036] The interference signal processing module is used to detect whether interference occurs after the camera transmitter of the machine emits light. When interference occurs, the light detection signal module and the camera transmitter are instructed to repeatedly perform the process of detecting external light, emitting light and detecting interference until there is no interference.

[0037] The data processing module is used to output the image when there is no interference.

[0038] Optionally, the device further comprises:

[0039] The camera receiving end, the trigger signal end and the exposure time end of the camera receiving end are connected to the interference signal processing module;

[0040] The interference signal processing module is also used to turn on or off the anti-multi-machine function; when the anti-multi-machine function is turned on, the camera receiving end is configured to a trigger mode, which is used to start the exposure process according to the agreed duration after the camera receives an external trigger command, and output a frame of image after the exposure is completed; when the anti-multi-machine function is turned off, the camera receiving end is configured to a continuous mode, which is used to enable the camera to automatically and continuously perform multiple exposure processes according to the agreed duration and continuously output multiple frames of images.

[0041] Optionally, each exposure process includes four exposures, and outputting a frame of image also includes outputting a depth value.

[0042] (III) Beneficial effects

[0043] The beneficial effects of the present invention are as follows: the anti-interference implementation method and device of the TOF camera based on time-sharing of the present invention, the anti-interference distance measurement method of which senses external light through the detection light signal module, thereby separating it from the self-luminescence. It can effectively suppress the interference of the interference light signal on the reflected light signal, so that the accuracy of the depth camera detection result can be improved, and the application scenarios and scope of application of multiple mobile robots in collaboration can be effectively expanded. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a structural schematic diagram of a time-sharing based TOF camera anti-interference implementation device according to a preferred embodiment of the present invention;

[0045] Figure 2 A schematic diagram of a detection signal of a detection optical signal module in a preferred embodiment of the present invention;

[0046] Figure 3 A waveform diagram of a preferred embodiment of the present invention when there is no interference during the interference detection process;

[0047] Figure 4Waveform schematic diagram when there is interference in the interference detection process of the preferred embodiment of the present invention;

[0048] Figure 5 Flow schematic diagram of the anti-interference implementation method of the TOF camera based on time sharing in the preferred embodiment of the present invention;

[0049] Figure 6 Flow schematic diagram of the detection of the interference signal processing module in the preferred embodiment of the present invention.

[0050]

Explanation of reference numerals

[0051] 10. Interference signal processing module; 11. Data processing module; 12. Laser receiving end; 13. Detection optical signal module; 14. Camera transmitting end. Detailed implementation manners

[0052] In order to better explain the present invention for easier understanding, the present invention will be described in detail below with reference to the accompanying drawings through specific implementation manners.

[0053] The anti-interference implementation method and device of the TOF camera based on time sharing proposed in the embodiment of the present invention senses external light through the detection optical signal module to separate it from its own light emission. Compared with the prior art, it solves the technical problem that when two or more mobile robots work together, there is interference between cameras, causing the mobile robot to misjudge commands and posing a safety hazard. It can effectively suppress the interference of the interference optical signal on the reflected optical signal, improve the accuracy of the depth camera detection result, and effectively expand the application scenarios and scope of application of multi-robot cooperation of mobile robots.

[0054] In order to better understand the above technical solution, the exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0055] See Figure 1 , Figure 1 is the structural schematic diagram of the anti-interference implementation device of the TOF camera based on time sharing in the embodiment of the present invention. The embodiment of the present invention provides an anti-interference implementation device of the TOF camera based on time sharing, including a detection optical signal module 13, a camera transmitting end 14, an interference signal processing module 10, a camera receiving end 12, and a data processing module 11. Among them:

[0056] The optical signal detection module 13 is used to detect whether there is external light when the TOF camera starts to expose or before starting to expose. In this embodiment, the optical signal detection module 13 can use a PD (Photo Diode) device, or an APD (Avalanche Photo Diode) or a SIPM (Silicon photomultiplier). Different devices can be selected according to the actual scenario. The principle of the devices is to convert the external optical signal into a voltage signal, but the gains of the devices are different. The principle of the detection signal of the optical signal detection module 13 is as Figure 2 shown. In this embodiment, taking the APD as an example, the signal detected by the APD passes through an operational amplifier and then is sent to a comparator to output a pulse of 0 or 1.

[0057] The camera transmitter 14 (laser emission module) is used to not emit light when the optical signal detection module detects external light; and emit light when the optical signal detection module detects no external light.

[0058] The interference signal processing module 10 is used to detect whether interference occurs after the camera transmitter 14 of the local machine emits light. When there is interference, it instructs the optical signal detection module 13 and the camera transmitter 14 to repeatedly execute the processes of detecting external light, emitting light, and detecting interference until there is no interference. In implementation, the interference signal processing module 10 is generally a programmable chip such as a CPLD (Complex Programmable Logic Device) or an FPGA. It can be selected according to the actual hardware conditions. The detection process of the interference signal processing module 10 is as Figure 6 shown.

[0059] The camera receiver 12, its trigger signal terminal SYNC and exposure time terminal exp are connected to the interference signal processing module 10. In this embodiment, the interference signal processing module 10 has the function of whether to enable the anti-multi-camera function. The anti-multi-camera function can be controlled to be enabled or disabled through software. When the anti-multi-camera function is enabled, the camera receiver 12 is configured in a trigger mode, and is used to start the exposure process according to the agreed duration after the camera receives an external trigger command, and output a frame of image after the exposure ends; when the anti-multi-camera function is disabled, the camera receiver 12 is configured in a continuous mode, and is used to make the camera automatically and continuously perform multiple exposure processes according to the agreed duration and continuously output multiple frames of images. In implementation, the camera receiver 12 can be an image sensor composed of a charge-coupled device (CCD), a complementary metal oxide semiconductor (CMOS), a single-photon avalanche diode (SPAD), etc.

[0060] The data processing module is used to output an image when there is no interference.

[0061] On the other hand, Figure 5Schematic flow chart of the anti-interference implementation method of the time-sharing TOF camera in the embodiments of the present invention. The anti-interference implementation method of the time-sharing TOF camera in the embodiments of the present invention includes the following steps:

[0062] S1: When the TOF camera starts to expose or before starting to expose, detect whether there is external light through the optical signal detection module. During implementation, before starting to expose includes: at a moment Tus in advance of the moment of starting to expose.

[0063] S2: If external light is detected, the TOF camera does not emit light; if no external light is detected, the TOF camera emits light.

[0064] S3: After the TOF camera emits light, detect whether interference occurs through the interference signal processing module, and repeatedly execute steps S2 and S3 until there is no interference.

[0065] This embodiment is not limited to depth cameras based on the iTOF principle. It is also applicable to dToF and structured light cameras. This embodiment is described by taking a depth camera based on the CW modulation and demodulation principle as an example. Refer to Figure 3 , during the implementation of this embodiment, each exposure process includes four exposures, and each frame of image includes 4 Taps; outputting a frame of image also includes outputting a depth value. The CW depth camera generally uses four exposures to calculate the depth value, and the calculation formula is as follows:

[0066]

[0067] Among them, Q1, Q2, Q3, and Q4 are the intensity values of the four exposures of the TOF camera, C is the speed of light, 3×10 8 m / s; F is the modulation frequency of the TOF camera.

[0068] Refer to Figure 3 and Figure 4 , during the implementation of this embodiment, detecting whether interference occurs through the interference signal processing module includes the following steps:

[0069] The interference signal processing module detects whether multiple Taps (shutters, or exposures) within a frame overlap. If overlap occurs, it is determined that there is interference; if interference is detected again, the interference frame is retained, and then steps S2 and S3 are executed until there is no overlap among multiple Taps within a frame, and it is determined that there is no interference.

[0070] The interference detection process is as shown in Figure 3As shown in the figure. The rising edge of the trigger signal SYNC is aligned with the rising edge of EXP, indicating that the rising edge trigger is valid. One rising edge module 12 will output one frame of image. In this depth camera, one frame is the four exposures of Q1, Q2, Q3, and Q4. The width of one pulse is the exposure duration Texp. Q1, Q2, Q3, and Q4 are the exposure continuous states of the image sensor. Ta is the data readout time after exposure, and Tb is the readout time of the last exposure and the frame rest time. At the moment of the exposure rising edge, detect whether there is light in the APD signal in advance by T, that is, determine whether the APD module is triggered.

[0071] When it is detected that there is external light, detect whether there is external light again after a delay of T duration until the preset detection times or detection duration are reached, and then execute step S3. The T duration can be adjusted according to the actual situation. If the defined T time is too large, the avoidance effect cannot be achieved. If T is defined too small, missed detections will occur. This time needs to be comprehensively considered according to the usage method of the camera. For example, the camera may use high and low integration ranging, such as 1 ms for high integration and 0.1 ms for low integration. At this time, the maximum value of T should be defined as the time of low integration. That is, detect whether there is light entering the PD 100 us in advance at the SYNC rising edge. If there is light entering, then enter step S2. The CPLD needs to control the camera receiving end 12 not to output an image at this moment and output the image after a delay of t time.

[0072] In case of interference Figure 4 As shown in the figure, the two cameras expose separately. If the exposure times overlap in time, the waveform of the APD will become the waveform related to the camera exposure time, as shown by the dotted line in the figure. The delayed time t is the duration of the fourth falling edge of the APD waveform, that is, the interfering camera is controlled within the Tb interval to ensure that N cameras do not interfere with each other. The calculation formula for the tolerance camera number N for this method is as follows:

[0073] The frame rate of the depth camera is x fps, the single exposure time is Texp, and the exposure interval time is Ta. If the interfering camera is controlled within Tb, the calculation formula for Tb is as follows:

[0074]

[0075] Then the value range of N:

[0076]

[0077] where N is a positive integer of 0 or 1, 2, 3... n.

[0078] S4: Output the image.

[0079] See Figure 6, in this embodiment, when the interference signal processing module enables the anti-multi-camera function, the camera receiving end is configured in the trigger mode; when the anti-multi-camera function is not enabled, the camera receiving end is configured in the continuous mode; when the camera receiving end is in the trigger mode: after the camera receives an external trigger command, it starts to expose for a predefined duration, and outputs an image after the exposure ends; when the camera receiving end is in the continuous mode: the camera automatically and continuously exposes for a predefined duration and continuously outputs images (steam image output mode).

[0080] The anti-interference implementation method of the TOF camera based on time sharing proposed in the embodiment of the present invention senses external light through the APD detecting optical signal module, so as to separate it from its own light emission. It can effectively suppress the interference of the interference optical signal on the reflected optical signal, improve the accuracy of the depth camera detection result, and effectively expand the application scenarios and applicable scope of multi-robot cooperation. The present invention uses a time-sharing method and does not require repeated modification of the frequency, and can be conveniently and simply implemented through software.

[0081] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0082] In the present invention, unless otherwise clearly defined and limited, the terms "installed", "connected", "connected", "fixed" and other terms should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0083] In the present invention, unless otherwise clearly defined and limited, the first feature being "on" or "under" the second feature can be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on" the second feature can be that the first feature is directly above or obliquely above the second feature, or merely indicates that the first feature has a higher horizontal height than the second feature. The first feature being "under", "beneath" and "under" the second feature can be that the first feature is directly below or obliquely below the second feature, or merely indicates that the first feature has a lower horizontal height than the second feature.

[0084] In the description of this specification, the descriptions of terms such as "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0085] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for anti-interference implementation of a time-division TOF camera, characterized in that, it includes: S1: When the TOF camera starts to expose or before starting to expose, detect whether there is external light through the light signal detection module; S2: If external light is detected, the TOF camera does not emit light; If no external light is detected, the TOF camera emits light; S3: After the TOF camera emits light, detect whether interference occurs through the interference signal processing module, and repeatedly execute steps S2 and S3 until there is no interference; S4: Output an image.

2. The method for anti-interference implementation of a time-division TOF camera according to claim 1, characterized in that, in the step S3, detecting whether interference occurs through the interference signal processing module includes the following steps: The interference signal processing module detects whether multiple Taps within one frame overlap. If overlap occurs, it is determined that there is interference; If interference is detected again, retain the interference frame, and then execute steps S2 and S3 until multiple Taps within one frame do not overlap, and it is determined that there is no interference.

3. The method for anti-interference implementation of a time-division TOF camera according to claim 1, characterized in that, the step S2 further includes: when external light is detected, detect whether there is external light again after a delay of T duration until the preset detection times or detection duration is reached, and then execute step S3.

4. The method for anti-interference implementation of a time-division TOF camera according to claim 2, characterized in that, the step S4 further includes: when the anti-multi-camera function of the interference signal processing module is enabled, configure the camera receiving end as the trigger mode; when the anti-multi-camera function is not enabled, configure the camera receiving end as the continuous mode; when the camera receiving end is in the trigger mode: after the camera receives an external trigger command, start to expose according to the agreed duration, and output one frame of image after the exposure ends; when the camera receiving end is in the continuous mode: the camera automatically and continuously exposes according to the agreed duration and continuously outputs images.

5. The method for anti-interference implementation of a time-division TOF camera according to claim 1, characterized in that, in the step S1, before starting to expose includes: at the moment T us earlier than the moment of starting to expose.

6. The method for anti-interference implementation of a time-division TOF camera according to claim 4, characterized in that, the number of the multiple Taps is 4; each exposure process includes four exposures, and the output of one frame of image further includes outputting a depth value; the TOF camera uses four exposures to calculate the depth value, and the calculation formula is as follows: Among them, Q1, Q2, Q3, and Q4 are the intensity values of four exposures of the TOF camera, C is the speed of light, 3×10 8 m / s; F is the modulation frequency of the TOF camera.

7. The method for anti-interference implementation of a time-division TOF camera according to claim 6, characterized in that, the calculation formula for the number N of cameras tolerated by the method is as follows: The frame rate of the depth camera is x fps, the single exposure time is Texp, the exposure interval time is Ta, and the interfering camera is controlled within Tb, then the calculation formula for Tb is as follows: Then the value range of N: where N is a positive integer of 0 or 1, 2, 3... n, and Tb is the readout time and frame rest time of the last exposure.

8. A device for anti-interference implementation of a time-division TOF camera, characterized in that, it includes: A light signal detection module is used to detect whether there is external light when the TOF camera starts to expose or before it starts to expose; The camera transmitting end is used for not emitting light when the light signal detection module detects that there is external light; and emitting light when the light signal detection module detects that there is no external light; The interference signal processing module is used to detect whether interference occurs after the camera transmitter of the machine emits light. When interference occurs, the light detection signal module and the camera transmitter are instructed to repeatedly perform the process of detecting external light, emitting light and detecting interference until there is no interference. The data processing module is used to output the image when there is no interference.

9. The time-sharing based TOF camera anti-interference implementation device as claimed in claim 8, It is characterized in that Also includes: A camera receiving end, wherein a trigger signal end and an exposure time end of the camera receiving end are connected to a module for processing interference signals; The interference signal processing module is also used to turn on or off the anti-multi-machine function; when the anti-multi-machine function is turned on, the camera receiving end is configured to a trigger mode, which is used to start an exposure process according to an agreed duration after the camera receives an external trigger command, and output a frame of image after the exposure is completed; when the anti-multi-machine function is turned off, the camera receiving end is configured to a continuous mode, which is used to enable the camera to automatically and continuously perform multiple exposure processes according to the agreed duration and continuously output multiple frames of images.

10. The time-sharing based TOF camera anti-interference implementation device as claimed in claim 7, It is characterized in that Each exposure process includes four exposures, and outputting a frame of image also includes outputting a depth value.

Images