Anti-interference method and device based on depth camera

Abstract

The present application relates to the technical field of mobile robots, and in particular to an anti-interference method and device based on a depth camera. The method comprises: S100, when or before exposure is started, detecting whether external light exists; and if there is external light, switching the modulation frequency, wherein the external light comprises optical signals of other depth cameras; S200, when exposure is started, transmitting an optical signal to an object to be measured, wherein the optical signal comprises a modulation signal of the current modulation frequency; and receiving in real time a reflected signal reflected by said object; S300, demodulating the received reflected signal, and determining whether the received reflected signal is valid data; and S400, if the received reflected signal is valid data, acquiring the distance of said object on the basis of the received valid data. The present application has the beneficial effects that the interference of interference optical signals on reflected optical signals is effectively inhibited, and the accuracy of a measurement result of the depth camera is improved.

Description

An anti-interference method and device based on depth camera

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on December 27, 2023, with application number 202311828811.6 and invention name “Anti-interference method and device based on depth camera”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of mobile robot technology, and in particular to an anti-interference method and device based on a depth camera. Background Art

[0004] In the field of mobile robots, machines need to perform functions such as obstacle avoidance, docking, and navigation. With the promotion of Industry 4.0, unmanned factories have become mainstream. The use of perception sensors to achieve these functions has become inevitable. Among the current mainstream solutions on the market, structured light and time-of-flight (TOF) perception solutions dominate. Structured light's ranging is limited to a baseline, with a range of 3 meters being the limit, and accuracy degrades significantly with increasing distance. Improving accuracy requires increasing the baseline, which makes the structured light camera larger and imposes significant limitations on vehicle mounting orientation. TOF perception solutions, on the other hand, have a range of up to 10 meters without significant accuracy degradation. They also offer the advantages of better timeliness, high frequency, and 3D imaging. Unmanned factories inevitably involve the operation of over a hundred vehicles. TOF cameras typically operate at a wavelength of 940 nanometers. When multiple TOF cameras are operating together, multi-machine interface (MII) can affect the performance of obstacle avoidance, docking, and navigation functions, leading to misinterpretation of commands by the robots and posing safety risks. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present application provides an anti-interference method and device based on a depth camera, which solves the technical problem that when two or more robots work together, there is interference between the cameras, causing the robots to misjudge the instructions and create safety hazards.

[0007] (2) Technical solution

[0008] In order to achieve the above objectives, the main technical solutions adopted in this application include:

[0009] In a first aspect, an embodiment of the present application provides an anti-interference method based on a depth camera, comprising:

[0010] S100, when starting exposure or before starting exposure, detecting whether there is external light in real time;

[0011] If there is external light, the modulation frequency of the transmitted light signal is switched;

[0012] S200, when starting exposure, transmitting a light signal to the object under test; the light signal includes: a modulation signal of the current modulation frequency; and receiving a reflection signal reflected by the object under test in real time;

[0013] S300, demodulating the received reflected signal to generate a demodulated signal, and determining whether the received reflected signal is valid data;

[0014] S400 : When the received reflected signal is valid data, obtain the distance of the measured object according to the received valid data.

[0015] Optionally, the reflected signal and the demodulated signal have the same modulation frequency;

[0016] The demodulated signal r(t) is obtained by formula 1, which is: ω=2πf;

[0017] Where A is the modulation amplitude, ω is the angular velocity, is the signal phase, f is the modulation frequency; t is the interval between receiving the reflected signal and transmitting the modulated signal;

[0018] The modulation signal c(t) is obtained by formula 2, which is: c(t)=cos(ωt);

[0019] Where ω is the angular velocity and t is the interval between receiving the reflected signal and transmitting the modulated signal.

[0020] Optionally, the S300 includes:

[0021] S300-1. Demodulate the received reflected signal to generate a demodulated signal;

[0022] S300-2, performing cross-correlation processing on the demodulated signal and the modulated signal to generate a result signal;

[0023] S300-3. Determine whether the result signal is valid data according to the result signal.

[0024] Optionally, the result signal is obtained by formula 3, which is:

[0025] Where τ is the delay phase of the cross-correlation between the modulation signal and the demodulation signal, T is the signal period, A is the modulation amplitude, ω is the angular velocity, is the signal phase, t is the interval between receiving the reflected signal and transmitting the modulated signal;

[0026] When the modulation frequencies of the modulated signal and the demodulated signal are different, S(t) is 0, and the modulated signal is an interference signal and is discarded.

[0027] Optionally, before the depth camera starts exposure includes: a moment Tus before the camera exposure moment.

[0028] Optionally, the depth processing module switching the modulation frequency includes:

[0029] The difference between the modulation frequency after switching and the modulation frequency before switching is less than 5 MHz.

[0030] Optionally, the depth camera uses a 4-shutter principle to perform depth measurement. In this case, the shutter angles are 0°, 90°, 180°, and 270°, respectively, and intensity values ​​of four exposures of the depth camera are obtained respectively.

[0031] In a second aspect, an embodiment of the present application provides an anti-interference device based on a depth camera, comprising:

[0032] A detection unit, configured to detect whether there is external light when the depth camera starts to expose or before the depth camera starts to expose; if there is external light, send a detection signal to the interference processing module;

[0033] an interference processing unit, configured to receive the detection signal, determine whether to switch the modulation frequency, and send a signal to the depth processing unit; if the modulation frequency is to be switched, the difference between the modulation frequency before switching and the modulation frequency after switching is less than 5 MHz;

[0034] A transmitting unit, configured to transmit a laser light source toward the object being measured; the laser light source includes a modulation signal of a current modulation frequency;

[0035] A receiving unit, used for receiving the reflection signal of the laser spot on the object to be measured;

[0036] The depth processing unit is used to determine whether to switch the modulation frequency of the modulation signal based on the signal sent by the interference processing unit; when the receiving unit receives the reflected signal on the object under test, it obtains and demodulates the reflected signal, filters the valid data, and outputs an image based on the valid data.

[0037] Optionally, the detection unit includes a PD device, an APD device, or a SIPM device;

[0038] The receiving unit includes a CCD or CMOS chip;

[0039] The interference processing unit includes a programmable chip; the programmable chip includes: CPLD or FPGA.

[0040] Optionally, the modulation signal includes a plurality of light pulses with the same pulse interval and varying modulation amplitude; the wavelength of the light pulses includes infrared light.

[0041] (3) Beneficial effects

[0042] The beneficial effects of the present application are: an anti-interference method and device based on a depth camera of the present application, by judging whether there is external light and judging whether to switch the modulation frequency to avoid interference, can effectively suppress the interference of the interference light signal on the reflected light signal, improve the accuracy of the depth camera detection results, and effectively expand the technical field of mobile robot-to-platform collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] FIG1 is a flow chart of an anti-interference method based on a depth camera;

[0044] FIG2 is a structural diagram of a depth camera in an anti-interference method and apparatus based on a depth camera;

[0045] FIG3 is a schematic diagram of a detection unit in an anti-interference method and device based on a depth camera;

[0046] FIG4 is a flow chart of an interference processing unit detection method and device in an anti-interference method based on a depth camera. DETAILED DESCRIPTION

[0047] In order to better explain the present application and facilitate understanding, the present application is described in detail below through specific implementation methods in conjunction with the accompanying drawings.

[0048] The embodiments of the present application propose an anti-interference method and device based on a depth camera, which avoids interference by determining whether there is external light and whether to switch the modulation frequency.

[0049] To better understand the above technical solutions, exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a clearer and more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0050] Example 1

[0051] This embodiment provides an anti-interference method based on a depth camera, as shown in FIG1 , including:

[0052] S100, when exposure starts or before exposure starts, where before exposure starts includes the moment Tus before the camera exposure moment, detecting whether there is external light (external light can be sensed by APD);

[0053] S150: If no external light is detected, the device will emit light normally;

[0054] If external light is detected, the detected pulse waveform needs to be spectrally converted (Fourier transform) to obtain frequency information. If the number of frequency peaks in the frequency information is greater than 1, it is determined that external light has interfered with the camera, and the modulation frequency is switched to emit light.

[0055] If there is no external light interfering with the camera or there is no external light, steps S100 and S150 are repeated;

[0056] S200, when exposure begins, emitting a light signal toward the object being measured; (the wavelength of the emitted light signal is generally 940 nanometers, and other infrared light may also be used) the light signal including: a modulation signal of the current modulation frequency; and receiving a reflection signal reflected by the object being measured in real time;

[0057] S300-1. Demodulate the received reflected signal to generate a demodulated signal;

[0058] S300-2, performing correlation processing on the demodulated signal and the modulated signal to generate a result signal;

[0059] S300-3, judging whether the result signal is valid data;

[0060] S400: Obtain the distance of the measured object and output an image based on the received valid data.

[0061] Optionally, the anti-interference method based on the depth camera mentioned in this embodiment is not limited to the depth camera based on the iTOF principle, but is also applicable to dTOF and structured light cameras.

[0062] Taking a CW modulation and demodulation-based depth camera as an example, a CW depth camera typically uses four exposures to calculate depth values, where the depth value represents the distance between the subject and the camera. Each pixel has a corresponding depth value, which represents the distance from the camera. Smaller depth values ​​indicate closer objects, while larger depth values ​​indicate farther objects. Depth values ​​are calculated using Formula 4:

[0063] Among them, Q1, Q2, Q3, and Q4 are the intensity values ​​of the four camera exposures, C is the speed of light, and f is the modulation frequency of the depth camera.

[0064] A continuous light wave, or light signal, is emitted toward the scene of interest (the light signal includes a modulated signal at the current modulation frequency). The signal reflects off the scene and returns to the camera. A lens system focuses the signal onto a sensor within the camera, which then records the reflected signal.

[0065] The transmitted light signal is modulated in amplitude and phase, the latter depending on the 3D structure of the observed scene, as the phase shift depends on the distance the signal traveled before reaching the sensor. The reflected light signal is then correlated (compared) with the modulated signal emitted directly from the camera (the correlation signal). This process is called cross-correlation. The two signals will have the same modulation frequency, while the received input signal will be phase-shifted. Using a sinusoidal signal and ignoring nonlinear effects, the demodulated signal r(t) is given by Equation 1: ω=2πf;

[0066] Where A is the modulation amplitude, ω is the angular velocity, is the signal phase, f is the modulation frequency of the TOF camera; t is the interval between receiving the reflected signal and transmitting the modulated signal.

[0067] The modulation signal c(t) is obtained by formula 2, which is: c(t)=cos(ωt);

[0068] Where ω is the angular velocity and t is the interval between receiving the reflected signal and transmitting the modulated signal.

[0069] The result of the modulated signal and the demodulated signal is a cross-correlation function, and the result signal is obtained by formula 3, which is:

[0070] Among them, τ is the delay phase of the cross-correlation between the adjustment signal and the demodulation signal, T is the signal period, A is the modulation amplitude, ω is the angular velocity, is the signal phase, t is the interval between receiving the reflected signal and transmitting the modulated signal;

[0071] Formula 3 can be simplified to Formula 5, which is:

[0072] By changing the result of ωt, that is, the angle, multiple results can be obtained. TOF cameras generally use the 4-shutter principle (4-shutter principle: the principle of depth measurement by using 4 clock phases in the depth camera. This principle is based on the time of flight principle and calculates the depth of the object by measuring the time it takes for light to be emitted from the camera to reflect on the object and then return to the camera. Specifically, when performing depth measurement, the TOF camera will emit 4 laser pulses in sequence, and each pulse has a different phase. When the laser pulse hits the object, part of the light will be reflected back by the object and received by the detector at the receiving end of the TOF camera. By processing and calculating the light signals of these 4 phases, the depth information of the object can be obtained. This 4-shutter principle can improve the measurement accuracy and anti-interference ability of the depth camera), that is, ωt is equal to 0°, 90°, 180°, and 270° respectively, and the intensity values ​​obtained by the camera are Q1, Q2, Q3, and Q4 respectively. The depth value is calculated using formula 4.

[0073] When cameras with the same modulation frequency in a scene receive each other's light intensity, the signal-to-noise ratio will decrease and the distance measurement will fluctuate. If their modulation frequencies are different, the expression of the interference signal is formula 6, which is:

[0074] Substitute it into formula 4, that is

[0075] That is, the demodulated signal will only respond to the modulation frequency it transmits.

[0076] In a TOF camera, a series of corrections are required before true distance measurement can be achieved. For example, if the modulation frequency used by the camera is f, when the detection unit (CPLD) detects external light interference, the modulation frequency is switched to f1. Where: |f-f1| < 5MHz;

[0077] Among them, f1 has not been corrected by the system, but the difference between f1 and f is small enough, and the accuracy of the ranging performance of the two is similar, which is acceptable in the field of mobile robots.

[0078] This embodiment provides an anti-interference method based on a depth camera. By determining whether there is external light and whether to switch the modulation frequency to avoid interference, it can effectively suppress the interference of interfering light signals on reflected light signals, improve the accuracy of depth camera detection results, and effectively expand the technical field of mobile robot-to-platform collaboration.

[0079] Example 2

[0080] This embodiment provides an anti-interference device based on a TOF camera, as shown in FIG2 , comprising: a detection unit, a transmitting unit, a receiving unit, a depth processing unit, and an interference processing unit.

[0081] The detection unit is used to detect whether there is external light (the external light includes other depth camera light signals) when the depth camera starts to expose or before the exposure starts (before the exposure starts, the Tus time is pre-set) by the PD device; if there is external light, the detection signal is sent to the interference processing module. It can use PD (Photo Diode) devices or APD (Avalanche Photo Diode) or SIPM (Silicon photomultiplier), and different devices are selected according to the actual scenario. The principle of the devices is to convert external light signals into voltage signals, but their gains are different. (Gain refers to the amplification factor between the input signal and the output signal. In photodiodes, avalanche photodiodes and silicon photomultipliers, gain can be understood as the degree to which the input light signal is amplified. A higher gain means that the input light signal can be amplified to a larger amplitude, thereby improving the detection sensitivity and measurement accuracy of the signal.) Taking APD as an example, the principle of the detection signal of the detection unit is shown in Figure 3. The signal detected by the APD is amplified by the operational amplifier and outputs a pulse signal of 0 or 1 through the comparator.

[0082] The interference processing unit is used to receive the detection signal and determine whether to turn on the multi-machine working mode, that is, whether to switch the modulation frequency, and send a signal to the depth processing unit. When there is external light, the multi-machine function is turned on and a signal is sent to the depth processing unit to switch the modulation frequency to a nearby frequency. It generally uses CPLD (Complex Programmable Logic Device) or other programmable chips such as FPGA, and is selected according to actual hardware conditions. The entire detection process is shown in Figure 4. After the depth camera emits a light signal, it can also repeatedly detect whether interference occurs through the interference processing unit. When interference occurs, the depth processing unit can also be notified to switch the modulation frequency. The transmitting unit is used to emit a laser light source to the object to be measured; the laser light source includes a modulation signal of the current modulation frequency.

[0083] The receiving unit is used to receive the reflected signal of the laser spot on the object being measured. It can use a CCD or CMOS chip. It can use an image sensor composed of a charge coupled device (CCD), complementary metal oxide semiconductor (CMOS), single photon avalanche diode (SPAD), etc.

[0084] The depth processing unit is used to determine whether to switch the modulation frequency of the modulation signal based on the signal sent by the interference processing unit; the difference between the modulation frequency before switching and the modulation frequency after switching is less than 5MHz; when the receiving unit receives the reflected signal on the object under test, it obtains and demodulates the reflected signal, filters the valid data, and outputs an image based on the valid data.

[0085] For example, when the TOF depth camera starts to expose or in advance of the Tus time, the detection unit's detection PD detects whether there is external light;

[0086] If there is no external light, the depth camera will emit light normally;

[0087] If external light is detected, the detected pulse waveform needs to be sent to a programmable chip such as CPLD or FPGA in the interference processing unit for spectrum conversion, namely Fourier transform, to obtain frequency information. The frequency with a larger peak can be found in the spectrum. When the number of peaks found is greater than 1, it is determined that external light has interfered with the camera. The interference processing unit then sends a signal to the depth processing unit to switch the modulation frequency of the transmitted modulation signal.

[0088] If there is no external light or no external light interferes with the camera, repeat the above steps.

[0089] After the camera emits light, the transmitting unit transmits a light signal to the object being measured; the light signal includes: a modulation signal of the current modulation frequency; and receives a reflection signal reflected by the object being measured in real time;

[0090] After receiving the reflected signal, the receiving unit sends it to the deep processing unit, which demodulates the received reflected signal to generate a demodulated signal and determines whether the received reflected signal is valid data;

[0091] When the received reflected signal is valid data, the distance of the measured object is obtained according to the received valid data, and an image is generated.

[0092] This embodiment provides an anti-interference device based on a depth camera. By determining whether there is external light and whether to switch the modulation frequency to avoid interference, it can effectively suppress the interference of interference light signals on reflected light signals, improve the accuracy of depth camera detection results, and effectively expand the technical field of mobile robot-to-platform collaboration.

[0093] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0094] In this application, unless otherwise specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0095] In this application, unless otherwise expressly specified or limited, when a first feature is “on” or “below” a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above”, or “above” a second feature, it may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is “below”, “below”, or “below” a second feature, it may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0096] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0097] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An anti-interference method based on a depth camera, characterized in that, Including: S100. When starting to expose or before starting to expose, detect whether there is external light in real time; If there is external light, switch the modulation frequency of the emitted optical signal; S200. When starting to expose, emit an optical signal to the object to be measured; the optical signal includes: a modulation signal with the current modulation frequency; and receive the reflected signal reflected by the object to be measured in real time; S300. Demodulate the received reflected signal to generate a demodulated signal, and determine whether the received reflected signal is valid data; S400. When the received reflected signal is valid data, obtain the distance of the object to be measured according to the received valid data.

2. The anti-interference method based on a depth camera according to claim 1, characterized in that The reflected signal and the demodulated signal have the same modulation frequency; The demodulated signal r(t) is obtained through Formula 1, and Formula 1 is: ω = 2πf; where A is the modulation amplitude and ω is the angular velocity, is the signal phase, f is the modulation frequency; t is the time interval between receiving the reflected signal and emitting the modulation signal; The modulation signal c(t) is obtained through Formula 2, and Formula 2 is: c(t) = cos(ωt); where ω is the angular velocity and t is the time interval between receiving the reflected signal and emitting the modulation signal.

3. The anti-interference method based on a depth camera according to claim 2, characterized in that, The S300 includes: S300-1. Demodulate the received reflected signal to generate a demodulated signal; S300-2. Perform cross-correlation processing on the demodulated signal and the modulation signal to generate a result signal; S300-3. Judge whether it is valid data according to the result signal.

4. The anti-interference method based on a depth camera according to claim 3, characterized in that The result signal is obtained through Formula 3, and the Formula 3 is as follows: Among them, τ is the delay phase of the cross-correlation between the modulation signal and the demodulation signal, T is the signal period, A is the modulation amplitude, and ω is the angular velocity. is the signal phase, t is the time interval between receiving the reflected signal and emitting the modulation signal; When the modulation frequencies of the modulation signal and the demodulated signal are different, if S(t) is 0, then the modulation signal is an interference signal and is discarded.

5. The anti-interference method based on a depth camera according to claim 1, wherein Before the depth camera starts to expose, it includes: the moment Tus time before the camera exposure moment.

6. The anti-interference method based on a depth camera according to claim 1, wherein The depth processing module switching the modulation frequency includes: The difference between the switched modulation frequency and the modulation frequency before switching is less than 5 MHz.

7. The anti-interference method based on a depth camera according to claim 1, characterized in that The depth camera uses the 4-shutter principle for depth measurement. At this time, the shutter angles are 0°, 90°, 180°, and 270° respectively, and the intensity values of the four exposures of the depth camera are obtained respectively.

8. An anti-interference device based on a depth camera, characterized in that, Including: A detection unit for detecting whether there is external light when the depth camera starts to expose or before starting to expose; If there is external light, send a detection signal to the interference processing module; An interference processing unit for receiving the detection signal, judging whether to switch the modulation frequency, and sending a signal to the depth processing unit; if the modulation frequency needs to be switched, the difference between the modulation frequency before switching and the modulation frequency after switching is less than 5 MHz; An emission unit for emitting a laser light source to the object to be measured; the laser light source includes a modulation signal with the current modulation frequency; A receiving unit for receiving the reflected signal of the laser spot on the object to be measured; A depth processing unit, configured to determine whether to switch the modulation frequency of a modulation signal according to a signal sent by a received interference processing unit; after the receiving unit receives a reflected signal on the object to be measured, acquire and demodulate the reflected signal, screen valid data, and output an image according to the valid data.

9. The anti-interference device based on a depth camera according to claim 8, wherein the detection unit includes a PD device or an APD device or a SIPM device; the receiving unit includes a CCD or a CMOS chip; the interference processing unit includes a programmable chip; the programmable chip includes: a CPLD or an FPGA.

10. The anti-interference device based on a depth camera according to claim 8, wherein The modulation signal includes a plurality of optical pulses with the same pulse interval and varying modulation amplitude; the wavelength of the optical pulses includes infrared light.

Images