An anti-interference distance measurement method and a depth camera

Abstract

The present application relates to an anti-interference distance measurement method and a depth camera, the method comprising: a first depth camera receiving a reflected light signal reflected by a measured object in real time, and detecting a light signal of other depth cameras in real time / during exposure of the reflected light signal; if the light signal of other depth cameras is detected, determining whether the light signal interferes with valid data of the reflected light signal received by the first depth camera; the valid data is data read out by the first depth camera during exposure of the received reflected light signal; the detector receiving the transmitted light signal and the detector detecting the light signal of other depth cameras in the first depth camera are independent of each other; if there is interference, a light signal acquisition strategy is used to acquire valid data of the reflected light signal without interference; and the distance of the measured object is obtained according to the acquired valid data without interference. The above method can improve the accuracy of the detection result of the depth camera.

Description

Technical Field

[0001] This invention relates to the field of signal anti-interference technology, and in particular to an anti-interference distance measurement method and a depth camera. Background Technology

[0002] There are many methods for depth camera ranging in existing technologies. Existing Time-of-Flight (ToF) ranging methods work by emitting light pulses / signals (usually invisible light) onto the object being observed, then receiving the light signal reflected back from the object, and calculating the distance between the object and the camera by detecting the round-trip time of the light signal. In ToF technology, the technique that directly measures the time of flight of light is called dToF (direct-TOF); the technique that periodically modulates the emitted light signal, measures the phase delay of the reflected light signal relative to the emitted light signal, and then calculates the time of flight from the phase delay is called iToF (indirect-TOF).

[0003] Based on the modulation and demodulation methods, depth cameras can be divided into continuous wave (CW) modulation and demodulation methods and pulse modulated (PM) modulation and demodulation methods. In the field of mobile robotics, depth cameras are used for recognition, obstacle avoidance, and navigation, offering advantages such as good timeliness, high frame rate, and 3D imaging. However, they also have drawbacks. In mobile robotics, when two or more machines work together, interference can occur between the cameras. Because iToF-based measurement devices need to actively emit light, when multiple iToF devices work simultaneously at close range, the modulated light from other cameras can be received and interpreted by the current camera, leading to incorrect depth values. This can cause the robot to misinterpret commands, creating safety hazards. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an interference-resistant distance measurement method and a depth camera.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0008] In a first aspect, embodiments of the present invention provide an interference-resistant distance measurement method, the distance measurement method being used in a depth camera of an AGV, the method comprising:

[0009] The first depth camera receives the reflected light signal from the object being measured in real time and detects the light signals from other depth cameras in real time / detects the light signals from other depth cameras during the exposure of the reflected light signal;

[0010] If the light signals from other depth cameras are detected, it is determined whether the light signals interfere with the valid data of the reflected light signals received by the first depth camera; the valid data is the data read out by the first depth camera after exposing the received reflected light signals; the detector in the first depth camera that receives the emitted light signals and the detector that detects the light signals from other depth cameras are independent of each other;

[0011] If interference is present, an optical signal acquisition strategy is used to acquire valid data of the interference-free reflected optical signal;

[0012] The distance to the object being measured is obtained based on the collected, interference-free, and valid data.

[0013] Optionally, if light signals from other depth cameras are detected, it is determined whether the light signals interfere with the valid data of the reflected light signals received by the first depth camera, including:

[0014] The first depth camera determines whether the time point at which it detects the light signals from other depth cameras falls within the exposure time period of the received reflected light signals;

[0015] If so, then interference is confirmed.

[0016] Optionally, an optical signal acquisition strategy is used to acquire effective data of the interference-free reflected optical signal, including:

[0017] Discard the valid data of the current reflected light signal / current frame.

[0018] Alternatively, obtain the delay time period t, and after delaying by t, re-receive the reflected light signal reflected by the object under test;

[0019] Alternatively, discard the valid data of the current reflected light signal, obtain the delay time period t, and re-receive the reflected light signal reflected by the object under test after the delay time t.

[0020] The t satisfies the following formula:

[0021]

[0022]

[0023] N is a positive integer, shutter1 represents the single exposure time, Ta represents the exposure interval time, i.e. the data readout time after exposure, x is the frame rate of the depth camera, in fps; Tb is an intermediate parameter or the last readout time in a frame and the inter-frame rest time.

[0024] Optionally, based on the collected, interference-free, and valid data, the distance to the measured object is obtained, including:

[0025] The distance to the object being measured is obtained using the following formula;

[0026]

[0027] d is the distance between the depth camera and the object being measured;

[0028] C is the speed of light, and F is the modulation frequency of the depth camera, measured in MHz.

[0029] Q1, Q2, Q3, and Q4 are all valid data of interference-free reflected light signals.

[0030] Secondly, embodiments of the present invention also provide a depth camera applied to an AGV, the depth camera comprising:

[0031] Laser emission module, first detector, second detector, and processor;

[0032] The laser light-emitting module is used to emit light signals;

[0033] The first detector is used to receive the reflected light signal reflected by the object being measured in real time;

[0034] The second detector is used to receive light signals from other depth cameras in real time;

[0035] The processor is electrically connected to the laser emission module, the first detector, and the second detector. It is used to determine whether the light signal detected by the second detector from other depth cameras interferes with the effective data of the reflected light signal received by the first depth camera if interference is present. If interference exists, a light signal acquisition strategy is adopted to acquire the effective data of the reflected light signal without interference. Based on the acquired effective data of the reflected light signal without interference, the distance of the object being measured is obtained.

[0036] The effective data of the reflected light signal is the data read out by the processor after exposing the received reflected light signal;

[0037] The first detector and the second detector are independent of each other, and the distances in both the vertical and horizontal directions meet the preset distance thresholds.

[0038] Optionally, the depth camera is: a dToF depth camera, an iTOF depth camera, or a structured light camera;

[0039] Both the first detector and the second detector are photodiode detectors;

[0040] The processor includes an image sensor electrically connected to the first detector.

[0041] Optionally, the processor is specifically used to determine whether the time point at which the light signal from other depth cameras is detected falls within the exposure time period of the reflected light signal from the object being measured.

[0042] If so, discard the current valid data of the reflected light signal.

[0043] Alternatively, obtain the delay time period t, and after delaying by t, re-receive the reflected light signal reflected by the object under test;

[0044] Alternatively, discard the valid data of the current reflected light signal, obtain the delay time period t, and re-receive the reflected light signal reflected by the object under test after the delay time t.

[0045] The t satisfies the following formula:

[0046]

[0047]

[0048] N is a positive integer, shutter1 represents the single exposure time, Ta represents the exposure interval time, i.e. the data readout time after exposure, x is the frame rate of the depth camera, in fps; Tb is an intermediate parameter or the last readout time in a frame and the inter-frame rest time.

[0049] Optionally, the processor is further configured to obtain the distance to the object being measured according to the following formula;

[0050]

[0051] d is the distance between the depth camera and the object being measured;

[0052] C is the speed of light, and F is the modulation frequency of the depth camera, measured in MHz.

[0053] Q1, Q2, Q3, and Q4 are all valid data of reflected light signals at different time points without interference.

[0054] Optionally, the laser light-emitting module is specifically used for

[0055] An optical signal is emitted toward the object being measured. The frame signal of the optical signal includes multiple pulse signals with the same pulse interval and varying modulation frequency.

[0056] Thirdly, embodiments of the present invention also provide an AGV, which includes: a support body, on which a depth camera as described in any of the second aspects is fixed.

[0057] (III) Beneficial Effects

[0058] The method of this invention uses a separate detector to sense external light signals, thereby separating them from its own emission. At the same time, the separate detector detects the light signals reflected by the object being measured. The identification of the detected light signals can effectively suppress interference from interfering light signals on the reflected light signals, thereby improving the accuracy of the depth camera detection results and effectively expanding the application scenarios of mobile robots. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the structure of a depth camera provided in an embodiment of the present invention;

[0060] Figure 2 This is a flowchart illustrating an anti-interference distance measurement method according to an embodiment of the present invention.

[0061] Figure 3 This is a schematic diagram illustrating interference in the reflected light signal.

[0062] Figure 4 This is a schematic diagram illustrating the interference-free situation in the reflected light signal and the avoidance processing during its acquisition.

[0063] Figure 5 This is a schematic diagram of adjacent frames. Detailed Implementation

[0064] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] In the following embodiments, some embodiments use "light pulse" and some embodiments use "light signal". Light pulse is a form of light signal.

[0066] The exposure period of the light signal can be understood as the process by which the image sensor accumulates the light signal (i.e., during the light emission operation period).

[0067] Example 1

[0068] like Figure 1 As shown, this embodiment of the invention provides an anti-interference distance measurement method. The execution subject of this method can be a depth camera, and the distance measurement method of this embodiment can be used in the depth camera of an AGV. Its specific implementation method includes the following steps:

[0069] S101, The first depth camera receives the reflected light signal from the object being measured in real time and detects the light signals from other depth cameras in real time.

[0070] In other embodiments, the step may also be: the first depth camera receives the reflected light signal reflected by the object being measured in real time, and detects the light signals of other depth cameras during the exposure of the reflected light signal; here the detector for detecting the light signals of other depth cameras and the detector for detecting the reflected light signal are different, so the light signals of other depth cameras can be detected based on the exposure time information of the reflected light signal.

[0071] S102. If the light signals of other depth cameras are detected, determine whether the light signals interfere with the valid data of the reflected light signals received by the first depth camera; if there is interference, proceed to step S103, otherwise repeat step S101.

[0072] In this step, the effective data of the reflected light signal can be the data read out by the first depth camera after exposing the received reflected light signal; the detector in the first depth camera that receives the emitted light signal and the detector that detects the light signals of other depth cameras are independent of each other.

[0073] S103. If interference exists, an optical signal acquisition strategy shall be adopted to acquire effective data of interference-free reflected optical signals.

[0074] The optical signal acquisition strategy in this step may include: discarding the current frame, delaying for a period of time t, discarding the current frame and delaying for a period of time t, etc., which can be configured according to actual needs. This embodiment is only an example.

[0075] S104. The first depth camera obtains the distance to the object being measured based on the effective data of the interference-free reflected light signal collected.

[0076] The method in this embodiment uses a separate detector to sense external light signals, thereby separating them from the light emitted by the device itself. At the same time, the separate detector detects the light signals reflected by the object being measured. The identification of the detected light signals can effectively suppress the interference of interfering light signals on the reflected light signals, thereby improving the accuracy of the depth camera detection results and effectively expanding the application scenarios of mobile robots.

[0077] To better understand the implementation process of the above method, a detailed explanation is provided below. In the specific implementation process, this embodiment provides a detailed description of each step, but this is not a limiting description. The method of this embodiment may include the following steps:

[0078] A201. After the first depth camera is started, its laser emission module emits a light signal to the object being measured. The wavelength of the light signal can be 940nm, or other bands of infrared light.

[0079] The first depth camera emits light signals to the object being measured. The frame signal of the light signal may include multiple pulse signals with the same pulse interval and varying modulation frequency.

[0080] A202. The first depth camera receives the reflected light signal from the object being measured in real time and detects the light signals from other depth cameras in real time. If no light is detected, repeat step A202. If light is detected, proceed to step A203.

[0081] A203. If the light signal of other depth cameras is detected, the first depth camera determines whether the time point at which the light signal of other depth cameras is detected falls within the exposure time period of the reflected light signal reflected by the object being measured.

[0082] If yes, then interference is confirmed; otherwise, there is no interference, and step A202 is repeated.

[0083] A204. If interference exists, the first depth camera shall use an optical signal acquisition strategy to acquire effective data of interference-free reflected light signals.

[0084] For example, the first depth camera acquires a time delay period t, and after a delay of t, re-receives the light signal reflected by the object being measured;

[0085] The t satisfies the following formula:

[0086]

[0087]

[0088] N is a positive integer, shutter1 represents the single exposure time, Ta represents the exposure interval time, i.e. the data readout time after exposure, x is the frame rate of the depth camera, in fps; Tb is an intermediate parameter or the last readout time in a frame and the inter-frame rest time.

[0089] A205. Based on the effective data of the interference-free reflected light signal collected, the distance of the measured object is obtained using the following formula.

[0090]

[0091] d is the distance between the depth camera and the object being measured;

[0092] C is the speed of light, and F is the modulation frequency of the depth camera, measured in MHz.

[0093] Q1, Q2, Q3, and Q4 are all valid data of interference-free reflected light signals.

[0094] The method in this embodiment, combined with specific formulas, provides a concrete step for identifying the detected light signal. This effectively suppresses interference from interfering light signals on the reflected light signal, thereby improving the accuracy of the depth camera detection results.

[0095] Example 2

[0096] This invention provides a depth camera, such as Figure 1 As shown, the depth camera in this embodiment includes: a laser emission module 11, a first detector 12, a second detector 15, and a processor;

[0097] The processor may include: an image sensor 16 and a data processing module 10; the first detector 12 is electrically connected to the image sensor 16 and the data processing module 10, the second detector 15 is electrically connected to the data processing module 10, and the laser emission module is electrically connected to the data processing module 10.

[0098] Both the first detector 12 and the second detector 15 can be PDs (Photo Diode Detectors). In a specific implementation, the second detector 15 needs to be wirelessly close to the first detector 12 in both the horizontal and vertical directions, that is, the distance in the horizontal and vertical directions needs to meet a preset threshold.

[0099] The image sensor 16 in the processor can also be a single or combined image sensor such as a charge-coupled device (CCD), complementary metal-oxide semiconductor (CMOS), avalanche diode (AD), or single-photon avalanche diode (SPAD). This embodiment does not limit it and the sensor can be selected according to actual needs.

[0100] There is a certain distance between the depth camera and the object being measured 14. Typically, multiple depth cameras are placed on an AGV. As a result, there is light signal interference between adjacent depth cameras, leading to inaccurate distance measurements.

[0101] After the laser emission module 11 emits a light signal to the object being measured, both the first detector 12 and the second detector 15 detect the light signal. The first detector 12 detects the reflected light signal from the object being measured, while the second detector 15 detects the reflected light signal from other depth cameras after it has been reflected by the object being measured. Figure 1 As shown, the wavelengths of the light signals emitted by other depth cameras are exactly the same as those emitted by the current depth camera, which will cause interference to the current depth camera.

[0102] In this embodiment, any depth camera on the forklift / AGV can emit light signals of any wavelength, but the light signals emitted by adjacent depth cameras that cause interference are of the same wavelength.

[0103] When the processor detects a light signal on the second detector 15, it determines whether it interferes with the reflected light signal used by the current depth camera to calculate distance. If interference is detected, the processor discards this frame of data or delays acquisition. Otherwise, it does not perform any processing and acquires the next frame of light signal / light pulse.

[0104] The aforementioned depth camera can be an iTOF-based depth camera. It is also applicable to dToF and structured light cameras. This embodiment uses a CW modulation / demodulation principle depth camera as an example to explain the method. CW depth cameras typically use four exposures to calculate the depth value. The calculation formula is:

[0105]

[0106] C is the speed of light, 3*10^8 m / s; F is the modulation frequency of the depth camera, in MHz.

[0107] Q1, Q2, Q3, and Q4 are the intensity values ​​of the reflected light signals from four uninterrupted exposures by the camera. These intensity values ​​represent the valid data of the reflected light signals.

[0108] like Figure 3 or Figure 4 As shown, Q1, Q2, Q3, and Q4 represent the continuous exposure states of the image sensor. Figure 3 In the diagram, Ta represents the data readout time after exposure, and Tb represents the readout time of the last exposure in the current frame and the frame rest time. It's understandable that there's a frame rest time between adjacent frames, corresponding to the sensor's rest time. Figure 5 As shown.

[0109] from Figure 3 and Figure 4 It can be directly concluded that the first and second detectors operate in parallel. The operating status of the second detector 15 is detected during the exposure period of the image sensor 16; the result of 15 is either 0 or 1, where 0 represents light and 1 represents no light. Figure 3 The results of the second detector 15, i.e., PD detection, indicate that external light (pulse light emitted by other cameras or light signals with the same wavelength as 11 in the working environment) appears during the exposure of the image sensor 16. The collected data will be interfered with, and the depth data d calculated by substituting the data into formula (1) will be lost or deviate greatly from the true value.

[0110] exist Figure 4 The topmost randomly delayed light signal represents a state unaffected by the light signal detected by the second detector. In this state, the time point with light detected by the second detector 15 is within Tb, which does not affect the image sensor 16 in acquiring data during exposure. Therefore, no operation is required in this state. Figure 4The reflected light signal detected by the first detector in the middle layer and the light signal detected by the second detector below interfere with each other. Figure 3 The state, that is, the exposure period of Q1 corresponding to the signal detected by PD.

[0111] In practice, when interference is detected, the processing of interference depth data frames can be done in the following ways: if the user does not need the interference frame, the current frame is discarded immediately upon detection of interference data. If the user needs to use part or all of the data in the interference frame, the optical signal is collected again after a delay.

[0112] Figure 4 The top part shows a timing diagram that can avoid interference; when the first detector 12 is detected to have a response at the same time as the exposure, the processor controls the image sensor 16 to randomly delay for a period of time t, and t is calculated as follows:

[0113] The depth camera has a frame rate of x fps, a single exposure time of shutter1 (in seconds / milliseconds), and an exposure interval of Ta. The formula for calculating Tb is as follows:

[0114]

[0115] The range of values ​​for the delay time t is as follows:

[0116]

[0117] Where N is an integer, which can be 0, 1, 2, 3, ... Data can be collected with a random delay within this range.

[0118] That is, if the processor detects light signals from other depth cameras on the second detector, it determines whether the light signals interfere with the valid light signal data received by the first depth camera; if interference exists, it adopts a light signal acquisition strategy to acquire the valid light signal data without interference; and based on the acquired valid light signal data without interference, it obtains the distance of the object being measured.

[0119] Example 3

[0120] like Figure 3 As shown, this embodiment also provides an AGV, including: a support body, on which at least one depth camera is fixed, the depth camera performing the steps of any of the methods described in Embodiment 1 and Embodiment 2 above.

[0121] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0122] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0124] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. An anti-interference distance measurement method, characterized in that, The distance measurement method is used in the first depth camera of the AGV, and the method includes: The first detector of the first depth camera receives the reflected light signal from the object being measured in real time, and the second detector of the first depth camera detects the light signals of other depth cameras during the exposure period when the reflected light signal is received. If the second detector detects light signals from other depth cameras, it determines whether the light signals interfere with the valid data of the reflected light signals received by the first depth camera; that is, whether the time point at which the first depth camera detects the light signals from other depth cameras falls within the exposure time period for receiving the reflected light signals reflected by the object being measured. The valid data is the data read out by the first depth camera after exposing the received reflected light signal; the first detector in the first depth camera that receives the reflected light signal and the second detector that detects the light signals of other depth cameras are independent of each other and the distances in the vertical and horizontal directions both meet the preset distance thresholds; If interference exists within the exposure time period, a light signal acquisition strategy is used to acquire effective data of interference-free reflected light signals. Based on the collected interference-free and valid data, the distance to the measured object is obtained; An optical signal acquisition strategy is employed to acquire effective data of interference-free reflected optical signals, including: discarding effective data of the current reflected optical signal or the current frame; Based on the collected, interference-free, and valid data, the distance to the measured object is obtained, including: The distance to the object being measured is obtained using the following formula; ； d is the distance between the depth camera and the object being measured; C is the speed of light, and F is the modulation frequency of the depth camera, measured in MHz. Q1, Q2, Q3, and Q4 are all valid data of interference-free reflected light signals.

2. The method according to claim 1, characterized in that, The method of acquiring effective data of interference-free reflected light signals using an optical signal acquisition strategy also includes: Obtain the delay time period t, and after the delay t, re-receive the reflected light signal reflected by the object under test; Alternatively, discard the valid data of the current reflected light signal, obtain the delay time period t, and re-receive the reflected light signal reflected by the object under test after the delay time t. The t satisfies the following formula: ； ； N is a positive integer, shutter1 represents the single exposure time, and Ta represents the exposure interval time, i.e., the data readout time after exposure. 1 represents the frame rate of the depth camera, measured in fps; Tb represents the inter-frame rest time.

3. A depth camera, characterized in that, The depth camera, used in AGVs, includes: Laser emission module, first detector, second detector, and processor; The laser light-emitting module is used to emit light signals; The first detector is used to receive the reflected light signal reflected by the object being measured in real time; The second detector is used to detect light signals from other depth cameras during the exposure of the reflected light signal; The processor is electrically connected to the laser emission module, the first detector, and the second detector. It is used to detect light signals from other depth cameras via the second detector and determine whether these light signals interfere with the valid data of the reflected light signals received by the depth cameras. Specifically, it determines whether the time point at which the light signals from other depth cameras are detected falls within the exposure time period for receiving the reflected light signals from the object being measured. If interference exists within the exposure time period, a light signal acquisition strategy is used to acquire valid data of the reflected light signals without interference. Based on the acquired valid data of the reflected light signals without interference, the distance to the object being measured is obtained. The acquisition of valid data of the reflected light signals without interference using the light signal acquisition strategy includes discarding the current valid data of the reflected light signals or the current frame. The valid data of the reflected light signal is the data read out by the processor after exposing the received reflected light signal; The first detector and the second detector are independent of each other, and the distances in both the vertical and horizontal directions meet the preset distance thresholds; Based on the collected, interference-free, and valid data, the distance to the measured object is obtained, including: The distance to the object being measured is obtained using the following formula; ； d is the distance between the depth camera and the object being measured; C is the speed of light, and F is the modulation frequency of the depth camera, measured in MHz. Q1, Q2, Q3, and Q4 are all valid data of interference-free reflected light signals.

4. The depth camera according to claim 3, characterized in that, The depth camera is: dToF depth camera, iTOF depth camera; Both the first detector and the second detector are photodiode detectors; The image sensor, which is electrically connected to the first detector, is also electrically connected to the processor.

5. The depth camera according to claim 3, characterized in that, Specifically, the processor is used to determine whether the time point at which the light signals from other depth cameras are detected falls within the exposure time period for receiving the reflected light signals from the object being measured. If so, obtain the delay time period t, and after delaying by t, re-receive the reflected light signal reflected by the object under test; Alternatively, discard the valid data of the current reflected light signal, obtain the delay time period t, and re-receive the reflected light signal reflected by the object under test after the delay time t. The t satisfies the following formula: ； ； N is a positive integer, shutter1 represents the single exposure time, and Ta represents the exposure interval time, i.e., the data readout time after exposure. The frame rate of the depth camera, measured in fps; Tb is the inter-frame rest time.

6. The depth camera according to claim 3, characterized in that, The laser emission module is specifically used for An optical signal is emitted toward the object being measured. The optical signal includes multiple pulse signals with the same pulse interval and varying modulation frequency.

7. An AGV, characterized in that, include: A support body, on which a depth camera as described in any one of claims 3 to 6 is fixed.

Images