A depth measuring device and system

Abstract

The application relates to a depth measuring device and system, which comprises a central processor, a TOF unit, a microcontroller and an RGB sensor, wherein: the central processor is used for generating TOF control signals and RGB control signals to the TOF unit and the RGB sensor respectively and simultaneously sending a correction signal to the microcontroller; the TOF unit is used for emitting different modulation light beams according to the TOF control signals, collecting reflected modulation light beams to generate at least two frames of raw phase images, and transmitting the images to the central processor according to the TOF control signals, and simultaneously generating a feedback signal to the microcontroller; the microcontroller is used for correcting the feedback signal according to the correction signal to generate an RGB trigger signal; the RGB sensor is used for collecting an RGB image according to the RGB trigger signal and sending the image to the central processor according to the RGB control signal; and the central processor is further used for calculating the at least two frames of raw phase images to obtain a TOF depth image to synchronously output the RGB image and the TOF depth image. The embodiment of the application can improve the synchronization effect of the image.

Description

Technical Field

[0001] This application relates to the field of image acquisition and processing technology, and in particular to a depth measurement device and system. Background Technology

[0002] Depth cameras can be categorized into several types based on their working principles, including Time-of-Flight (TOF) and structured light. Depth cameras are increasingly being applied to various fields of human production and daily life, such as robotics, interactive games, Augmented Reality (AR), Virtual Reality (VTOF), Autonomous Driving, and 3D modeling.

[0003] Among various optical 3D measurement technologies, Time-of-Flight (TOF) depth measurement technology stands out due to its high stability and practicality. TOF depth measurement technology includes two types: one commonly known as pulse ranging, which calculates the distance between the measured object (or the detection area of ​​the measured object) and the TOF depth measuring device by measuring the time interval between the emission and reception of the laser pulse emitted by the TOF depth measuring device; the other commonly known as phase difference ranging, which calculates the distance between the measured object (or the detection area of ​​the measured object) and the TOF depth measuring device by measuring the phase change generated by the laser emitted by the TOF depth measuring device making one round trip to the measured object.

[0004] On the one hand, to improve detection accuracy and range, TOF depth measurement devices generally employ multi-frequency fusion for depth detection. However, when using multi-frequency fusion algorithms, multiple frames of images need to be acquired for phase unwrapping. When processing the images, the acquired multiple frames require spatial and temporal alignment. Spatial alignment can be achieved through image calibration, but temporal alignment is difficult to achieve.

[0005] On the other hand, to improve the texture of TOF depth measurement devices, RGB cameras are typically used to acquire RGB images for texture mapping. However, in existing technologies, the RGB camera and the TOF depth measurement device use their respective timestamps for synchronization. This approach has several drawbacks. First, the difference in exposure times between the two devices leads to timestamp offsets, ultimately causing synchronization errors. Second, because SOC chips have task priority scheduling, and the synchronization task between the RGB camera and the TOF depth measurement device is relatively low in priority, it is difficult to achieve synchronized output between the two devices. Summary of the Invention

[0006] In view of this, embodiments of this application provide a depth measurement device and system that can solve at least one technical problem in the related art.

[0007] In a first aspect, one embodiment of this application provides a depth measurement device, comprising: an RGB sensor, a TOF sensor, a central processing unit and a microcontroller, a color sensor, a TOF unit including a projection module and a TOF sensor, a central processing unit and a microcontroller, wherein: the central processing unit is used to send TOF control signals and RGB control signals to the TOF unit and the RGB sensor respectively, and simultaneously send correction signals to the microcontroller; the TOF unit is used to receive the TOF control signals and simultaneously generate a gating signal or a vertical synchronization signal to the microcontroller; wherein, triggered by the TOF control signals, the projection module is used to emit at least two different modulated beams toward the target scene, and the TOF sensor is used to... The system acquires different modulated light beams reflected from the target scene to generate at least two raw phase images, and transmits the raw phase images to the central processing unit according to the TOF control signal; the microcontroller is used to correct the gating signal or vertical synchronization signal according to the correction signal to generate an RGB trigger signal to trigger the RGB sensor to start working; the RGB sensor is used to acquire RGB images according to the RGB trigger signal and send the RGB images to the central processing unit according to the RGB control signal; the central processing unit is also used to perform phase unwrapping and depth fusion calculation on the at least two raw phase images to obtain a TOF depth image, so as to synchronously output the TOF depth image and the RGB image.

[0008] Secondly, one embodiment of this application provides a depth measurement system, including: multiple depth measurement devices as described in the first aspect embodiment.

[0009] The beneficial effect of this application embodiment is that, while the depth measuring device realizes the measurement extension, exposure synchronization is achieved through software and hardware synchronization, thereby improving the synchronization effect of the image. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a schematic diagram of the structure of a depth measuring device provided in an embodiment of this application;

[0012] Figure 2 This is a schematic diagram of the image shape obtained by two exposure methods according to an embodiment of this application;

[0013] Figure 3This is a timing diagram of the operation of each module of a depth measuring device provided in an embodiment of this application;

[0014] Figure 4 This is a timing diagram of the operation of various modules of a main device provided in an embodiment of this application;

[0015] Figure 5 This is a timing diagram of the operation of each module when the device is synchronously triggered, provided in one embodiment of this application;

[0016] Figure 6 This is another timing diagram of the operation of each module when the slave device is synchronously triggered, provided in one embodiment of this application;

[0017] Figure 7 Another embodiment of this application provides a schematic diagram of a connection method between multiple depth measuring devices;

[0018] Figure 8 This is a schematic diagram illustrating another connection method between multiple depth measuring devices provided in an embodiment of this application;

[0019] Figure 9 This is a schematic diagram of the structure of a depth measurement system provided in one embodiment of this application;

[0020] Figure 10 This is a schematic diagram of another depth measurement system provided in an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] The term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items, as well as all possible combinations, and includes such combinations.

[0023] The terms "one embodiment" or "some embodiments" described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0024] Furthermore, in the description of this application, "a plurality of" means two or more. The terms "first" and "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0026] Figure 1 This is a schematic diagram of a depth measuring device provided in one embodiment of this application. Figure 1 As shown, the depth measurement device includes an RGB sensor 11, a TOF unit 12, a central processing unit 13, and a microcontroller unit (MCU) 14, wherein:

[0027] The central processing unit 13 is used to send TOF control signals to the TOF unit 12 to control the TOF unit 12 to start working and synchronously send RGB control signals to the RGB sensor 11, and at the same time send correction signals to the microcontroller 14.

[0028] The TOF unit 12 includes a projection module and a TOF sensor, used to receive TOF control signals and simultaneously generate feedback signals to the microcontroller 14. Under the trigger of the TOF control signal, the projection module emits at least two different modulated light beams towards the target scene. The TOF sensor is exposed at least twice to collect the different modulated light beams reflected back from the target scene to generate at least two raw phase images, i.e., one modulated light beam corresponds to at least one raw phase image. The raw phase images are then transmitted to the central processing unit 13 according to the TOF control signal. The feedback signal includes a gating signal or a vertical synchronization signal, and the raw phase image is the raw data from the TOF sensor converting the collected light signals into digital signals.

[0029] The microcontroller 14 is used to receive correction signals and feedback signals, and also to correct the feedback signals according to the correction signals to generate RGB trigger signals and transmit them to the RGB sensor 11.

[0030] The RGB sensor 11 is used to receive RGB trigger signals, and also to perform exposure according to the RGB trigger signals to acquire RGB images of the target scene, and send the RGB images to the central processing unit 13 according to the RGB control signals; wherein, the exposure time of the RGB sensor 11 is aligned with the total time of at least two exposures of the TOF sensor.

[0031] The central processing unit 13 is used to synchronously receive at least two frames of raw phase images and RGB images according to the TOF control signal and RGB control signal, and to perform phase unwinding and depth fusion calculation on the received at least two frames of raw phase images to obtain a TOF depth image so as to synchronously output the TOF depth image and RGB image.

[0032] It should be noted that the time interval between at least two exposures of the TOF sensor can be the same or different, but the exposure time of the RGB sensor 11 must be aligned with the total exposure time of the TOF sensor at least twice.

[0033] In one embodiment, the microcontroller 14 may be equipped with a real-time system for receiving various signals in real time to generate trigger signals to each sensor. That is, the microcontroller 14 is a single-task processor, used only for handling synchronous triggering tasks in this application, so that the microcontroller 14 can receive various signals and provide real-time feedback trigger signals. In one embodiment, the microcontroller 14 receives correction signals and feedback signals in real time, and corrects the feedback signals according to the correction signals to generate RGB trigger signals, which are used to activate the RGB sensor 11.

[0034] Under the triggering of the TOF control signal, the synchronization method of the projection module and the TOF sensor in the TOF unit 12 includes: the TOF sensor in the TOF unit 12 receives the TOF control signal, generates a TOF exposure signal to control the exposure of the subject, and synchronously outputs a projection trigger signal to the projection module to drive the projection module to emit modulated beams of different frequencies toward the target scene. Preferably, the modulated beams include continuous waves of different frequencies or pulse beams with different pulse periods.

[0035] In one embodiment, to ensure that the exposure time of the TOF sensor at least twice is aligned with the exposure time of the RGB sensor 11, a corresponding timestamp can be assigned to the image obtained by the TOF sensor or the RGB sensor 11 at each exposure, so that the raw phase image and the RGB image can be synchronously transmitted to the central processing unit 13 according to the timestamp. For example, one frame and two frames of raw phase images and RGB images can be transmitted synchronously according to the timestamp, since the total exposure time of the two frames of raw phase images and the one frame of RGB image are aligned, the corresponding timestamps are assigned to the two frames of raw phase images and the one frame of RGB image respectively when the exposure is completed, so that the one frame of RGB image can be transmitted synchronously while transmitting the two frames of raw phase images.

[0036] In one embodiment, the exposure method of the TOF sensor or RGB sensor 11 is either rolling shutter exposure or global exposure. When the exposure method is rolling shutter exposure, it is also necessary to ensure that the exposure center of the TOF sensor at least twice is aligned with the exposure center of the RGB sensor 11 to avoid poor alignment between the raw phase image and the RGB image. It should be noted that the exposure center can be understood as the center of the image obtained after the sensor exposure.

[0037] Specifically, when the RGB sensor 11 uses a rolling shutter exposure, the resulting image is a parallelogram; while when the TOF sensor uses a global exposure, the resulting image is rectangular, such as... Figure 2 As shown, the images obtained from the two exposure methods are not aligned. Therefore, the central processing unit 13 needs to adjust the exposure center of the image to align it, thus obtaining an aligned image. Specifically, the exposure centers of the images obtained from different exposure methods are compared in advance. If the exposure centers are not aligned, an offset is calculated, and the image is adjusted according to the offset to align it and synchronously sent to the central processing unit 13. It should be noted that the above-mentioned offset calculation can be performed in advance and saved for use in subsequent image synchronization.

[0038] In one embodiment, the TOF depth image is obtained by performing phase unwinding and depth fusion calculations on at least two raw phase images. The exposure time corresponding to the TOF depth image is a certain time between the exposure time periods of the two raw phase images. Therefore, in order to ensure the synchronous output effect of the RGB image and the TOF depth image, the central processing unit 13 also needs to correct the exposure center of the RGB image and the TOF depth image to align the exposure center.

[0039] In one embodiment, the TOF sensor includes at least one pixel, and each pixel includes at least three or more taps (charge accumulation elements for acquiring electrical signals generated by light beams reflected back from the target scene). The exposure time of each tap is fixed and unchanging within consecutive frame periods. To obtain the number of winding cycles by unwinding the different modulated light beams reflected back from the target scene, the TOF sensor needs to acquire at least one raw phase image corresponding to different modulated light beams in consecutive frame periods and transmit it to the central processing unit 13. The central processing unit 13 performs unwinding calculations based on the raw phase images corresponding to different modulated light beams to obtain the number of winding cycles for each different modulated light beam; uses the number of winding cycles to calculate the depth value obtained based on different modulated light beams; and finally assigns corresponding weights to the depth values ​​corresponding to different modulated light beams for fusion calculation to obtain the TOF depth image.

[0040] In one embodiment, to improve depth accuracy, each pixel's taps preferably acquire the charge generated by reflected light or background light in a rotating mode. Taking three taps a, b, and c as an example to implement the rotating mode, in the first frame period, the first tap, second tap, and third tap sequentially activate the charge accumulation signal; in the second frame period, the second tap, third tap, and first tap sequentially activate the charge accumulation signal. The TOF sensor acquires at least two raw phase images under different modulated light beams according to the above method and transmits them to the central processing unit 13; taking two different modulated light beams as an example, the TOF sensor needs to acquire four raw phase images, that is, the TOF sensor needs to be exposed four times. The central processing unit 13 correspondingly superimposes the raw phase images obtained based on different frame periods according to different modulated light beams to eliminate noise and obtain a high-precision raw phase image, thereby improving depth accuracy; for example, superimposing raw phase images obtained by the same modulated light based on different frame periods to eliminate noise. That is, when the projection module projects n different modulated beams, the TOF sensor can acquire at least n frames of raw phase images; to improve depth accuracy, the TOF sensor can acquire at least 2n frames of raw phase images under each of the n different modulated beams for noise cancellation.

[0041] It should be noted that the rotation mode of the taps is not limited to the above method. It can also be that the third tap, the first tap, and the second tap are turned on in sequence. When each pixel includes more than 3 taps, the same principle applies. This will not be elaborated here.

[0042] For ease of understanding, Figures 3 to 6 The embodiment is illustrated by taking the example of a TOF sensor acquiring at least two raw phase images under at least two different modulated light beams and transmitting them to the central processing unit 13 to generate a TOF depth map.

[0043] Figure 3 Based on the timing diagram of each module of the depth measurement device provided in this application, combined with Figure 1 and Figure 3 As shown, when the TOF unit 12 receives the TOF control signal, the TOF sensor in the TOF unit generates a TOF exposure signal (denoted as TOFexposure) and performs at least four exposures based on the TOF exposure signal to acquire at least four frames of raw phase images. Simultaneously, a projection trigger signal is sent to the projection module to emit at least two different modulated beams towards the target scene. A high-level TOF strobe signal (denoted as TOF strobe out) is also generated. The TOF strobe signal is monitored by the real-time system mounted on the microcontroller 14. When the real-time system mounted on the microcontroller 14 detects the TOF strobe signal, it corrects the TOF strobe signal based on the correction signal received synchronously from the central processing unit 13, generating a high-level RGB trigger signal (denoted as RGBtriggle in) and sending it to the RGB sensor 11 to control the RGB sensor 11 to perform one exposure (denoted as RGBexposure) to acquire the RGB image of the target scene. It should be noted that this embodiment only uses the TOF gating signal as an example for illustration. In some other embodiments, the TOF unit 12 may also generate a vertical synchronization (VSYNC) signal or other synchronization signals to achieve synchronous exposure when receiving the TOF control signal. This is not limited here.

[0044] This application embodiment also provides a depth measurement system, which includes multiple depth measurement devices (hereinafter referred to as measurement devices). One measurement device is selected as the master device, operating in master mode, while the remaining measurement devices are slave devices. The master device can generate a synchronization trigger signal (denoted as Ext Triggle out) to the slave devices to trigger them to work synchronously. The synchronization trigger signal can be generated by any one of the RGB sensor 11, TOF sensor, or microcontroller 14 in the master device. Preferably, when the microcontroller 14 is equipped with a real-time system, the synchronization trigger signal (denoted as Ext Triggle out) can be generated by the microcontroller 14 in the master device to trigger the slave devices to work synchronously, ensuring accurate and stable synchronization among multiple devices.

[0045] Specifically, Figure 4 The diagram shown is the timing diagram of each module of the main equipment, combined with... Figure 1 and Figure 4As shown, when the TOF strobe signal (denoted as TOF strobe out) of the master device's TOF unit 12 is fed back to the microcontroller 14, the microcontroller 14 can send an RGB trigger signal (RGB triggle in) to the RGB sensor 11 and simultaneously send a synchronization trigger signal (ExtTriggle out) to the slave device to achieve synchronous operation between the master and slave devices. In one embodiment, to avoid interference between the master and slave devices, the rising edge of the synchronization trigger signal sent to the slave device is delayed by a delay time ΔT from the rising edge of the TOF control signal received by the master device's TOF unit or the TOF exposure signal corresponding to the first exposure of the TOF sensor. The configuration of ΔT needs to be greater than the exposure time of a single frame phase of the TOF sensor and less than the time interval between two frames, where the time interval between two frames depends on the number of master and slave devices.

[0046] Furthermore, since this application requires the acquisition of at least two raw phase images for TOF depth image synthesis, inserting multiple measurement devices between the two raw phase images allows for simultaneous operation, maximizing the utilization of the camera's inherent characteristics to achieve multi-device synchronization while avoiding interference. Specifically, assuming the maximum exposure time of the TOF sensor is 600µs and the idle time between each IR image frame is 8ms, theoretically, approximately 13 measurement devices can be inserted during this idle time to work together. More or fewer devices can be inserted, depending on the actual exposure time of the TOF sensor, and no limitation is imposed here.

[0047] Figure 5 This is a timing diagram showing the operation of each module when the device is synchronously triggered. Specifically, it combines... Figure 1 and Figure 5 As shown, the microcontroller 14 of the device receives a synchronous trigger signal (Ext Triggle in) sent by the master device to generate a TOF trigger signal (denoted as TOF triggle in, which can be the TOF gating signal or vertical distribution signal mentioned above) so that the TOF unit 12 can perform at least four exposures to obtain at least four frames of raw phase images, and at the same time generates an RGB trigger signal to the RGB sensor 11 so that the RGB sensor 11 can perform one exposure to obtain an RGB image.

[0048] exist Figure 5 Based on the timing diagram shown. Figure 6 The diagram shown illustrates a complete timing sequence for controlling the operation of this machine by receiving an external trigger signal from a measuring device. (Combined with...) Figure 1 and Figure 6As shown, the microcontroller 14 receives an external trigger signal (denoted as Ext Triggle in) sent by the synchronization device to control the exposure of the local RGB sensor 11 and TOF unit 12, and outputs an output signal (ExtTriggle out) to trigger the synchronous exposure of the next measuring device, so that the exposure of other devices does not interfere with the local exposure.

[0049] It should be noted that the master and slave devices can also generate corresponding trigger signals at regular intervals through their respective microcontrollers 14 and send them to the TOF unit 12 and the RGB sensor 11 to achieve data synchronization between the master and slave devices.

[0050] In addition to simultaneously sending TOF trigger signals and RGB trigger signals to the TOF unit 12 and RGB sensor 11 respectively, the microcontroller 14 of the slave device can also, in the same single-device operating mode, first send a TOF trigger signal to the TOF unit 12, and then generate an RGB trigger signal based on the TOF trigger signal to achieve synchronous output. It should be understood that, regardless of the method, the exposure center should be calibrated to ensure synchronous output.

[0051] In one embodiment, the connection method between multiple measuring devices may include a star configuration or a chain configuration. Specifically, one measuring device is selected as the master device, and the remaining measuring devices are slave devices. For example, Figure 7 As shown, in a chained configuration, the signal output of the master device is connected to the signal input of a slave device, the signal output of one slave device is connected to the signal input of the next slave device, and so on, until the signal output of the last slave device is connected to the signal input of the master device, thus forming a chain structure. Figure 8 As shown, in star configuration, the signal input terminals of each slave device are connected to the signal output terminals of the master device.

[0052] In some embodiments, the depth measurement system may include a server in addition to multiple measuring devices described above, with each measuring device connected to the server. The server acts as the host, and each measuring device acts as a slave device. This configuration facilitates the transmission of images acquired by multiple measuring devices to the server for synchronous processing. The connection between the measuring devices and the server can be wired or wireless. As one possible implementation, such as... Figure 9 As shown, the connection between multiple measuring devices is in a chain pattern, with each measuring device connected to the server. As another possible implementation, such as... Figure 10 As shown, the connection between multiple measuring devices is in a star configuration, with each measuring device connected to the server.

[0053] In one embodiment, each frame of image acquired by the measuring device needs to be timestamped. The depth measurement system performs the following method to synchronize the timestamps of images from multiple measuring devices or the system time, thereby achieving synchronized image output. It should be noted that when performing the following method, the server in the depth measurement system is the master device, and each measuring device is a slave device of the master device.

[0054] S1: The server sends its own time and request signal to multiple measuring devices to request the recording of the time of each measuring device.

[0055] S2: Each measuring device returns its own time and sets its own time to the time T sent by the server.

[0056] S3: The server records the RTT (round-trip time) of multiple measuring devices; for measuring devices with abnormal RTT time deviation, repeat steps S1, S2 and S3; wherein, RTT time is the time used when the server sends its own time to the measuring device, and the measuring device sets its own time to the time sent by the server and sends a setting success signal to the server.

[0057] S4: The server sends the RTT to the measuring device, and then the measuring device sets its own time to T+RTT / 2. Here, only the timestamp of the image is modified or the system time of the measuring device is changed directly.

[0058] In some embodiments, the system time of each measuring device can be calibrated periodically.

[0059] In some embodiments, further, the system time of each measuring device can be corrected based on the trigger signal received by each measuring device to achieve synchronization. More specifically, this includes:

[0060] S10: The microcontroller in each measuring device receives the rising edge of the synchronization trigger signal (Ext triggle in), triggers the corresponding frame synchronization function normally, and records the system time at this time as Ts;

[0061] S20: Poll and read the level of the current synchronization trigger signal of each measuring device; calculate the duration t ms of the high level (unit: milliseconds); if t is greater than the preset threshold, it is judged as a time synchronization signal. At the same time, if |T0+(n-1)×NT-Ts|>NT, then set n=floor[(Ts-T0+NT / 2) / NT]; then uniformly configure the system time of each measuring device to T0+(n-1)×NT+t; where floor[] means round down, the preset threshold is preferably 10, T0 is the initial start time of the system (can be uniformly defined as January 1, 1970, 0 minutes 0 seconds 0 milliseconds 0 microseconds), and n represents the number of time synchronization signals received.

[0062] It should be noted that this embodiment reuses the Ext triggle in and Ext triggle out signals. When the system receives the Ext triggle in time synchronization signal, the device configures the system time to T0+NT. N is the period length of time synchronization of multiple measurement devices. The unit can be set to 1 / 30 second or frame time. N should not be too large or too small. It is recommended that the range be [1000, 10000].

[0063] This application's embodiments achieve exposure synchronization through hardware and software synchronization, improving the synchronization effect of images. Furthermore, in some embodiments, a microcontroller equipped with a real-time system can provide real-time feedback on various data and signals, achieving control precision down to the microsecond level, ensuring stable and accurate synchronization.

[0064] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A depth measuring device, characterized in that, It includes an RGB sensor, a TOF unit comprising a projection module and a TOF sensor, a central processing unit, and a microcontroller, wherein: The central processing unit is used to send TOF control signals and RGB control signals to the TOF unit and the RGB sensor respectively, and simultaneously send correction signals to the microcontroller; The TOF unit is used to receive the TOF control signal and simultaneously generate a gating signal or a vertical synchronization signal to the microcontroller; wherein, under the trigger of the TOF control signal, the projection module is used to emit at least two different modulated beams toward the target scene, and the TOF sensor is used to collect the different modulated beams reflected back from the target scene to generate at least two raw phase images, and transmit the raw phase images to the central processing unit according to the TOF control signal. The microcontroller is used to correct the gating signal or vertical synchronization signal according to the correction signal to generate an RGB trigger signal to trigger the RGB sensor to start working; The RGB sensor is used to acquire RGB images according to the RGB trigger signal and send the RGB images to the central processing unit according to the RGB control signal; The central processing unit is also used to perform phase unwrapping and depth fusion calculations on the at least two raw phase images to obtain a TOF depth image, so as to synchronously output the TOF depth image and the RGB image; the exposure time of the RGB sensor acquiring the RGB image is aligned with the total exposure time of the TOF sensor acquiring the at least two raw phase images.

2. The depth measuring device as described in claim 1, characterized in that, The microcontroller is equipped with a real-time system, which is used to receive signals from various sources in real time and generate trigger signals to the RGB sensor in real time.

3. The depth measuring device as described in claim 1, characterized in that, The central processing unit is also used to align the exposure centers of the at least two raw phase images and the RGB image; the central processing unit is also used to synchronously output the depth image and the RGB image after aligning their exposure centers.

4. A depth measurement system, characterized in that, include: Multiple depth measuring devices as described in any one of claims 1 to 3.

5. The depth measurement system as described in claim 4, characterized in that, Any one of the depth measuring devices is a master device, and the other depth measuring devices are slave devices. The master device is used to generate a synchronization trigger signal to the microcontroller of the slave device so that the microcontroller of the slave device triggers the slave device to work synchronously. The synchronization trigger signal is generated by any one of the RGB sensor, TOF sensor or microcontroller in the master device.

6. The depth measurement system as described in claim 4, characterized in that, Any one of the depth measuring devices is a master device, and the other depth measuring devices are slave devices. The master device and the slave device generate synchronization trigger signals at regular intervals through their respective microcontrollers and send them to their respective RGB sensors and TOF sensors to achieve data synchronization.

7. The depth measurement system according to any one of claims 4 to 6, characterized in that, The connection methods for the multiple depth measurement devices include star mode or chain mode.

8. The depth measurement system as described in claim 5, characterized in that, The rising edge of the synchronization trigger signal sent to the slave device differs from the rising edge of the TOF trigger signal received by the TOF unit of the master device or from the rising edge of the TOF exposure signal of the first exposure of the TOF sensor by a delay time. T.

9. The depth measurement system as described in claim 8, characterized in that, It also includes a server, and multiple depth measuring devices are connected to the server. The server is the host, and the multiple depth measuring devices are slave devices of the host. The host receives the RGB images and depth images output by each depth measuring device.

10. The depth measurement system as described in claim 9, characterized in that, The system time or image timestamp of each depth measurement device is synchronized.

11. The depth measurement system as described in claim 10, characterized in that, The system time calibration for each of the aforementioned depth measuring devices includes: The server is used to send its own time T and a request signal to multiple depth measuring devices to request the recording of the time of each depth measuring device. Each of the aforementioned depth measurement devices is used to return its own time according to the request signal, and set its own time to the own time T sent by the server; The server is also used to record the round-trip time (RTT) of multiple depth measuring devices and send the RTT to the depth measuring devices so that the depth measuring devices set their own time to T+RTT / 2. The RTT is the time used by the server to send its own time T to the depth measuring devices, and by the depth measuring devices to set their own time to the time T sent by the server and send a setting success signal to the server.

Images