Self-feedback temperature compensation method, apparatus and system for depth camera

Abstract

A self-feedback temperature compensation method, apparatus and system for a depth camera. The self-feedback temperature compensation method for a depth camera comprises: under a target initialization condition, triggering a TOF depth sensor to acquire a reference phase of a light source which is conducted by an optical fiber; under a trigger condition of a fixed frequency, triggering the TOF depth sensor to acquire the current phase of the light source which is conducted by the optical fiber; on the basis of the current phase and the reference phase, acquiring a phase deviation of the TOF depth sensor; and on the basis of the phase deviation, acquiring a delay time required by a programmable delay chip, so as to correct a phase offset between a laser emitter and the TOF depth sensor. The self-feedback temperature compensation method for a depth camera solves a labor cost problem caused by complex temperature calibration of a TOF depth sensor and an error problem caused by a common temperature compensation algorithm fitting curve, and improves the correction accuracy and efficiency of a phase offset.

Description

A self-feedback temperature compensation method, device and system for a depth camera

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on August 8, 2023, with application number 202310994335.9 and invention name “A self-feedback temperature compensation method, device and system for a depth camera”, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of image processing technology, and in particular to a self-feedback temperature compensation method, device, and system for a depth camera. Background Art

[0004] Depth cameras are generally classified according to their depth measurement principles: Time of Flight (TOF), Structured Light, and Stereo Vision. Currently, common TOF (Time of Flight) depth cameras all exhibit a certain phase shift as temperature changes.

[0005] Phase offset means that the measured distance is no longer accurate. To ensure the measurement accuracy of the depth camera, phase offset compensation is required. However, the temperature drift of the distance measurement of TOF depth cameras has always been a difficult problem to solve in this field. The common compensation solution is to calibrate the temperature and phase changes or depth changes. Through calibration, a fitting function of the phase or depth change with temperature is obtained, and then the fitting function is used to compensate for the phase offset.

[0006] However, due to the defects of the above-mentioned phase compensation scheme, such as complex operation, long time period, need for dedicated high and low temperature control equipment, separate temperature calibration of different sensors, certain errors due to fitting functions, and lag in temperature sensor detection, the accuracy of phase offset compensation is low.

[0007] Summary of the Invention

[0008] In view of this, the embodiments of the present application provide a self-feedback temperature compensation method, device and system for a depth camera to solve the labor cost problem caused by the complex temperature calibration of the TOF depth sensor and the error problem caused by the fitting curve of the common temperature compensation algorithm.

[0009] In a first aspect, embodiments of the present application provide a self-feedback temperature compensation method for a depth camera. The method is performed by a controller of a self-feedback temperature compensation system for a depth camera. The system includes a laser emitter, an optical fiber, a time-of-flight depth sensor, a controller, and a programmable delay chip. The optical fiber is used to transmit the light source of the laser emitter to the time-of-flight depth sensor.

[0010] The method comprises:

[0011] triggering the TOF depth sensor under target initialization conditions to obtain a reference phase of the light source conducted by the optical fiber;

[0012] Under a fixed frequency triggering condition, triggering the TOF depth sensor to obtain a current phase of the light source conducted by the optical fiber;

[0013] Acquire a phase deviation of the TOF depth sensor according to the current phase and the reference phase;

[0014] According to the phase deviation, the delay time required by the programmable delay chip is obtained to correct the phase offset between the laser emitter and the TOF depth sensor.

[0015] According to the above technical means, the embodiment of the present application uses optical fiber to construct an optical feedback loop from the inside of the TOF depth sensor, transmits the light source of the laser emitter to the TOF depth sensor, calculates the phase offset, and adjusts the delayed illumination of the light source of the laser emitter through a programmable delay chip to solve the phase offset problem caused by temperature and improve the accuracy of phase offset compensation.

[0016] In combination with the first aspect, in one embodiment, the receiving end of the optical fiber is connected to the edge of the effective light distribution area of ​​the laser transmitter, and the emitting end of the optical fiber is connected to the edge of the photosensitive area of ​​the TOF depth sensor.

[0017] According to the above technical means, the receiving end of the optical fiber is connected to the edge of the effective light distribution area of ​​the laser emitter, and the emitting end of the optical fiber is connected to the edge of the photosensitive area of ​​the TOF depth sensor; thereby ensuring that the optical fiber can effectively transmit the light source of the laser emitter to the TOF depth sensor, and the TOF depth sensor effectively receives the light source of the laser emitter to correct the phase offset.

[0018] In combination with the first aspect, in one embodiment, the target initialization condition is to initialize the delay time of the programmable delay chip to zero delay, and the TOF depth sensor operates at a fixed temperature.

[0019] According to the above technical means, deep phase calibration is performed under the target initialization condition to ensure the accuracy of the reference phase.

[0020] In combination with the first aspect, in one embodiment, raw phase image data of the TOF depth sensor is acquired;

[0021] acquiring IR intensity image data of the laser emitter according to the original phase image data;

[0022] The data in the IR intensity image data that is greater than a first set threshold is determined as a pixel group x corresponding to the light source conducted by the optical fiber on the TOF depth sensor.

[0023] According to the above technical means, before performing depth phase calibration, it is necessary to determine the pixel group x, which is the pixel group imaged by the light source transmitted by the optical fiber on the TOF depth sensor, to determine the reference phase.

[0024] In combination with the first aspect, in one embodiment, the light source of the laser transmitter is transmitted to the TOF depth sensor through the optical fiber, and the TOF depth sensor is triggered to collect multiple frames of image data;

[0025] An average phase corresponding to the pixel group x on the multiple frames of image data is obtained, and the average phase is determined as a reference phase.

[0026] In combination with the first aspect, in one embodiment, the TOF depth sensor is triggered at a fixed frequency, and the light source of the laser transmitter is transmitted to the TOF depth sensor through the optical fiber to obtain the current phase of the pixel group x.

[0027] According to the above technical means, the TOF depth sensor is triggered at a fixed frequency to determine the current phase of the pixel group x, so as to calculate the phase offset between the laser emitter and the TOF depth sensor.

[0028] In combination with the first aspect, in one embodiment, when the phase deviation is a positive value, the delay time of the programmable delay chip is reduced;

[0029] When the phase deviation is a negative value, the delay time of the programmable delay chip is increased.

[0030] According to the above technical means, the embodiment of the present application uses optical fiber to construct an optical feedback loop from the inside of the TOF depth sensor, transmits the light source of the laser emitter to the TOF depth sensor, calculates the phase offset, and solves the phase offset problem caused by temperature by adjusting the delayed illumination of the light source of the laser emitter through a programmable delay chip. When the phase deviation is positive, the controller controls the next trigger delay of the programmable delay chip to decrease; when the phase deviation is negative, the controller controls the next trigger delay of the programmable delay chip to increase, thereby improving the accuracy of phase offset compensation.

[0031] In a second aspect, an embodiment of the present application provides a self-feedback temperature compensation device for a depth camera, the device being configured to implement the self-feedback temperature compensation method for a depth camera as described above, the device comprising:

[0032] a reference phase acquisition module, configured to trigger the TOF depth sensor under target initialization conditions to acquire a reference phase of the light source conducted by the optical fiber;

[0033] a current phase acquisition module, configured to trigger the TOF depth sensor under a fixed frequency trigger condition to acquire the current phase of the light source conducted by the optical fiber;

[0034] A phase deviation acquisition module, configured to acquire a phase deviation of the TOF depth sensor based on the current phase and the reference phase;

[0035] The phase offset correction module is used to obtain the delay time required by the programmable delay chip according to the phase deviation to correct the phase offset between the laser emitter and the TOF depth sensor.

[0036] In a third aspect, an embodiment of the present application provides a self-feedback temperature compensation system for a depth camera, the system comprising a laser emitter, an optical fiber, a TOF depth sensor, a controller, and a programmable delay chip;

[0037] The two ends of the optical fiber are respectively connected to the laser emitter and the TOF depth sensor, and the optical fiber is used to transmit the light source of the laser emitter to the TOF depth sensor; the TOF depth sensor is used to image the light source transmitted by the optical fiber;

[0038] Two ends of the programmable delay chip are respectively connected to the controller and the laser emitter, and the programmable delay chip is used to delay triggering the laser emitter according to the instruction of the controller;

[0039] The controller is used to:

[0040] triggering the TOF depth sensor under target initialization conditions to obtain a reference phase of the light source conducted by the optical fiber;

[0041] Under a fixed frequency triggering condition, triggering the TOF depth sensor to obtain a current phase of the light source conducted by the optical fiber;

[0042] Acquire a phase deviation of the TOF depth sensor according to the current phase and the reference phase;

[0043] According to the phase deviation, the delay time required by the programmable delay chip is obtained to correct the phase offset between the laser emitter and the TOF depth sensor.

[0044] In a fourth aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned self-feedback temperature compensation method for a depth camera.

[0045] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned self-feedback temperature compensation method for a depth camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Fig. 1 is a schematic structural diagram of a self-feedback temperature compensation system for a depth camera according to an exemplary embodiment.

[0048] Fig. 2 is a schematic diagram showing the installation position of an optical fiber according to an exemplary embodiment.

[0049] Fig. 3 is a schematic diagram of a half-width area of ​​a light distribution curve of a laser transmitter according to an exemplary embodiment.

[0050] FIG4 is a flow chart showing a method for self-feedback temperature compensation of a depth camera according to an exemplary embodiment.

[0051] FIG5 is a flowchart showing a method for self-feedback temperature compensation of a depth camera according to an exemplary embodiment.

[0052] Fig. 6 is a schematic diagram showing a specific implementation of a self-feedback temperature compensation method for a depth camera according to an exemplary embodiment.

[0053] Fig. 7 is a structural block diagram showing a self-feedback temperature compensation device for a depth camera according to an exemplary embodiment.

[0054] FIG8 shows a structural block diagram of a computer device according to an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solution of this application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0056] Figure 1 is a schematic diagram of a self-feedback temperature compensation system for a depth camera according to an exemplary embodiment. As shown in Figure 1, the system includes a laser transmitter, an optical fiber, a TOF depth sensor (i.e., the TOF sensor in Figure 1), a controller, and a programmable delay chip;

[0057] The two ends of the optical fiber are respectively connected to the laser emitter and the TOF depth sensor. The optical fiber is used to transmit the light source of the laser emitter to the TOF depth sensor. The TOF depth sensor is used to image the light source transmitted by the optical fiber.

[0058] The two ends of the programmable delay chip are respectively connected to the controller and the laser emitter, and the programmable delay chip is used to delay triggering the laser emitter according to the instruction of the controller;

[0059] The controller is used to:

[0060] Under target initialization conditions, performing depth phase calibration on the TOF depth sensor to obtain a reference phase of the light source transmitted by the optical fiber;

[0061] Under a fixed frequency trigger condition, triggering the TOF depth sensor to obtain a current phase of the light source conducted by the optical fiber;

[0062] Obtaining a phase deviation of the TOF depth sensor according to the current phase and the reference phase;

[0063] According to the phase deviation, the delay time required by the programmable delay chip is obtained to correct the phase offset between the laser emitter and the TOF depth sensor.

[0064] Optionally, the receiving end of the optical fiber is connected to an edge position of the effective light distribution area of ​​the laser transmitter, and the emitting end of the optical fiber is connected to an edge position of the light-sensitive area of ​​the TOF depth sensor.

[0065] Optionally, please refer to the installation position diagram of the optical fiber shown in Figure 2. The optical fiber in Figure 2 is the above-mentioned optical fiber. The receiving end of the optical fiber collects light from the light source surface of the laser emitter. Considering the impact on the laser emitter, the edge position of the effective light distribution area of ​​the laser emitter is selected as much as possible. Generally, the area within the half-width of the light distribution curve of the laser emitter is selected. Here, please refer to the half-width area diagram of the light distribution curve of the laser emitter shown in Figure 3; the horizontal axis of Figure 3 is the angle of the vertically emitting laser (vcsel), in degrees, and the vertical axis is the normalized illumination intensity; the solid line H represents the light intensity distribution of the vertically emitting laser (vcsel); the dotted line V represents the light intensity distribution of the vertically emitting laser (vcsel) in the vertical direction, and the box is the half-width area of ​​the light distribution curve of the laser emitter.

[0066] As shown in FIG2 , the emitting end of the optical fiber is connected to the TOF depth sensor (ie, the tof sensor in FIG2 ) and is installed at the edge of the photosensitive area of ​​the TOF depth sensor.

[0067] Optionally, the system also includes a MOS tube and a MOS tube driver; the programmable delay chip is connected to the MOS tube driver, the MOS tube and the laser emitter in sequence; the MOS tube driver is used to drive the MOS tube according to the delay time of the programmable delay chip to trigger the laser emitter to emit a light source.

[0068] Optionally, the laser emitter may be a vertical emitting laser (VCSEL) or a horizontal emitting laser; and the controller may be an FPGA chip.

[0069] Optionally, the optical fiber may be a single-mode optical fiber; the single-mode optical fiber generally has a core diameter of 10 μm and an outer diameter of 125 μm; the pixel size of the TOF depth sensor is generally about 5 μm, so the optical fiber corresponds to at least 4 effective TOF depth sensor pixels.

[0070] Optionally, when in application, the controller synchronously triggers the programmable delay chip and the TOF depth sensor. The programmable delay chip is in an initial zero delay state. The optical fiber transmits the light source emitted by the laser emitter to the TOF depth sensor. The TOF depth sensor images the transmitted light source to obtain image data. The controller calculates and analyzes the image data to realize delay calibration and delay deviation calculation to obtain a phase deviation. At this time, the controller controls the programmable delay chip to trigger the laser emitter with a specified delay time according to the phase deviation. The programmable delay chip drives the MOS tube through the MOSDriver (MOS tube driver) to trigger the laser emitter. When the specified delay time arrives, the MOS tube triggers the laser emitter, thereby realizing the correction of the phase offset between the laser emitter and the TOF depth sensor.

[0071] In summary, the embodiments of the present application use optical fibers to construct an optical feedback loop from the inside of the TOF depth sensor, transmit the light source of the laser emitter to the TOF depth sensor, calculate the phase offset, use a programmable delay chip to control the illumination time of the laser emitter, and correct the phase offset caused by temperature between the laser and the TOF depth sensor; solve the labor cost problem caused by the complex temperature calibration of the TOF depth sensor and the error problem caused by the fitting curve of the common temperature compensation algorithm, improve the accuracy and efficiency of the phase offset correction, reduce errors and save waste of human resources.

[0072] FIG4 is a flowchart illustrating a method for self-feedback temperature compensation for a depth camera according to an exemplary embodiment. The method is executed by a controller of the self-feedback temperature compensation system for a depth camera shown in FIG1 . The system includes a laser transmitter, an optical fiber, a time-of-flight depth sensor, a controller, and a programmable delay chip. The optical fiber is used to transmit light from the laser transmitter to the time-of-flight depth sensor. As shown in FIG4 , the method may include the following steps:

[0073] Step S401: Under target initialization conditions, trigger the TOF depth sensor to obtain a reference phase of the light source transmitted by the optical fiber.

[0074] In one possible implementation, after connecting both ends of the optical fiber to a laser emitter and a TOF depth sensor respectively, the optical fiber can transmit the light source emitted by the laser emitter to the TOF depth sensor; at this time, under the condition of target initialization, the TOF depth sensor images the light source transmitted by the optical fiber to obtain image data, performs depth phase calibration on the TOF depth sensor based on the image data, triggers the TOF depth sensor, and thereby obtains a reference phase of the light source transmitted by the optical fiber.

[0075] Step S402: Under a fixed frequency triggering condition, trigger the TOF depth sensor to obtain the current phase of the light source transmitted by the optical fiber.

[0076] In one possible implementation, after obtaining a reference phase of the light source transmitted by the optical fiber, the current phase of the light source transmitted by the optical fiber is calculated, and phase offset correction is performed based on the offset between the current phase and the reference phase. The current phase is generated when the TOF depth sensor is triggered under a fixed frequency trigger condition. At this time, an image of the light source transmitted by the optical fiber is imaged, and the current phase of the light source transmitted by the optical fiber is calculated based on the image data.

[0077] Step S403: Acquire a phase deviation of the TOF depth sensor according to the current phase and the reference phase.

[0078] In a possible implementation, after calculating the current phase and the reference phase of the light source transmitted by the optical fiber, the phase deviation of the TOF depth sensor can be obtained based on the deviation between the current phase and the reference phase.

[0079] Step S404: Obtain the delay time required by the programmable delay chip according to the phase deviation to correct the phase offset between the laser emitter and the TOF depth sensor.

[0080] In one possible implementation, after obtaining the phase deviation of the TOF depth sensor, the delay time of the programmable delay chip is adjusted according to the phase deviation to ensure that the current phase should be consistent with the reference phase, thereby correcting the phase deviation of the TOF depth sensor by adjusting the delay time.

[0081] In summary, the embodiments of the present application use optical fibers to construct an optical feedback loop from the inside of the TOF depth sensor, transmit the light source of the laser emitter to the TOF depth sensor, calculate the phase offset, use a programmable delay chip to control the illumination time of the laser emitter, and correct the phase offset caused by temperature between the laser and the TOF depth sensor; solve the labor cost problem caused by the complex temperature calibration of the TOF depth sensor and the error problem caused by the fitting curve of the common temperature compensation algorithm, improve the accuracy and efficiency of the phase offset correction, reduce errors and save waste of human resources.

[0082] FIG5 is a flowchart illustrating a method for self-feedback temperature compensation for a depth camera according to an exemplary embodiment. The method is executed by a controller of the self-feedback temperature compensation system for a depth camera shown in FIG1 . The system includes a laser transmitter, an optical fiber, a time-of-flight depth sensor, a controller, and a programmable delay chip. The optical fiber is used to transmit light from the laser transmitter to the time-of-flight depth sensor. As shown in FIG5 , the method may include the following steps:

[0083] Step S501: Under target initialization conditions, trigger the TOF depth sensor to obtain a reference phase of the light source transmitted by the optical fiber.

[0084] In a possible implementation, the receiving end of the optical fiber is connected to an edge of the effective light distribution area of ​​the laser transmitter, and the emitting end of the optical fiber is connected to an edge of the photosensitive area of ​​the TOF depth sensor.

[0085] In a possible implementation, the target initialization condition is to initialize the delay time of the programmable delay chip to zero delay, and the TOF depth sensor operates at a fixed temperature.

[0086] In one possible embodiment, before obtaining the reference phase of the light source transmitted by the optical fiber, raw phase image data of the TOF depth sensor is obtained; the raw phase image data is the raw phase image data formed by the TOF depth sensor when the optical fiber transmits the light source of the laser emitter to the TOF depth sensor;

[0087] Acquiring IR intensity image data of the laser emitter according to the original phase image data;

[0088] The data in the IR intensity image data that is greater than a first set threshold is determined as a pixel group x corresponding to the light source transmitted by the optical fiber on the TOF depth sensor.

[0089] Optionally, in this embodiment, before calculating the reference phase, after the optical fiber is installed, the laser emitter is covered to ensure that the light source received by the TOF depth sensor is only the light source transmitted by the optical fiber; the phase data of the TOF depth sensor is collected in a dark room, and the IR intensity image data of the laser emitter is calculated. The data greater than a first set threshold value is selected from the IR intensity image data, and the data greater than the first set threshold value (the first set threshold value can be set according to the specific application scenario) is the specific pixel group x formed on the TOF depth sensor corresponding to the light source transmitted by the optical fiber.

[0090] Alternatively, the IR intensity image data may be calculated using the following formula: A = sqrt[(DCS1 - DCS3)] 2 +(DCS2-DCS4) 2 ];

[0091] Among them, DCS1, DCS2, DCS3, and DCS4 respectively represent the optical fiber transmitting the light source of the laser transmitter to the TOF depth sensor, and the original phase image data formed by the TOF depth sensor; A represents the calculated IR intensity image data.

[0092] In a possible implementation, step S501 includes:

[0093] transmitting the light source of the laser transmitter to the TOF depth sensor through the optical fiber, and triggering the TOF depth sensor to collect multiple frames of image data;

[0094] An average phase corresponding to the pixel group x on the multiple frames of image data is obtained, and the average phase is determined as a reference phase.

[0095] Optionally, after determining the pixel group x corresponding to the light source transmitted by the optical fiber on the TOF depth sensor, the phase of the pixel group x is calculated to obtain a reference phase; when calculating the reference phase, the controller must initialize the delay time of the programmable delay chip to 0 under room temperature conditions, and the controller triggers the TOF depth sensor to collect image data. The TOF depth sensor trigger signal triggers the laser emitter for modulated illumination through the programmable delay chip. The TOF depth sensor collects multiple times and calculates the average phase of the current pixel group x, thereby obtaining the reference phase (the average phase is the reference phase).

[0096] Alternatively, the phase may be calculated using the following formula: P = atan2(DCS1-DCS3, DCS2-DCS4);

[0097] Among them, DCS1, DCS2, DCS3, and DCS4 respectively represent the original phase image data formed by the TOF depth sensor when the optical fiber transmits the light source of the laser transmitter to the TOF depth sensor; P represents the calculated phase data of the pixel group x; the average phase can be obtained by averaging the phase data corresponding to each multi-frame image data.

[0098] Step S502: Under a fixed frequency triggering condition, trigger the TOF depth sensor to obtain the current phase of the light source transmitted by the optical fiber.

[0099] In a possible implementation, step S502 includes:

[0100] The TOF depth sensor is triggered at a fixed frequency, and the light source of the laser transmitter is transmitted to the TOF depth sensor through the optical fiber to obtain the current phase of the pixel group x.

[0101] In order to correct the phase offset between the laser emitter and the TOF depth sensor, it is necessary to calculate the current phase of the light source transmitted by the optical fiber, that is, the current phase of the pixel group x; when calculating the current phase of the pixel group x, the TOF depth sensor is triggered at a fixed frequency (for example, the fixed frequency is 30 Hz), and the current phase of the pixel group x can be calculated based on the image data of the TOF depth sensor.

[0102] Step S503: Determine the phase deviation of the TOF depth sensor according to the difference between the current phase and the reference phase.

[0103] In one possible implementation, as the laser, MOS tube, and TOF depth sensor all experience a certain phase drift with use or as the ambient temperature rises, the deviation between the current phase and the reference phase of the pixel group x is detected at this time; that is, the phase deviation of the TOF depth sensor can be obtained by taking the difference between the current phase and the reference phase.

[0104] Step S504: When the phase deviation is a positive value, reducing the delay time of the programmable delay chip.

[0105] Step S505: When the phase deviation is a negative value, increase the delay time of the programmable delay chip.

[0106] In one possible implementation, the controller simultaneously triggers the programmable delay chip. When the phase deviation is positive, the controller controls the programmable delay chip to decrease the next trigger delay; when the phase deviation is negative, the controller controls the programmable delay chip to increase the next trigger delay.

[0107] Please refer to the specific implementation diagram of the self-feedback temperature compensation method of a depth camera shown in Figure 6. In actual application, after determining the pixel group x, the delay of the programmable delay chip is set to zero, and the controller synchronously triggers the programmable delay chip and the TOF depth sensor (and the tof sensor in Figure 6). The programmable delay chip triggers the laser emitter illumination, and the optical fiber transmits the light source to the TOF depth sensor, and the TOF depth sensor collects multiple frames of image data; at this time, the controller calculates the phase of the pixel group x position based on the multiple frames of image data as the reference phase; after obtaining the reference phase, the programmable delay chip delays the setting delay (correction has not been achieved at this time, so the initial delay can be set as needed), and triggers the laser emitter illumination. When the measurement of the current phase begins, the controller triggers the TOF depth sensor at a fixed frequency, the programmable delay chip triggers the laser emitter for illumination, the optical fiber transmits the light source to the TOF depth sensor, and the TOF depth sensor collects the current image data; at this time, the controller calculates the current phase of the pixel group x position based on the current image data; at this time, the calculation of the reference phase and the current phase has been completed, and the difference operation is performed on the reference phase and the current phase. If the difference between the current phase and the reference phase is greater than or equal to zero, the delay of the programmable delay chip is reduced, and the feedback is fed back to the programmable delay chip for the next trigger; if the difference between the current phase and the reference phase is less than zero, the delay of the programmable delay chip is increased, and the feedback is fed back to the programmable delay chip for the next trigger.

[0108] In summary, the embodiments of the present application use optical fibers to construct an optical feedback loop from the inside of the TOF depth sensor, transmit the light source of the laser emitter to the TOF depth sensor, calculate the phase offset, use a programmable delay chip to control the illumination time of the laser emitter, and correct the phase offset caused by temperature between the laser and the TOF depth sensor; solve the labor cost problem caused by the complex temperature calibration of the TOF depth sensor and the error problem caused by the fitting curve of the common temperature compensation algorithm, improve the accuracy and efficiency of the phase offset correction, reduce errors and save waste of human resources.

[0109] FIG7 is a block diagram of a self-feedback temperature compensation device for a depth camera according to an exemplary embodiment. The device is used to implement the self-feedback temperature compensation method for a depth camera as described above;

[0110] The device includes:

[0111] A reference phase acquisition module 701 is configured to trigger the TOF depth sensor under target initialization conditions to acquire a reference phase of the light source transmitted by the optical fiber;

[0112] A current phase acquisition module 702 is configured to trigger the TOF depth sensor under a fixed frequency trigger condition to acquire the current phase of the light source conducted by the optical fiber;

[0113] A phase deviation acquisition module 703 is configured to acquire a phase deviation of the TOF depth sensor according to the current phase and the reference phase;

[0114] The phase offset correction module 704 is used to obtain the delay time required by the programmable delay chip according to the phase deviation, so as to correct the phase offset between the laser emitter and the TOF depth sensor.

[0115] In a possible implementation, the receiving end of the optical fiber is connected to an edge of the effective light distribution area of ​​the laser transmitter, and the emitting end of the optical fiber is connected to an edge of the photosensitive area of ​​the TOF depth sensor.

[0116] In a possible implementation, the target initialization condition is to initialize the delay time of the programmable delay chip to zero delay, and the TOF depth sensor operates at a fixed temperature.

[0117] In one possible embodiment, the device is further used for:

[0118] Obtaining raw phase image data of the TOF depth sensor;

[0119] Acquiring IR intensity image data of the laser emitter according to the original phase image data;

[0120] The data in the IR intensity image data that is greater than a first set threshold is determined as a pixel group x corresponding to the light source transmitted by the optical fiber on the TOF depth sensor.

[0121] In a possible implementation manner, the reference phase acquisition module 701 is further configured to:

[0122] transmitting the light source of the laser transmitter to the TOF depth sensor through the optical fiber, and triggering the TOF depth sensor to collect multiple frames of image data;

[0123] An average phase corresponding to the pixel group x on the multiple frames of image data is obtained, and the average phase is determined as a reference phase.

[0124] In a possible implementation, the current phase acquisition module 702 is further configured to:

[0125] The TOF depth sensor is triggered at a fixed frequency, and the light source of the laser transmitter is transmitted to the TOF depth sensor through the optical fiber to obtain the current phase of the pixel group x.

[0126] In a possible implementation, the phase offset correction module 704 is further configured to:

[0127] When the phase deviation is positive, the delay time of the programmable delay chip is reduced;

[0128] When the phase deviation is a negative value, the delay time of the programmable delay chip is increased.

[0129] In summary, the embodiments of the present application use optical fibers to construct an optical feedback loop from the inside of the TOF depth sensor, transmit the light source of the laser emitter to the TOF depth sensor, calculate the phase offset, use a programmable delay chip to control the illumination time of the laser emitter, and correct the phase offset caused by temperature between the laser and the TOF depth sensor; solve the labor cost problem caused by the complex temperature calibration of the TOF depth sensor and the error problem caused by the fitting curve of the common temperature compensation algorithm, improve the accuracy and efficiency of the phase offset correction, reduce errors and save waste of human resources.

[0130] Please refer to Figure 8, which is a schematic diagram of a computer device provided according to an exemplary embodiment of the present application. The computer device includes a memory and a processor. The memory is used to store a computer program. When the computer program is executed by the processor, the above-mentioned method of time series data compression is implemented.

[0131] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0132] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this application. The processor executes the non-transitory software programs, instructions, and modules stored in the memory to perform various functional applications and data processing of the processor, thereby implementing the methods in the above-mentioned method embodiments.

[0133] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0134] In an exemplary embodiment, a computer-readable storage medium is further provided, storing at least one computer program, which is loaded and executed by a processor to implement all or part of the steps of the above method. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, or an optical data storage device.

[0135] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0136] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A self-feedback temperature compensation method for a depth camera, characterized in that: The method is performed by a controller of a self-feedback temperature compensation system of a depth camera, the system comprising a laser transmitter, an optical fiber, a TOF depth sensor, a controller and a programmable delay chip, the optical fiber being used to transmit the light source of the laser transmitter to the TOF depth sensor; The method comprises: Under target initialization conditions, triggering the TOF depth sensor to obtain a reference phase of the light source conducted by the optical fiber; Under a trigger condition of a fixed frequency, triggering the TOF depth sensor to obtain a current phase of the light source conducted by the optical fiber; Acquire a phase deviation of the TOF depth sensor according to the current phase and the reference phase; According to the phase deviation, the delay time required by the programmable delay chip is obtained to correct the phase offset between the laser emitter and the TOF depth sensor.

2. The method according to claim 1, characterized in that The receiving end of the optical fiber is connected to the edge of the effective light distribution area of ​​the laser transmitter, and the transmitting end of the optical fiber is connected to the edge of the photosensitive area of ​​the TOF depth sensor.

3. The method according to claim 1, characterized in that The target initialization condition is to initialize the delay time of the programmable delay chip to zero delay, and the TOF depth sensor operates at a fixed temperature.

4. The method according to claim 1, characterized in that Before acquiring the reference phase of the light source transmitted by the optical fiber, the method further comprises: Acquire raw phase image data of the TOF depth sensor; Acquiring IR intensity image data of the laser emitter according to the original phase image data; The data in the IR intensity image data that is greater than a first set threshold is determined as a pixel group x corresponding to the light source transmitted by the optical fiber on the TOF depth sensor.

5. The method according to claim 4, characterized in that The step of obtaining a reference phase of a light source transmitted by the optical fiber comprises: Conducting the light source of the laser transmitter to the TOF depth sensor through the optical fiber, and triggering the TOF depth sensor to collect multiple frames of image data; An average phase corresponding to the pixel group x on the multiple frames of image data is obtained, and the average phase is determined as a reference phase.

6. The method according to claim 4, characterized in that The triggering of the TOF depth sensor under the triggering condition of the fixed frequency to obtain the current phase of the light source conducted by the optical fiber includes: The TOF depth sensor is triggered at a fixed frequency, and the light source of the laser transmitter is transmitted to the TOF depth sensor through the optical fiber to obtain the current phase of the pixel group x.

7. The method according to any one of claims 1 to 6, characterized in that: The step of obtaining the delay time required by the programmable delay chip according to the phase deviation includes: When the phase deviation is a positive value, reducing the delay time of the programmable delay chip; When the phase deviation is a negative value, the delay time of the programmable delay chip is increased.

8. A self-feedback temperature compensation device for a depth camera, characterized in that: The device is used to implement the self-feedback temperature compensation method for a depth camera according to any one of claims 1 to 7, and the device comprises: A reference phase acquisition module, used to trigger the TOF depth sensor under target initialization conditions to acquire a reference phase of the light source conducted by the optical fiber; A current phase acquisition module, used to trigger the TOF depth sensor under a fixed frequency trigger condition to acquire the current phase of the light source conducted by the optical fiber; A phase deviation acquisition module, used to acquire the phase deviation of the TOF depth sensor according to the current phase and the reference phase; The phase shift correction module is used to obtain the delay time required by the programmable delay chip according to the phase deviation, so as to correct the phase shift between the laser emitter and the TOF depth sensor.

9. A self-feedback temperature compensation system for a depth camera, characterized in that: The system includes a laser transmitter, an optical fiber, a TOF depth sensor, a controller and a programmable delay chip; The two ends of the optical fiber are respectively connected to the laser emitter and the TOF depth sensor, and the optical fiber is used to transmit the light source of the laser emitter to the TOF depth sensor; the TOF depth sensor is used to image the light source transmitted by the optical fiber; Two ends of the programmable delay chip are respectively connected to the controller and the laser emitter, and the programmable delay chip is used to delay triggering the laser emitter according to the instruction of the controller; The controller is used to: Under target initialization conditions, triggering the TOF depth sensor to obtain a reference phase of the light source conducted by the optical fiber; Under a trigger condition of a fixed frequency, triggering the TOF depth sensor to obtain a current phase of the light source conducted by the optical fiber; Acquire a phase deviation of the TOF depth sensor according to the current phase and the reference phase; According to the phase deviation, the delay time required by the programmable delay chip is obtained to correct the phase offset between the laser emitter and the TOF depth sensor.

10. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement a self-feedback temperature compensation method for a depth camera as described in any one of claims 1 to 7.

Images