Distance deviation value correction method and device of DTOF module, equipment and storage medium

Abstract

The invention discloses a distance deviation value correction method for a DTOF module, and the method comprises the steps: obtaining a reflected echo signal after the DTOF module carrying an attenuation piece emits a laser signal, and obtaining a distance deviation value of each pixel point and the confidence coefficient of each pixel point; as for the first pixel column with the confidence coefficient being 0, if the first pixel column has no continuous N pixel points with the confidence coefficient being 1, the distance deviation value of the reference column serves as the distance deviation value of the first pixel column; for a second pixel column with the confidence coefficient of 0, if N continuous pixel points with the confidence coefficient of 1 exist in the first center position of the second pixel column, the second pixel column is a second pixel column with the confidence coefficient of 1; if yes, based on the distance deviation value of the pixel point at the second center position of the reference column, the distance deviation value of the pixel point at the first center position of the second pixel column and the distance deviation values of all the pixel points of the second pixel column, the distance deviation value of the pixel point with the confidence coefficient being 0 in the second pixel column is obtained through calculation. By adopting the method, the accumulation phenomenon can be effectively avoided in a limited space, and distance deviation value calibration is completed.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of laser detection, and in particular, to a method, device, equipment and storage medium for correcting the distance deviation value of a DTOF module. Background Art

[0002] With the rapid development of 3D imaging technology, 3D depth imaging has been widely used in many fields such as computer vision, augmented reality (AR), and autonomous driving. Among them, direct time of flight (DTOF), as an active depth perception technology based on the measurement of the propagation time of light pulses, is one of the current mainstream depth calculation methods. DTOF calculates the distance of light propagation by measuring the time difference between the emission of a light beam and its reflection back to the sensor by a target object, thereby generating three-dimensional depth information. Array DTOF uses a large-scale pixel array to obtain a complete 3D depth image by independently calculating the distance of each pixel point in the sensor.

[0003] In the actual application of a large-area array high-power DTOF depth camera, due to the transmission and processing delays in the internal circuit of the chip, there may be deviations in the delay times of different pixel points, resulting in inconsistent measurement data. If such pixel-level non-uniformity errors are not calibrated and corrected, it will cause systematic distortion in the depth image. Traditional calibration methods need to obtain reference data at a known distance, but in actual implementation, they are often restricted by space. For example, in short-distance high-power ranging, high-intensity echo photons will trigger the pileup effect of avalanche photodiodes (APDs), that is, the overlap of photons in the time domain will cause incorrect timestamp parsing by the leading-edge discrimination circuit, resulting in non-linear deviation of the ranging value. Summary of the Invention

[0004] Based on the above problems, the embodiments of the present application provide a method, device, equipment and storage medium for correcting the distance deviation value of a DTOF module, aiming to solve the problem of calibrating the distance deviation value of the entire pixel plane by the depth camera while avoiding the influence of the pileup effect in a limited space.

[0005] In a first aspect, the embodiments of the present application provide a method for correcting the distance deviation value of a DTOF module, including:

[0006] After the DTOF module equipped with an attenuation sheet emits a laser signal, an echo signal reflected by a target object is acquired at the current distance, so as to obtain the distance deviation value of each pixel point and the confidence level of each pixel point. Wherein, the distance deviation value represents the distance deviation between the actual current distance and the detection distance. If the confidence level of a pixel point is credible, the confidence level is marked as 1. If the confidence level of a pixel point is not credible, the confidence level is marked as 0;

[0007] For a first pixel column with a confidence level of 0, if there are no N consecutive pixel points with a confidence level of 1 in the first pixel column, all the distance deviation values of the reference column are used as the distance deviation values of the first pixel column; where N is an integer greater than 2;

[0008] For a second pixel column with a confidence level of 0, if there are N consecutive pixel points with a confidence level of 1 at the first central position of the second pixel column, based on the distance deviation value of the pixel point at the second central position of the reference column, the distance deviation value of the pixel point at the central position of the second pixel column, and the distance deviation values of all pixel points of the second pixel column, the distance deviation value of the pixel point with a confidence level of 0 in the second pixel column is calculated;

[0009] Wherein, the reference column is the pixel column closest to the first pixel column or the second pixel column and with the confidence levels of all pixel points being 1.

[0010] Based on the first aspect, in a possible implementation manner, calculating the distance deviation value of the pixel point with a confidence level of 0 in the second pixel column based on the distance deviation value of the pixel point at the second central position of the reference column, the distance deviation value of the pixel point at the central position of the second pixel column, and the distance deviation values of all pixel points of the second pixel column specifically includes:

[0011] Accumulate and average the distance deviation values of the pixel points at the first central position of the second pixel column to obtain the first deviation reference value of the second pixel column;

[0012] Accumulate and average the distance deviation values of the pixel points with a confidence level of 1 at the second central position of the reference column to obtain the second deviation reference value of the reference column;

[0013] Subtract the second deviation reference value from the distance deviation values of all pixel points of the second pixel column to obtain the relative deviation values of all pixel points of the second pixel column;

[0014] Add the relative deviation values of all pixel points of the second pixel column to the first deviation reference value to calculate the distance deviation value of the pixel point with a confidence level of 0 in the second pixel column.

[0015] Based on the first aspect, in a possible implementation manner, the step of adding the relative deviation values of all the pixel points in the second pixel column to the first deviation reference value to calculate the distance deviation value of the pixel points with a confidence level of 0 in the second pixel column includes:

[0016] For the second pixel column, add the relative deviation value of each row of pixel points to the first deviation reference value to obtain the deviation correction value of each row of pixel points;

[0017] For each row of pixel points in the second pixel column, if the pixel point has a confidence level of 1, do not replace the distance deviation value of the current row of pixel points; if the pixel point has a confidence level of 0, replace the distance deviation value of the pixel point with the deviation correction value corresponding to the pixel point.

[0018] Based on the first aspect, in a possible implementation manner, the second center position is:

[0019] If there is only one set of N consecutive pixel points with a confidence level of 1 in the first center position of the second pixel column, record the coordinate range of the N consecutive pixel points with a confidence level of 1 in the second pixel column, and use it as the second center position of the reference column;

[0020] If there are at least two sets of N consecutive pixel points with a confidence level of 1 in the first center position of the second pixel column, record the coordinate range of the first center position of the second pixel column, and use it as the second center position of the reference column.

[0021] Based on the first aspect, in a possible implementation manner, when N = 3,

[0022] If there is only one set of 3 consecutive pixel points with a confidence level of 1 in the first center position of the second pixel column, record the coordinate range A1 - A2 of the 3 consecutive pixel points with a confidence level of 1 in the second pixel column, and use it as the second center position A1 - A2 of the reference column;

[0023] If there are at least two sets of 3 consecutive pixel points with a confidence level of 1 in the first center position of the second pixel column, record the coordinate range B1 - B2 of the first center position of the second pixel column, and use it as the second center position B1 - B2 of the reference column, where B1 ≤ A1 < A2 ≤ B2.

[0024] In a second aspect, an embodiment of the present application further provides a distance deviation value correction device for a DTOF module, including:

[0025] The echo signal acquisition unit is configured to obtain the echo signal reflected by the target object at the current distance after the DTOF module equipped with the attenuation sheet emits the laser signal, so as to obtain the distance deviation value of each pixel point and the confidence of each pixel point. Wherein, the distance deviation value represents the distance deviation between the actual current distance and the detection distance. If the confidence of a pixel point is reliable, the confidence is marked as 1. If the confidence of a pixel point is not reliable, the confidence is marked as 0;

[0026] The processing unit is configured to, for the first pixel column with a confidence of 0, if there are no N consecutive pixel points with a confidence of 1 in the first pixel column, use all the distance deviation values of the reference column as the distance deviation values of the first pixel column; where N is an integer greater than 2;

[0027] The processing unit is further configured to, for the second pixel column with a confidence of 0, if there are N consecutive pixel points with a confidence of 1 at the first central position of the second pixel column, calculate the distance deviation value of the pixel points with a confidence of 0 in the second pixel column based on the distance deviation value of the pixel point at the second central position of the reference column, the distance deviation value of the pixel point at the central position of the second pixel column, and the distance deviation values of all the pixel points in the second pixel column;

[0028] Wherein, the reference column is the pixel column closest to the first pixel column or the second pixel column and with the confidence of all pixel points being 1.

[0029] Based on the second aspect, in a possible implementation, the device further includes:

[0030] The processing unit is further configured to accumulate and average the distance deviation values of the pixel points at the first central position of the second pixel column to obtain the first deviation reference value of the second pixel column;

[0031] The processing unit is further configured to accumulate and average the distance deviation values of the pixel points with a confidence of 1 at the second central position of the reference column to obtain the second deviation reference value of the reference column;

[0032] The processing unit is further configured to subtract the second deviation reference value from the distance deviation values of all the pixel points in the second pixel column to obtain the relative deviation values of all the pixel points in the second pixel column;

[0033] The processing unit is further configured to add the first deviation reference value to the relative deviation values of all the pixel points in the second pixel column to calculate the distance deviation value of the pixel points with a confidence of 0 in the second pixel column.

[0034] Based on the second aspect, in a possible implementation, the device further includes:

[0035] The processing unit is further configured to, for the second pixel column, add the relative deviation value of each row of pixel points to the first deviation reference value to obtain the deviation correction value of each row of pixel points;

[0036] The processing unit is further configured to, for each row of pixel points in the second pixel column, if the pixel point has a confidence level of 1, the distance deviation value of the current row of pixel points is not replaced; if the pixel point has a confidence level of 0, the distance deviation value of the pixel point is replaced with the deviation correction value corresponding to the pixel point.

[0037] Each functional unit of the second aspect is used to implement the method described in the first aspect and any possible implementation manner of the first aspect.

[0038] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0039] A central processing unit, a memory, and an input / output interface;

[0040] The memory is a transient storage memory or a persistent storage memory;

[0041] The central processing unit is configured to communicate with the memory and execute the instruction operations in the memory to execute the method described in the first aspect and any possible implementation manner of the first aspect.

[0042] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect and any possible implementation manner of the first aspect is executed.

[0043] From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages:

[0044] In the embodiments of the present application, in a limited scenario space, the method of adding an attenuation sheet to the DTOF module can be adopted to control the echo energy, so as to reduce the ranging deviation caused by the pileup phenomenon; however, the addition of the attenuation sheet may cause abnormal reception of photons in the vignetting area of the DTOF sensor. Therefore, in the embodiments of the present application, the received echo signals are used to screen out the untrustworthy pixel points affected by the vignetting area or echo energy attenuation; then, the distance deviation value of the pixel points with a confidence level of 1 in the reference column is used as a reference, so that the pixel points with a confidence level of 0 in the second pixel column can be corrected and matched with the overall ranging deviation trend of the reference column, and finally, the ranging deviation of the entire pixel plane is ensured to be corrected. The embodiments of the present application can not only correct the wrong distance deviation value, eliminate the distance error caused by the pileup phenomenon, but also reduce the complexity of experimental calibration, so that the large-area high-power DTOF depth camera can be stably applied in a complex environment, and improve the accuracy and consistency of the depth image. Description of the Drawings

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to the provided drawings.

[0046] Figure 1 It is a schematic diagram of a vignetting area provided by the embodiments of the present application;

[0047] Figure 2 It is a schematic diagram of a distance deviation value correction system for a DTOF module provided by the embodiments of the present application;

[0048] Figure 3 It is a schematic diagram of a distance deviation value correction method flow for a DTOF module provided by the embodiments of the present application;

[0049] Figure 4 It is a schematic diagram of the arrangement of pixel points within a pixel column provided by the embodiments of the present application;

[0050] Figure 5 It is a comparison schematic diagram of pixel columns before and after correction provided by the embodiments of the present application;

[0051] Figure 6 It is a schematic diagram of the structure of a distance deviation value correction device for a DTOF module provided by the embodiments of the present application;

[0052] Figure 7 It is a schematic diagram of the structure of an electronic device provided by the embodiments of the present application. Detailed Embodiments

[0053] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0054] In the actual application of a large-area array high-power DTOF depth camera, due to the transmission and processing delays in the internal circuit of the chip, there may be deviations in the delay times of different pixel points, resulting in inconsistent measurement data. If such pixel-level non-uniformity errors are not calibrated and corrected, systematic distortion will occur in the depth image. Traditional calibration methods usually require obtaining reference data at a known distance, but in actual implementation, they are often restricted by space. For example, in short-distance high-power ranging, the echo signal intensity is high, and the number of photon triggers increases significantly. However, after each photon event is detected, the avalanche diode detector takes several nanoseconds to return to its normal operating state, and this period of time is also called the dead time. During the dead time, the detector cannot respond to new photon events.

[0055] Due to the high photon trigger frequency under high-power ranging conditions and the existence of dead time in the detector, photon events are prone to pileup effects, resulting in a shorter calculated photon flight time and causing errors in the detection distance calculated based on the flight time. During the depth camera calibration process, if there are deviations in the calculated detection distance, the distance deviation value (offset) used for depth image restoration will also be incorrect, affecting the accuracy and reliability of the depth information.

[0056] Based on this, the embodiments of the present application provide a method for correcting the distance deviation value of a DTOF module. Considering the limited scene space during the calibration of the depth camera, the embodiments of the present application choose to control the echo energy by adding an attenuation film to the DTOF module to reduce the ranging deviation caused by the pileup phenomenon. However, the addition of the attenuation film may affect the dark corner area of the DTOF sensor, resulting in ineffective reception of photons by the pixels in this area, thereby affecting the reliability of the ranging data. Refer to Figure 1 , Figure 1It shows the difference between the vignetting area and the central area in DTOF sensor imaging. Specifically, the central area of the sensor (the white part marked as "1" in the figure) can normally receive light intensity information, that is, the intensity of the reflected echo signal is normal. Therefore, the pixel confidence in this central area is relatively high (confidence = 1), and the distance deviation values of the pixel points in this area can be directly used. The vignetting area (the gray part marked as "0" in the figure) is located around the sensor. Affected by the optical system and the attenuation film (such as the vignetting effect of the lens), the energy of the echo signal is attenuated, and even some pixels cannot effectively receive photons. Due to the small number of photon triggers, the ranging data is unreliable and the confidence is low. Therefore, this area needs to be compensated and corrected.

[0057] In summary, after the DTOF module equipped with an attenuation film emits a laser signal in the embodiment of the present application, the distance deviation value of each pixel point is obtained by acquiring the echo signal, and the confidence of each pixel point is judged; for the first pixel column with a confidence of 0, if there are no N consecutive pixel points with a confidence of 1 in this first pixel column, then all the distance deviation values of the reference column are used as the distance deviation values of the first pixel column; for the second pixel column with a confidence of 0, if there are N consecutive pixel points with a confidence of 1 at the first central position of this second pixel column, then based on the distance deviation values of the pixel points at the second central position of the reference column, the distance deviation values of the pixel points at the central position of the second pixel column, and the distance deviation values of all the pixel points in the second pixel column, the distance deviation values of the pixel points with a confidence of 0 in the second pixel column are calculated; where N is an integer greater than 2; the reference column is the pixel column closest to the first pixel column or the second pixel column and with the confidence of all pixel points being 1.

[0058] It can be understood that after analyzing the confidence of each pixel point according to the echo signal in the embodiment of the present application, the distance deviation value of the pixel point with a confidence of 0 is further corrected to obtain the corrected correct distance deviation value. Considering that there is similarity in the distance deviation values of the receiving-end sensor of the DTOF module between different rows and columns, that is, the relative distance deviation relationship of the pixels in different rows of adjacent columns is similar, the embodiment of the present application utilizes this characteristic to correct the pixel points with low confidence. In addition, since the central position of the sensor can normally receive photons, the correct distance deviation value of the pixel point with a confidence of 0 in the vignetting area of the sensor can be calculated by combining the relative deviation relationship of adjacent columns and the distance deviation value of the central position.

[0059] By the above method, the embodiment of the present application corrects the distance deviation values in the pixel column with a confidence of 0 by using the distance deviation values of the reference column, and finally effectively eliminates the distance error caused by the photon pile-up phenomenon, thereby improving the accuracy of depth measurement. At the same time, the present application does not rely on a large-space calibration scene and is applicable to the calibration and optimization of depth cameras in restricted environments.

[0060] Based on the method for correcting the distance deviation value of the above DTOF module, an embodiment of the present application provides a system for correcting the distance deviation value of a DTOF module, which can be seen in Figure 2 , Figure 2 The system for correcting the distance deviation value of the DTOF module shown can include multiple terminal devices 201 and multiple servers 202. Among them, a communication connection is established between any terminal device and any server. The terminal device 201 can be various types of terminal devices. For example, the terminal device 201 includes any one or more of a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart vehicle, and a smart wearable device. A game client, a multimedia playback client, a social client, a browser client, an information flow client, an education client, etc. can run in the terminal device 201. The server 202 can be a server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal device 201 and the server 202 can be directly or indirectly communicatively connected through wired or wireless communication methods, and the embodiments of the present application do not limit this here.

[0061] In a feasible embodiment, the method for correcting the distance deviation value of the above DTOF module can be only executed by Figure 2 the terminal device 201 in the system for correcting the distance deviation value of the DTOF module shown. The terminal device 201 includes a laser device, a single-photon avalanche diode array, etc. in the DTOF module transmitter. The specific execution process is as follows: The terminal device 201 controls the laser device in the transmitter to emit a laser signal, and after the laser device in the transmitter emits the laser signal to the target object, the terminal device 201 acquires the echo signal reflected by the target object to the single-photon avalanche diode array in the receiver. After that, the terminal device 201 obtains the distance deviation value of each pixel point through the acquired echo signal and judges the confidence of each pixel point. For the first pixel column with a confidence of 0, if there are no continuous N pixel points with a confidence of 1 in this first pixel column, then all the distance deviation values of the reference column are used as the distance deviation values of the first pixel column; at the same time, for the second pixel column with a confidence of 0, if there are continuous N pixel points with a confidence of 1 at the first central position of this second pixel column, then based on the distance deviation value of the pixel point at the second central position of the reference column, the distance deviation value of the pixel point at the central position of the second pixel column, and the distance deviation values of all the pixel points of the second pixel column, the distance deviation value of the pixel point with a confidence of 0 in the second pixel column is calculated.

[0062] Optionally, the method for correcting the distance deviation value of the above DTOF module can also be executed only by Figure 2 the server 202 in the distance deviation value correction system of the DTOF module shown. At this time, the server 202 includes a laser device, a single-photon avalanche diode array, etc. in the DTOF module transmitter; the specific execution process can refer to the execution process of the terminal device 201 and will not be elaborated here.

[0063] In another feasible embodiment, the method for correcting the distance deviation value of the above DTOF module can run in the distance deviation value correction system of the DTOF module. The distance deviation value correction system of the DTOF module can include a terminal device and a server. Specifically, the method for correcting the distance deviation value of the DTOF module can be jointly completed by Figure 2 the terminal device 201 and the server 202 included in the distance deviation value correction system of the DTOF module shown. Among them, the terminal device 201 can include a laser device, a single-photon avalanche diode array, etc. in the transmitter. The specific execution process is as follows: The terminal device 201 can control the laser device in the transmitter to emit a laser signal, and after the laser device in the transmitter emits the laser signal to the target object, obtain the return light signal reflected by the target object to the single-photon avalanche diode array in the receiver. Further, the terminal device 201 will analyze the echo signal to obtain the distance deviation value of each pixel point and the confidence of each pixel point, and then send the distance deviation value of each pixel point and the confidence of each pixel point to the server 202. Then, the server 202 analyzes and processes the distance deviation value of each pixel point and the confidence of each pixel point. Specifically, for the first pixel column with a confidence of 0, if there are no N consecutive pixel points with a confidence of 1 in the first pixel column, the server 202 can use all the distance deviation values of the reference column as the distance deviation values of the first pixel column; where N is an integer greater than 2. Finally, for the second pixel column with a confidence of 0, if there are N consecutive pixel points with a confidence of 1 at the first central position of the second pixel column, the server 202 can calculate the distance deviation value of the pixel point with a confidence of 0 in the second pixel column based on the distance deviation value of the pixel point at the second central position of the reference column, the distance deviation value of the pixel point at the central position of the second pixel column, and the distance deviation values of all the pixel points in the second pixel column. Optionally, the server 202 can also send the corrected distance deviation value of the pixel point with a confidence of 0 to the terminal device 201.

[0064] Based on the above method for correcting the distance deviation value of the DTOF module and the system for correcting the distance deviation value of the DTOF module, an embodiment of the present application provides a method for correcting the distance deviation value of the DTOF module. It should be noted that the method for correcting the distance deviation value of the DTOF module in the embodiments of the present invention is mainly illustrated by taking a direct time-of-flight detection system as an example. The method for correcting the distance deviation value of the DTOF module in the embodiments of the present application can also be applied to other scenarios for measuring distance, which is not limited herein.

[0065] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of a method for correcting the distance deviation value of the DTOF module provided by an embodiment of the present application. Figure 3 The method for correcting the distance deviation value of the DTOF module shown can be executed by Figure 2 the server or terminal device shown in Figure 3 The method for correcting the distance deviation value of the DTOF module shown may include steps S301 - S303:

[0066] S301: After the DTOF module equipped with an attenuation sheet emits a laser signal, obtain the echo signal reflected by the target object at the current distance, so as to obtain the distance deviation value of each pixel point and the confidence level of each pixel point. Among them, the distance deviation value represents the distance deviation between the actual current distance and the detected distance. If the confidence level of a pixel point is credible, the confidence level is marked as 1. If the confidence level of a pixel point is not credible, the confidence level is marked as 0;

[0067] In the embodiments of the present application, the emitting end of the DTOF module for emitting laser signals has a laser device. Exemplarily, the laser device can be a vertical cavity surface emitting laser, a helium-neon laser, a free electron laser, etc. The target object refers to an object that reflects the laser signal. The target object can be a pre-set object for reflecting the laser signal, or an object randomly contacted by the laser signal. The target object can also be an object with a surface that meets the preset reflectivity. In practical applications, the reflectivity of the target object can be reduced to the lowest level, such as a reflectivity of 1.5%, and then observe the echo signal reflected by the target object. Optionally, if an attenuation sheet has been paired with the emitting end of the module currently, but if the stacking phenomenon still exists, an attenuation sheet with a higher attenuation ratio can be replaced (for example, increasing the attenuation level of a neutral density filter), or multiple layers of attenuation sheets can be used to further reduce the laser energy output.

[0068] After obtaining the echo signal, a histogram can be generated to analyze whether the number of photons is affected by pileup. The histogram of the echo signal is used to characterize the mapping relationship between the number of photons and the time bins. The specific way to generate the histogram of the echo signal can be to detect the number of photons triggering the single-photon avalanche diode array in the echo signal and record the time bins corresponding to each number of photons; then, based on the multiple numbers of photons and the time bins corresponding to each number of photons, the histogram of the echo signal can be generated. Among them, the time bin corresponding to any number of photons refers to the time point when the photons with the corresponding number of photons trigger the single-photon avalanche diode array, and its unit is bin.

[0069] After obtaining the histogram corresponding to the echo signal, analyze the histogram generated by the echo signal. When the peak value of the histogram is lower than the first preset threshold, it indicates that the echo photons do not trigger the pileup effect. Among them, the number of photons refers to the number of photons triggering the single-photon avalanche diode array in the echo signal; the single-photon avalanche diode array can be composed of one or more single-photon avalanche diodes. The present application does not limit the specific value of the first preset threshold, which can be determined according to actual needs. For example, the first preset threshold is set to 1000, that is, when the number of photons received by each pixel is greater than 1000, it is considered that the pileup phenomenon may occur, affecting the ranging accuracy.

[0070] Through the above method, the influence of the over-strong energy of the echo signal received by the receiving end due to limited space can be solved.

[0071] After determining that the DTOF module is not affected by the pileup phenomenon, the untrusted pixel points affected by the vignetting area or the echo energy attenuation can be further screened out according to the number of photons received by each pixel point; optionally, if the confidence of a pixel point is trusted, the confidence is marked as 1, and if the confidence of a pixel point is untrusted, the confidence is marked as 0. For a single-photon avalanche diode sensor, the confidence of a pixel point can be based on the photon detection count. For example, the confidence threshold can be set above 50-100 photons to ensure sufficient signal strength. The specific value is not limited in the embodiments of the present application and can be set according to actual needs. It should be noted that the method of distinguishing trusted and untrusted pixel points is not limited to using 0 and 1 as marks, and other methods that can clearly distinguish trusted and untrusted, such as different identifiers, colors, or other methods suitable for specific application scenarios, can also be used for distinguishing and marking.

[0072] S302: For the first pixel column with a confidence of 0, if there are no N consecutive pixel points with a confidence of 1 in the first pixel column, then all the distance deviation values of the reference column are used as the distance deviation values of the first pixel column; where N is an integer greater than 2.

[0073] When there are pixel points with a confidence level of 0 in a certain pixel column, it means that the distance deviation values of some pixel points in this pixel column are unreliable. Therefore, it is possible to determine whether there is reliable distance deviation value data to assist in correcting the distance deviation values of the untrusted pixel points in this column by checking whether there are N consecutive pixel points with a confidence level of 1 in this pixel column. Among them, the value of N can be set according to actual needs. Specifically, N can be an integer greater than 2, indicating that only when at least N consecutive pixel points have a confidence level of 1, the data of this column is considered partially credible, otherwise the distance deviation values of the entire reference column will be directly used as the replacement data for this pixel column.

[0074] The purpose of the embodiment of this application is to ensure the reliability of the distance deviation value data of the pixel points in the pixel column. When there is a lack of sufficient reliable pixel point data in a certain column of ranging data as a whole, the data of the reference column is directly used for replacement to improve the accuracy and consistency of the depth image.

[0075] S303: For the second pixel column with a confidence level of 0, if there are N consecutive pixel points with a confidence level of 1 at the first central position of the second pixel column, then based on the distance deviation value of the pixel points at the second central position of the reference column, the distance deviation value of the pixel points at the central position of the second pixel column, and the distance deviation values of all pixel points of the second pixel column, calculate the distance deviation value of the pixel points with a confidence level of 0 in the second pixel column.

[0076] In the embodiment of this application, it is possible to determine whether there are N consecutive pixel points with a confidence level of 1 in a specific first central position area in the second pixel column. Here, the division of the first central area can be set in combination with the actual application. Exemplarily, the preset central area is based on the center of the current pixel column, and a specified number of rows are extended upward and downward from the center point to form a symmetric central area. Exemplarily, if the current pixel column contains 100 rows, the central position is the 50th row, and 5 rows are extended upward and downward from the central position. Finally, the formed central area range is [45, 55].

[0077] When there are N consecutive pixel points with a confidence level of 1 in the first central position of the second pixel column, if so, it is considered that part of the ranging data of this column is credible and can be used to calculate the correction value of the invalid pixel points. When calculating, comprehensively utilize the data in three aspects, that is, the distance deviation value of the pixel points at the second central position of the reference column, the distance deviation value of the pixel points at the central position of the second pixel column, and the distance deviation values of all pixel points of the second pixel column. The corrected distance deviation value of the pixel points with a confidence level of 0 in the second pixel column can be deduced to achieve precise correction of the invalid pixel points and match the overall change trend of the distance deviation values between the pixel points of the reference column, effectively reducing the ranging error and improving the accuracy of the depth image.

[0078] In the embodiment of the present application, aiming at the ranging deviation problem that may be caused by limited scene space during the calibration of a depth camera, a method of adding an attenuation sheet to the DTOF module is adopted to control the echo energy, so as to reduce the ranging deviation caused by the pileup effect; however, the addition of the attenuation sheet may cause abnormal reception of photons in the vignetting area of the DTOF sensor. Therefore, the embodiment of the present application screens out the untrustworthy pixel points affected by the vignetting area or echo energy attenuation according to the received echo signal; then, using the distance deviation value of the pixel points with a confidence level of 1 in the reference column as a reference, the pixel points with a confidence level of 0 in the second pixel column can be corrected and matched with the overall ranging deviation trend of the reference column, and finally, the ranging deviation of the entire pixel plane is ensured to be corrected. The embodiment of the present application can not only correct the wrong distance deviation value and eliminate the distance error caused by the pileup phenomenon, but also reduce the complexity of experimental calibration, enabling the large-area high-power DTOF depth camera to be stably applied in a complex environment and improving the accuracy and consistency of the depth image.

[0079] In a feasible embodiment, based on the distance deviation value of the pixel points at the second center position of the reference column, the distance deviation value of the pixel points at the center position of the second pixel column, and the distance deviation value of all pixel points in the second pixel column, the distance deviation value of the pixel points with a confidence level of 0 in the second pixel column is calculated, which specifically includes: accumulating and averaging the distance deviation values of the pixel points at the first center position of the second pixel column to obtain the first deviation reference value of the second pixel column; accumulating and averaging the distance deviation values of the pixel points with a confidence level of 1 at the second center position of the reference column to obtain the second deviation reference value of the reference column; subtracting the second deviation reference value from the distance deviation values of all pixel points in the second pixel column to obtain the relative deviation values of all pixel points in the second pixel column; adding the relative deviation values of all pixel points in the second pixel column to the first deviation reference value to calculate the distance deviation value of the pixel points with a confidence level of 0 in the second pixel column.

[0080] Please refer to Figure 4 , assuming that the value of N is 3, and the first center position of the second pixel column is from the 3rd row to the 5th row. Taking Figure 4 the leftmost column 1 as an example, there are pixel points with a confidence level of 0 in column 1, but there are no 3 consecutive pixel points with a confidence level of 1; continue to search in column 2, and it is found that there are pixel points with a confidence level of 0 and there are 3 consecutive pixel points with a confidence level of 1; continue to search in column 3, and it is known through traversal that all pixel points in column 3 have a confidence level of 1. Therefore, column 3 can be used as the reference column to serve as a reference for the correction of the pixel points with a confidence level of 0 in the pixel columns to be corrected (that is, column 1 and column 2 where there are pixel points with a confidence level of 0).

[0081] It should be noted that the principle for the existence of a deviation reference value in a pixel column is that there are N (currently N = 3) consecutive pixel points with a confidence level of 1 in the pixel column. RegardingFigure 4 In the first column of Figure 4 , if there are pixel points with a confidence level of 0, but there are no three consecutive pixel points with a confidence level of 1, and if all pixel points in a certain pixel column have a confidence level of 0, it can be considered that there is no deviation reference value (the deviation reference value can also be called the "central pixel reference") for correcting untrusted pixel points in this column; therefore, for Figure 5 the first column in Figure 5 , it is necessary to replace the distance deviation values corresponding to all pixel points in this column with the distance deviation values of the reference column. Exemplarily, reference can be made to

[0082] For Figure 4 the second column of pixel columns in

[0083] , if there are pixel points with a confidence level of 0 and there are three consecutive pixel points with a confidence level of 1, the distance deviation values of the three consecutive pixel points with a confidence level of 1 in the second column can be accumulated and averaged to obtain the first deviation reference value offsetcenter of the second column, and record the coordinate range where these three consecutive pixel points with a confidence level of 1 are located in the second column (for example, the row coordinate range where these three consecutive pixel points with a confidence level of 1 are located in the second column: row 3 to row 5). Figure 4 Taking the third column reference column as a reference, calculate the average of the accumulated distance deviation values of the pixel points at the second central position of the third column (for example, in the same row coordinate range as the three consecutive pixel points with a confidence level of 1 in the second column, row 3 - row 5) to obtain the second deviation reference value offsetmean of the third column. Subtract the second deviation reference value offsetmean of the third column from the distance deviation value of each pixel point in the third column to obtain the relative deviation value of all pixel points in the third column, that is, [relative deviation value offsetrelative] = [distance deviation value offset of each pixel point in the third column] - [second deviation reference value offsetmean of the third column]. After obtaining the relative deviation value, it can be applied to Figure 4 the second column in

[0084] to obtain the correct distance deviation value. Specifically, [distance deviation value of the corrected second column (also called the "distance correction value")] = [relative deviation value offsetrelative] + [first deviation reference value offsetcenter of the second column].

[0084] Please refer to Figure 5 , Figure 5If there are 3 consecutive pixels with a confidence level of 1 in the 3rd to 5th rows of the 2nd column, then calculate the mean of the distance deviation values of the 3 consecutive pixels with a confidence level of 1: (173 + 152 + 131) / 3 = 152, that is, the first deviation reference value offsetcenter = 152. Then calculate the mean of the distance deviation values of the pixels with row coordinates from 3 to 5 in the 3rd column (the reference column): (170 + 150 + 130) / 3 = 150, that is, the second deviation reference value offsetmean = 150. Next, calculate the distance deviation values of all pixels in the 2nd column minus the second deviation reference value: [200, 180, 170, 150, 130, 170, 190] – 150, to obtain the relative deviation value offsetrelative of the 2nd column = [50, 30, 20, 0, -20, 20, 40]. And by adding the relative deviation values of all pixels in the second pixel column to the first deviation reference value, the distance deviation value of the corrected 2nd column can be obtained: [50, 30, 20, 0, -20, 20, 40] + 152 = [202, 182, 172, 152, 132, 172, 192].

[0085] In the embodiment of the present application, through the comparison calculation between the reference column and the pixel column to be corrected, pixel-level error compensation is realized, ensuring the integrity and reliability of the detection distance data. At the same time, in the case where a reliable deviation reference value cannot be obtained, the ranging data of the reference column is directly adopted, ensuring the consistency of the depth image and improving the stability of the large-area array DTOF depth camera.

[0086] In another optional embodiment provided by the embodiment of the present application, adding the relative deviation values of all pixels in the second pixel column to the first deviation reference value to calculate the distance deviation value of the pixel with a confidence level of 0 in the second pixel column includes: for the second pixel column, adding the relative deviation value of each row of pixels to the first deviation reference value to obtain the deviation correction value of each row of pixels; for each row of pixels in the second pixel column, if it is a pixel with a confidence level of 1, then do not replace the distance deviation value of the current row of pixels; if it is a pixel with a confidence level of 0, then replace the distance deviation value of the pixel with the deviation correction value of the corresponding pixel.

[0087] Based on the above Figure 4 corresponding example, for Figure 4 each row of the 2nd column, calculate the sum of the relative deviation value offsetrelative of the pixels in this row and the first deviation reference value offsetcenter to obtain the deviation correction value of this row. For the pixels with a confidence level of 1 in the 2nd column, their original distance deviation values can be retained without replacement. For example Figure 4The confidence of the pixel at the 3rd row and 2nd column is 1, and the original distance deviation value of this pixel is 173. However, since the confidence of this pixel is 1, the original distance deviation value of 173 will not be replaced. For pixels with a confidence of 0, the original distance deviation value is replaced with the deviation correction value corresponding to the row to achieve error correction. For example Figure 4 The confidence of the pixel at the 1st row and 2nd column is 0, and the original distance deviation value of this pixel is -10. After calculating the distance correction value for the 2nd column, the distance correction value of 202 corresponding to the 1st row can be selected and used to replace the original distance deviation value of -10.

[0088] Based on the calculation of the deviation correction value and combined with the confidence determination in the embodiments of the present application, the distance deviation value data of reliable pixels remains unchanged, while the distance deviation value of unreliable pixels is corrected, thereby optimizing the quality of the depth image and improving the ranging accuracy and stability of the large-area array DTOF depth camera.

[0089] Optionally, based on one or more of the above Figure 3 corresponding embodiments, in another feasible embodiment, to improve the accuracy of distance deviation value correction, the selection of the central position of the second pixel column can be optimized. Specifically, the second central position is as follows: If there is only one set of N consecutive pixels with a confidence of 1 in the first central position of the second pixel column, record the coordinate range of the N consecutive pixels with a confidence of 1 in the second pixel column and use it as the second central position of the reference column; If there are at least two sets of N consecutive pixels with a confidence of 1 in the first central position of the second pixel column, record the coordinate range of the first central position of the second pixel column and use it as the second central position of the reference column.

[0090] Exemplarily, assume a pixel column to be corrected with a confidence of 0. The number of rows of this pixel column to be corrected is 100, and the preset central region is defined as [45, 55] (that is, the central position of the row is the 50th row, and it extends 5 rows upward and downward respectively). The judgment condition for the existence of the central pixel reference is the existence of 3 consecutive pixels with a confidence of 1 (i.e., N = 3).

[0091] When calculating the first deviation reference value of the second pixel column and determining the second central position of the reference column, there are the following 3 cases. For specific reference, please refer to the following examples:

[0092] Case 1: Assume that only the 48th - 50th rows in the pixel column to be corrected meet the condition of "three consecutive pixels with a confidence of 1". Then, sum the distance deviation values of the pixels with a confidence of 1 in the 48th - 50th rows of this column and divide by 3 to obtain the first deviation reference value offsetcenter of the pixel column to be corrected; record the row coordinate range [48, 50] of the consecutive pixels with a confidence of 1 in this column and use it as the second central position of the reference column. Then select the nearest reference column where all pixels have a confidence of 1. When calculating the second pixel reference value offsetmean of the reference column, the distance deviation values of the pixels within the second central position [48, 50] of the reference column are used as the calculation basis. Among them, calculating the second pixel reference value of the reference column can refer to the example in Figure 4 , which will not be elaborated in this application.

[0093] Case 2: Assume that in the first central position of the pixel column to be corrected, the 45th - 46th rows, the 48th - 50th rows, and the 53rd - 55th rows are pixels with a confidence of 1. The 48th - 50th rows and the 53rd - 55th rows meet the condition of "three consecutive pixels with a confidence of 1". Then, sum the distance deviation values of the pixels with a confidence of 1 in the 48th - 50th rows and the 53rd - 55th rows of this column and divide by 6 to obtain the first deviation reference value offsetcenter of the column to be corrected; record the coordinate range of the first central position of the pixel column to be corrected as [45, 55], and use [45, 55] as the second central position of the reference column. Then select the nearest reference column where all pixels have a confidence of 1. When calculating the second pixel reference value offsetmean of the reference column, the distance deviation values of the pixels within the second central position [45, 55] of the reference column are used as the calculation basis.

[0094] Case 3: Assume that in the first central position of the pixel column to be corrected, the 45th - 46th rows, the 48th - 50th rows, and the 53rd - 55th rows are pixels with a confidence of 1. The 48th - 50th rows and the 53rd - 55th rows both meet the condition of "three consecutive pixels with a confidence of 1". Then, sum the distance deviation values of all pixels with a confidence of 1 in the first central position of the pixel column to be corrected (i.e., the pixels in the 45th - 46th rows, the 48th - 50th rows, and the 53rd - 55th rows), and divide by the number of all pixels with a confidence of 1 in the first central position (i.e., 8), so as to obtain the first deviation reference value of the pixel column to be corrected; record the coordinate range [45, 55] of the first central position of the pixel column to be corrected and use it as the second central position of the reference column. Then select the nearest reference column where all pixels have a confidence of 1. When calculating the second pixel reference value offsetmean of the reference column, the distance deviation values of the pixels within the second central position [45, 55] of the reference column are used as the calculation basis.

[0095] Through the above method, the second central position of the reference column can be flexibly determined in the embodiments of the present application, making the correction calculation of the distance deviation value more targeted, reducing the ranging error, and improving the accuracy of the depth image. In different confidence distribution cases, this method can effectively select an appropriate reference area (i.e., the second central position of the reference column) to optimize the ranging correction effect.

[0096] In a feasible embodiment, when N = 3, if there is only one group of 3 consecutive pixels with a confidence of 1 in the first central positions of the second pixel column, record the coordinate range A1 - A2 of the 3 consecutive pixels with a confidence of 1 in the second pixel column, and use it as the second central position A1 - A2 of the reference column; if there are at least two groups of 3 consecutive pixels with a confidence of 1 in the first central positions of the second pixel column, record the coordinate range B1 - B2 of the first central positions of the second pixel column, and use it as the second central position B1 - B2 of the reference column, where B1 ≤ A1 < A2 ≤ B2.

[0097] In the embodiments of the present application, if there is only one group of N (N = 3) consecutive pixels with a confidence of 1 in the first central positions of the second pixel column, record the coordinate range A1 - A2 of this group of pixels, and select the same coordinate range A1 - A2 in the reference column as the reference area (i.e., the second central position). If there are multiple groups (at least two groups) of N (N = 3) consecutive pixels with a confidence of 1, the reference area (i.e., the second central position) of the reference column will be extended to the entire range B1 - B2 of the first central positions, where B1 ≤ A1 < A2 ≤ B2, ensuring that the reference area is larger, so as to include more valid data to correct the ranging error. It can be understood that for a single group of 3 consecutive pixels with a confidence of 1, the reference area of the reference column is smaller and is suitable for local correction. For multiple groups of 3 consecutive pixels with a confidence of 1, the selection range of the reference area of the reference column can be extended to provide more comprehensive data support, making the correction effect more stable and accurate, and avoiding the problem of unstable data in a single area. The embodiments of the present application enable reasonable selection of the reference area in different scenarios (different confidence distributions), improving the reliability and accuracy of the ranging data of the depth image.

[0098] To implement the distance deviation value correction method of the DTOF module in the embodiments of the present application, the embodiments of the present application also provide a distance deviation value correction device for the DTOF module, as Figure 6 shown. The device 600 includes:

[0099] The echo signal acquisition unit 601 is configured to obtain the echo signal reflected by the target object at the current distance after the DTOF module equipped with the attenuation sheet emits the laser signal, so as to obtain the distance deviation value of each pixel point and the confidence level of each pixel point. Wherein, the distance deviation value represents the distance deviation between the actual current distance and the detection distance. If the confidence level of a pixel point is reliable, the confidence level is marked as 1. If the confidence level of a pixel point is not reliable, the confidence level is marked as 0;

[0100] The processing unit 602 is configured to, for the first pixel column with a confidence level of 0, if there are no N consecutive pixel points with a confidence level of 1 in the first pixel column, use all the distance deviation values of the reference column as the distance deviation values of the first pixel column; where N is an integer greater than 2;

[0101] The processing unit 602 is further configured to, for the second pixel column with a confidence level of 0, if there are N consecutive pixel points with a confidence level of 1 at the first central position of the second pixel column, calculate the distance deviation values of the pixel points with a confidence level of 0 in the second pixel column based on the distance deviation values of the pixel points at the second central position of the reference column, the distance deviation values of the pixel points at the central position of the second pixel column, and the distance deviation values of all the pixel points in the second pixel column;

[0102] Wherein, the reference column is the pixel column closest to the first pixel column or the second pixel column and with the confidence levels of all pixel points being 1.

[0103] Based on the second aspect, in a possible implementation manner, the device 600 further includes:

[0104] The processing unit 602 is further configured to accumulate and average the distance deviation values of the pixel points at the first central position of the second pixel column to obtain the first deviation reference value of the second pixel column;

[0105] The processing unit 602 is further configured to accumulate and average the distance deviation values of the pixel points with a confidence level of 1 at the second central position of the reference column to obtain the second deviation reference value of the reference column;

[0106] The processing unit 602 is further configured to subtract the second deviation reference value from the distance deviation values of all the pixel points in the second pixel column to obtain the relative deviation values of all the pixel points in the second pixel column;

[0107] The processing unit 602 is further configured to add the relative deviation values of all the pixel points in the second pixel column to the first deviation reference value to calculate the distance deviation values of the pixel points with a confidence level of 0 in the second pixel column.

[0108] Based on the second aspect, in a possible implementation manner, the device 600 further includes:

[0109] The processing unit 602 is further configured to, for the second pixel column, add the relative deviation value of each row of pixel points to the first deviation reference value to obtain a deviation correction value for each row of pixel points;

[0110] The processing unit 602 is further configured to, for each row of pixel points in the second pixel column, if the pixel point has a confidence level of 1, the distance deviation value of the current row of pixel points is not replaced; if the pixel point has a confidence level of 0, the distance deviation value of the pixel point is replaced with the corresponding deviation correction value.

[0111] Based on the second aspect, in a possible implementation,

[0112] The position recording unit 603 is configured to, if there is only one set of N consecutive pixel points with a confidence level of 1 in the first central position of the second pixel column, record the coordinate range of the N consecutive pixel points with a confidence level of 1 in the second pixel column and use it as the second central position of the reference column;

[0113] The position recording unit 603 is further configured to, if there are at least two sets of N consecutive pixel points with a confidence level of 1 in the first central position of the second pixel column, record the coordinate range of the first central position of the second pixel column and use it as the second central position of the reference column.

[0114] Based on the second aspect, in a possible implementation, the apparatus 600 further includes:

[0115] The position recording unit 603 is further configured to, if there is only one set of 3 consecutive pixel points with a confidence level of 1 in the first central position of the second pixel column, record the coordinate range A1 - A2 of the 3 consecutive pixel points with a confidence level of 1 in the second pixel column and use it as the second central position A1 - A2 of the reference column;

[0116] The position recording unit 603 is further configured to, if there are at least two sets of 3 consecutive pixel points with a confidence level of 1 in the first central position of the second pixel column, record the coordinate range B1 - B2 of the first central position of the second pixel column and use it as the second central position B1 - B2 of the reference column, where B1 ≤ A1 < A2 ≤ B2.

[0117] Each functional unit of the second aspect is used to implement the methods described in the above first aspect and any possible implementation manner of the first aspect. Among them, the echo signal acquisition unit 601, the processing unit 602, and the position recording unit 603 in the device 600 can be implemented by software or by hardware. In practical applications, the processing unit 602 can be implemented by a processor in an electronic device in combination with a communication interface, and the echo signal acquisition unit 601 and the position recording unit 603 can be implemented by a communication interface in a distance deviation value correction device of a DTOF module.

[0118] It should be noted that when the distance deviation value correction device of the DTOF module provided in the above embodiment performs distance deviation value correction, only the above division of each program module is used as an example. In practical applications, the above processing can be allocated to different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the distance deviation value correction device of the DTOF module provided in the above embodiment and the embodiment of the distance deviation value correction method of the DTOF module belong to the same concept. For the specific implementation process, please refer to the method embodiment, which will not be elaborated here.

[0119] Based on the hardware implementation of the above program module, and in order to implement a distance deviation value correction method of a DTOF module provided in an embodiment of the present application, an embodiment of the present application also provides an electronic device, such as Figure 7 shown, the electronic device 700 includes:

[0120] A central processing unit 701, a memory 702, and an input / output interface 703;

[0121] The memory 702 is a transient storage memory or a persistent storage memory;

[0122] The central processing unit 701 is configured to communicate with the memory 702 and execute the instruction operations in the memory 702 to execute any one of the above distance deviation value correction methods of the DTOF module.

[0123] Of course, in practical applications, each component in the electronic device 700 is coupled together through a bus system 704. It can be understood that the bus system 704 is used to realize the connection and communication between these components. The bus system 704 includes not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for the sake of clear description, in Figure 7 all kinds of buses are labeled as the bus system 704.

[0124] The memory 702 in the embodiments of the present application is used to store various types of data to support the operation of the electronic device 700. Examples of such data include: any computer program for operating on the electronic device 700.

[0125] It can be understood that when the processor in the above-described electronic device executes a computer program, it can also implement the functions of each unit in the corresponding device embodiments described above, which will not be elaborated here. Exemplarily, the computer program can be divided into one or more modules / units, and one or more modules / units are stored in the memory and executed by the processor to complete the various embodiments of the present application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device. For example, the computer program can be divided into the respective units in the above-described electronic device, and each unit can implement the specific functions described in the corresponding electronic device description above.

[0126] The electronic device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the processor and the memory are only examples of the electronic device, and do not constitute a limitation on the electronic device. It may include more or fewer components, or combine certain components, or different components. For example, the electronic device may further include input / output devices, network access devices, a bus, etc.

[0127] The processor may be a central processing unit (CPU, Central Processing Unit), or may also be other general-purpose processors, digital signal processors (DSP, Digital Signal Processor), application specific integrated circuits (ASIC, Application Specific Integrated Circuit), field-programmable gate arrays (FPGA, Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc. The processor is the control center of the electronic device, and connects all parts of the entire electronic device through various interfaces and lines.

[0128] The memory can be used to store computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory and invoking the data stored in the memory, the processor realizes various functions of the electronic device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can include high-speed random access memory and can also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0129] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it executes the method for correcting the distance deviation value of the DTOF module described in any one of the above.

[0130] An embodiment of the present application also provides a computer program product, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, it is used to implement the method for correcting the distance deviation value of the DTOF module described in the first aspect or any specific implementation manner of the first aspect of the embodiments of the present application.

[0131] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here.

[0132] In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings, direct couplings, or communication connections to each other can be through some interfaces, indirect couplings, or communication connections of devices or units, and can be in electrical, mechanical, or other forms.

[0133] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in various embodiments of the present application may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0135] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

Claims

1. A distance deviation value correction method for a DTOF module, characterized in that: include: After the DTOF module equipped with the attenuation sheet emits a laser signal, the echo signal reflected by the target object is obtained at the current distance, so as to obtain the distance deviation value of each pixel and the confidence of each pixel, wherein the distance deviation value represents the distance deviation between the actual current distance and the detection distance. If the confidence of a pixel is credible, the confidence is marked as 1, and if the confidence of a pixel is unreliable, the confidence is marked as 0; For a first pixel column with a confidence level of 0, if the first pixel column does not have N consecutive pixels with a confidence level of 1, all distance deviation values ​​of the reference column are used as distance deviation values ​​of the first pixel column; wherein N is an integer greater than 2; For a second pixel column with a confidence level of 0, if there are N consecutive pixels with a confidence level of 1 at a first center position of the second pixel column, a distance deviation value of pixels with a confidence level of 0 in the second pixel column is calculated based on the distance deviation value of the pixel at the second center position of the reference column, the distance deviation value of the pixel at the center position of the second pixel column, and the distance deviation values ​​of all pixels in the second pixel column; The reference column is a pixel column that is closest to the first pixel column or the second pixel column and in which confidence levels of all pixels are all 1.

2. The method according to claim 1, characterized in that The calculating of the distance deviation value of the pixel point with a confidence level of 0 in the second pixel column based on the distance deviation value of the pixel point at the second center position of the reference column, the distance deviation value of the pixel point at the center position of the second pixel column, and the distance deviation values ​​of all the pixel points in the second pixel column specifically includes: Accumulating and averaging the distance deviation values ​​of the pixel points at the first center position of the second pixel column to obtain a first deviation reference value of the second pixel column; Accumulating and averaging the distance deviation values ​​of the pixel points at the second center position of the reference column with a confidence level of 1 to obtain a second deviation reference value of the reference column; Subtract the second deviation reference value from the distance deviation value of all pixels in the second pixel column to obtain relative deviation values ​​of all pixels in the second pixel column; The relative deviation values ​​of all pixels in the second pixel column are added to the first deviation reference value to calculate the distance deviation value of the pixel with a confidence level of 0 in the second pixel column.

3. The method according to claim 2, characterized in that The step of adding the relative deviation values ​​of all pixels in the second pixel column to the first deviation reference value to calculate the distance deviation value of pixels with a confidence level of 0 in the second pixel column includes: For the second pixel column, a deviation correction value of each row of pixels is obtained by adding the relative deviation value of each row of pixels to the first deviation reference value; For each row of pixels in the second pixel column, if the confidence level of the pixel is 1, the distance deviation value of the pixel in the current row will not be replaced; if the confidence level of the pixel is 0, the distance deviation value of the pixel will be replaced with the deviation correction value corresponding to the pixel.

4. The method according to claim 1, characterized in that: The second center position is: If there is only one set of N consecutive pixels with a confidence level of 1 in the first center position of the second pixel column, then the coordinate range of the N consecutive pixels with a confidence level of 1 in the second pixel column is recorded and used as the second center position of the reference column; If there are at least two groups of N consecutive pixel points with a confidence level of 1 in the first center position of the second pixel column, the coordinate range of the first center position of the second pixel column is recorded and used as the second center position of the reference column.

5. The method according to claim 4, characterized in that When N=3, If there is only one set of three consecutive pixels with a confidence level of 1 in the first center position of the second pixel column, the coordinate range A1-A2 of the three consecutive pixels with a confidence level of 1 in the second pixel column is recorded and used as the second center position A1-A2 of the reference column; If there are at least two groups of three consecutive pixels with a confidence level of 1 in the first center position of the second pixel column, the coordinate range B1-B2 of the first center position of the second pixel column is recorded and used as the second center position B1-B2 of the reference column, where B1≤A1 <A2≤B2。 6. A distance deviation value correction device for a DTOF module, characterized in that: include: The echo signal acquisition unit is used to obtain the echo signal reflected by the target object at the current distance after the DTOF module equipped with the attenuation sheet emits a laser signal, so as to obtain the distance deviation value of each pixel point and the confidence of each pixel point, wherein the distance deviation value represents the distance deviation between the actual current distance and the detection distance, and if the confidence of a pixel point is credible, the confidence is marked as 1, and if the confidence of a pixel point is unreliable, the confidence is marked as 0; a processing unit, configured to, for a first pixel column having a confidence level of 0, use all distance deviation values ​​of the reference column as distance deviation values ​​of the first pixel column if the first pixel column does not have N consecutive pixels having a confidence level of 1; wherein N is an integer greater than 2; The processing unit is further configured to calculate, for a second pixel column having a confidence level of 0, a distance deviation value of pixels having a confidence level of 0 in the second pixel column based on the distance deviation value of the pixel at the second center position of the reference column, the distance deviation value of the pixel at the center position of the second pixel column, and the distance deviation values ​​of all pixels in the second pixel column, if there are N consecutive pixels having a confidence level of 1 at the first center position of the second pixel column; The reference column is a pixel column that is closest to the first pixel column or the second pixel column and in which confidence levels of all pixels are all 1.

7. The device according to claim 6, characterized in that The device also includes: The processing unit is further used to accumulate and average the distance deviation values ​​of the pixel points at the first center position of the second pixel column to obtain a first deviation reference value of the second pixel column; The processing unit is further configured to accumulate and average the distance deviation values ​​of the pixel points at the second center position of the reference column with a confidence level of 1 to obtain a second deviation reference value of the reference column; The processing unit is further configured to subtract the second deviation reference value from the distance deviation value of all pixels in the second pixel column to obtain relative deviation values ​​of all pixels in the second pixel column; The processing unit is further configured to add the relative deviation values ​​of all pixels in the second pixel column to the first deviation reference value to calculate the distance deviation value of the pixel with a confidence level of 0 in the second pixel column.

8. The device according to claim 6, characterized in that The device also includes: The processing unit is further configured to obtain, for the second pixel column, a deviation correction value of each row of pixels by adding the relative deviation value of each row of pixels to the first deviation reference value; The processing unit is also used to, for each row of pixels in the second pixel column, if the pixel has a confidence level of 1, not replace the distance deviation value of the pixel in the current row; if the pixel has a confidence level of 0, replace the distance deviation value of the pixel with a deviation correction value corresponding to the pixel.

9. An electronic device, characterized in that: include: CPU, memory and input / output interface; The memory is a short-term storage memory or a persistent storage memory; The central processing unit is configured to communicate with the memory and execute instruction operations in the memory to perform the distance deviation value correction method of the DTOF module described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the distance deviation value correction method of the DTOF module as described in any one of claims 1 to 5 is performed.

Images