A method and apparatus for calibrating depth data

Abstract

The application provides a depth data calibration method and device, which comprises the following steps: adopting Zhang's calibration method to calibrate a lens of a distance measuring module to obtain intrinsic parameters and distortion parameters of the lens; calculating a calibration factor corresponding to each pixel in a pixel array according to the intrinsic parameters and the distortion parameters of the lens and saving the calibration factor; obtaining an original measurement distance sensed by each pixel during distance measurement; and calling the calibration factor corresponding to each pixel to calibrate the original measurement distance to obtain a corresponding first calibration measurement distance. After calibrating the lens of the distance measuring module, the calibration factor of the pixel is obtained, so that the original measurement distance can be calibrated by the calibration factor during distance measurement of the area array sensor, and the accuracy of the area array depth distance measurement is improved.

Description

Technical Field

[0001] This invention relates to the field of depth sensing technology, and in particular to a depth data calibration method and apparatus. Background Technology

[0002] Depth sensing chips based on dToF (direct time of flight) have been widely used in devices such as LiDAR and mobile phones. dToF technology works by emitting pulsed light into a scene, using a high-performance photoelectric sensor to receive the pulsed light reflected back from the target object, recording the flight time of each received light signal, and performing histogram statistics based on the flight times of multiple pulsed light signals. The depth of the target object is then determined by the flight time that occurs most frequently.

[0003] A dTOF area array sensor consists of multiple SPADs (single photon avalanche diodes) arranged on a rectangular plane. It detects the radial distance of an object by the time-of-flight of each effective pixel. However, due to the tolerances in the manufacturing and assembly process of the lens in the ranging module, and the fact that each pixel has a corresponding spatial detection direction based on its own position, if the raw data measured by each pixel is directly used as the ranging result, there will be a discrepancy with the true radial distance, which will lead to ranging error and affect the accuracy of the area array ranging. Summary of the Invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a depth data calibration method and apparatus to improve the accuracy of area array depth ranging.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] The first aspect of this invention provides a depth data calibration method, comprising the following steps:

[0007] The lens of the ranging module was calibrated using Zhang's calibration method to obtain the lens's intrinsic parameters and distortion parameters.

[0008] Based on the lens's intrinsic distortion parameters, calculate and save the calibration factor corresponding to each pixel in the pixel array;

[0009] During ranging, the raw measured distance is obtained from the sensing of each pixel;

[0010] The original measurement distance is calibrated by calling the calibration factor corresponding to each pixel to obtain the corresponding first calibrated measurement distance.

[0011] In one embodiment, the step of using Zhang's calibration method to calibrate the lens of the ranging module and obtain the lens's intrinsic and distortion parameters includes:

[0012] Project the area light source onto the preset chessboard grid;

[0013] The checkerboard grayscale image is obtained by acquiring and converting the pixel array in the ranging module;

[0014] The Zhang calibration method was used to calibrate the lens of the checkerboard grayscale image, and the intrinsic parameters of the lens were calculated.

[0015] In one embodiment, the step of acquiring and converting the checkerboard grayscale image through the pixel array in the ranging module includes:

[0016] The light signal reflected back from the checkerboard pattern within a preset exposure time is received by the pixel array, and the light signal is incident on the pixel array from different angles;

[0017] A histogram is generated based on the light signal acquired from each pixel;

[0018] The cumulative photon energy in the histogram of each pixel is used as the grayscale value corresponding to each pixel, and a checkerboard grayscale image is obtained.

[0019] In one embodiment, the calibration factor for each pixel is calculated according to the following formula:

[0020]

[0021] r distorted =r*(1+k1*r) 2 +k2*r 4 +k3*r 6 )

[0022]

[0023] F x =s x ×F

[0024] F y =s y ×F

[0025] Among them, the lens's intrinsic parameters include Cx, Cy, and F. x F y The distortion parameters include radial distortion parameters k1, k2, and k3;

[0026] Cx and Cy are the coordinates of the lens optical center, F x F is the lens focal length in pixels along the x-axis. yis the lens focal length in pixels along the y-axis; F is the lens focal length, s x and s y , respectively, represent the number of pixels per millimeter along the x-axis and y-axis; r is the distance between the ideal image point and the imaging center. distorted denoted as , where is the distance between the actual image point and the imaging center, m and n are the coordinates of each pixel in the pixel coordinate system, cosθ is the calibration factor for each pixel, and θ is the incident angle corresponding to each pixel.

[0027] In one embodiment, the step of calling the calibration factor corresponding to each pixel to perform lens calibration on the original measurement distance to obtain the corresponding first calibrated measurement distance specifically involves:

[0028] Lens calibration is performed on the original measurement distance of each pixel using the formula Z = D * cosθ, where Z is the first calibration measurement distance for each pixel, D is the original measurement distance for each pixel, cosθ is the calibration factor for each pixel, and θ is the incident angle corresponding to each pixel.

[0029] In one embodiment, after performing lens calibration on the original measurement distance by calling the calibration factor corresponding to each pixel to obtain the corresponding first calibrated measurement distance, the method further includes:

[0030] The distance to the preset reflector is measured to obtain the original measured distance of each pixel to the reflector, wherein the reflector is parallel to the ranging module and is spaced at a preset distance.

[0031] The offset value of each pixel relative to the reflector is calculated and saved based on the preset distance, the original measured distance of each pixel to the reflector, and the calibration factor.

[0032] In one embodiment, the offset value of each pixel relative to the reflector is calculated using the following formula:

[0033]

[0034] Where D0 is the original measured distance of each pixel to the reflector, and Z... GT The distance is a preset value, cosθ is the calibration factor for each pixel, and θ is the incident angle corresponding to each pixel.

[0035] In one embodiment, after calculating and saving the offset value of each pixel relative to the reflector, the method further includes:

[0036] During ranging, the raw measured distance of each pixel to the target object is obtained;

[0037] The original measured distance of the target object is calibrated by calling the calibration factor corresponding to each pixel to obtain the first calibrated measured distance of the target object.

[0038] The offset value of each pixel relative to the reflector is used to perform offset calibration on the corresponding first calibration measurement distance of the target object, thereby obtaining the corresponding second calibration measurement distance of the target object.

[0039] In one embodiment, the step of using the offset value of each pixel to perform offset calibration on the first calibration measurement distance to obtain the corresponding second calibration measurement distance specifically involves:

[0040] The offset calibration of the first calibration measurement distance of the target object for each pixel is performed using the formula Z' = Z + offset * cosθ, where Z = D * cosθ, Z is the first calibration measurement distance of the target object, Z' is the second calibration measurement distance, offset is the offset value of each pixel relative to the reflector, D is the original measurement distance of the target object for each pixel, cosθ is the calibration factor of each pixel, and θ is the incident angle corresponding to each pixel.

[0041] A second aspect of the present invention provides a depth data calibration apparatus, comprising:

[0042] The calibration module is used to calibrate the lens of the ranging module using Zhang's calibration method to obtain the lens's intrinsic parameters and distortion parameters.

[0043] The calculation and storage module is used to calculate and save the calibration factor corresponding to each pixel in the pixel array based on the lens's intrinsic parameters and distortion parameters.

[0044] The calibration module is used to acquire the original measurement distance sensed by each pixel during ranging, and to call the calibration factor corresponding to each pixel to perform lens calibration on the original measurement distance to obtain the corresponding first calibrated measurement distance.

[0045] The beneficial effects of the present invention are as follows: It provides a depth data calibration method and apparatus, which obtains the pixel calibration factor after lens calibration of the ranging module, so that the original measurement distance can be calibrated by the calibration factor when the area array sensor measures the distance, thereby improving the accuracy of area array depth ranging. Attached Figure Description

[0046] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0047] Figure 1 This is a flowchart of the depth data calibration method in an embodiment of the present invention;

[0048] Figure 2This is a schematic diagram illustrating the principle of lens distortion in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of a planar array depth ranging method according to an embodiment of the present invention;

[0050] Figure 4 This is a structural diagram of the depth data calibration device in an embodiment of the present invention. Detailed Implementation

[0051] To make the technical problems, technical solutions, and beneficial effects of the embodiments of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0052] It should be noted that when a component is referred to as "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as "connected to" another component, it can be directly connected to or indirectly connected to that other component. Furthermore, a connection can be for both fixing and circuit connection purposes.

[0053] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0054] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0055] The depth data calibration provided in this embodiment of the invention is applied to an area array depth sensing system based on dToF (direct time offlight) technology. This area array depth sensing system includes at least a controller, a transmitter, and a receiver. The controller is connected to both the transmitter and the receiver. The transmitter emits a probe beam towards a target object, and at least a portion of the probe beam is reflected by the target object to form reflected light. The receiver includes a pixel array composed of multiple pixels, used to receive the reflected light from the target object. The controller synchronously controls the emission and reception of light, performs histogram statistics on the photons received by the receiver by time bin, and then calculates the time of flight of the photons using the histogram to obtain the depth value of each pixel, thus constructing a depth map of the target object.

[0056] Specifically, the transmitter may include a driver and a light source, etc. The light source may be a light-emitting diode (LED), a laser diode (LD), an edge-emitting laser (EEL), a vertical-cavity surface-emitting laser (VCSEL), a picosecond laser, etc. Under the driving control of the driver, the light source emits a detection beam outward. The detection beam may be visible light, infrared light, ultraviolet light, etc. At least a portion of the detection beam is emitted toward the target object, and the reflected light generated by the reflection of at least a portion of the detection beam by the target object is received by the receiver.

[0057] The receiver may include a pixel array and receiving optical elements, which may be one or more combinations of lenses, microlens arrays, mirrors, etc. The receiving optical elements receive reflected light and guide it to the pixel array. The pixel array includes multiple pixels that collect photons. In one embodiment, the pixel array consists of multiple single-photon avalanche photodiodes (SPADs). The SPADs can respond to the incident single photon and output a photon signal indicating the arrival time of the received photon at each SPAD. Of course, in other embodiments, photoelectric conversion devices such as avalanche photodiodes, photomultiplier tubes, silicon photomultiplier tubes, etc., may also be used.

[0058] Currently, dToF area array depth sensors typically consist of multiple SPADs arranged on a plane to form a pixel array of corresponding resolution for depth detection of target objects. Due to the presence of the lens in the ranging module, tolerances during its production and assembly introduce lens distortion. Furthermore, there is a mapping relationship between each pixel in the pixel array and a real-world pixel, meaning each pixel has a corresponding spatial detection direction. This results in a discrepancy between the raw depth data detected by the pixels and the actual radial distance, affecting the accuracy of area array depth detection and making it difficult to output an accurate depth map. Therefore, this invention describes a depth data calibration method applied to area array depth sensing systems to address this problem. After depth data calibration, measurement errors caused by lens and pixel positions can be effectively eliminated, improving the accuracy of area array depth ranging.

[0059] like Figure 1 As shown, Figure 1 This is a flowchart of a depth data calibration method in one embodiment of the present invention. The method specifically includes the following steps:

[0060] S101. The range measuring module is calibrated using Zhang's calibration method to obtain the lens's intrinsic parameters and distortion parameters.

[0061] In this embodiment, the lens parameters of the ranging module are first calculated using Zhang's calibration method, specifically including the lens's intrinsic parameters and distortion parameters. Since the area array depth sensing system needs to map the ranging results of each pixel to spatial coordinates to construct a point cloud during actual use, the process of mapping points (Xw, Yw, Zw) in the 3D camera coordinate system to points u(x, y) on the 2D camera plane needs to be completed using the intrinsic parameters obtained through calibration. Specifically, the intrinsic parameters to be calibrated include the lens optical center coordinates Cx and Cy, and the lens focal length F in pixel units along the x-axis. x And the lens focal length F in pixels along the y-axis. y .

[0062] Meanwhile, the shape of the lens in the ranging module can cause radial distortion, such as Figure 2 As shown, for a ray with an incident angle of θ, u is the projection point on the 2D camera plane under ideal, distortion-free conditions. However, due to lens distortion, the actual projection point on the 2D camera plane will be u′. This distortion also needs to be calibrated by obtaining the lens distortion parameters to ensure the accuracy of depth measurement.

[0063] In one embodiment, step S101 includes:

[0064] Project the area light source onto the preset chessboard grid;

[0065] The checkerboard grayscale image is obtained by acquiring and converting the pixel array in the ranging module;

[0066] The Zhang calibration method was used to calibrate the lens of the checkerboard grayscale image, and the intrinsic parameters of the lens were calculated.

[0067] In this embodiment, when using Zhang's calibration method to calculate lens parameters, a pre-set checkerboard pattern is used as the calibration object. A surface array light source is projected onto the checkerboard pattern. Specifically, the surface array light source can be a 940nm infrared light source, or it can be a light source of other wavelengths; this embodiment does not limit this. The light projected onto the checkerboard pattern is reflected and received by the pixel array in the ranging module. The pixel array collects the light signal reflected from the checkerboard pattern within a certain exposure time to obtain histogram data. The histogram data is then further converted to obtain a grayscale image of the checkerboard pattern. Through the conversion between depth detection data and grayscale image data, the ranging module captures the image of the checkerboard pattern. Then, Zhang's calibration method is used to calibrate the lens of the ranging module using the grayscale image of the checkerboard pattern, calculating the intrinsic parameters and distortion parameters of the ranging module lens. The specific calculation process is existing technology and will not be elaborated here.

[0068] In one embodiment, a checkerboard grayscale image is obtained by acquiring and converting pixels from a ranging module, including:

[0069] The light signal reflected back from the checkerboard pattern within a preset exposure time is received by the pixel array, and the light signal is incident on the pixel array from different angles;

[0070] A histogram is generated based on the light signal acquired from each pixel;

[0071] The cumulative photon energy in the histogram of each pixel is used as the grayscale value corresponding to each pixel, and a checkerboard grayscale image is obtained.

[0072] In this embodiment, when acquiring and converting the checkerboard grayscale image, the pixel array receives the light signal reflected back from the checkerboard within a preset exposure time. The light signal is incident on the pixel array from different angles. That is, by changing the relative angle between the ranging module and the checkerboard, the light signal reflected back from different directions is captured, so as to convert the corresponding checkerboard grayscale images in different directions and obtain richer coordinate information to improve the accuracy and reliability of lens calibration.

[0073] After performing time-to-digital conversion and counting on the light signal acquired by each pixel, a corresponding histogram is generated. The cumulative value of all photon energies in the histogram is used as the gray value of each pixel, thereby converting to a checkerboard grayscale image. The gray value of each pixel in the checkerboard grayscale image is determined by the cumulative photon energy of that pixel, thus realizing the conversion from histogram data to grayscale image data.

[0074] For example, each pixel has 32 time bins. After performing TDC counting on each bin based on the received light signal, a photon count value from 0 to 255 can be obtained. A histogram for a single pixel can be generated based on the photon count values ​​of the 32 bins. The photon count values ​​of the 32 bins in the histogram are accumulated to obtain the grayscale value corresponding to that pixel. The grayscale value matrix of all pixels can then form a checkerboard grayscale value. Since white has a high reflectivity and black has a low reflectivity in the checkerboard formed by alternating black and white squares, the cumulative amplitude of the histogram for pixels corresponding to white areas is larger than that for pixels corresponding to black areas within a certain exposure time. This is consistent with the grayscale value change pattern corresponding to black and white areas. This embodiment utilizes this to convert the histogram data into grayscale image data, enabling the rangefinder to acquire grayscale images of the checkerboard, so as to calibrate the lens of the rangefinder and obtain the lens's intrinsic parameters and distortion parameters.

[0075] S102. Calculate and save the calibration factor corresponding to each pixel in the pixel array based on the lens's intrinsic parameters and distortion parameters.

[0076] In this embodiment, after lens calibration to obtain intrinsic distortion parameters, a corresponding calibration factor is calculated and saved for the position of each pixel in the pixel array. This calibration factor is related to the incident angle of each pixel so that it can be efficiently and flexibly called during actual ranging.

[0077] Specifically, such as Figure 2 As shown, for a ray with an incident angle of θ, the ideal image point u(x, y) and the distorted actual image point u′(x, y) are... distorted y distorted The corresponding relationship is:

[0078] x distorted =x(1+k1*r 2 +k2*r 4 +k3*r 6 )

[0079] y distorted =y(1+k1*r 2 +k2*r 4 +k3*r 6 )

[0080] r 2 =x 2 +y 2

[0081] Let K = 1 + k1*r 2 +k2*r 4 +k3*r 6 Let r be the distance between the ideal image point and the imaging center, then:

[0082]

[0083] We can obtain:

[0084] r distorted =r*K=r*(1+k1*r) 2 +k2*r 4 +k3*r 6 )

[0085] r distorted This refers to the distance between the actual image point and the imaging center; in this embodiment, it is the distance between each pixel and the imaging center. Meanwhile, since... Therefore, it can be deduced that the calibration factor cosθ can be derived from r. distorted The solutions k1, k2, and k3 are obtained.

[0086] When calculating for each pixel in the pixel array, let the coordinates of a SPAD in the pixel coordinate system be (m, n), then Where Cx and Cy are the coordinates of the lens optical center, and s x and s y s represents the number of pixels per millimeter along the x-axis and y-axis, respectively. x and s y This can be achieved through the internal parameter F. x F y And the focal length F is calculated, where F x =s x ×F,F y =s y ×F.

[0087] Therefore, the distance r between each pixel and the imaging center can be calculated from the pixel coordinates in the pixel coordinate system. distorted Further combining the lens intrinsic parameters Cx, Cy, and F obtained from calibration x F y The radial distortion parameters k1, k2, and k3 can be used to calculate the corresponding calibration factor cosθ, which can then be used as the calibration basis for area array depth ranging to improve ranging accuracy.

[0088] S103. During ranging, obtain the original measured distance sensed by each pixel;

[0089] S104. Call the calibration factor corresponding to each pixel to perform lens calibration on the original measurement distance to obtain the corresponding first calibrated measurement distance.

[0090] In this embodiment, during actual ranging, the ranging module constructs a histogram based on the histogram data acquired by each pixel. After peak finding and determining the flight time of the peak value, the corresponding original measurement distance can be calculated. Then, the pre-stored calibration factors for each pixel are used to perform lens calibration on the original measurement distance to obtain the first calibrated measurement distance. This allows each pixel in the area array sensor to undergo precise ranging calibration through calibration factors, improving the accuracy of area array depth ranging.

[0091] The specific lens calibration method is to perform lens calibration on the original measurement distance of each pixel using the formula Z = D * cosθ, where Z is the first calibration measurement distance of each pixel, D is the original measurement distance of each pixel, cosθ is the calibration factor of each pixel, and θ is the incident angle corresponding to each pixel.

[0092] like Figure 3 As shown, the projection points of points P1 and P2 on target plane 1 fall on SPAD1 and SPAD2 respectively, and their theoretical ranging results are both Z1. Specifically, SPAD1 measures the flight distance P1C from point P1 to point C as the original measurement distance D1, and SPAD2 measures the flight distance P2C from point P2 to point C as the original measurement distance D2. Therefore, after lens calibration, the ranging result of SPAD1 is Z1 = D1·cosθ1, and the ranging result of SPAD2 after lens calibration is Z1 = d2·cosθ2. Similarly, the calibration ranging process for pixels at other positions on other target planes can be completed. Therefore, after completing the lens parameter calibration and calculating the calibration factor for each pixel, calibration can be quickly and conveniently performed during ranging to eliminate ranging deviations caused by lens distortion and the different spatial orientations corresponding to each pixel, ensuring the reliability of the ranging results.

[0093] In one embodiment, after step S104, the method further includes:

[0094] The distance to the preset reflector is measured to obtain the original measured distance of each pixel to the reflector, wherein the reflector is parallel to the ranging module and is spaced at a preset distance.

[0095] The offset value of each pixel relative to the reflector is calculated and saved based on the preset distance, the original measured distance of each pixel to the reflector, and the calibration factor.

[0096] In this embodiment, since there is a certain inconsistency between each SPAD pixel and TDC in the pixel array, when measuring the same distance, there will be a certain distance offset between the depth results obtained by different pixels. Moreover, the offset between different SPAD pixels is randomly distributed. Therefore, it is not possible to perform surface fitting and correction on the array error using a small number of parameters. In this embodiment, the offset value of each pixel is further calibrated and saved in order to eliminate the offset error during distance measurement.

[0097] In practice, distance is measured using a reflector with a known true distance. For example, the reflector is fixed 1 meter away from the ranging module, and the reflector and ranging module are kept parallel in the measurement environment. The parallelism between the ranging module and the reflector can be determined by observing the point cloud in real time. To eliminate random errors and improve calibration accuracy, multiple frames of measurement results can be collected and averaged when calculating the original measurement distance of each pixel to the reflector. For example, 40 frames of original measurement results can be collected for each pixel and the averaged as its original measurement distance to the reflector. The difference between the original measurement distance and the true distance for each pixel is the offset value of that pixel. Therefore, based on the preset distance, the original measurement distance of each pixel to the reflector, and the calibration factor, the offset value of each pixel to the reflector can be calculated and saved for quick retrieval during subsequent distance measurements.

[0098] Specifically, the offset of each pixel relative to the reflector is calculated using the following formula:

[0099]

[0100] Where D0 is the original measured distance of each pixel to the reflector, and Z... GT The distance is a preset value, cosθ is the calibration factor for each pixel, and θ is the incident angle corresponding to each pixel.

[0101] In one embodiment, after calculating and saving the offset value of each pixel relative to the reflector, the method further includes:

[0102] During ranging, the raw measured distance of each pixel to the target object is obtained;

[0103] The original measured distance of the target object is calibrated by calling the calibration factor corresponding to each pixel to obtain the first calibrated measured distance of the target object.

[0104] The offset value of each pixel relative to the reflector is used to perform offset calibration on the corresponding first calibration measurement distance of the target object, thereby obtaining the corresponding second calibration measurement distance of the target object.

[0105] In this embodiment, when the ranging module is actually measuring the distance, it not only performs lens calibration on the original measurement distance using the calibration factor calculated after lens calibration, but also further calls the offset value of each pixel to the reflector based on the lens calibration result to perform offset calibration on the first calibrated measurement distance, thereby obtaining the second calibrated measurement distance after secondary calibration. This eliminates the depth detection error caused by pixel inconsistency and further improves the accuracy of area array depth ranging.

[0106] Specifically, the first calibration measurement distance of the target object for each pixel is offset and calibrated using the formula Z' = Z + offset * cosθ, where Z = D * cosθ, Z is the first calibration measurement distance of the target object, Z' is the second calibration measurement distance, offset is the offset value of each pixel relative to the reflector, D is the original measurement distance of the target object for each pixel, cosθ is the calibration factor for each pixel, and θ is the incident angle corresponding to each pixel.

[0107] It should be noted that there is no necessary order between the above steps. Those skilled in the art will understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0108] The present invention also provides a depth data calibration device, such as... Figure 4 As shown in the figure, this is a structural diagram of a depth data calibration device according to an embodiment of the present invention. It includes a calibration module 401, a calculation and storage module 402, and a calibration module 403, which are connected sequentially. The calibration module 401 is used to calibrate the lens of the ranging module using the Zhang calibration method to obtain the intrinsic distortion parameters of the lens. The calculation and storage module 402 is used to calculate and save the calibration factor corresponding to each pixel in the pixel array based on the intrinsic distortion parameters of the lens. The calibration module 403 is used to obtain the original measurement distance sensed by each pixel during ranging, and to call the calibration factor corresponding to each pixel to perform lens calibration on the original measurement distance to obtain the corresponding first calibrated measurement distance. Since the depth data calibration process has been described in detail in the above method embodiments, please refer to the corresponding method embodiments above for details, and will not be repeated here.

[0109] In summary, this invention provides a depth data calibration method and apparatus. The method includes: calibrating a ranging module using the Zhang calibration method to obtain the lens's intrinsic distortion parameters; calculating and storing the calibration factor corresponding to each pixel in the pixel array based on the lens's intrinsic distortion parameters; acquiring the original measurement distance sensed by each pixel during ranging; and using the calibration factor corresponding to each pixel to perform lens calibration on the original measurement distance to obtain a corresponding first calibrated measurement distance. By obtaining the pixel calibration factors after calibrating the ranging module, the original measurement distance can be calibrated using the calibration factors during ranging by the area array sensor, thereby improving the accuracy of area array depth ranging.

[0110] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or purpose, should be considered within the scope of protection of the present invention.

Claims

1. A depth data calibration method, characterized in that, Includes the following steps: The lens of the ranging module was calibrated using Zhang's calibration method to obtain the lens's intrinsic parameters and distortion parameters. Based on the coordinates of each pixel in the pixel coordinate system and the intrinsic and distortion parameters of the lens, calculate and save the calibration factor corresponding to each pixel in the pixel array; During ranging, the raw measured distance is obtained from the sensing of each pixel; The original measurement distance is calibrated by calling the calibration factor corresponding to each pixel to obtain the corresponding first calibrated measurement distance; The lens calibration of the ranging module using Zhang's calibration method is performed to obtain the lens's intrinsic parameters and distortion parameters, including: Project the area light source onto the preset chessboard grid; The light signal reflected back from the checkerboard pattern within a preset exposure time is received by the pixel array, and the light signal is incident on the pixel array from different angles; A histogram is generated based on the light signal acquired from each pixel; The cumulative value of photon energy in the histogram of each pixel is used as the gray value corresponding to each pixel, and a checkerboard grayscale image is obtained. The Zhang calibration method was used to calibrate the lens of the chessboard grayscale image, and the intrinsic distortion parameters of the lens were calculated. After performing lens calibration on the original measurement distance by calling the calibration factor corresponding to each pixel to obtain the corresponding first calibrated measurement distance, the method further includes: The distance to the preset reflector is measured to obtain the original measured distance of each pixel to the reflector, wherein the reflector is parallel to the ranging module and is spaced at a preset distance. Based on the preset distance, the original measured distance of each pixel to the reflector, and the calibration factor, the offset value of each pixel to the reflector is calculated and saved; The offset of each pixel relative to the reflector is calculated using the following formula: in, The original measured distance of each pixel to the reflector. For the preset distance, Calibration factor for each pixel, This represents the incident angle corresponding to each pixel.

2. The depth data calibration method according to claim 1, characterized in that, The calibration factor for each pixel is calculated using the following formula: Among them, the lens's internal parameters include Distortion parameters include radial distortion parameters. ; For the lens optical center coordinates, x Lens focal length along the axial direction, expressed in pixels. for y Lens focal length along the axial direction, expressed in pixels; F For the lens focal length, and They are respectively x Axial direction and y The number of pixels per millimeter along the axial direction; r is the distance between the ideal image point and the imaging center. Let m and n be the distance between the actual image point and the image center, and m and n be the coordinates of each pixel in the pixel coordinate system. Calibration factor for each pixel, This represents the incident angle corresponding to each pixel.

3. The depth data calibration method according to claim 1, characterized in that, The step of calling the calibration factor corresponding to each pixel to perform lens calibration on the original measurement distance to obtain the corresponding first calibrated measurement distance is as follows: Through formula Z=D Lens calibration is performed on the raw measured distance for each pixel, where, Z The first calibration measurement distance for each pixel, D is the original measurement distance for each pixel. Calibration factor for each pixel, This represents the incident angle corresponding to each pixel.

4. The depth data calibration method according to claim 1, characterized in that, After calculating and saving the offset value of each pixel relative to the reflector, the process also includes: During ranging, the raw measured distance of each pixel to the target object is obtained; The original measured distance of the target object is calibrated by calling the calibration factor corresponding to each pixel to obtain the first calibrated measured distance of the target object. The offset value of each pixel relative to the reflector is used to perform offset calibration on the corresponding first calibration measurement distance of the target object, thereby obtaining the corresponding second calibration measurement distance of the target object.

5. The depth data calibration method according to claim 4, characterized in that, The step of using the offset value of each pixel to perform offset calibration on the first calibration measurement distance to obtain the corresponding second calibration measurement distance is as follows: Through formula Z'=Z For each pixel of the target object, the corresponding first calibration measurement distance is used for offset calibration, where... Z=D , Z The first calibration measurement distance for the target object. Z' For the second calibration measurement distance, The offset value of each pixel relative to the reflector. D The original measured distance of the target object for each pixel. Calibration factor for each pixel, This represents the incident angle corresponding to each pixel.

6. A depth data calibration device, characterized in that, include: The calibration module is used to calibrate the lens of the ranging module using Zhang's calibration method to obtain the lens's intrinsic parameters and distortion parameters. The calculation and storage module is used to calculate and save the calibration factor corresponding to each pixel in the pixel array based on the coordinates of each pixel in the pixel coordinate system and the intrinsic and distortion parameters of the lens. The calibration module is used to acquire the original measurement distance sensed by each pixel during ranging, and to call the calibration factor corresponding to each pixel to perform lens calibration on the original measurement distance to obtain the corresponding first calibrated measurement distance; The calibration module is specifically used to project an area array light source onto a preset checkerboard grid; and to receive light signals reflected back from the checkerboard grid within a preset exposure time through a pixel array, wherein the light signals are incident on the pixel array from different angles. A histogram is generated based on the light signal acquired from each pixel; The cumulative photon energy in the histogram of each pixel is used as the gray value corresponding to each pixel to obtain a checkerboard grayscale image; the Zhang calibration method is used to calibrate the checkerboard grayscale image to calculate the intrinsic parameters and distortion parameters of the lens. It also includes an offset calibration module for measuring the distance of a preset reflector to obtain the original measured distance of each pixel to the reflector, wherein the reflector is parallel to the ranging module and spaced at a preset distance; and for calculating and saving the offset value of each pixel to the reflector based on the preset distance, the original measured distance of each pixel to the reflector, and a calibration factor. The offset of each pixel relative to the reflector is calculated using the following formula: in, The original measured distance of each pixel to the reflector. For the preset distance, Calibration factor for each pixel, This represents the incident angle corresponding to each pixel.

Images