An iToF correction image acquisition and correction method, system, storage medium, and device

Abstract

This invention relates to the field of image correction technology, specifically to an iToF image acquisition and correction method, system, storage medium, and device. The method includes: acquiring Raw Data by moving an iToF camera relative to a checkerboard pattern and taking images; calculating four phase differences based on the Raw Data; deblurring based on the Raw Data and the four phase differences to obtain optimized images IR0 and IR1; pixel enhancement of IR0 and IR1 to obtain Optimized_IR0 and Optimized_IR1; searching for checkerboard patterns (rows, cols) with matching corner points using a checkerboard search method; and calibrating the camera using the Zhang Zhengyou calibration method based on (rows, cols). This solution can utilize blurred images for calibration and extract two sets of images from a single image, improving FPS speed, accelerating calibration, and reducing calibration time.

Description

Technical Field

[0001] This invention relates to the field of image correction technology, specifically to an iToF corrected image acquisition and correction method, system, storage medium, and device. Background Technology

[0002] During image acquisition, distortion can occur due to the characteristics of camera lenses and image sensors. This distortion can cause image loss and affect subsequent image processing and analysis tasks. Therefore, distortion correction is necessary before image processing.

[0003] In camera distortion correction, the Zhang Zhengyou calibration method is often used, which requires at least three images from different angles to perform distortion correction. These three images from different angles need to be captured from a static checkerboard pattern. If the captured images have blurriness or shadows, it will affect the accuracy of the correction. In addition, the accurate acquisition of the minimum three images takes time, which increases the calibration time.

[0004] Therefore, there is an urgent need for an iToF calibration image acquisition and calibration method, system, storage medium, and device that can use blurred images for calibration and parse two sets of images from a single image, thereby increasing FPS (frames per second) speed, accelerating calibration, and reducing calibration time. Summary of the Invention

[0005] One of the objectives of this invention is to provide an iToF calibration image acquisition and calibration method that can use blurred images for calibration and extract two sets of images from a single image, thereby improving FPS speed, accelerating calibration, and reducing calibration time.

[0006] The basic solution provided by this invention is an iToF corrected image acquisition and correction method, which includes the following:

[0007] S1. Move the iToF camera or the checkerboard, or move the iToF camera or the checkerboard simultaneously;

[0008] The S2 and iToF cameras take pictures and obtain raw data.

[0009] S3. Calculate the four phase differences based on the raw data; the four phase differences include: C0, C... 90 C 180 C 270 ; these are the phase differences at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees, respectively;

[0010] S4. Based on the Raw Data and the four phase differences, perform deblurring to obtain two optimized images: IR0 and IR1;

[0011] S5. Perform pixel enhancement on IR0 and IR1 to obtain the enhanced images: Optimized_IR0 and Optimized_IR1;

[0012] S6. Use the checkerboard search method to search for checkerboard squares (rows, cols) with matching corner points in Optimized_IR0 and Optimized_IR1;

[0013] S7. Based on (rows, cols), perform camera calibration using Zhang Zhengyou's calibration method.

[0014] The beneficial effects of Basic Solution 1: When the lens moves quickly, the exposure time (integration time) affects the image, resulting in blurriness. Blurry images are usually unusable. However, in this solution, the iToF camera or the checkerboard is moved first, or both are moved simultaneously. The iToF camera takes a picture to obtain the raw data, i.e., the blurred image. Because the iToF camera and the checkerboard move relative to each other, the blurred image contains image information of the checkerboard at different positions.

[0015] Then, based on the raw data, four phase differences are calculated; these four phase differences include: C0, C... 90 C 180 C 270 It refers to the phase difference at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees. Because iToF uses infrared light and time-of-flight to calculate depth and distance, it uses two capacitors on the photosensitive module (sensor) to receive infrared light. The receiving process of the capacitors involves four delays, divided into 0-360 degrees, namely 0 degrees, 90 degrees, 180 degrees, and 270 degrees (four phases). The four phase delays (0 degrees, 90 degrees, 180 degrees, and 270 degrees) can be obtained by the difference between the two capacitor values. Usually, the sum of the capacitor values ​​is used as the IR image.

[0016] Then, based on the raw data and four phase differences, deblurring is performed to obtain two optimized images: IR0 and IR1. Thus, two sets of images are parsed from the same image frame, which speeds up the acquisition of image frames, increases the FPS to twice the speed, speeds up the calculation of three chessboard grids at different angles, and reduces calibration time.

[0017] Then, this scheme performs pixel enhancement on IR0 and IR1 to obtain enhanced images: Optimized_IR0 and Optimized_IR1. The checkerboard pattern of the enhanced image is easier to search, further reducing calibration time.

[0018] Finally, the checkerboard search method is used to search for checkerboard grids (rows, cols) with matching corner points in Optimized_IR0 and Optimized_IR1. Based on (rows, cols), the Zhang Zhengyou calibration method is used to perform camera calibration.

[0019] In summary, this solution can utilize blurred images for calibration, find clear checkerboard images within blurred checkerboard images, and extract two sets of images from a single image, thereby improving FPS speed, accelerating calibration, and reducing calibration time.

[0020] Furthermore, C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 Where A0, B0, A 90 B 90 A 180 B 180 A 270 B 270 These are the capacitance values ​​of two capacitors A and B on the iToF photosensitive module at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0021] Beneficial effects: iToF uses infrared light and time-of-flight to calculate depth and distance. Its photosensitive module (sensor) uses two capacitors, A and B, to receive the infrared light. The receiving process involves four delays, divided into four phases from 0 to 360 degrees: 0 degrees, 90 degrees, 180 degrees, and 270 degrees. Therefore, the capacitors are defined with capacitance values ​​A0, B0, and A1 respectively. 90 B 90 A 180 B 180 A 270 B 270 The difference between two capacitance values ​​is defined according to four phase delays (0 degrees, 90 degrees, 180 degrees, and 270 degrees). The sum of the capacitance values ​​is typically used as the IR image, such as... Therefore, in this scheme, the four phases are C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 .

[0022] Compared with existing technologies, this solution cannot be used with dToF, structured light, RGB, or other cameras, nor can it be used with relatively stationary objects. It requires iToF or checkerboard to generate blur in a moving state before correction can be performed. Among them, the light storage principle of RGB, structured light, and dToF only requires one capacitor, while iToF uses two capacitors for storage, so they are different in hardware. In addition, iToF has delayed light reception technology, so four delayed phases require four image receptions.

[0023] Furthermore, S4 includes:

[0024] The four phase differences are added together with two sets of opposite phases, 0 degrees and 180 degrees, and 90 degrees and 270 degrees, to obtain two sets of images T0 and T1 after addition. 90 T0 = ​​C0 + C 180 With T 90 =C 90 +C 270 ;

[0025] For T0 and T respectively 90 The absolute values ​​of the pixels are abs_T0 and abs_T. 90 : abs_T0 = abs(T0), abs_T 90 =abs(T 90 );

[0026] Calculate T0 and T 90 The average value of pixels, and T0 and T 90 Pixels smaller than the average value are set to 0, and two optimized images are obtained: IR0 and IR1.

[0027] Beneficial effects: The reason why this solution can deblurred pixels is that 0 degrees and 180 degrees are out of phase. When the phase value of 0 degrees is 1, the phase value of 180 degrees is -1. When the object is stationary, the sum of the values ​​of 0 degrees and 180 degrees is 0. However, this solution utilizes the phase delay during movement, which causes the object to have different pixel coordinates. Therefore, the sum of the values ​​will not be 0. By using fuzzy resolution to process the non-zero pixel coordinates, the checkerboard boundary can be obtained. Similarly, the same principle applies to 90 degrees and 270 degrees. Furthermore, the calculation and judgment of the average value can reduce noise in the environment.

[0028] Furthermore, S5 includes:

[0029] Hough transformation is used to search for line segments on IR0 and IR1 to obtain (p, θ), where P is the distance from the origin (0,0) and θ is the line rotation angle in radians.

[0030] The pixel search is performed using the linear equation y = ax + b to obtain the enhanced images: Optimized_IR0 and Optimized_IR1. The pixel search involves enhancing pixels that meet preset requirements.

[0031] Beneficial effect: The Hough transform is used for enhancement, making the chessboard pattern of the enhanced image easier to search.

[0032] The second objective of this invention is to provide an iToF calibration image acquisition and calibration system that can use blurred images for calibration and extract two sets of images from a single image, thereby improving FPS speed, accelerating calibration, and reducing calibration time.

[0033] This invention provides a second basic solution: an iToF corrected image acquisition and correction system, comprising: an iToF camera, a checkerboard pattern, and a processor;

[0034] Both the iToF camera and the checkerboard pattern are connected to the processor;

[0035] Move the iToF camera or the checkerboard pattern, or move the iToF camera or the checkerboard pattern simultaneously, and the iToF camera will take a picture, obtain the raw data, and send it to the processor;

[0036] The processor is used to calculate four phase differences based on the raw data; the four phase differences include: C0, C... 90 C 180 C 270 It refers to the phase difference at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0037] It is also used to deblur based on the raw data and four phase differences to obtain two optimized images: IR0 and IR1;

[0038] It is also used to perform pixel enhancement on IR0 and IR1 to obtain the enhanced images: Optimized_IR0 and Optimized_IR1;

[0039] It is also used to search for matching corner points in Optimized_IR0 and Optimized_IR1 using a checkerboard search method (rows, cols);

[0040] It is also used to control the iToF camera to perform camera calibration based on (rows, cols) and the Zhang Zhengyou calibration method.

[0041] The beneficial effects of Basic Scheme 2: When the lens moves quickly, the exposure time (integration time) affects the image, resulting in blurriness. Blurry images are usually unusable. However, in this scheme, the iToF camera or the checkerboard is moved first, or both are moved simultaneously. The iToF camera takes a picture to obtain the raw data, i.e., the blurred image. Because the iToF camera and the checkerboard move relative to each other, the blurred image contains the image information of the checkerboard at different positions.

[0042] Then, based on the raw data, four phase differences are calculated; these four phase differences include: C0, C... 90 C 180 C 270 It refers to the phase difference at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees. Because iToF uses infrared light and time-of-flight to calculate depth and distance, it uses two capacitors on the photosensitive module (sensor) to receive infrared light. The receiving process of the capacitors involves four delays, divided into 0-360 degrees, namely 0 degrees, 90 degrees, 180 degrees, and 270 degrees (four phases). The four phase delays (0 degrees, 90 degrees, 180 degrees, and 270 degrees) can be obtained by the difference between the two capacitor values. Usually, the sum of the capacitor values ​​is used as the IR image.

[0043] Then, based on the raw data and four phase differences, deblurring is performed to obtain two optimized images: IR0 and IR1. Thus, two sets of images are parsed from the same image frame, which speeds up the acquisition of image frames, increases the FPS to twice the speed, speeds up the calculation of three chessboard grids at different angles, and reduces calibration time.

[0044] Then, this scheme performs pixel enhancement on IR0 and IR1 to obtain enhanced images: Optimized_IR0 and Optimized_IR1. The checkerboard pattern of the enhanced image is easier to search, further reducing calibration time.

[0045] Finally, the checkerboard search method is used to search for checkerboard grids (rows, cols) with matching corner points in Optimized_IR0 and Optimized_IR1. Based on (rows, cols), the Zhang Zhengyou calibration method is used to perform camera calibration.

[0046] In summary, this solution can utilize blurred images for calibration, find clear chessboard images within blurred chessboard images, and extract two sets of images from a single image, thereby increasing FPS (frames per second) speed, accelerating calibration, and reducing calibration time.

[0047] Furthermore, C0 = A0 - B0, C90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 Where A0, B0, A 90 B 90 A 180 B 180 A 270 B 270 These are the capacitance values ​​of two capacitors A and B on the iToF photosensitive module at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0048] Beneficial effects: iToF uses infrared light and time-of-flight to calculate depth and distance. Its photosensitive module (sensor) uses two capacitors, A and B, to receive the infrared light. The receiving process involves four delays, divided into four phases from 0 to 360 degrees: 0 degrees, 90 degrees, 180 degrees, and 270 degrees. Therefore, the capacitors are defined with capacitance values ​​A0, B0, and A1 respectively. 90 B 90 A 180 B 180 A 270 B 270 The difference between two capacitance values ​​is defined according to four phase delays (0 degrees, 90 degrees, 180 degrees, and 270 degrees). The sum of the capacitance values ​​is typically used as the IR image, such as... Therefore, in this scheme, the four phases are C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 .

[0049] Compared with existing technologies, this solution cannot be used with dToF, structured light, RGB, or other cameras, nor can it be used with relatively stationary objects. It requires iToF or checkerboard to generate blur in a moving state before correction can be performed. Among them, the light storage principle of RGB, structured light, and dToF only requires one capacitor, while iToF uses two capacitors for storage, so they are different in hardware. In addition, iToF has delayed light reception technology, so four delayed phases require four image receptions.

[0050] Furthermore, the processor includes: a deblurring module;

[0051] The defuzzing module is used to obtain inputs C0 and C. 90 C 180 and C 270 The outputs IR0 and IR1, and the specific processing includes:

[0052] The four phase differences are added together with two sets of opposite phases, 0 degrees and 180 degrees, and 90 degrees and 270 degrees, to obtain two sets of images T0 and T1 after addition. 90 T0 = ​​C0 + C 180 With T 90 =C 90 +C 270 ;

[0053] For T0 and T respectively 90 The absolute values ​​of the pixels are abs_T0 and abs_T. 90 : abs_T0 = abs(T0), abs_T 90 =abs(T 90 );

[0054] Calculate T0 and T 90 The average value of pixels, and T0 and T 90 Pixels smaller than the average value are set to 0, and two optimized images are obtained: IR0 and IR1.

[0055] Beneficial effects: The reason why this solution can deblurred pixels is that 0 degrees and 180 degrees are out of phase. When the phase value of 0 degrees is 1, the phase value of 180 degrees is -1. When the object is stationary, the sum of the values ​​of 0 degrees and 180 degrees is 0. However, this solution utilizes the phase delay during movement, which causes the object to have different pixel coordinates. Therefore, the sum of the values ​​will not be 0. By using fuzzy resolution to process the non-zero pixel coordinates, the checkerboard boundary can be obtained. Similarly, the same principle applies to 90 degrees and 270 degrees. Furthermore, the calculation and judgment of the average value can reduce noise in the environment.

[0056] Furthermore, the processor includes: a pixel enhancement module;

[0057] The pixel enhancement module is used to acquire inputs IR0 and IR1 and output Optimized_IR0 and Optimized_IR1. The specific processing includes:

[0058] Hough transformation is used to search for line segments on IR0 and IR1 to obtain (p, θ), where P is the distance from the origin (0,0) and θ is the line rotation angle in radians.

[0059] The pixel search is performed using the linear equation y = ax + b to obtain the enhanced images: Optimized_IR0 and Optimized_IR1. The pixel search involves enhancing pixels that meet preset requirements.

[0060] Beneficial effect: The Hough transform is used for enhancement, making the chessboard pattern of the enhanced image easier to search.

[0061] The third objective of this invention is to provide a storage medium that can perform calibration using blurred images and extract two sets of images from a single image, thereby improving FPS speed, accelerating calibration, and reducing calibration time.

[0062] The present invention provides a third basic solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the iToF corrected image acquisition and correction method.

[0063] Beneficial effects of basic scheme three: This scheme provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the iToF correction image acquisition and correction method, which facilitates the application and promotion of the iToF correction image acquisition and correction generation method.

[0064] The fourth objective of this invention is to provide a device that can perform calibration using blurred images and extract two sets of images from a single image, thereby increasing FPS speed, accelerating calibration, and reducing calibration time.

[0065] The present invention provides a basic solution four: a device that uses the above-mentioned iToF correction image acquisition and correction method.

[0066] The beneficial effects of basic scheme four: This scheme can use blurred images for calibration, find clear chessboard images in blurred chessboard images, and extract two sets of images from one image, thereby improving FPS speed, speeding up calibration, and reducing calibration time. Attached Figure Description

[0067] Figure 1 This is a flowchart illustrating an embodiment of an iToF corrected image acquisition and correction method according to the present invention;

[0068] Figure 2 This is a schematic diagram of an image captured by an iToF camera in an embodiment of an iToF corrected image acquisition and correction method according to the present invention;

[0069] Figure 3 This is a schematic diagram illustrating the signal transmission of iToF, which uses infrared light and time-of-flight to calculate depth and distance.

[0070] Figure 4 The waveform diagram shows four delays.

[0071] Figure 5 A phase piecewise linear plot showing the four phase differences;

[0072] Figure 6 This is a schematic diagram of the deblurring process in an embodiment of an iToF corrected image acquisition and correction method of the present invention;

[0073] Figure 7 This is a schematic diagram of an optimized image in an embodiment of an iToF corrected image acquisition and correction method of the present invention;

[0074] Figure 8 The image is a line segment to be searched;

[0075] Figure 9 To obtain an image of the searched line segment by performing pixel enhancement using Hough transform;

[0076] Figure 10 This is a schematic diagram of the pixel enhancement process in an embodiment of an iToF corrected image acquisition and correction method of the present invention;

[0077] Figure 11 This is a schematic diagram of an enhanced image in an embodiment of an iToF corrected image acquisition and correction method of the present invention;

[0078] Figure 12 This is a schematic diagram of the structure of an embodiment of the device of the present invention. Detailed Implementation

[0079] The following detailed description illustrates the specific implementation method:

[0080] Example 1

[0081] This embodiment is basically as shown in the appendix. Figure 1 As shown: An iToF corrected image acquisition and correction method, including the following:

[0082] S1. Move the iToF camera or the checkerboard, or move the iToF camera or the checkerboard simultaneously;

[0083] The S2 and iToF cameras take pictures and obtain raw data. The images captured by the iToF camera are as follows: Figure 2 As shown;

[0084] S3. Calculate the four phase differences based on the raw data; the four phase differences include: C0, C... 90 C 180 C 270 ; these are the phase differences at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees, respectively;

[0085] Specifically, C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 Where A0, B0, A 90 B 90 A 180 B 180 A 270 B 270 The capacitance values ​​of two capacitors A and B on the iToF sensor at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0086] like Figure 3 As shown, iToF uses infrared light and time-of-flight to calculate depth and distance. It uses two capacitors on the photosensitive module (sensor) to receive the infrared light. The two capacitors are defined as A and B respectively. Figure 3 In this diagram, Target is the object being photographed, Transmitter is the transmitter, Receiver is the receiver, Modulation Block is the modulation module, Correlation Block is the correlation module, ADC is the digital-to-analog converter, Processing is the processing module, and Phase is the phase.

[0087] like Figure 4 and Figure 5 As shown, the capacitor's receiving process involves four delays, divided into four phases from 0 to 360 degrees: 0 degrees, 90 degrees, 180 degrees, and 270 degrees. Therefore, the capacitor's capacitance values ​​are defined as A0, B0, A... 90 B 90 A 180 B 180 A 270 B 270 ; Figure 4 In the diagram, IntegTime is the integration time, which is the time required to acquire the signal; Radiated IRSignal is the radiated infrared signal, which is the transmitter signal; Reflected IR Signal is the reflected IR signal, which is the receiver signal; and 4-PhaseControl Signal are the four phase control signals. Figure 5 In this context, "Voltage" refers to voltage.

[0088] The difference between two capacitance values ​​is defined according to four phase delays (0 degrees, 90 degrees, 180 degrees, and 270 degrees). The sum of the capacitance values ​​is typically used as the IR image, such as...

[0089] Therefore, in this scheme, the four phases are C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 .

[0090] S4. Based on the original data and the four phase differences, perform deblurring to obtain two optimized images: IR0 and IR1;

[0091] The reason why this solution can deblurred pixels is that 0 degrees and 180 degrees are out of phase. When the phase value of 0 degrees is 1, the phase value of 180 degrees is -1. When the object is stationary, the sum of the values ​​of 0 degrees and 180 degrees is 0. However, this solution utilizes the phase delay during movement, which causes the object to have different pixel coordinates. Therefore, the sum of the values ​​will not be 0. By using fuzzy resolution to process the non-zero pixel coordinates, the checkerboard boundary can be obtained. Similarly, the same principle applies to 90 degrees and 270 degrees.

[0092] like Figure 6 As shown, the specific defuzzification process is as follows:

[0093] The four phase differences are added together with two sets of opposite phases, 0 degrees and 180 degrees, and 90 degrees and 270 degrees, to obtain two sets of images T0 and T1 after addition. 90 T0 = ​​C0 + C 180 With T 90 =C 90 +C 270 This allows two images to be obtained from a set of raw data, doubling the FPS (frames per second) and enabling the output of two images from a single raw image.

[0094] For T0 and T respectively 90 The absolute values ​​of the pixels are abs_T0 and abs_T. 90 : abs_T0 = abs(T0), abs_T 90 =abs(T 90 );

[0095] Calculate T0 and T 90 The average value of pixels, and T0 and T 90Pixels with values ​​below the average are set to 0, and two optimized images, IR0 and IR1, are obtained to reduce noise in the environment. The optimized image is shown below. Figure 7 As shown.

[0096] S5. Perform pixel enhancement on IR0 and IR1 to obtain the enhanced images: Optimized_IR0 and Optimized_IR1;

[0097] Specifically, this scheme uses Hough transform for pixel enhancement. The Hough transform searches for line segments and outputs (p, θ), where P is the distance from the origin (0,0) (the top left corner of the image), and θ is the line rotation angle in radians. Figure 8 and Figure 9 As shown, for an image ( Figure 8 The image of the searched line segment is obtained by performing pixel enhancement using Hough transform. Figure 9 );

[0098] like Figure 10 As shown, the specific pixel enhancement process is as follows:

[0099] Hough transformation is used to search for line segments on IR0 and IR1 to obtain (p, θ), where P is the distance from the origin (0,0) and θ is the line rotation angle in radians.

[0100] The linear equation y = ax + b is used for pixel search to obtain the enhanced images: Optimized_IR0 and Optimized_IR1. Pixel search involves enhancing pixels that meet preset requirements, such as pixel value > 1, then pixel value = pixel value * 1.5. The checkerboard pattern in the enhanced image is easier to search. The enhanced image is as follows: Figure 11 As shown;

[0101] S6. Using the checkerboard search method, search for checkerboard grids (rows, cols) with matching corner points in Optimized_IR0 and Optimized_IR1, and record them in memory; where the checkerboard grids (rows, cols) with matching corner points are the checkerboard grids in the image where the corner points match the checkerboard grids.

[0102] S7. Based on (rows, cols), perform camera calibration using Zhang Zhengyou's calibration method.

[0103] This solution can use blurred images for calibration, find clear checkerboard images in blurred checkerboard images, and extract two sets of images from a single image, thereby improving FPS speed, speeding up calibration, and reducing calibration time.

[0104] like Figure 12As shown, this embodiment also provides a device comprising: at least one processor, at least one storage medium, and at least one bus. In this embodiment, it further includes at least one communication interface. The bus is used to enable direct communication between these components, the communication interface is used for signaling or data communication with other node devices, and the storage medium stores machine-readable instructions executable by the processor. When the device is running, the processor communicates with the storage medium via the bus, and the machine-readable instructions are executed by the processor to implement the steps of the iToF corrected image acquisition and correction method described above.

[0105] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described iToF corrected image acquisition and correction method.

[0106] If the methods described above are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0107] Example 2

[0108] This embodiment provides an iToF corrected image acquisition and correction system, including: an iToF camera, a checkerboard pattern, and a processor;

[0109] Both the iToF camera and the checkerboard pattern are connected to the processor;

[0110] Move the iToF camera or the checkerboard pattern, or move the iToF camera or the checkerboard pattern simultaneously, and the iToF camera will take a picture, obtain the raw data, and send it to the processor;

[0111] The processor is used to calculate four phase differences based on the raw data; the four phase differences include: C0, C... 90 C 180 C 270It represents the phase difference at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees; where C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 Where A0, B0, A 90 B 90 A 180 B 180 A 270 B 270 The capacitance values ​​of two capacitors A and B on the iToF photosensitive module at four phases: 0 degrees, 90 degrees, 180 degrees and 270 degrees.

[0112] It is also used to deblur based on the raw data and four phase differences to obtain two optimized images: IR0 and IR1;

[0113] Specifically, the processor includes: a deblurring module;

[0114] The defuzzing module is used to obtain inputs C0 and C. 90 C 180 and C 270 The outputs IR0 and IR1, and the specific processing includes:

[0115] The four phase differences are added together with two sets of opposite phases, 0 degrees and 180 degrees, and 90 degrees and 270 degrees, to obtain two sets of images T0 and T1 after addition. 90 T0 = ​​C0 + C 180 With T 90 =C 90 +C 270 ;

[0116] For T0 and T respectively 90 The absolute values ​​of the pixels are abs_T0 and abs_T. 90 : abs_T0 = abs(T0), abs_T 90 =abs(T 90 );

[0117] Calculate T0 and T 90 The average value of pixels, and T0 and T 90 Pixels smaller than the average value are set to 0, and two optimized images are obtained: IR0 and IR1;

[0118] It is also used to perform pixel enhancement on IR0 and IR1 to obtain the enhanced images: Optimized_IR0 and Optimized_IR1;

[0119] Specifically, the processor also includes: a pixel enhancement module;

[0120] The pixel enhancement module is used to acquire inputs IR0 and IR1 and output Optimized_IR0 and Optimized_IR1. The specific processing includes:

[0121] Hough transformation is used to search for line segments on IR0 and IR1 to obtain (p, θ), where P is the distance from the origin (0,0) and θ is the line rotation angle in radians.

[0122] The pixel search is performed using the linear equation y = ax + b to obtain the enhanced images: Optimized_IR0 and Optimized_IR1, where the pixel search is used to enhance pixels that meet the preset requirements;

[0123] It is also used to search for matching corner points in Optimized_IR0 and Optimized_IR1 using a checkerboard search method (rows, cols);

[0124] It is also used to control the iToF camera to perform camera calibration based on (rows, cols) and the Zhang Zhengyou calibration method.

[0125] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for acquiring and correcting iToF-corrected images, characterized in that, Includes the following: S1. Move the iToF camera or the checkerboard, or move the iToF camera or the checkerboard simultaneously; The S2 and iToF cameras take pictures and obtain raw data. S3. Calculate the four phase differences based on the Raw Data; wherein the four phase differences comprise: C0, C 90 , C 180 , C 270 ; respectively the phase difference at the four phases 0 degrees, 90 degrees, 180 degrees and 270 degrees; S4. Based on the Raw Data and the four phase differences, perform deblurring to obtain two optimized images: IR0 and IR1; The S4 includes: The four phase differences are added together with two sets of opposite phases, 0 degrees and 180 degrees, and 90 degrees and 270 degrees, to obtain two sets of images T0 and T1 after addition. 90 T0 = ​​C0 + C 180 With T 90 =C 90 +C 270 ; For T0 and T respectively 90 The absolute values ​​of the pixels are abs_T0 and abs_T. 90 abs_T0 = abs(T0), abs_T 90 =abs(T) 90 ); Calculate T0 and T 90 The average value of pixels, and T0 and T 90 Pixels smaller than the average value are set to 0, and two optimized images are obtained: IR0 and IR1; S5. Perform pixel enhancement on IR0 and IR1 to obtain the enhanced images: Optimized_IR0 and Optimized_IR1; S6. Use the checkerboard search method to search for checkerboard squares (rows, cols) with matching corner points in Optimized_IR0 and Optimized_IR1. S7. Based on (rows, cols), perform camera calibration using Zhang Zhengyou's calibration method.

2. The iToF corrected image acquisition and correction method according to claim 1, characterized in that, The C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 Where A0, B0, A 90 B 90 A 180 B 180 A 270 B 270 These are the capacitance values ​​of two capacitors A and B on the iToF photosensitive module at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

3. The iToF corrected image acquisition and correction method according to claim 1, characterized in that, The S5 includes: Hough transformation is used to search for line segments on IR0 and IR1 to obtain (p, θ), where P is the distance from the origin (0,0) and θ is the line rotation angle in radians. The pixel search is performed using the linear equation y=ax+b to obtain the enhanced images: Optimized_IR0 and Optimized_IR1. The pixel search involves enhancing pixels that meet preset requirements.

4. An iToF corrected image acquisition and correction system, characterized in that, include: iToF camera, checkerboard pattern, and processor; Both the iToF camera and the checkerboard pattern are connected to the processor; Move the iToF camera or the checkerboard pattern, or move the iToF camera or the checkerboard pattern simultaneously, and the iToF camera will take a picture, obtain the raw data, and send it to the processor; The processor is used to calculate four phase differences based on the raw data; the four phase differences include: C0, C... 90 C 180 C 270 It refers to the phase difference at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees. The processor includes: a deblurring module; The defuzzing module is used to obtain inputs C0 and C. 90 C 180 and C 270 The outputs IR0 and IR1, and the specific processing includes: The four phase differences are added together with two sets of opposite phases, 0 degrees and 180 degrees, and 90 degrees and 270 degrees, to obtain two sets of images T0 and T1 after addition. 90 T0 = ​​C0 + C 180 With T 90 =C 90 +C 270 ; For T0 and T respectively 90 The absolute values ​​of the pixels are abs_T0 and abs_T. 90 abs_T0 = abs(T0), abs_T 90 =abs(T) 90 ); Calculate T0 and T 90 The average value of pixels, and T0 and T 90 Pixels smaller than the average value are set to 0, and two optimized images are obtained: IR0 and IR1; It is also used to deblur based on the raw data and four phase differences to obtain two optimized images: IR0 and IR1; It is also used to perform pixel enhancement on IR0 and IR1 to obtain the enhanced images: Optimized_IR0 and Optimized_IR1; It is also used to search for checkerboard grids (rows, cols) with matching corner points in Optimized_IR0 and Optimized_IR1 using a checkerboard search method. It is also used to control the iToF camera to perform camera calibration based on (rows, cols) using the Zhang Zhengyou calibration method.

5. The iToF corrected image acquisition and correction system according to claim 4, characterized in that, The C0 = A0 - B0, C 90 =A 90 -B 90 C 180 =A 180 -B 180 C 270 =A 270 -B 270 Where A0, B0, A 90 B 90 A 180 B 180 A 270 B 270 These are the capacitance values ​​of two capacitors A and B on the iToF photosensitive module at four phases: 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

6. The iToF corrected image acquisition and correction system according to claim 4, characterized in that, The processor includes: a pixel enhancement module; The pixel enhancement module is used to acquire inputs IR0 and IR1 and output Optimized_IR0 and Optimized_IR1. The specific processing includes: Hough transformation is used to search for line segments on IR0 and IR1 to obtain (p, θ), where P is the distance from the origin (0,0) and θ is the line rotation angle in radians. The pixel search is performed using the linear equation y=ax+b to obtain the enhanced images: Optimized_IR0 and Optimized_IR1. The pixel search involves enhancing pixels that meet preset requirements.

7. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the iToF corrected image acquisition and correction method as described in any one of claims 1-3.

8. An electronic device, characterized in that, The iToF corrected image acquisition and correction method as described in any one of claims 1-3 is adopted.

Images