Lens alignment method for SPAD dof module and SPAD dof module

Abstract

The embodiment of the invention discloses a lens alignment method for an SPAD (Single Program Amplifier) dtof module and the SPAD dtof module, which are used for improving the frame rate of a lens AA. According to the method provided by the embodiment of the invention, the first data array is rapidly acquired by using the high frame rate characteristic of the PTM mode, and the ROI region containing the edge or high contrast pattern is rapidly determined based on the first data array. And then, switching to a PCM mode in the ROI region is carried out, and local data acquisition is carried out by utilizing the advantage of pixel-level high-precision exposure so as to efficiently acquire a second data array. Through the mode, the acquisition range in the PCM mode is obviously reduced, so that the data acquisition time cost is reduced. In the process of adjusting the position of the lens thread for multiple times, definition evaluation and comparison of each frame of image can be completed more efficiently, so that the position of the receiving end lens with the optimal definition is locked quickly, and accurate alignment of the lens is realized.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of electronic image sensors, and in particular, to a lens alignment method for a SPAD dtof module and a SPAD dtof module. Background Art

[0002] During the assembly process of a direct time-of-flight (dtof) module, due to the cumulative tolerance effect of multi-link assembly processes, the imaging quality is easily affected. Therefore, it is necessary to adopt the lens active alignment (AA) technology to ensure the imaging quality. The AA technology uses a light panel to emit uniform light, projects the test card pattern onto the image sensor, and enables the assembly machine to adjust the lens position according to the imaging effect of the sensor, thereby optimizing the imaging quality.

[0003] In current mainstream solutions, complementary metal oxide semiconductor image sensors (CIS) adopt a global shutter mechanism, which can achieve complete imaging in a single exposure, and have advantages such as short exposure time and high frame rate. However, due to the sharp increase in data throughput, single photon avalanche diode (SPAD) image sensors generally use a rolling shutter for zonal exposure, that is, only part of the area is turned on each time, and finally combined into a complete image after sequential exposure, resulting in a significant reduction in the frame rate of SPAD image sensors.

[0004] In addition, to improve the photosensitivity efficiency, each pixel of the SPAD sensor usually consists of multiple SPAD cells. In the photon counting mode (PCM), the exposure is unitized by SPAD cells. For example, when a pixel consists of 3×3 SPAD cells, the frame rate in the PCM mode is only 1 / 9 of the pixel-level exposure. This exponentially reduced frame rate severely restricts the production line efficiency and affects the mass production application of SPAD sensors in dtof modules. Summary of the Invention

[0005] Based on the above problems, the embodiments of the present application provide a lens alignment method for a SPAD dtof module and a SPAD dtof module, aiming to improve the frame rate of lens AA.

[0006] In a first aspect, an embodiment of the present application provides a lens alignment method for a SPAD dtof module. The dtof module includes a SPAD array and a receiving lens. The working modes of the SPAD include a photon counting PCM mode and a photon timing (PTM, Photon Timing Mode) mode. The method includes:

[0007] In the PTM mode, emit a laser at a test card, turn on all SPAD cells, and obtain first data collected by pixels formed by all SPAD cells. The first data of all pixels constitutes a first data array;

[0008] Based on the first data array, obtain the boundary position between the low reflectivity region and the high reflectivity region in the test card, and determine the boundary position as the region of interest (ROI) in the SPAD array;

[0009] In the PCM mode, turn on the SPAD cells corresponding to the ROI region, denoted as ROI-SPAD cells, emit a laser at the test card, and obtain second data collected by all ROI-SPAD cells. The second data of all ROI-SPAD cells constitutes a second data array;

[0010] Calculate the clarity corresponding to the current receiving lens position according to the second data array;

[0011] If the clarity does not reach the preset clarity, control the lens to move. After each movement, new first and second data arrays will be obtained in the PTM mode and the PCM mode respectively, so as to calculate the clarity corresponding to the receiving lens position again, and adjust the receiving lens to the receiving lens position with the highest clarity to achieve lens alignment.

[0012] In one embodiment, the step of obtaining the boundary position between the low reflectivity region and the high reflectivity region in the test card based on the first data array and determining the boundary position as the ROI region in the SPAD array includes:

[0013] Calculate the difference in the number of photons between each pixel and its adjacent pixels according to the number of photons of each pixel in the first data array;

[0014] Determine the pixels with the difference in the number of photons greater than a preset threshold as the boundary position between the low reflectivity region and the high reflectivity region in the test card;

[0015] Determine the boundary position as the ROI region in the SPAD array.

[0016] In one embodiment, the clarity is evaluated by the modulation transfer function (MTF), where the closer the MTF value is to 1, the clearer it is.

[0017] In one embodiment, the color with low reflectivity is black, and the color with high reflectivity is white.

[0018] In one embodiment, one pixel includes 3×3 SPAD units or 2×2 SPAD units.

[0019] In one embodiment, the method further includes: if the clarity does not reach the preset clarity, but the number of movements reaches the preset number or the movement time reaches the preset time, determining the lens position with the highest clarity based on the clarity calculated after all lens movements, so as to calibrate the lens.

[0020] In one embodiment, the size of the test card is proportional to the size of the SPAD array.

[0021] In one embodiment, the test card includes a white background and a plurality of black rectangular blocks, and the plurality of black rectangular blocks are located on two diagonals of the test card.

[0022] In one embodiment, the shape of the test card is rectangular, and the border of the black rectangular block is not parallel to the border of the test card.

[0023] In a second aspect, an embodiment of the present application further provides an SPAD dtof module, including: an SPAD array, a receiving end lens, and a controller;

[0024] The controller is configured to execute the lens alignment method for the SPAD dtof module described in any one of the above.

[0025] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0026] By combining the PTM mode and the PCM mode, this application improves the speed and accuracy of lens AA while taking into account the image processing efficiency and imaging accuracy. Considering that the frame rate of the PTM mode can reach over 10 fps under the same sensor, while the PCM mode is only about 1 fps, the embodiments of this application utilize the high frame rate characteristic of the PTM mode to quickly acquire the overall contour of the test card to obtain a first data array, and detect the boundary position between the low reflectivity region and the high reflectivity region based on the first data array to determine the ROI region. Subsequently, in the PCM mode, only pixel-level high-precision exposure is performed on the ROI region to obtain a more accurate second data array. The ROI region can significantly reduce the amount of single-frame data. Furthermore, when calculating the clarity corresponding to the current lens thread position based on the second image data, the calculation complexity can be reduced, effectively alleviating the problem of the low frame rate of the PCM mode, making the clarity calculation of each lens thread position more efficient, and thus being able to quickly determine the receiving-end lens position with the highest clarity to calibrate the lens. The embodiments of this application not only reduce the time required for lens AA, but also increase the hourly production rate (UPH, Unit Per Hour) of the production line, providing stronger feasibility for the mass production application of SPAD sensors in dtof modules. Brief Description of the Drawings

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0028] Figure 1 Schematic diagram of a lens alignment system provided by an embodiment of this application;

[0029] Figure 2 Schematic diagram of the focusing effect of a lens provided by an embodiment of this application;

[0030] Figure 3 Schematic diagram of the flow of a lens alignment method for a SPAD dtof module provided by an embodiment of this application;

[0031] Figure 4 Schematic diagram of the imaging effect of a test card on a SPAD sensor provided by an embodiment of this application;

[0032] Figure 5 Schematic diagram of the imaging effect of the hypotenuse image in the PTM mode and the PCM mode provided by an embodiment of this application. Detailed Description of the Embodiments

[0033] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0034] Please refer to Figure 1 , in the assembly process of the camera component in the dtof module, there are multiple parts such as an image sensor, a lens, a lens holder, and a circuit board. In the multiple assembly processes of the parts, due to the assembly technology limitations, the accumulated tolerances will become larger and larger, and finally they will all be manifested in the shooting effect of the camera, affecting the imaging quality. For example, the position of the clearest shooting may deviate from the center of the screen, and the clarity of the four corners is uneven.

[0035] Therefore, it is necessary to adopt the lens AA technology to ensure the imaging quality. The AA process is a technology that actively aligns according to the actual situation of the semi-finished products to be assembled during the assembly process, and then assembles the next part in place, so as to effectively reduce the assembly tolerance of the entire module, and then improve the consistency of the camera modules on the entire production line. The lens AA is the most important link in the entire AA process, directly affecting the final imaging effect.

[0036] As Figure 1 shown, during the lens AA process, both the CIS sensor and / or the single-photon avalanche diode direct time-of-flight (SPAD dtof, Single-Photon Avalanche Diode direct Time-of-Flight Sensor) sensor are used as passive photosensitive devices. The test card is composed of special black and white patterns. The black pattern part has a low transmittance, while the white pattern part has a high transmittance, and it is assembled in front of the light board. In a feasible implementation manner, uniform light is emitted by the light board, and the pattern is projected onto the photosensitive surface of the sensor. The assembly machine dynamically adjusts the assembly of the lens (focal length, tilt, etc.) based on the imaging effect of the sensor.

[0037] Please refer to Figure 2 , taking the lens focusing process in the lens AA link as an example, the quality of the lens focusing effect determines the clarity of the image. As Figure 2 shown, Figure 2It contains 15 images numbered from No.1 to No.15, which are partial images obtained after each focusing operation, showing the imaging effects of the lens on the same target at different focal length moving positions. It can be understood that if a fixed black-and-white pattern is photographed, as the lens is gradually adjusted to the appropriate focal length, the boundary of the black-and-white pattern will gradually become clear. On the AA machine platform, the set software program analyzes the clarity of the images frame by frame, and at the same time controls the focal length of the lens through the robotic arm gripper until it is adjusted to the clearest position, then the lens is fixed with glue to maintain the current focal length. Thus, it can be seen that the adjustment of the lens focal length depends on the image output, and each image output requires the sensor to output an image. If the imaging speed of the sensor is slow, the imaging and analysis of each frame will be delayed, and the accumulation of the delay of each frame image will lead to an increase in the time-consuming of the lens AA adjustment process, ultimately resulting in a decrease in the overall production line efficiency. Therefore, the speed of the sensor outputting an image is an important indicator of the lens AA efficiency.

[0038] In the above lens AA system, the frame rate of the sensor outputting an image determines the assembly and debugging time of the device under test on the production line machine platform, while the UPH determines the production cost. Currently, CIS cameras usually use global shutters, and a complete image can be generated with one exposure. The exposure time determines the frame rate of the image output, so CIS cameras have the advantages of short exposure time and high frame rate. However, for SPAD dtof sensors, due to the huge amount of data received by the SPAD dtof module, the data transmission has high requirements for the interface. In order to reduce the pressure on the data interface, a rolling shutter (also known as partitioned exposure) is adopted, and only a part of the entire column is opened for each exposure. After the exposure is completed, this part of the data of this exposure is transmitted, and then the next block is exposed. A complete image frame is composed of multiple sub-frames. Therefore, compared with the CIS device with global exposure, the frame rate of the SPAD dtof array sensor drops significantly.

[0039] A SPAD dtof array sensor supports two operating modes: PCM and PTM. In the PCM mode, the time-to-digital converter (TDC) inside the chip does not work, and only the number of photons received during the exposure time is recorded. In the PTM mode, the TDC inside the chip works, and while exposure is taking place, it also records the arrival time of photons (ambient noise and signal light) to form a time-photon number histogram, and finally calculates the flight time of photons based on the histogram. To improve the photosensitivity efficiency and pay less attention to pixel resolution, each pixel of the SPAD sensor usually consists of multiple SPAD cells. In the PTM mode, the exposure image is output in units of pixels. For example, one pixel consists of 3×3 SPAD cells. In the PCM mode, the exposure image is output in units of SPAD cells, so the frame rate is only 1 / 9 of the pixel-level exposure. Such an exponentially reduced frame rate severely restricts the production line efficiency and affects the mass production application of the SPAD sensor in the dtof module.

[0040] Generally, to compress the data volume in the PTM mode and improve efficiency, a digital signal processing (DSP) module is built into the chip to calculate and statistically analyze the positions of the background noise and signals in the histogram. The built-in DSP greatly compresses the data volume. Specifically, taking a 576×768 SPAD dtof array sensor as an example (the actual effective pixel number is 192×256), the PTM mode has a relatively high image output frame rate, usually reaching more than 10 frames per second, while the frame rate of the PCM mode is relatively low, only about 1 frame per second.

[0041] For the above reasons, in each embodiment of this application, the imaging efficiency differences of different operating modes under the same sensor conditions are comprehensively considered. Therefore, in actual implementation, first, utilize the high frame rate characteristic of the PTM mode to quickly obtain the first data array, and based on this first data array, quickly determine the ROI region containing edges or high-contrast patterns. Subsequently, switch to the PCM mode for the ROI region, and utilize its advantage of high-precision pixel-level exposure to perform local data acquisition to efficiently obtain the second data array.

[0042] By the above method, the acquisition range in the PCM mode is significantly reduced, thereby reducing the time cost of data acquisition. During the process of adjusting the lens position multiple times, it is possible to more efficiently complete the clarity evaluation and comparison of each frame of the image, and then quickly lock the receiving-end lens position with the optimal clarity, realizing the precise alignment of the lens.

[0043] The following further describes each embodiment of this application in detail with reference to the accompanying drawings.

[0044] An embodiment of the present application provides a lens alignment method for a SPAD dtof module. The dtof module includes a SPAD array and a receiving lens. The working modes of the SPAD include a photon counting PCM mode and a photon timing PTM mode. As Figure 3 shown, the method includes steps S301 - S305:

[0045] S301: In the PTM mode, emit laser light towards a test card, turn on all SPAD cells, and obtain first data collected by pixels formed by all SPAD cells. The first data of all pixels constitutes a first data array;

[0046] The test card is an imaging reference board with a specific high - contrast pattern, used to form an imaging image with obvious boundary features. During the AA process of the lens, the focus accuracy is usually reflected by calculating the sharpness of the black - and - white edges of a specific pattern or image inside the test card. In actual use, in order to cover the entire field of view angle of the lens, multiple groups of hypotenuse patterns as Figure 4 shown are usually used, but the hypotenuse patterns do not fill the entire sensor imaging surface. By calculating the sharpness of the hypotenuse and adjusting the lens assembly in real time according to the calculation result, the focus can be made clear.

[0047] In step S301, emit a laser beam towards the test card in the PTM mode. Turn on all SPAD cells in the SPAD array to make them participate in the acquisition of image signals synchronously. After each SPAD cell receives the laser signal reflected by the test card, it outputs corresponding timestamp data respectively, and finally a first data array corresponding to multiple pixels is obtained. The first data array can be used for subsequent ROI region extraction.

[0048] S302: Based on the first data array, obtain the boundary position between the low - reflectivity region and the high - reflectivity region in the test card, and determine the boundary position as the region of interest ROI in the SPAD array;

[0049] By analyzing the first data array collected by all SPAD pixels in the PTM mode, identify the boundary position between the low - reflectivity region and the high - reflectivity region (i.e., the edge with the largest contrast change), and then select the identified boundary region as the ROI region, which can make the accuracy and efficiency of the next imaging higher.

[0050] S303: In the PCM mode, turn on the SPAD cells corresponding to the ROI region, denoted as ROI - SPAD cells, emit laser light towards the test card, and obtain second data collected by all ROI - SPAD cells. The second data of all ROI - SPAD cells constitutes a second data array;

[0051] The exposure unit in PCM mode is the SPAD pixel, which is more effective than PTM in recognizing the clarity of the hypotenuse. For the specific imaging effects under the two working modes, please refer to Figure 5 . Figure 5 The PTM image on the left represents the image collected in PTM mode. In PTM mode, the exposure and image output are in pixels, and the frame rate is higher. However, the boundary transition between the black low-reflectivity area and the white high-reflectivity area shows obvious stepped serrations, and the boundary transition is not delicate enough. Therefore, the image resolution is limited, and it is suitable for preliminary identification of the image contour or positioning of the ROI area. Figure 5 The PCM image on the right is the image collected in PCM mode. In PCM mode, the exposure is collected in units of SPAD pixels. Compared with PTM, PCM provides higher resolution, and the boundary transition between the black low-reflectivity area and the white high-reflectivity area is more natural, making it more suitable for high-precision requirement scenarios such as clarity analysis and focus evaluation. Therefore, for the final clarity calculation, it is necessary to perform clarity calculation based on the PCM image, and then control the machine to assemble the lens according to the clarity result.

[0052] In the previous step, the working mode of the SPAD dtof sensor has been switched to PCM mode, and the ROI area with edge features in the image corresponding to the test card has been identified based on the first data array. Therefore, in this step, only the SPAD units corresponding to the ROI area are turned on, that is, only the part where the low-reflectivity area and the high-reflectivity area of the test card transition is turned on, and the SPAD units in the pure low-reflectivity area and the pure high-reflectivity area are not turned on. The SPAD units in the turned-on ROI area are collectively referred to as ROI-SPAD units.

[0053] Subsequently, continue to irradiate the test card with a laser light source to trigger the ROI-SPAD units to expose. During the exposure, each ROI-SPAD unit performs photon counting on the incident light signal, counts the number of photons received within the set exposure time, and outputs it in the form of a count value to obtain the second data output by each ROI-SPAD unit. The second data of all ROI-SPAD units together constitute the second data array, which is used for subsequent image clarity evaluation and lens focus adjustment processing.

[0054] It should be noted that in the prior art, SPAD array imaging usually relies on multiple exposures, and a complete image is formed by accumulating multiple sub-frames. Since all SPAD pixels in the entire array are involved in the exposure, the data volume is large and the frame rate is low, which affects the processing efficiency. In the embodiment of the present application, only the ROI-SPAD units located at the transition boundary between the low reflectivity region and the high reflectivity region are turned on to participate in the exposure, significantly reducing the number of exposures required for each frame of image. For example, in the case where the original exposure requires N times, the effective number of exposures can be reduced to about N / 10 by this method, thereby significantly increasing the image output frame rate (up to about 10 times the original frame rate), realizing faster generation of grayscale images, and significantly improving the efficiency of image acquisition and analysis during the lens active alignment process.

[0055] S304: Calculate the clarity corresponding to the current receiving end lens position according to the second data array;

[0056] Perform image clarity analysis on the second data array obtained in the above step S303, and calculate the image clarity at the current lens position based on the pixel brightness change or gray level gradient change in the second data array. The methods for calculating clarity include but are not limited to edge sharpness calculation, variance, Tenengrad gradient method, and Laplacian gradient method. This clarity is used to evaluate the imaging quality of the current receiving end lens focusing state and serves as a basis for judging whether to further adjust the lens position subsequently.

[0057] S305: If the clarity does not reach the preset clarity, control the lens to move. After each movement, a new first data array and a new second data array will be obtained respectively in the PTM mode and the PCM mode, so as to calculate the clarity corresponding to the receiving end lens position again, and adjust the receiving end lens to the receiving end lens position with the highest clarity to achieve lens alignment.

[0058] If the clarity obtained from the current second data array does not meet the set threshold, it means that the current focal length position still needs to be adjusted. At this time, the robotic arm or electric platform of the AA device can be controlled to finely adjust the position of the receiving end lens (such as moving up and down or rotating the focusing thread), and the step size of each adjustment can be a fixed value.

[0059] It can be understood that each time the lens is moved, a new round of data is collected, that is, rapid imaging, boundary recognition, ROI update are performed again in the PTM mode, and high-precision imaging of the ROI region is performed again in the PCM mode to generate a new second data array. After multiple test card data acquisitions, record the clarity corresponding to each lens position, and finally determine the lens position with the highest clarity value, and move the receiving end lens to the lens position corresponding to the highest clarity value to achieve lens alignment and fix the lens (such as dispensing and curing).

[0060] More specifically, the technical solution of the present application can be understood in combination with the following examples: In the present application, the sensor first operates in the PTM mode. Thanks to the built-in DSP, the data volume can be greatly compressed, reducing the amount of data transmitted, thereby improving the overall image output speed and enabling the sensor to quickly complete the output of a full-frame image.

[0061] For example, an image of 192×256 pixels is acquired in the PTM mode. Using image processing algorithms such as edge detection or brightness gradient analysis, the hypotenuse region of the test card pattern in the image can be located, that is, the boundary between the low-reflectivity region and the high-reflectivity region in the image. Please refer to Figure 4 , Figure 4 . The abscissa (H: 768 SPAD / 256 pixel) represents the photosensitive structure in the horizontal direction of the image, and the ordinate (V: 576 SPAD / 192 pixel) represents the photosensitive structure in the vertical direction of the image. In this example, the projection of the test card pattern on the sensor imaging surface consists of several squares with hypotenuses. Each square contains 4 hypotenuses. If the test pattern contains a total of 5 squares, there are a total of 20 hypotenuse regions. Determine the positions corresponding to the above 20 hypotenuse regions in the image coordinate system as the ROI regions, and in the subsequent PCM mode, only activate the SPAD units located within these ROI regions and turn off the SPAD units in the remaining non-ROI regions. Since the ROI regions only account for about one-tenth of the entire imaging region, in the PCM mode, only about 1 / 10 of the SPAD units need to be turned on to participate in exposure and data acquisition, greatly reducing the acquisition and processing time of a single-frame image.

[0062] Under this condition, although the PCM mode is usually limited by the data volume and readout structure, and its conventional frame rate is much lower than that of the PTM mode, through the ROI region control strategy of the present application, the frame rate in the PCM mode can be increased to be equivalent to that of the PTM mode, for example, maintained at about 10 frames per second (assuming that the frame rate of the complete image in the PTM mode is 10 fps). This significantly improves the data acquisition efficiency during the lens AA process, shortens the focusing cycle, and improves the focusing accuracy and processing throughput of the production line.

[0063] In the solution provided by the present application, the image output frame rates of the SPAD dtof sensor in the PTM mode and the PCM mode can both reach 10 frames per second, that is, it only takes about 100 milliseconds for each frame of image to complete the output from the start of exposure. For the SPAD sensor, its mode switching time is very short, only about 1 millisecond, and this switching process can be completed in parallel during the execution of image operations (such as sharpness analysis), so it will not additionally increase the overall duration of the imaging cycle.

[0064] Based on this, the image acquisition and clarity calculation process for a single lens position can be completed in approximately 200 milliseconds. Among them, approximately 100 milliseconds are used for image exposure and output in the ROI region (acquiring the first data array in the PTM mode and the second data array in the PCM mode respectively), and another approximately 100 milliseconds are used for image analysis and clarity calculation. Therefore, the overall processing frame rate of the lens alignment process can reach 5 frames per second.

[0065] Compared with the traditional solution, if the entire lens AA process runs completely in the PCM mode, limited by the low frame rate of the PCM mode, its imaging efficiency is only about 1 frame per second. In contrast, by quickly switching between the PTM mode and the PCM mode, the present application effectively improves the operation efficiency of the lens AA process to about 5 times that of the original solution, significantly shortens the lens focusing cycle, greatly improves the UPH, and thus effectively reduces the assembly cost per unit module on the mass production line.

[0066] In summary, by combining the PTM mode and the PCM mode, the present application improves the speed and accuracy of lens AA while taking into account the image processing efficiency and imaging accuracy. Considering that the frame rate of the PTM mode can reach more than 10fps under the same sensor, while the PCM mode is only about 1fps, the embodiment of the present application utilizes the high frame rate characteristic of the PTM mode to quickly acquire the overall contour of the test card to obtain the first data array, and detects the boundary position between the low reflectivity region and the high reflectivity region based on the first data array to determine the ROI region. Subsequently, in the PCM mode, only pixel-level high-precision exposure is performed on the ROI region to obtain a more accurate second data array. The ROI region can significantly reduce the amount of single-frame data. Furthermore, when calculating the clarity corresponding to the current lens thread position based on the second image data, the calculation complexity can be reduced, effectively alleviating the problem of the low frame rate of the PCM mode, making the clarity calculation of each lens thread position more efficient, and thus being able to quickly determine the receiving-end lens position with the highest clarity to calibrate the lens. The embodiment of the present application not only reduces the time required for lens AA, but also increases the production per hour (UPH, Unit Per Hour) of the production line, providing stronger feasibility for the mass production application of SPAD sensors in dtof modules.

[0067] In one embodiment, obtaining the boundary position between the low reflectivity region and the high reflectivity region in the test card based on the first data array and determining the boundary position as the ROI region of interest in the SPAD array includes: calculating the difference in the number of photons between each pixel and its adjacent pixel according to the number of photons of each pixel in the first data array; determining the pixels with the difference in the number of photons greater than a preset threshold as the boundary position between the low reflectivity region and the high reflectivity region in the test card; and determining the boundary position as the ROI region of interest in the SPAD array.

[0068] In an embodiment of the present application, according to the number of photons recorded by each pixel in the first data array, the difference in the number of photons between the pixel and its adjacent pixels is calculated respectively. Exemplarily, one or more adjacent pixels in the horizontal direction, vertical direction or diagonal direction of each pixel can be selected for difference calculation to obtain the local brightness change amplitude.

[0069] Next, the position of the pixel with the photon number difference greater than the preset threshold is determined as the boundary position of the test card in the imaging area. Here, the boundary position corresponds to the boundary transition position between the low reflectivity area and the high reflectivity area in the test card pattern. The above preset threshold can be adjusted according to the dynamic range, noise level or illumination conditions of the actual image, as long as the accuracy and stability of boundary extraction can be ensured. The embodiment of the present application does not limit this.

[0070] Finally, the boundary position corresponding to the pixel determined to be at the edge transition is determined as the ROI area, which is used to guide the range of SPAD units enabled in the PCM mode. Thus, the area with high contrast and obvious edge features in the image can be effectively extracted as the key area for image sharpness analysis, thereby significantly reducing the data acquisition burden in the PCM mode while ensuring the imaging accuracy, and improving the response speed and efficiency of the overall focusing process.

[0071] In one embodiment, the method of the embodiment of the present application further includes: if the sharpness does not reach the preset sharpness, but the number of movements reaches the preset number or the movement time reaches the preset time, based on the sharpness calculated after all lens movements, the lens position with the highest sharpness is determined to calibrate the lens.

[0072] After each movement of the lens, based on the second data array collected at the current lens position, the corresponding image sharpness value is calculated, and the lens position and its corresponding sharpness are stored. To prevent the lens from fine-tuning repeatedly for a long time or infinitely, a termination condition can be set: if the number of movements exceeds the preset number (for example, 20 times) or the total movement time exceeds the preset time (for example, 10 seconds), the further lens movement operation is terminated. When the above termination condition is triggered, all the previously recorded lens positions and their corresponding sharpness values are traversed, the lens position with the highest sharpness is selected therefrom, and the lens is moved to the lens position with the highest sharpness to complete the final lens calibration.

[0073] On the basis of ensuring the optimization of image quality, the embodiment of the present application further enhances the fault tolerance of the assembly process, effectively avoids infinite focusing attempts under specific imaging conditions that are not ideal or in scenarios with structural limitations, and improves the stability of the lens active alignment process and the processing efficiency of the overall production line.

[0074] In one embodiment, the sharpness is evaluated using the modulation transfer function (MTF). Among them, the closer the MTF value is to 1, the sharper the image is.

[0075] The MTF function can be used to reflect the modulation and restoration ability of an image system for signals with different spatial frequencies, as well as to evaluate the resolution and edge sharpness performance of an optical imaging system. In the embodiment of the present application, a bevel pattern is used as the test target area in the test card. The edge spread function (ESF) can also be extracted by the bevel method and further converted into the line spread function (LSF) to obtain the MTF curve. Among them, the MTF value of any pixel is usually between 0 and 1. The closer the MTF is to 1, the stronger the image contrast retention ability is, and the sharper the image is.

[0076] In one embodiment, the test card pattern for lens alignment is composed of regions with different reflectivities. Among them, the color of the low-reflectivity region is black, and the color of the high-reflectivity region is white. The above black and white regions have a significant light reflection contrast, forming an image boundary with a significant difference in gray values on the photosensitive surface of the sensor, which can be used as a basis for boundary region extraction and sharpness analysis.

[0077] In an alternative embodiment, in the SPAD dToF sensor, each image pixel is composed of multiple SPAD units. Specifically, a pixel may include 3×3 SPAD units, or 2×2 SPAD units. Although a pixel composed of more SPAD units can equivalently expand the photosensitive area, making the imaging data more refined, the frame rate will also decrease due to pixel splitting and increased exposure data. Through the above integrated design of SPAD units in the embodiment of the present application, a certain balance is achieved among the frame rate, sensitivity, and resolution. While expanding the effective photosensitive area, improving the photon collection efficiency and ranging accuracy, it can also reduce the impact of the random noise of a single SPAD on the final image data, thereby improving the overall imaging quality.

[0078] In one embodiment, the size of the test card is proportional to the size of the SPAD array, that is, the size of the test card in the actual physical space has a proportional relationship with the size of the SPAD array (such as the imaging field of view, the aspect ratio of the photosensitive surface) to ensure that the test card pattern is completely presented in the sensor imaging area and will not cause the pattern to exceed the field of view or be distorted due to size mismatch. For example, if the SPAD array is 4:3, the test card is designed as a 4:3 ratio rectangle, so as to ensure that the imaging area is centered and the boundary is complete, which is conducive to subsequent boundary detection and ROI extraction.

[0079] In another embodiment, the test card includes a white background and a plurality of black rectangular blocks, and the plurality of black rectangular blocks are located on two diagonals of the test card. Here, the white background serves as a high reflectivity region, and the black rectangular blocks serve as low reflectivity regions, and a clear contrast between light and dark is formed between the high reflectivity region and the low reflectivity region. The plurality of black rectangular blocks are distributed along two diagonals of the test card, so that a high-contrast image boundary extending from the center of the image to the four corners can be formed in the sensor imaging image. Such a rectangular block layout can not only cover the middle part of the picture, but also cover the edge and corner regions, which helps to comprehensively analyze the clarity of different positions in the imaging region during the AA process of the lens, thereby improving the consistency of the focusing accuracy within the entire imaging field of view.

[0080] In yet another embodiment, the test card is rectangular in shape, and the border of the black rectangular block is not parallel to the border of the test card.

[0081] Please refer to Figure 4 , the rectangular shape of the test card facilitates keeping the arrangement consistent with the photosensitive surface of the SPAD dtof sensor, helps to achieve complete coverage of the image during imaging, and facilitates the positioning and fixing operations of the test card on the assembly platform. In this rectangular test card, the border of the black rectangular block is not parallel to the border of the test card, that is, the black block is placed at a certain angle of rotation, so that a non-horizontal or non-vertical hypotenuse structure is formed in the image. The purpose is to construct an inclined edge to meet the prerequisite for MTF calculation based on the hypotenuse method. Compared with the edge parallel to the pixel arrangement direction, the hypotenuse can avoid the aliasing error caused by image sampling alignment and more accurately reflect the modulation ability of the imaging system in terms of spatial frequency. Therefore, the black rectangular block not parallel to the border of the test card is conducive to achieving a more stable and sensitive image clarity evaluation, effectively improving the accuracy and robustness of the lens alignment algorithm.

[0082] Correspondingly, an embodiment of the present application further provides an SPAD dtof module, including: an SPAD array, a receiving-end lens, and a controller; wherein, the SPAD array is configured to receive the laser signal reflected back by an external object and generate a corresponding data array according to different working modes (such as photon timing mode PTM or photon counting mode PCM). The receiving-end lens is configured to image the pattern of the external test card onto the photosensitive surface of the SPAD array; the controller is configured to execute the lens alignment method for the SPAD dtof module described in any one of the above. Through the structural design of the SPAD dtof module, the lens alignment accuracy and imaging consistency can be significantly improved, meeting the dual requirements of high-performance imaging devices for fast focusing and high-quality imaging.

[0083] To implement the lens alignment method for the SPAD dtof module in the embodiments of the present application, the embodiments of the present application further provide a lens alignment device for the SPAD dtof module, and the device includes:

[0084] An acquisition unit, configured to emit a laser to a test card in the PTM mode, turn on all SPAD units, obtain first data collected by pixels formed by all SPAD units, and the first data of all pixels forms a first data array;

[0085] A boundary detection unit, configured to obtain a boundary position between a low reflectivity region and a high reflectivity region in the test card based on the first data array, and determine the boundary position as an interested ROI region in the SPAD array;

[0086] An acquisition unit, configured to turn on the SPAD units corresponding to the ROI region, denoted as ROI-SPAD units, emit a laser to the test card in the PCM mode, obtain second data collected by all ROI-SPAD units, and the second data of all ROI-SPAD units forms a second data array;

[0087] A processing unit, configured to calculate the clarity corresponding to the current receiving end lens position according to the second data array;

[0088] A lens calibration unit, configured to, if the clarity does not reach a preset clarity, control the lens to move. After each movement, new first data arrays and new second data arrays are respectively obtained in the PTM mode and the PCM mode, so as to calculate the clarity corresponding to the receiving end lens position again, and adjust the receiving end lens to the receiving end lens position with the highest clarity to achieve lens alignment.

[0089] In one embodiment, the obtaining a boundary position between a low reflectivity region and a high reflectivity region in the test card based on the first data array and determining the boundary position as an interested ROI region in the SPAD array includes:

[0090] Calculating the difference in the number of photons between each pixel and its adjacent pixels according to the number of photons of each pixel in the first data array;

[0091] Determining the pixels with the difference in the number of photons greater than a preset threshold as the boundary position between the low reflectivity region and the high reflectivity region in the test card;

[0092] Determining the boundary position as an interested ROI region in the SPAD array.

[0093] In one embodiment, the clarity is evaluated by the modulation transfer function MTF, where the closer the MTF value is to 1, the clearer it is.

[0094] In one embodiment, the color with low reflectivity is black, and the color with high reflectivity is white.

[0095] In one embodiment, the pixel includes 3×3 SPAD units or 2×2 SPAD units.

[0096] In one embodiment, the method further includes: if the clarity does not reach the preset clarity, but the number of movements reaches the preset number or the movement time reaches the preset time, determining the lens position with the highest clarity based on the clarity calculated after all lens movements, so as to calibrate the lens.

[0097] In one embodiment, the size of the test card is proportional to the size of the SPAD array.

[0098] In one embodiment, the test card includes a white background and a plurality of black rectangular blocks, and the plurality of black rectangular blocks are located on two diagonals of the test card.

[0099] In one embodiment, the shape of the test card is rectangular, and the borders of the black rectangular blocks are not parallel to the border of the test card.

[0100] It should be noted that: when the above-mentioned lens alignment device for the SPAD dtof module performs lens alignment, only the above division of each program module is used for illustration. In practical applications, the above processing can be allocated to different program modules according to needs, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the above-mentioned lens alignment device for the SPAD dtof module provided in the embodiment belongs to the same concept as the embodiment of the lens alignment method for the SPAD dtof module. For the specific implementation process, please refer to the method embodiment, which will not be elaborated here.

[0101] Based on the hardware implementation of the above program module, and in order to implement a lens alignment method for a SPAD dtof module provided in an embodiment of the present application, an embodiment of the present application further provides a computer device, which includes:

[0102] A central processing unit, a memory, and an input / output interface;

[0103] The memory is a transient storage memory or a persistent storage memory;

[0104] The central processing unit is configured to communicate with the memory and execute the instruction operations in the memory to execute any one of the above-mentioned lens alignment methods for the SPAD dtof module.

[0105] Of course, in practical applications, the various components in the computer device are coupled together through a bus system. It can be understood that the bus system is used to realize the connection and communication between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.

[0106] The memory in the embodiments of the present application is used to store various types of data to support the operation of the computer device. Examples of such data include: any computer program for operating on the computer device.

[0107] It can be understood that when the processor in the computer device described above executes a computer program, it can also implement the functions of each unit in the corresponding device embodiments described above, which will not be elaborated here. Exemplarily, the computer program can be divided into one or more modules / units. One or more modules / units are stored in the memory and executed by the processor to complete the various embodiments of the present application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the computer device. For example, the computer program can be divided into the various units in the above computer device, and each unit can implement the specific functions as described in the corresponding computer device above.

[0108] The computer device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the processor and the memory are only examples of the computer device, and do not constitute a limitation on the computer device. It may include more or fewer components, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, a bus, etc.

[0109] The processor can be a central processing unit (CPU, Central Processing Unit), or other general-purpose processors, digital signal processors (DSP, Digital Signal Processor), application-specific integrated circuits (ASIC, Application Specific Integrated Circuit), off-the-shelf programmable gate arrays (FPGA, Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The processor is the control center of the computer device, and connects various parts of the entire computer device through various interfaces and lines.

[0110] The memory can be used to store computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory and invoking the data stored in the memory, the processor can implement various functions of the computer device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can include high-speed random access memory and can also include non-volatile memory, such as hard disks, memory, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0111] The embodiments of the present application also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it executes the method for aligning the lens of the SPAD dtof module described in any one of the above.

[0112] The embodiments of the present application also provide a computer program product, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, it is used to implement the method for aligning the lens of the SPAD dtof module described in the first aspect or any specific implementation manner of the first aspect of the embodiments of the present application.

[0113] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0114] In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0115] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, in each embodiment of the present application, each functional unit may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0117] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

Claims

1. A lens alignment method for a SPAD dtof module, characterized in that: The dtof module includes a SPAD array and a receiving end lens, the working mode of the SPAD includes a photon counting PCM mode and a photon timing PTM mode, and the method includes: In the PTM mode, a laser is emitted to the test card, all SPAD units are turned on, and first data collected by pixels formed by all SPAD units are obtained, and the first data of all pixels form a first data array; Based on the first data array, obtaining a boundary position between a low reflectivity region and a high reflectivity region in the test card, and determining the boundary position as a region of interest (ROI) in the SPAD array; In the PCM mode, the SPAD units corresponding to the ROI area are turned on, which are recorded as ROI-SPAD units, and lasers are emitted to the test card to obtain second data collected by all ROI-SPAD units, and the second data of all ROI-SPAD units constitute a second data array; Calculating the definition corresponding to the current receiving end lens position according to the second data array; If the clarity does not reach the preset clarity, the lens movement is controlled. After each movement, a new first data array and a new second data array are obtained in the PTM mode and the PCM mode respectively, so as to calculate the clarity corresponding to the receiving end lens position again, and adjust the receiving end lens to the receiving end lens position with the highest clarity to achieve lens alignment.

2. The method according to claim 1, characterized in that The method of acquiring a boundary position between a low reflectivity region and a high reflectivity region in the test card based on the first data array, and determining the boundary position as an ROI region of interest in the SPAD array, comprises: Calculate the difference between the number of photons of each pixel and its adjacent pixel according to the number of photons of each pixel in the first data array; Determine the pixel whose photon number difference is greater than a preset threshold as the boundary position between the low reflectivity area and the high reflectivity area in the test card; The boundary position is determined as the ROI region of interest in the SPAD array.

3. The method according to claim 1, characterized in that The clarity is evaluated using a modulation transfer function (MTF), wherein the closer the MTF value is to 1, the clearer the image is.

4. The method according to claim 1, characterized in that: The color with low reflectivity is black, and the color with high reflectivity is white.

5. The method according to claim 1, characterized in that The one pixel includes 3×3 SPAD units, or 2×2 SPAD units.

6. The method according to claim 1, characterized in that The method also includes: if the clarity does not reach the preset clarity, but the number of movements reaches the preset number of times or the movement time reaches the preset time, based on the clarity calculated after all lenses move, determining the lens position with the highest clarity to calibrate the lens.

7. The method according to claim 1, characterized in that The size of the test card is proportional to the size of the SPAD array.

8. The method according to claim 1, characterized in that: The test card includes a white background and a plurality of black rectangular blocks, and the plurality of black rectangular blocks are located on two diagonal lines of the test card.

9. The method according to claim 8, characterized in that The test card is in a rectangular shape, and the frame of the black rectangular block is not parallel to the frame of the test card.

10. A SPAD dtof module, characterized in that: include: SPAD array, receiving end lens, controller; The controller is used to execute the lens alignment method for a SPAD dtof module as described in any one of claims 1 to 9.

Images