Methods, devices, and computer equipment for calibrating parameters of image acquisition equipment

Abstract

This application relates to a parameter calibration method, apparatus, and computer device for an image acquisition device. The method includes at least one of the following two steps: a) determining a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on a first gain matrix and a trained gain mapping relationship; b) determining a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on a first defocus conversion coefficient, a second defocus conversion coefficient, and a trained defocus mapping relationship. The obtained gain matrix and / or defocus conversion coefficient are written into the memory of the image acquisition device for use in optical shadow correction and phase detection autofocus during image signal processing. By using the trained gain mapping relationship and the trained defocus mapping relationship, gain matrices and defocus conversion coefficients corresponding to various calibration modes and object distances can be generated, reducing calibration costs and time consumption.

Description

Technical Field

[0001] This application relates to the field of camera module technology, and in particular to a parameter calibration method, apparatus and computer equipment for an image acquisition device. Background Technology

[0002] A camera module is a miniature image acquisition device that integrates optical imaging and electronic signal processing. Camera modules are widely used in smartphones, security monitoring, automotive electronics, drones, medical endoscopes, industrial inspection, and AR / VR devices. Taking smartphone camera modules as an example, with the increasing diversity and shorter update cycles in the smartphone camera module market, various products such as telephoto lenses, internal focusing, large aperture, and large image sensors have been mass-produced. The assembly process for smartphone camera modules is also becoming increasingly demanding. During assembly, PDAF calibration, gain matrix calibration, and phase difference offset calibration are typically required. PDAF calibration is a crucial step in ensuring the proper functioning of phase detection autofocus. PDAF calibration yields the defocus conversion coefficient (DCC), and the DCC directly affects the image quality output by the camera module.

[0003] Current technologies for PDAF and gain matrix calibration of camera modules, especially PDAF calibration, typically require first calibrating the camera module under uniform lighting conditions to obtain a gain map, followed by phase difference (PD) mapping, to calibrate the DCC (defocus conversion factor). However, camera modules often require separate PDAF and gain matrix calibrations for various calibration modes and object distances. Performing separate calibrations for each mode and object distance results in high calibration costs and long calibration times for the camera module. Summary of the Invention

[0004] Therefore, it is necessary to provide a parameter calibration method, apparatus, and computer equipment for an image acquisition device to address the aforementioned technical problems.

[0005] In a first aspect, this application provides a parameter calibration method for an image acquisition device. The method includes: acquiring a first gain matrix and a first defocus conversion coefficient of the image acquisition device in a first calibration mode and at a first object distance, and a second defocus conversion coefficient in the same calibration mode and at a second object distance; performing at least one of the following two steps: a) determining a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; b) determining a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient, and the trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance; and writing the obtained gain matrix and / or defocus conversion coefficient into a memory in the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage.

[0006] In one embodiment, the method further includes: obtaining a first phase difference offset of the image acquisition device in the first calibration mode and at the first object distance; determining a phase difference offset corresponding to a target condition based on the first defocus conversion coefficient and the first phase difference offset, combined with at least one paired defocus conversion coefficient across modes or across object distances, and an offset mapping relationship; wherein the target condition differs from the first condition formed by the first calibration mode and the first object distance only in the calibration mode or object distance, and the paired defocus conversion coefficient corresponds to the target condition.

[0007] In one embodiment, the gain mapping relationship is implemented through a gain calibration model group, which includes at least one of a first gain calibration model, a second gain calibration model, and a third gain calibration model, and each model has the following functions: the first gain calibration model is used to perform consistency correction on the input gain matrix under the same calibration mode and the same object distance, and output the consistency-corrected gain matrix; the second gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other calibration mode under the same object distance; the third gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other object distance under the same calibration mode.

[0008] In one embodiment, the first gain calibration model, the second gain calibration model, and the third gain calibration model are implemented by different output heads of the same neural network, or by sub-networks trained independently.

[0009] In one embodiment, the defocus mapping relationship is implemented based on a coefficient calibration model group, which includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and each has the following functions: the coefficient calibration model is used to map the defocus conversion coefficients corresponding to any two object distances to the defocus conversion coefficients corresponding to other two object distances under the same calibration mode; the coefficient conversion algorithm is used to map the known defocus conversion coefficients to the defocus conversion coefficients corresponding to the same object distance and other calibration modes based on the corresponding mode conversion matrix under the same object distance.

[0010] In one embodiment, the gain mapping relationship is determined by: collecting sample gain matrices covering all calibration modes and all object distances; after data preprocessing, training each model in the gain calibration model group; the training includes at least: training a first gain calibration model using samples of the same calibration mode and object distance; and / or training a second gain calibration model using samples of the same object distance across calibration modes; and / or training a third gain calibration model using samples of the same calibration mode across object distances; to determine the gain mapping relationship.

[0011] In one embodiment, the coefficient calibration model is determined by: collecting multiple sets of input-output sample pairs, each set of sample pairs containing: sample defocus conversion coefficients corresponding to any two object distances under the same calibration mode after standardization as input samples; sample defocus conversion coefficients corresponding to other two object distances under the same calibration mode as expected output samples; and training the coefficient calibration model based on the input samples and expected output samples to calculate the defocus conversion coefficients between different object distances under the same calibration mode during the inference phase.

[0012] In one embodiment, the coefficient conversion algorithm is implemented through the mode conversion matrix. The mode conversion matrix is ​​determined by fitting the sample defocus conversion coefficients corresponding to different calibration modes as training data under the same object distance, and is used in the inference stage to convert the defocus conversion coefficient of any calibration mode into the defocus conversion coefficient corresponding to another calibration mode.

[0013] Secondly, this application also provides a parameter calibration device for an image acquisition device, the device comprising: an acquisition module, configured to acquire a first gain matrix and a first defocus conversion coefficient of the image acquisition device in a first calibration mode and at a first object distance, and a second defocus conversion coefficient in the same calibration mode and at a second object distance; a first calibration module, configured to determine a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; and / or a second calibration module, configured to determine a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient, and the trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance; and a storage module, configured to write the obtained gain matrix and / or defocus conversion coefficient into a memory in the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage.

[0014] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the parameter calibration method for any of the image acquisition devices described in the first aspect.

[0015] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the parameter calibration method for any of the image acquisition devices described in the first aspect.

[0016] The parameter calibration method, apparatus, and computer equipment for the aforementioned image acquisition device first acquire a first gain matrix and a first defocus conversion coefficient when the image acquisition device is in a first calibration mode and at a first object distance, and a second defocus conversion coefficient when it is in the same calibration mode and at a second object distance. At least one of the following two steps is performed: a) Based on the first gain matrix and a trained gain mapping relationship, a gain matrix corresponding to the first target calibration mode and / or the first target object distance is determined; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance. b) Based on the first defocus conversion coefficient, the second defocus conversion coefficient, and the trained defocus mapping relationship, a defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance is determined; wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first and second object distances. The obtained gain matrix and / or defocus conversion coefficient are written into the memory of the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage. With only the first gain matrix, the first defocus conversion coefficient, and the second defocus conversion coefficient obtained, the gain matrix and defocus conversion coefficient corresponding to various calibration modes and object distances can be generated through trained gain mapping relationships and trained defocus mapping relationships, thereby reducing calibration costs and calibration time. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a parameter calibration method for an image acquisition device in one embodiment;

[0018] Figure 2 This is a flowchart illustrating the training method for the calibration module in one embodiment;

[0019] Figure 3 This is a flowchart illustrating the training method of a coefficient calibration model in one embodiment;

[0020] Figure 4 This is a flowchart illustrating a gain matrix calibration method in one embodiment;

[0021] Figure 5 This is a flowchart illustrating a defocus conversion coefficient calibration method in one embodiment;

[0022] Figure 6 This is a flowchart illustrating a phase difference offset calibration method in one embodiment;

[0023] Figure 7 This is a schematic diagram of an FCN model in one embodiment;

[0024] Figure 8 This is a schematic diagram illustrating a specific gain matrix calibration in one embodiment;

[0025] Figure 9 This is a schematic diagram illustrating the calibration of the defocus conversion coefficient and phase difference offset in one embodiment;

[0026] Figure 10 This is a structural block diagram of the parameter calibration device of an image acquisition device in one embodiment;

[0027] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] A camera module is a miniature image acquisition device that integrates optical imaging and electronic signal processing. Camera modules are widely used in smartphones, security monitoring, automotive electronics, drones, medical endoscopes, industrial inspection, and AR / VR devices. Taking smartphones as an example, with the increasing diversity and shorter update cycles in the smartphone camera module market, various products such as telephoto lenses, internal focusing, large aperture lenses, and large image sensors have been mass-produced. The assembly process for smartphone camera modules is also becoming increasingly demanding. During assembly, PDAF calibration, gain map calibration, and phase difference offset calibration are typically required. PDAF calibration is a crucial step in ensuring the proper functioning of phase detection autofocus. PDAF calibration yields the defocus conversion coefficient (DCC), and the DCC directly affects the image quality output by the camera module.

[0030] Currently, for the DCC (Discrete Conversion Coefficient), camera module manufacturers commonly use traditional PDAF (Pulse-Detection Algorithm) algorithms provided by third-party platforms such as Qualcomm, MediaTek, and HiSilicon for DCC calibration. This algorithm typically requires first calibrating the camera module's gain map under uniform lighting, followed by phase-difference (PD) image generation to complete the DCC calibration. However, camera modules often need to calibrate the DCC separately for various calibration modes and object distances. Using the aforementioned traditional PDAF algorithm for multiple calibration modes and object distances is costly and time-consuming. Specifically, the cost mainly includes three aspects: First, phase-difference PD image generation and re-image processing are extremely time-consuming. Phase-difference PD image generation requires at least 11 images, and separate image generation is needed for different calibration modes and object distances. Re-image generation is required every time environmental variables such as calibration mode or object distance are changed. Second, the calculation steps of traditional PDAF algorithms are cumbersome, requiring image processing of multiple images, which is extremely time-consuming. Thirdly, a single calibration system cannot simultaneously accommodate both large and small object distances for calibration plates. Multiple calibration systems are required for calibrating various object distances, leading to high costs. Similarly, gain map calibration requires separate calibration for different calibration modes and object distances. Offset calibration also requires separate calibration for different calibration modes and object distances. Performing separate calibration for each calibration mode and object distance results in high calibration costs and long calibration times for the camera module.

[0031] The traditional PDAF algorithm, for any calibration mode and object distance, follows this calibration process: First, a phase difference (PD) image is acquired using the camera module. This PD image is then separated to determine the left and right PD images. Gain correction is performed on the PD images using a gain map, which represents a mapping table dynamically adjusting the light sensitivity of the phase detection shield pixel. For example, gain correction is performed on the left PD image using the left gain map and on the right PD image using the right gain map. The gain-corrected left and right PD images are then downsampled (vertically loaded) to improve computational efficiency, and the phase difference (PD) is calculated. Finally, the defocus conversion factor (DCC) is fitted. The focus position is calculated using the following formula: TargetPosition = CurrentPosition + PD × DCC, where TargetPosition represents the target motor position, i.e., the focus position; CurrentPosition represents the current motor position; PD represents the phase difference, used to measure image sharpness, and has a direction; the defocus conversion factor DCC represents the ratio of defocus to PD, measured in DAC / pd. Camera module manufacturers calibrate the DCC for each camera module and write it into the module's EEPROM, representing the proportional coefficient for converting pixel-level phase difference into motor drive values. Defocus represents the physical distance the lens needs to move from its current position to the focus position. Vertical loading refers to an image processing technique typically used to reduce image data volume and improve subsequent processing efficiency.

[0032] Understandably, based on the above formula, when calculating the target motor position, PD compensation is also required, as follows: TargetPosition = CurrentPosition + DCC×(PD+offset), where the phase difference offset represents the offset of the phase difference PD point, which is used to compensate for the PD value in order to achieve accurate focusing.

[0033] After obtaining the gain map, DCC (Defocus Conversion Coefficient), and offset through calibration, the camera module acquires a PD (Disc Point) image during actual autofocus operation. This image is then split into a left PD image and a right PD image. Gain correction is applied to the left PD image using the left gain map, and to the right PD image using the right gain map. Using the corrected left and right PD images, the PD value of the current image is calculated in real-time. Multiplying the PD value by the DCC value determines the focus position, i.e., the physical distance the defocus lens needs to move from the current position to the focus point. During this process, the offset can be used to compensate for the PD value, thus achieving autofocus. Taking a camera module applied to a mobile phone as an example, the phone has pre-set gain map, DCC, and offset parameters after camera module calibration. When a mobile phone is used for normal photography, during the autofocus process, it acquires a PD image, performs gain correction on the PD image using a gain map matrix, calculates the PD value, multiplies the PD value by the defocus conversion factor (DCC), and obtains the defocus displacement of the camera module. The PD value is then compensated for using offset adjustment, thus completing PDAF focusing.

[0034] In one embodiment, such as Figure 1 As shown, a parameter calibration method for an image acquisition device is provided, including the following steps:

[0035] Step 101: Obtain the first gain matrix and the first defocus conversion coefficient of the image acquisition device in the first calibration mode and at the first object distance, and the second defocus conversion coefficient in the same calibration mode and at the second object distance.

[0036] Image acquisition devices can be camera modules or devices such as mobile phones and tablets that include camera modules. The image acquisition device is one that requires gain mapping, DCC (defocus conversion coefficient), and offset calibration. If the image acquisition device is a camera module, it can consist of components such as a lens, image sensor, and motor, achieving image acquisition through optical imaging and electronic signal processing. The camera module can be installed in a mobile phone, a security monitoring system, or a vehicle; this embodiment does not specifically limit the application scenario of the camera module.

[0037] When calibrating using the method provided in this embodiment, it is first necessary to use traditional calibration techniques to calibrate the first gain matrix and the first defocus conversion coefficient in the first calibration mode at the first object distance, and the second defocus conversion coefficient in the same calibration mode at the second object distance. The same calibration mode is the same as the first calibration mode. The first calibration mode can be any calibration mode among multiple calibration modes, and the first object distance and the second object distance can be any two object distances among multiple object distances.

[0038] The calibration mode refers to different calibration techniques used when calibrating an image acquisition device. Examples of calibration modes include QPD calibration mode and SPD calibration mode. This embodiment does not specifically limit the number of calibration modes; it only requires that all calibration modes can calibrate the parameters of the image acquisition device. QPD calibration mode is a four-phase detection autofocus technology that achieves 2x2 phase detection autofocus (PDAF) across the entire sensor image array, achieving 100% coverage, high sensitivity, small defocus error, and a small required DCC value. SPD calibration mode is a segmented pixel detection technology that divides the imaging pixel into two (left / right or top / bottom), with each sub-pixel independently sensing light. Only about 50% of the light is used for phase detection, resulting in low sensitivity, large defocus error, and a large required DCC value. The accuracy of DCC is inversely proportional to the signal-to-noise ratio of phase detection. The QPD calibration mode results in a high signal-to-noise ratio (SNR) and a low DCC value; the SPD calibration mode results in a low SNR and a high DCC value. This calibration mode can also be used with other phase-detection autofocus technologies.

[0039] Object distance refers to the distance from the subject to the optical center (or principal plane) of the lens. Examples of object distances include 5cm, 6cm, 10cm, and 80cm. This embodiment does not specifically limit the object distances; only calibration is required. In this invention, the working distance of the image card is considered the object distance, and its value is equal to the distance from the surface of the image card to the entrance pupil of the lens.

[0040] First, the calibration environment is set to the first calibration mode and the first object distance. Under these conditions, the image acquisition device is calibrated using a traditional calibration method to obtain the first gain matrix and the first defocus conversion coefficient. Then, the calibration environment is adjusted to the first calibration mode and the second object distance. Under these conditions, the image acquisition device is calibrated using a traditional calibration method to obtain the second defocus conversion coefficient.

[0041] After obtaining the first gain matrix, the first defocus conversion coefficient, and the second defocus conversion coefficient, at least one of the following steps 102 and 103 shall be performed. That is, only step 102 may be performed, only step 103 may be performed, or both steps 102 and 103 may be performed simultaneously.

[0042] Step 102: Based on the first gain matrix and the trained gain mapping relationship, determine the gain matrix corresponding to the first target calibration mode and / or the first target object distance.

[0043] The first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance. That is, the first target calibration mode can be any calibration mode other than the first calibration mode, and the first target object distance can be any object distance other than the first object distance. Determine the gain matrix corresponding to the first target calibration mode and / or the first target object distance. That is, the gain matrix corresponding to any calibration mode other than the first calibration mode and any object distance can be determined, and the gain matrix corresponding to any object distance other than the first object distance and any calibration mode can also be determined.

[0044] The trained gain mapping relationship can be implemented through a trained neural network model and / or function, describing the mapping relationship of the gain matrix as a function of calibration mode and object distance. This is used to derive the gain matrix under unmeasured conditions based on the existing gain matrix. Specifically, by pre-obtaining the sample left gain matrix and sample right gain matrix under all calibration modes and all object distance conditions as a training dataset, the neural network model is trained using this training dataset, thereby obtaining the trained gain mapping relationship.

[0045] In this application, after obtaining the first gain matrix, based on the first gain matrix and the trained gain mapping relationship, the corresponding gain matrices under various calibration modes and object distance conditions can be determined. The first gain matrix can be the left gain matrix of the image acquisition device under the first calibration mode and the first object distance condition, or it can be the right gain matrix of the image acquisition device under the first calibration mode and the first object distance condition. That is, when calibrating the gain matrix of the image acquisition device using traditional calibration techniques, only the left gain matrix or the right gain matrix needs to be calibrated. The left gain matrix represents the matrix used for gain correction of the left PD image, and the right gain matrix represents the matrix used for gain correction of the right PD image.

[0046] Step 103: Based on the first defocus conversion coefficient, the second defocus conversion coefficient, and the trained defocus mapping relationship, determine the defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance.

[0047] The second target calibration mode differs from the first calibration mode, or the second target object distance differs from both the first and second object distances. That is, the second target calibration mode can be any calibration mode other than the first calibration mode. The second target object distance can be any object distance other than the first and second object distances. Determining the defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance means that the defocus conversion coefficient corresponding to any calibration mode other than the first calibration mode and any object distance can be determined, and the defocus conversion coefficient corresponding to any object distance other than the first and second object distances and any calibration mode can also be determined.

[0048] The trained defocus mapping relationship can be implemented through a trained neural network model and / or function. It describes the mapping relationship of the defocus conversion coefficient as a function of the calibration mode and / or object distance. It can be used to derive the DCC values ​​for other conditions based on one or two known DCC values. Specifically, using pre-acquired sample defocus conversion coefficients under all calibration modes and object distances as a training dataset, the neural network model is trained to obtain the trained defocus mapping relationship. Alternatively, based on the sample defocus conversion coefficients under all calibration modes and object distances, the functional relationship between sample defocus conversion coefficients under different calibration modes and object distances is determined, thus obtaining the trained defocus mapping relationship.

[0049] In this application, after obtaining the first defocus conversion coefficient and the second defocus conversion coefficient, the defocus conversion coefficients corresponding to various calibration modes and various object distance conditions can be determined based on the first defocus conversion coefficient, the second defocus conversion coefficient and the trained defocus mapping relationship.

[0050] Step 104: Write the obtained gain matrix and / or defocus conversion coefficient into the memory of the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage.

[0051] After obtaining the gain matrices for all calibration modes and object distance conditions, these matrices are written into the memory of the image acquisition device. Similarly, after obtaining the defocus conversion coefficients for all calibration modes and object distance conditions, these coefficients are also written into the memory of the image acquisition device. During actual use of the image acquisition device, the corresponding calibration mode and object distance conditions' gain matrix and defocus conversion coefficients are selected based on the current conditions for optical shading correction and phase-detection autofocus. Some parameters can be obtained directly using conventional methods, while others can be obtained through mapping relationships as described above; this application does not impose any restrictions on this.

[0052] This embodiment first obtains the first gain matrix and the first defocus conversion coefficient of the image acquisition device in the first calibration mode and at the first object distance, and the second defocus conversion coefficient in the same calibration mode and at the second object distance. At least one of the following two steps is performed: a) Based on the first gain matrix and the trained gain mapping relationship, determine the gain matrix corresponding to the first target calibration mode and / or the first target object distance; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance. b) Based on the first defocus conversion coefficient, the second defocus conversion coefficient, and the trained defocus mapping relationship, determine the defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first and second object distances. The obtained gain matrix and / or defocus conversion coefficient are written into the memory of the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage. With only the first gain matrix, the first defocus conversion coefficient, and the second defocus conversion coefficient obtained, the gain matrix and defocus conversion coefficient corresponding to various calibration modes and object distances can be generated through trained gain mapping relationships and trained defocus mapping relationships, thereby reducing calibration costs and calibration time.

[0053] In one embodiment, the method further includes the following steps:

[0054] Step 1a: Obtain the first phase difference offset of the image acquisition device in the first calibration mode and at the first object distance.

[0055] Using traditional calibration techniques, when calibrating in the first calibration mode and at the first object distance, in addition to obtaining other parameters, the first phase difference offset can also be obtained. That is, the first phase difference offset of the device in the first calibration mode and at the first object distance is obtained.

[0056] Step 1b: Based on the first defocus conversion coefficient and the first phase difference offset, combined with at least one paired defocus conversion coefficient across modes or across object distances, and the offset mapping relationship, determine the phase difference offset corresponding to the target conditions.

[0057] The target condition differs from the first condition consisting of the first calibration mode and the first object distance only in the calibration mode or the object distance. The first condition is the condition corresponding to the first defocus conversion coefficient and the first phase difference offset, wherein the paired defocus conversion coefficient also corresponds to the first condition, that is, the paired defocus conversion coefficient also corresponds to the phase difference offset corresponding to the target condition.

[0058] The paired defocus conversion coefficients across modes or object distances are the defocus conversion coefficients corresponding to the target condition. The phase difference offset corresponding to the target condition can be obtained from the already obtained pair of corresponding defocus conversion coefficients and phase difference offsets, as well as the defocus conversion coefficients corresponding to the target condition. By varying the paired defocus conversion coefficients across modes or object distances under different conditions, and combining them with the offset mapping relationship, the phase difference offset under the required calibration mode and the required object distance condition can be obtained.

[0059] The offset mapping relationship is a function for calculating the phase difference offset under different calibration modes and object distances. It is used to generate the phase difference offset under unmeasured conditions by combining the known phase difference offset and other parameters. The offset algorithm can be obtained based on physical optical characteristic analysis or statistical summarization of historical data. For example, based on the first defocus conversion coefficient and the first phase difference offset, combined with the second calibration mode and the defocus conversion coefficient under the first object distance condition, the phase difference offset under the second calibration mode and the first object distance condition can be obtained through the offset mapping relationship. For example, based on the first defocus conversion model and the first phase difference offset, combined with the first calibration mode and the defocus conversion coefficient under the third object distance condition, the phase difference offset under the first calibration mode and the third object distance condition can be obtained through the offset mapping relationship.

[0060] This embodiment, by acquiring only the first gain matrix, first defocus conversion coefficient, first phase difference offset, second defocus conversion coefficient, and second phase difference offset, can generate gain matrices, defocus conversion coefficients, and phase difference offsets corresponding to various calibration modes and object distances through trained gain mapping relationships, trained defocus mapping relationships, and offset mapping relationships. This reduces calibration costs and time. Furthermore, by replacing traditional calibration methods with model groups and algorithms, the number of image acquisitions and computational resource consumption can be reduced, avoiding delays caused by frequent image re-opening and multi-image processing.

[0061] In one embodiment, the above-mentioned gain mapping relationship can be implemented by a gain calibration model group, which may include at least one of a first gain calibration model, a second gain calibration model, and a third gain calibration model, and each model has the following functions.

[0062] The first gain calibration model is used to perform consistency correction on the input gain matrix under the same calibration mode and object distance conditions, and outputs a consistency-corrected gain matrix. Consistency correction refers to calculating the right gain matrix using the left gain matrix, or the left gain matrix using the right gain matrix, to maintain a predetermined consistency in the gain values ​​of the same pixel position in the left and right channels. For example, the first gain calibration model is used to predict the right gain matrix from the left gain matrix, or the left gain matrix from the right gain matrix, under the same calibration mode and object distance conditions. For instance, it can predict the right gain matrix under the first calibration mode and object distance conditions using the left gain matrix under the first calibration mode and object distance conditions.

[0063] The second gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other calibration mode under the same object distance condition. For example, the second gain calibration model is used to predict the left gain matrix (or right gain matrix) of other calibration modes under different calibration modes and the same object distance condition, using the left gain matrix (or right gain matrix) of any calibration mode. For instance, it can predict the left gain matrix (or right gain matrix) of other calibration modes and under the first object distance condition using the left gain matrix (or right gain matrix) of the first calibration mode and the first object distance condition.

[0064] The third gain calibration model is used to map the input gain matrix to at least one gain matrix corresponding to other object distances under the same calibration mode. For example, the third gain calibration model is used to predict the left gain matrix for other object distances using the left gain matrix for any object distance under the same calibration mode and different object distances; or to predict the right gain matrix for other object distances using the right gain matrix for any object distance. For instance, it can predict the left gain matrix for the first calibration mode and other object distances using the left gain matrix under the first calibration mode and the first object distance condition.

[0065] In this application, by using the trained gain mapping relationship, other unmeasured gain matrices can be obtained based on the obtained partial gain matrices. Compared with traditional calibration methods, this method greatly reduces the number of graphs generated and the calibration time, and improves calibration efficiency.

[0066] In one embodiment, before calibrating the camera module, a gain calibration module needs to be pre-trained to establish a gain mapping relationship. In this application, sample gain matrices covering all calibration modes and all object distances can be collected. After data preprocessing, each model in the gain calibration model group is trained separately. The training includes at least: training a first gain calibration model using samples of the same calibration mode and object distance; and / or training a second gain calibration model using samples of the same object distance across calibration modes; and / or training a third gain calibration model using samples of the same calibration mode across object distances; to determine the gain mapping relationship.

[0067] Specifically, such as Figure 2 As shown, the training method for this gain calibration module may include the following steps:

[0068] Step 201: Obtain the first initial dataset.

[0069] Obtain sample gain matrices for all calibration modes and all object distances. The first initial dataset includes the sample left gain matrix and sample right gain matrix under all calibration modes and all object distance conditions. The first initial dataset can be historical calibration data from the production line. This application provides an example with multiple calibration modes, including QPD calibration mode and SPD calibration mode. Multiple object distances are: 5cm, 6cm, 10cm, and 80cm. The sample left gain matrix and sample right gain matrix can be 12 × 16 matrices.

[0070] The first initial dataset includes the following sets of data: left and right gain matrices for QPD calibration mode under a 5cm object distance condition; left and right gain matrices for SPD calibration mode under a 5cm object distance condition; left and right gain matrices for QPD calibration mode under a 6cm object distance condition; left and right gain matrices for SPD calibration mode under a 6cm object distance condition; left and right gain matrices for QPD calibration mode under a 10cm object distance condition; left and right gain matrices for SPD calibration mode under a 10cm object distance condition; left and right gain matrices for QPD calibration mode under an 80cm object distance condition; and left and right gain matrices for SPD calibration mode under an 80cm object distance condition. Understandably, the first initial dataset includes multiple sets of the above data.

[0071] Step 202: Perform data preprocessing on the left gain matrix and right gain matrix of all samples in the first initial dataset to obtain a preprocessed dataset.

[0072] After obtaining the initial dataset, to reduce errors caused by data fluctuations and ensure the predicted gain matrix fluctuates within a certain range while suppressing outliers, data preprocessing is required for the sample left and right gain matrices. This preprocessing can include standardization, denoising, or normalization. For example, benchmark values ​​can be set for the sample left and right gain matrices of different calibration modes, and these benchmark values ​​can then be used to perform data preprocessing on the corresponding calibration modes' sample left and right gain matrices.

[0073] For the sample left and right gain matrices under QPD calibration mode, since their values ​​are around 2, the first reference value corresponding to the QPD calibration mode can be set to 2. During data preprocessing of the sample left and right gain matrices under QPD calibration mode, each value in the sample left gain matrix is ​​subtracted from the first reference value to obtain the preprocessed sample left gain matrix; similarly, each value in the sample right gain matrix is ​​subtracted from the first reference value to obtain the preprocessed sample right gain matrix.

[0074] For the sample left and right gain matrices under SPD calibration mode conditions, since their values ​​are around 1, the second reference value corresponding to the SPD calibration mode can be set to 1. During data preprocessing of the sample left and right gain matrices under SPD calibration mode conditions, each value in the sample left gain matrix is ​​subtracted from the second reference value to obtain the preprocessed sample left gain matrix; similarly, each value in the sample right gain matrix is ​​subtracted from the second reference value to obtain the preprocessed sample right gain matrix.

[0075] After data preprocessing, the preprocessed dataset is obtained based on the left gain matrix and the right gain matrix of all preprocessed samples.

[0076] Step 203: Based on the preprocessed dataset, construct the first training dataset, the second training dataset, and the third training dataset.

[0077] Since the gain calibration model group includes: a first gain calibration model corresponding to the left and right directions, a second gain calibration model corresponding to different calibration modes, and a third gain calibration model corresponding to different object distances, it is necessary to construct a first training dataset for the first gain calibration model corresponding to the left and right directions, a second training dataset for the second gain calibration model corresponding to different calibration modes, and a third training dataset for the third gain calibration model corresponding to different object distances, based on the preprocessed dataset.

[0078] Taking a set of data from the preprocessed dataset, including: the sample left and right gain matrices under QPD calibration mode and a 5cm object distance condition; the sample left and right gain matrices under SPD calibration mode and a 5cm object distance condition; the sample left and right gain matrices under QPD calibration mode and a 6cm object distance condition; the sample left and right gain matrices under SPD calibration mode and a 6cm object distance condition; the sample left and right gain matrices under QPD calibration mode and a 10cm object distance condition; the sample left and right gain matrices under SPD calibration mode and a 10cm object distance condition; the sample left and right gain matrices under QPD calibration mode and a 80cm object distance condition; and the sample left and right gain matrices under SPD calibration mode and a 80cm object distance condition, as an example, the following explanation will be provided:

[0079] The first training dataset includes multiple sets of sample left and right gain matrices under the same calibration mode and object distance conditions. Specifically, the first training dataset is a training set composed of sample left and right gain matrices with the same calibration mode and object distance conditions selected from the preprocessed dataset. The first training dataset is used to model a stable correspondence between the left and right gain matrices and train a first gain calibration model describing the response relationship between the left and right pixel channels. For example, based on a set of data from the aforementioned preprocessed dataset, for instance, the sample left and right gain matrices under the QPD calibration mode and 10cm object distance condition are used as a set of training data in the first training dataset. The sample left and right gain matrices under the SPD calibration mode and 10cm object distance condition are also used as a set of training data in the first training dataset.

[0080] The second training dataset includes multiple sets of left gain matrices and right gain matrices for samples under different calibration modes and the same object distance condition. Specifically, the second training dataset is a training set composed of multiple sets of left gain matrices and right gain matrices for samples under different calibration modes and the same object distance condition, selected from the preprocessed dataset. The second training dataset is used to reflect the influence of calibration mode changes on the gain matrix. For example, based on a set of data from the aforementioned preprocessed dataset, such as the left gain matrices of samples under QPD calibration mode and 10cm object distance condition, and the left gain matrices of samples under SPD calibration mode and 10cm object distance condition, are used as a set of training data in the second training dataset. Similarly, the right gain matrices of samples under QPD calibration mode and 10cm object distance condition, and the right gain matrices of samples under SPD calibration mode and 10cm object distance condition are used as a set of training data in the second training dataset.

[0081] The third training dataset includes multiple sets of left gain matrices and right gain matrices for samples under the same calibration mode and different object distances. Specifically, the third training dataset is a training set composed of multiple sets of left gain matrices and right gain matrices for samples under the same calibration mode and different object distances, selected from the preprocessed dataset. The third training dataset is used to reflect the influence of object distance changes on the gain matrix. For example, based on a set of data from the aforementioned preprocessed dataset, such as the left gain matrices of samples under the QPD calibration mode and a 10cm object distance, the left gain matrices of samples under the QPD calibration mode and a 5cm object distance, the left gain matrices of samples under the QPD calibration mode and a 6cm object distance, and the left gain matrices of samples under the QPD calibration mode and a 80cm object distance, the third training dataset is used as a set of training data. The right gain matrix of the sample under the SPD calibration mode and the 10cm object distance condition, the right gain matrix of the sample under the SPD calibration mode and the 5cm object distance condition, the right gain matrix of the sample under the SPD calibration mode and the 6cm object distance condition, and the right gain matrix of the sample under the SPD calibration mode and the 80cm object distance condition are used as a set of training data in the third training dataset.

[0082] Step 204: Train the first initial model based on the first training dataset to obtain the first gain calibration model corresponding to the left and right directions.

[0083] The first gain calibration model is trained using samples with the same calibration mode and object distance. Specifically, the first initial model is the initial machine learning model used to train the first gain calibration model. The first initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, in order to balance efficiency and memory constraints in practical application scenarios, the first initial model is an FCN model.

[0084] When training the first initial model using the first training dataset, an example is taken: a set of training data including the QPD calibration pattern and the sample left and right gain matrices under a 10cm object distance condition. The sample left gain matrix under the QPD calibration pattern and 10cm object distance condition is used as the model input, and the sample right gain matrix under the QPD calibration pattern and 10cm object distance condition is used as the model output true value; or the sample right gain matrix under the QPD calibration pattern and 10cm object distance condition is used as the model input, and the sample left gain matrix under the QPD calibration pattern and 10cm object distance condition is used as the model output true value. If the requirement is to derive the right gain matrix based on the left gain matrix, then the sample left gain matrix is ​​used as the input when training the model; if the requirement is to derive the left gain matrix based on the right gain matrix, then the sample right gain matrix is ​​used as the input when training the model. That is, the trained model corresponds one-to-one with the inference requirement.

[0085] The loss function of the first initial model is as follows:

[0086] ;

[0087] in, This represents the model's predicted value; y i represents the true value; n represents the number of samples.

[0088] The first training dataset is input into the first initial model for training. The model parameters are optimized through the loss function until the convergence condition is met, and the first gain calibration model is obtained.

[0089] Step 205: Train the second initial model based on the second training dataset to obtain the second gain calibration model corresponding to different calibration modes.

[0090] The second gain calibration model is trained using samples from the same object distance across calibration modes. Specifically, the second initial model is the initial machine learning model used to train the second gain calibration model. The second initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, to balance efficiency and memory constraints in practical applications, the second initial model is an FCN model.

[0091] When training the second initial model using the second training dataset, an example is taken: a set of training data including the left gain matrix of samples under the QPD calibration pattern and the 10cm object distance condition, and the left gain matrix of samples under the SPD calibration pattern and the 10cm object distance condition. The QPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model input, and the SPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model output true values; or, the SPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model input, and the QPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model output true values, to train the second initial model. If the requirement is to derive the SPD calibration pattern based on the QPD calibration pattern, then the QPD calibration pattern is used as the input when training the model; if the requirement is to derive the QPD calibration pattern based on the SPD calibration pattern, then the SPD calibration pattern is used as the input when training the model. That is, the trained model corresponds one-to-one with the inference requirement.

[0092] The loss function for the second initial model is as follows:

[0093] ;

[0094] in, This represents the model's predicted value; y i represents the true value; n represents the number of samples.

[0095] The second training dataset is input into the second initial model for training. The model parameters are optimized through the loss function until the convergence condition is met, thus obtaining the second gain calibration model.

[0096] Step 206: Train the third initial model based on the third training dataset to obtain the third gain calibration model corresponding to different object distances.

[0097] The third gain calibration model is trained using samples across object distances with the same calibration pattern. Specifically, the third initial model is the initial machine learning model used to train the third gain calibration model. The third initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, to balance efficiency and memory constraints in practical applications, the third initial model is an FCN model. Since the third gain calibration model predicts the gain matrices of multiple other object distances by inputting the gain matrix of any one object distance, the FCN model needs to be adjusted to meet the requirements of practical applications. In this embodiment, the FCN model uses a fully connected layer added to both the input and output, and a concat structure is used to enhance information extraction capabilities in order to improve the FCN model's information acquisition ability.

[0098] When training the third initial model based on the third training dataset, an example is taken: the left gain matrix of samples under the QPD calibration mode and a 10cm object distance condition, the left gain matrix of samples under the QPD calibration mode and a 5cm object distance condition, the left gain matrix of samples under the QPD calibration mode and a 6cm object distance condition, and the left gain matrix of samples under the QPD calibration mode and a 80cm object distance condition. The left gain matrix of samples under the QPD calibration mode and a 10cm object distance condition is used as the model input, and the left gain matrices of samples under the QPD calibration mode and a 5cm object distance condition, the QPD calibration mode and a 6cm object distance condition, and the QPD calibration mode and a 80cm object distance condition are used as the model's true values ​​to train the third initial model.

[0099] The loss function for the third initial model is as follows:

[0100] ;

[0101] in, This represents the model's predicted value; y i represents the true value; n represents the number of samples.

[0102] The third training dataset is input into the third initial model for training. The model parameters are optimized through the loss function until the convergence condition is met, thus obtaining the third gain calibration model.

[0103] The first, second, and third gain calibration models can be implemented by different output heads of the same neural network, or by independently trained sub-networks. Specifically, the first, second, and third gain calibration models can use the same neural network, with different output heads in this network outputting corresponding calibration parameters. Each of the first, second, and third gain calibration models can correspond to a sub-network, and each sub-network consists of a neural network.

[0104] This embodiment constructs a first initial dataset by collecting sample gain matrices covering all calibration modes and object distance conditions, and preprocesses them to form a high-quality preprocessed dataset. Based on this, a first training dataset, a second training dataset, and a third training dataset are decoupled, corresponding to three modeling dimensions: left-right channel relationship, cross-calibration mode change, and cross-object distance change, respectively. Then, a first gain calibration model, a second gain calibration model, and a third gain calibration model are trained to form a multi-dimensional collaborative gain matrix prediction system. In actual calibration, it is not necessary to repeatedly perform image acquisition and gain calculation for each calibration mode and object distance combination. Only a small amount of measured data is needed to derive the full-condition gain matrix through the gain calibration model group. This significantly reduces the number of PD image acquisitions, the frequency of environment switching, and the computational load, effectively reducing the time cost and equipment occupation cost of the calibration process, while improving parameter consistency and reproducibility.

[0105] In one embodiment, the defocus mapping relationship is implemented based on a coefficient calibration model group, which includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and each has the following functions;

[0106] The coefficient calibration model is used to predict the defocus conversion coefficients for other object distances under the same calibration mode and different object distances, using the defocus conversion coefficients for any two object distances. In other words, this model is used to map the defocus conversion coefficients corresponding to any two object distances to the defocus conversion coefficients corresponding to other two object distances under the same calibration mode. For example, the coefficient calibration model is used to predict the defocus conversion coefficients for other object distances under the same calibration mode and different object distances, using the defocus conversion coefficients for any two object distances. For example, using the first defocus conversion coefficient under the first calibration mode and the first object distance condition, and the second defocus conversion coefficient under the first calibration mode and the second object distance condition, the defocus conversion coefficients under the first calibration mode and the other object distance conditions are predicted.

[0107] The coefficient conversion algorithm is used to predict the defocus conversion coefficients of other calibration modes under different calibration modes and the same object distance, using the defocus conversion coefficient of any calibration mode. In other words, this algorithm maps known defocus conversion coefficients to defocus conversion coefficients corresponding to other calibration modes at the same object distance, based on the corresponding mode conversion matrix. For example, the coefficient conversion algorithm is used to predict the defocus conversion coefficients of other calibration modes under different calibration modes and the same object distance, using the defocus conversion coefficient of any calibration mode and a known mode conversion matrix. For instance, using the first defocus conversion coefficient under the first calibration mode and the first object distance, combined with the mode conversion matrix under the first object distance, the algorithm predicts the defocus conversion coefficients of other calibration modes and the first object distance.

[0108] In one embodiment, a set of coefficient calibration models needs to be trained before calibrating the camera module. For example... Figure 3 As shown, the training method for the coefficient calibration model may include the following steps:

[0109] Step 301: Collect multiple sets of input-output sample pairs. Each sample pair includes: the defocus conversion coefficients of samples corresponding to any two object distances under the same calibration mode after standardization as input samples; and the defocus conversion coefficients of samples corresponding to other two object distances under the same calibration mode as expected output samples.

[0110] Specifically, a second initial dataset can be obtained first. This second initial dataset includes the defocus conversion coefficients of samples under all calibration modes and object distance conditions. The second initial dataset can be historical calibration data from the production line. Taking the calibration modes including QPD and SPD calibration modes, and object distances including 5cm, 6cm, 10cm, and 80cm as an example, the data in the second initial dataset includes: defocus conversion coefficients of samples under QPD calibration mode and 5cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 5cm object distance; defocus conversion coefficients of samples under QPD calibration mode and 6cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 6cm object distance; defocus conversion coefficients of samples under QPD calibration mode and 10cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 10cm object distance; defocus conversion coefficients of samples under QPD calibration mode and 80cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 80cm object distance. Understandably, the second initial dataset includes multiple sets of the aforementioned data.

[0111] For example, the second initial dataset is configured in groups, each group containing: initial input samples and expected output samples; the initial input samples include: QPD calibration mode and sample defocus conversion coefficients under 10cm object distance conditions and QPD calibration mode and sample defocus conversion coefficients under 80cm object distance conditions; the expected output samples include: QPD calibration mode and sample defocus conversion coefficients under 5cm object distance conditions and QPD calibration mode and sample defocus conversion coefficients under 6cm object distance conditions.

[0112] For example, the second initial dataset is configured in groups, each group containing: initial input samples and expected output samples; the initial input samples include: the defocus conversion coefficient of the sample under SPD calibration mode and 10cm object distance condition, and the defocus conversion coefficient of the sample under SPD calibration mode and 80cm object distance condition; the expected output samples include: the defocus conversion coefficient of the sample under SPD calibration mode and 5cm object distance condition, and the defocus conversion coefficient of the sample under SPD calibration mode and 6cm object distance condition.

[0113] Based on the above, according to actual usage requirements, for the same calibration mode, the defocus conversion coefficients corresponding to any two object distances can be used as initial input samples, and the defocus conversion coefficients corresponding to other two object distances under the same calibration mode can be used as expected output samples.

[0114] Furthermore, the defocus conversion coefficients of the initial input samples in the second initial dataset can be standardized to obtain standard conversion coefficients. These standard conversion coefficients are the standardized sample defocus conversion coefficients and can be used as input samples. During the standardization process, the mean and standard deviation of the sample defocus conversion coefficients used as initial input samples are calculated respectively. Based on the mean and standard deviation, the sample defocus conversion coefficients for the corresponding calibration mode and object distance are standardized to obtain the standard conversion coefficients for the corresponding calibration mode and object distance.

[0115] Taking the standardization of the defocus conversion coefficient of the sample under QPD calibration mode and 10cm object distance as an example, the specific formula for standardization is as follows:

[0116] DCC after = (DCC before – DCC mean ) / DCC std ;

[0117] Among them, DCC after Standard conversion factor; DCC before DCC is the sample defocus conversion factor. mean The average values ​​are for QPD calibration mode and under a 10cm object distance condition; DCC std This represents the standard deviation under QPD calibration mode and a 10cm object distance condition. For each value in the matrix corresponding to the sample defocus conversion coefficient, the standard conversion coefficient is calculated using the above formula. Based on the above formula, the sample defocus conversion coefficients of all initial input samples are standardized to obtain the standard conversion coefficients for the input samples.

[0118] Step 302: The coefficient calibration model is trained based on the input samples and the expected output samples to calculate the defocus conversion coefficient between different object distances under the same calibration mode in the inference stage.

[0119] Specifically, the fourth initial model is trained based on the input samples and the expected output samples to obtain the coefficient calibration model. The fourth initial model is the initial machine learning model used to train the coefficient calibration model. The fourth initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, defocus conversion coefficient calibration can actually be viewed as time-series extrapolation; therefore, the fourth initial model is a Transformer model. However, the multi-head attention mechanism used by the Transformer model is time-consuming and memory-intensive, and its core self-attention value calculation is cumbersome, easily leading to overfitting on small datasets. Therefore, this embodiment optimizes the Transformer model as follows: First, the number of attention heads is reduced to 2 attention heads. Second, a channel attention mechanism is added to the existing Transformer model to improve its ability to extract channel information. Third, the sigmoid function of the channel attention mechanism is replaced with upsampling to make the model more suitable for the application scenario of this embodiment.

[0120] When training the fourth initial model based on input samples and expected output samples, the following example illustrates the process: Input samples include the QPD calibration mode and standard conversion coefficients under a 10cm object distance condition, as well as the QPD calibration mode and standard conversion coefficients under an 80cm object distance condition; expected output samples include the QPD calibration mode and sample defocus conversion coefficients under a 5cm object distance condition, as well as the QPD calibration mode and sample defocus conversion coefficients under a 6cm object distance condition. The fourth initial model is trained using these as inputs, and the QPD calibration mode and sample defocus conversion coefficients under a 5cm object distance condition and the QPD calibration mode and sample defocus conversion coefficients under a 6cm object distance condition as expected outputs. Corresponding to the standardization process for the initial input samples, before calculating the loss function using the model's initial predicted values ​​and expected output samples, the initial predicted values ​​are destandardized to obtain the model's predicted values.

[0121] The loss function for the fourth initial model is as follows:

[0122] ;

[0123] ;

[0124] ;

[0125] loss = λ1 × loss1 +λ2 × loss2 +λ3 × loss3;

[0126] in, This represents the model's predicted value; y i y represents the true value; abs represents the absolute value operation; n represents the number of samples. λ1, λ2, and λ3 represent the coefficients corresponding to each loss function, with preferred values ​​of λ1=0.3, λ2=0.5, and λ3=0.2. 19 y 20 y 27 y 28 These represent the true values ​​at indices 19, 20, 27, and 28 in the one-dimensional vector of DCC, respectively, under the measured object distance; corresponding to... , , , The target distance calculated by the model is the predicted value at the same index. The DCC one-dimensional vector is obtained by sampling along the straight line (horizontal centerline or radial line) passing through the optical center of the DCC matrix and arranging them in distance order. Index 20 is aligned with the center of the field of view, so the four points mentioned here are all located in the central region.

[0127] This embodiment uses model training to determine the nonlinear mapping relationship of the defocus conversion coefficient under the same calibration mode but different object distances. In subsequent calibration, only a small number of measured defocus conversion coefficient DCC values ​​are needed to derive the defocus conversion coefficient under other unmeasured conditions, which significantly reduces the number of image acquisitions and computational load, and reduces calibration cost and cycle while ensuring accuracy.

[0128] In one embodiment, the coefficient conversion algorithm is implemented through the mode conversion matrix. The mode conversion matrix is ​​determined by fitting sample defocus conversion coefficients corresponding to different calibration modes as training data at the same object distance. This matrix is ​​then used in the inference stage to convert the defocus conversion coefficients of any calibration mode into the defocus conversion coefficients corresponding to another calibration mode. Therefore, before calibrating the camera module, it is necessary to obtain mode conversion matrices corresponding to different object distances. These mode conversion matrices are used to convert the defocus conversion coefficients of one calibration mode into the defocus conversion coefficients of another calibration mode under the same object distance condition.

[0129] For example, let's take the example of obtaining the mode conversion matrix between the first and second calibration modes under a first object distance condition: Taking a first object distance of 10cm, and the first and second calibration modes being QPD and SPD calibration modes respectively, at this object distance, the sample defocus conversion coefficients for each camera module under QPD mode and SPD mode are extracted from historical production line data. For the same module, the sample defocus conversion coefficient under SPD mode is divided element-wise by the sample defocus conversion coefficient under QPD mode to obtain the individual mode conversion matrix for that module. Then, the average value is taken for the corresponding positions of the individual matrices of all modules to generate the final mode conversion matrix at a 10cm object distance. It is understandable that the median or other reasonable calculation methods can be used instead of the average value to calculate the mode conversion matrix.

[0130] Understandably, the mode conversion matrix between the first and second calibration modes under an object distance of 80cm, the mode conversion matrix between the first and second calibration modes under an object distance of 5cm, and the mode conversion matrix between the first and second calibration modes under an object distance of 6cm can also be obtained using the above method.

[0131] This embodiment obtains the mode conversion matrix corresponding to a certain object distance based on the defocus conversion coefficient in the first calibration mode and the defocus conversion coefficient in the second calibration mode. Based on the difference in defocus conversion coefficients of different calibration modes under the same object distance, it establishes the mode conversion rules, thereby obtaining the unmeasured mode parameters, saving calibration time and improving parameter calibration efficiency.

[0132] The following section details how to use the above mapping relationship and some measured parameters to obtain other unmeasured parameters.

[0133] In one embodiment, the gain matrix corresponding to multiple calibration modes and multiple object distance conditions is determined, specifically as follows: Figure 4 As shown, a gain matrix calibration method is provided, including the following steps:

[0134] This embodiment uses the gain calibration model, which includes three gain calibration models, as an example. The first calibration mode is the QPD calibration mode, the second calibration mode is the SPD calibration mode, the first object distance is 10cm, the second object distance is 80cm, the third object distance is 5cm, and the fourth object distance is 6cm as examples.

[0135] Step 401: Input the first gain matrix into the first gain calibration model to obtain the first calibration mode and the second gain matrix corresponding to the first object distance condition.

[0136] The first gain matrix can be the left gain matrix of the image acquisition device under the first calibration mode and the first object distance condition, or it can be the right gain matrix of the image acquisition device under the first calibration mode and the first object distance condition.

[0137] Taking the first gain matrix as the left gain matrix under the first calibration mode and the first object distance condition as an example, the first gain matrix is ​​the left gain matrix under the QPD calibration mode and the 10cm object distance condition. The first gain matrix is ​​input into the first gain calibration model, and the second gain matrix corresponding to the first calibration mode and the first object distance condition is output through the first gain calibration model, which is the right gain matrix under the QPD calibration mode and the 10cm object distance condition.

[0138] Step 402: Input the first gain matrix and the second gain matrix into the second gain calibration model to obtain the third gain matrix corresponding to the second calibration mode and the first object distance condition, and the fourth gain matrix corresponding to the second calibration model and the first object distance condition.

[0139] The first gain matrix, i.e. the left gain matrix under the QPD calibration mode and the 10cm object distance condition, is input into the second gain calibration model. The second gain calibration model outputs the third gain matrix, i.e. the left gain matrix under the SPD calibration mode and the 10cm object distance condition.

[0140] The second gain matrix, i.e. the right gain matrix under QPD calibration mode and 10cm object distance condition, is input into the second gain calibration model. The second gain calibration model outputs the fourth gain matrix, i.e. the right gain matrix under SPD calibration mode and 10cm object distance condition.

[0141] Step 403: Input the first gain matrix, the second gain matrix, the third gain matrix and the fourth gain matrix into the third gain calibration model to obtain multiple calibration modes and the corresponding gain matrices under multiple object distance conditions other than the first object distance.

[0142] The first gain matrix, i.e. the left gain matrix under the QPD calibration mode and the 10cm object distance condition, is input into the third gain calibration model. The third gain calibration model outputs the left gain matrix under the QPD calibration mode and the 5cm object distance condition, the left gain matrix under the QPD calibration mode and the 6cm object distance condition, and the left gain matrix under the QPD calibration mode and the 80cm object distance condition.

[0143] The second gain matrix, namely the right gain matrix under the QPD calibration mode and the 10cm object distance condition, is input into the third gain calibration model. The third gain calibration model outputs the right gain matrix under the QPD calibration mode and the 5cm object distance condition, the right gain matrix under the QPD calibration mode and the 6cm object distance condition, and the right gain matrix under the QPD calibration mode and the 80cm object distance condition.

[0144] Input the third gain matrix, namely the left gain matrix under the SPD calibration mode and the 10cm object distance condition, into the third gain calibration model. The third gain calibration model outputs the left gain matrix under the SPD calibration mode and the 5cm object distance condition, the left gain matrix under the SPD calibration mode and the 6cm object distance condition, and the left gain matrix under the SPD calibration mode and the 80cm object distance condition.

[0145] The fourth gain matrix, namely the right gain matrix under the SPD calibration mode and the 10cm object distance condition, is input into the third gain calibration model. The third gain calibration model outputs the right gain matrix under the SPD calibration mode and the 5cm object distance condition, the right gain matrix under the SPD calibration mode and the 6cm object distance condition, and the right gain matrix under the SPD calibration mode and the 80cm object distance condition.

[0146] Based on the above, the left and right gain matrices under all conditions are obtained. The above embodiment is only an example; in practice, the gain matrices corresponding to different object distances can be derived first, then the gain matrices corresponding to different calibration modes can be derived, and finally the left and right gain matrices can be derived, etc. This application does not impose any limitations. That is to say, the order and number of times the first, second, and third gain calibration models are used can be flexibly adjusted according to the actual calibration parameters obtained, as long as the left and right gain matrices under all conditions can be obtained by using the first, second, and third gain calibration models. Based on the same input, the accuracy of the results obtained by using the above three models, from high to low, is: third gain calibration model, second gain calibration model, and first gain calibration model.

[0147] This embodiment achieves gain transfer in the left and right directions by using the first gain calibration model, the second gain calibration model to achieve gain transfer across calibration modes, and the third gain calibration model to achieve gain transfer across object distances. Thus, starting from a portion of the measured gain matrix, a complete set of gain matrices covering multiple calibration modes and multiple object distance conditions is systematically generated.

[0148] In one embodiment, the defocus conversion coefficients corresponding to various calibration modes and various object distance conditions are determined, specifically as follows: Figure 5As shown, a method for calibrating the defocus conversion coefficient is provided, including the following steps:

[0149] This embodiment uses the coefficient mapping relationship implemented through a coefficient calibration model and a coefficient conversion algorithm as an example. The first calibration mode is QPD calibration mode, the second calibration mode is SPD calibration mode, the first object distance is 10cm, the second object distance is 80cm, the third object distance is 5cm, and the fourth object distance is 6cm as examples for illustration.

[0150] Step 501: Input the first defocus conversion coefficient and the second defocus conversion coefficient into the coefficient calibration model to obtain the third defocus conversion coefficient corresponding to the first calibration mode and the third object distance condition, and the fourth defocus conversion coefficient corresponding to the first calibration mode and the fourth object distance condition.

[0151] The first defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 10cm object distance condition. The second defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 80cm object distance condition. The first and second defocus conversion coefficients are input into the coefficient calibration model, which outputs the third and fourth defocus conversion coefficients. Specifically, the third defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 5cm object distance condition; the fourth defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 6cm object distance condition. It is understandable that before inputting the first and second defocus conversion coefficients into the coefficient calibration model, they need to be standardized. The standardization method is the same as described in step 301 above and will not be repeated here. Similarly, the original output of the coefficient calibration model is de-standardized to obtain the third and fourth defocus conversion coefficients.

[0152] Step 502: Obtain the first mode conversion matrix corresponding to the first object distance and the second mode conversion matrix corresponding to the second object distance.

[0153] The first mode conversion matrix is ​​the mode conversion matrix between the first calibration mode and the second calibration mode under an object distance of 10cm. The second mode conversion matrix is ​​the mode conversion matrix between the first calibration mode and the second calibration mode under an object distance of 80cm.

[0154] Step 503: Based on the first defocus conversion coefficient, the first mode conversion matrix, and the coefficient conversion algorithm, obtain the second calibration mode and the fifth defocus conversion coefficient corresponding to the first object distance condition.

[0155] After obtaining the first mode conversion matrix, the fifth defocus conversion coefficient is obtained based on the first defocus conversion coefficient, the first mode conversion matrix, and the coefficient conversion algorithm. The specific formula for the coefficient conversion algorithm is as follows:

[0156] DCC spd = k1 × DCC qpd ;

[0157] Among them, DCC qpd This represents the first defocus conversion factor, which is also the defocus conversion factor under QPD calibration mode and a 10cm object distance condition; DCC spd represents the fifth defocus conversion factor, which is the defocus conversion factor under SPD calibration mode and 10cm object distance conditions; k1 represents the first mode conversion matrix. The matrix size of the mode conversion matrix is ​​the same as the matrix size of the matrix corresponding to the defocus conversion factor.

[0158] Step 504: Based on the second defocus conversion coefficient, the second mode conversion matrix, and the coefficient conversion algorithm, obtain the sixth defocus conversion coefficient corresponding to the second calibration mode and the second object distance condition.

[0159] After obtaining the second mode conversion matrix, the sixth defocus conversion coefficient is obtained based on the second defocus conversion coefficient, the second mode conversion matrix, and the coefficient conversion algorithm. The specific formula for the coefficient conversion algorithm is as follows:

[0160] DCC spd = k2 × DCC qpd ;

[0161] Among them, DCC qpd This indicates the second defocus conversion factor, which is the defocus conversion factor under QPD calibration mode and 80cm object distance conditions; DCC spd represents the sixth defocus conversion factor, which is the defocus conversion factor under SPD calibration mode and 80cm object distance conditions; k2 represents the second mode conversion matrix. The matrix size of the mode conversion matrix is ​​the same as the matrix size of the matrix corresponding to the defocus conversion factor.

[0162] Step 505: Input the fifth and sixth defocus conversion coefficients into the coefficient calibration model to obtain the seventh defocus conversion coefficient corresponding to the second calibration mode and the third object distance condition, and the eighth defocus conversion coefficient corresponding to the second calibration mode and the fourth object distance condition.

[0163] After obtaining the fifth and sixth defocus conversion coefficients, they are input into the coefficient calibration model. The model then outputs the seventh and eighth defocus conversion coefficients. The seventh defocus conversion coefficient corresponds to the defocus conversion coefficient under the SPD calibration mode and a 5cm object distance condition; the eighth defocus conversion coefficient corresponds to the defocus conversion coefficient under the SPD calibration mode and a 6cm object distance condition. Understandably, before inputting the fifth and sixth defocus conversion coefficients into the coefficient calibration model, they need to be standardized. The standardization method is the same as described in step 301 above and will not be repeated here. Similarly, the original output of the coefficient calibration model is destandardized to obtain the seventh and eighth defocus conversion coefficients.

[0164] Optionally, when determining the seventh and eighth defocus conversion factors:

[0165] The third mode conversion matrix between the first and second calibration modes under a 5cm object distance condition can be obtained. Using the third defocus conversion coefficient, the third mode conversion matrix, and the coefficient conversion algorithm corresponding to the first calibration mode and the third object distance condition, the seventh defocus conversion coefficient corresponding to the second calibration mode and the third object distance condition can be obtained.

[0166] The fourth mode conversion matrix between the first and second calibration modes under a 6cm object distance condition can be obtained. Using the fourth defocus conversion coefficient, the fourth mode conversion matrix, and the coefficient conversion algorithm corresponding to the first calibration mode and the fourth object distance condition, the eighth defocus conversion coefficient corresponding to the second calibration mode and the fourth object distance condition can be obtained.

[0167] This application does not specifically limit the method for determining the seventh and eighth defocus conversion coefficients. Since the model yields better accuracy and takes less time than the coefficient matrix, the coefficient calibration model is preferred for determining the seventh and eighth defocus conversion coefficients.

[0168] Based on the above, the defocus conversion coefficient under all conditions is obtained.

[0169] This embodiment achieves cross-object distance extension of DCC under the same calibration mode by using DCC values ​​at two known object distances as input to call the coefficient calibration model and outputting DCC values ​​at other object distances under the same mode. By substituting the first defocus conversion coefficient and the first mode conversion matrix into the coefficient conversion algorithm to perform inter-mode conversion calculations and outputting DCC values ​​in the target mode, DCC migration from a known mode to a new calibration mode is achieved. Using the two DCC values ​​derived across modes as a new starting point, the coefficient calibration model is called again to perform cross-object distance extension, generating a DCC set under the complete object distance sequence, completing a systematic coverage from a small number of measured parameters to full-dimensional calibration. This avoids the redundant operations of repeatedly performing PD image acquisition, multi-image processing, and DCC fitting for each mode-object distance combination in traditional methods, significantly reducing the number of image acquisitions and computational load, and greatly reducing resource consumption while ensuring accuracy, thereby reducing calibration costs and time.

[0170] In one embodiment, the phase difference offset corresponding to multiple calibration modes and multiple object distance conditions is determined, specifically as follows: Figure 6 As shown, a phase difference offset calibration method is provided, which specifically includes the following steps:

[0171] Step 601: Based on the first phase difference offset, the first defocus conversion coefficient, the fifth defocus conversion coefficient, and the first offset conversion algorithm, obtain the third phase difference offset corresponding to the second calibration mode and the first object distance condition.

[0172] The offset mapping relationship is achieved through a first offset conversion algorithm between different modes and a second offset conversion algorithm between different object distances.

[0173] The following explanation uses the following calibration modes as examples: QPD calibration mode as the first calibration mode, SPD calibration mode as the second calibration mode, and object distances of 10cm, 80cm, 5cm, and 6cm as the third and fourth calibration modes.

[0174] The first phase difference offset represents the phase difference offset in QPD calibration mode and under a 10cm object distance condition. The first defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 10cm object distance condition. The fifth defocus conversion factor represents the defocus conversion factor in SPD calibration mode and under a 10cm object distance condition. The third phase difference offset represents the phase difference offset in SPD calibration mode and under a 10cm object distance condition.

[0175] The specific formula for the first offset conversion algorithm to calculate the third phase difference offset is as follows:

[0176] ;

[0177] in, This indicates the offset of the third phase difference; Indicates the fifth defocus conversion factor; Indicates the first defocus conversion factor; This represents the first phase difference offset.

[0178] Step 602: Based on the second phase difference offset, the second defocus conversion coefficient, the sixth defocus conversion coefficient, and the first offset conversion algorithm, obtain the fourth phase difference offset corresponding to the second calibration mode and the second object distance condition.

[0179] The second phase difference offset can be obtained using traditional methods while acquiring the second defocus conversion coefficient. Alternatively, it can be obtained using a conversion algorithm based on the first phase difference offset, the first defocus conversion coefficient, the second defocus conversion coefficient, and the second offset.

[0180] The second phase difference offset represents the phase difference offset in QPD calibration mode and under an 80cm object distance condition. The second defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under an 80cm object distance condition. The sixth defocus conversion factor represents the defocus conversion factor in SPD calibration mode and under an 80cm object distance condition. The fourth phase difference offset represents the phase difference offset in SPD calibration mode and under an 80cm object distance condition.

[0181] The specific formula for the first offset conversion algorithm to calculate the fourth phase difference offset is as follows:

[0182] ;

[0183] in, This indicates the fourth phase difference offset; Indicates the sixth defocus conversion factor; Indicates the second defocus conversion factor; This indicates the second phase difference offset.

[0184] Step 603: Based on the first phase difference offset, the first defocus conversion coefficient, the third defocus conversion coefficient, and the second offset conversion algorithm, obtain the fifth phase difference offset corresponding to the first calibration mode and the third object distance condition.

[0185] The first phase difference offset represents the phase difference offset in QPD calibration mode and under a 10cm object distance condition. The first defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 10cm object distance condition. The third defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 5cm object distance condition. The fifth phase difference offset represents the phase difference offset in QPD calibration mode and under a 5cm object distance condition.

[0186] The specific formula for the second offset conversion algorithm to calculate the fifth phase difference offset is as follows:

[0187] ;

[0188] in, This indicates the fifth phase difference offset; Indicates the third defocus conversion factor; Indicates the first defocus conversion factor; This represents the first phase difference offset.

[0189] Step 604: Based on the first phase difference offset, the first defocus conversion coefficient, the fourth defocus conversion coefficient, and the second offset conversion algorithm, obtain the sixth phase difference offset corresponding to the first calibration mode and the fourth object distance condition.

[0190] The first phase difference offset represents the phase difference offset in QPD calibration mode and under a 10cm object distance condition. The first defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 10cm object distance condition. The fourth defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 6cm object distance condition. The sixth phase difference offset represents the phase difference offset in QPD calibration mode and under a 6cm object distance condition.

[0191] The specific formula for the second offset conversion algorithm to calculate the sixth phase difference offset is as follows:

[0192] ;

[0193] in, This indicates the sixth phase difference offset; Indicates the fourth defocus conversion factor; Indicates the first defocus conversion factor; This represents the first phase difference offset.

[0194] Step 605: Based on the third phase difference offset, the fifth defocus conversion coefficient, the seventh defocus conversion coefficient, and the second offset conversion algorithm, obtain the seventh phase difference offset corresponding to the second calibration mode and the third object distance condition.

[0195] The third phase difference offset represents the phase difference offset in SPD calibration mode and under a 10cm object distance condition. The fifth defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 10cm object distance condition. The seventh defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 5cm object distance condition. The seventh phase difference offset represents the phase difference offset in SPD calibration mode and under a 5cm object distance condition.

[0196] The specific formula for the second offset conversion algorithm to calculate the seventh phase difference offset is as follows:

[0197] ;

[0198] in, This indicates the seventh phase difference offset; Indicates the seventh defocus conversion factor; Indicates the fifth defocus conversion factor; This indicates the offset of the third phase difference.

[0199] Step 606: Based on the third phase difference offset, the fifth defocus conversion coefficient, the eighth defocus conversion coefficient, and the second offset conversion algorithm, obtain the eighth phase difference offset corresponding to the second calibration mode and the fourth object distance condition.

[0200] The third phase difference offset represents the phase difference offset in SPD calibration mode and under a 10cm object distance condition. The fifth defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 10cm object distance condition. The eighth defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 6cm object distance condition. The eighth phase difference offset represents the phase difference offset in SPD calibration mode and under a 6cm object distance condition.

[0201] The specific formula for the second offset conversion algorithm to calculate the eighth phase difference offset is as follows:

[0202] ;

[0203] in, This indicates the eighth phase difference offset; This represents the eighth defocus conversion factor; Indicates the fifth defocus conversion factor; This indicates the offset of the third phase difference.

[0204] Steps 605 and 606 above obtain the seventh phase difference offset and the eighth phase difference offset through the second offset conversion algorithm. In actual use, the seventh phase difference offset and the eighth phase difference offset can also be calculated by the first offset conversion algorithm. This embodiment does not make specific limitations. It is only necessary to combine the first offset conversion algorithm and the second offset conversion algorithm to obtain the phase difference offset under all object distance conditions in the multi-calibration mode.

[0205] This embodiment constructs a dual-path offset conversion mechanism. The first offset conversion algorithm is used for offset migration across calibration modes, and the second offset conversion algorithm is used for offset migration across object distances. The algorithm-driven parameter migration mechanism replaces the traditional method of repeatedly performing phase difference offset calibration for each combination of calibration mode and object distance. This significantly reduces calibration resource consumption, calibration cost, and time consumption while ensuring focusing accuracy.

[0206] In one specific embodiment, a multi-object distance / multi-mode PDAF algorithm merging method includes the following steps:

[0207] The two calibration modes are QPD calibration mode and SPD calibration mode. The four object distances are 10cm, 80cm, 5cm and 6cm.

[0208] Step 1: Train the first gain calibration model corresponding to the left and right directions, the second gain calibration model corresponding to different calibration modes, and the third gain calibration model corresponding to different object distances. The gain matrix (Gain map) is mainly used for preprocessing the image for PDAF calculation. Therefore, the gain matrix includes the left gain matrix (LGM) and the right gain matrix (RGM). The gain matrix is ​​a 12×16 matrix. Based on the above, each calibration mode and each object distance corresponds to a left gain map and a right gain map.

[0209] Step 1.1: The first gain calibration model corresponding to the left and right directions shows opposite trends in the left and right gain maps, and the sum of corresponding positions is approximately 4. Therefore, an AI algorithm is used to calculate it, as follows:

[0210] a. Data preprocessing: Since the gain matrix value is approximately 2 in QPD calibration mode and approximately 1 in SPD calibration mode, based on the following formula, 'a' equals 2 in QPD calibration mode and 'a' equals 1 in SPD calibration mode. All gain matrices are preprocessed using the following formula.

[0211] GMafter = GMbefore - a;

[0212] Where GMafter represents the gain matrix after preprocessing; GMbefore represents the gain matrix before preprocessing. Data preprocessing can reduce errors caused by fluctuations in the data itself, making the calibration results fluctuate within a certain range and suppressing the occurrence of outliers.

[0213] b. Training the model: Left Gain map and Right Gain map. The input is a 1×12×16 Left Gain map, and the output is a 1×12×16 Right Gain map, with stable relationships between the data. Considering the efficiency of production line applications and memory limitations, the FCN algorithm is chosen as the prediction model.

[0214] c. Loss function design, the specific loss function is as follows:

[0215] ;

[0216] Step 1.2: The second gain calibration model corresponds to different calibration modes. The gain matrix value of the QPD calibration mode is around 2, and the gain matrix value of the SPD calibration mode is around 1, with a stable trend. The model input is the Left Gain map and Right Gain map of the first calibration mode, and the output is the Left Gain map and Right Gain map of the second calibration mode. The specific method is the same as in Step 1.1.

[0217] Step 1.3: Third gain calibration model for different object distances. Considering the object distance selection issue in DCC merging, QPD-10CM-LGM, QPD-10CM-RGM, SPD-10CM-LGM, and SPD-10CM-RGM were ultimately selected as measured inputs to infer the gain maps for the other three object distances. The steps are as follows:

[0218] a. Data preprocessing is the same as in step 1.1.

[0219] b. Training the model: Taking QPD-10CM-LGM as an example, the input of the third gain calibration model is a 1×12×16 QPD-10CM-LGM, but it needs to output the LGMs of the other three object distances simultaneously. The FCN (Fully Convolutional Network) model cannot meet this requirement, so the FCN model needs to be optimized. The optimized FCN model process is as follows: Figure 7 As shown, the core of the new model remains the FCN model, but fully connected layers are added at the beginning and end of the FCN model to improve the model's ability to acquire information. Furthermore, the input and output of this sub-model are concatenated in the concatenation layer to further enhance the model's ability to acquire information.

[0220] c. Loss function design, the specific loss function is as follows:

[0221] ;

[0222] In the third gain calibration model, the output of each sub-model is output at the end. Therefore, it is necessary to calculate the loss for each sub-model's output, denoted as loss1, loss2, and loss3 respectively. The final loss formula is as follows: loss = loss1 + loss2 + loss3. This embodiment can reduce the original process of sampling 8 images and calculating 16 gain maps to sampling 1 image and calculating 1 gain map, greatly improving the efficiency of PDAF calibration on the production line.

[0223] Step 2: Train the coefficient calibration model for the DCC (Discrete Conversion Coefficient). Based on the efficiency, accuracy, and profitability of the production line, a two-to-two algorithm is ultimately used for object distance merging. Considering the module's characteristics, the testing scheme for the inspection station, and data stability, DCC values ​​of 10cm and 80cm object distances are selected as the actual measured input data to predict DCC values ​​of 5cm and 6cm object distances. Details are as follows:

[0224] Step 2.1, data preprocessing, can reduce the impact of data fluctuations on accuracy and limit the results to a certain range, thus reducing the risk of model inference anomalies. The standardized formula is as follows:

[0225] DCC after = (DCC before – DCC mean ) / DCC std ;

[0226] Among them, DCC after For standardized DCC; DCC before DCC before standardization; DCC mean The average value of DCC; DCC std This represents the standard deviation of DCC.

[0227] Step 2.2, Model Training: Since there is currently no commercially available method for extrapolating PDAF data using AI models, this algorithm requirement is shifted to time-series extrapolation, and the Transformer algorithm is ultimately chosen for extrapolation. However, the multi-head attention mechanism used by the Transformer model is time-consuming and memory-intensive, and its core self-attention value calculation is cumbersome, making it prone to overfitting on small datasets. Therefore, the following optimizations are made to the Transformer algorithm: 1. The multi-head attention mechanism is reduced to a fixed number of 2 heads. 2. To improve the attention mechanism's ability to extract channel information, a channel attention mechanism is added. 3. The sigmoid function of the channel attention mechanism is replaced with upsampling, making the model more suitable for the task in this embodiment.

[0228] Step 2.3, Loss Function Design: The loss function is as follows:

[0229] ;

[0230] ;

[0231] ;

[0232] loss = λ1 × loss1 +λ2 × loss2 +λ3 × loss3;

[0233] in, This represents the model's predicted value; y i `abs` represents the true value; `abs` represents the absolute value operation; `n` represents the number of samples. `loss1` represents the mean squared error; `loss2` represents the DCC numerical precision error; `loss3` represents the center position weighted error. `λ1`, `λ2`, and `λ3` represent the coefficients corresponding to each loss function, with preferred values ​​of `λ1` = 0.3, `λ2` = 0.5, and `λ3` = 0.2. 19 y 20 y 27 y 28 These represent the true values ​​at indices 19, 20, 27, and 28 in the one-dimensional vector of DCC, respectively, under the measured object distance; corresponding to... , , , The target distance calculated by the model is the predicted value at the same index. The DCC one-dimensional vector is obtained by sampling along the straight line (horizontal centerline or radial line) passing through the optical center of the DCC matrix and arranging them in distance order. Index 20 is aligned with the center of the field of view, so the four points mentioned here are all located in the central region.

[0234] Step 3: Predict the defocus conversion coefficient (DCC) and phase difference offset for various calibration modes and object distances. Mathematically, the accuracy of the defocus conversion coefficient (DCC) is inversely proportional to the signal-to-noise ratio (SNR) of phase detection. The QPD calibration mode has a high SNR and a low DCC value; the SPD calibration mode has a low SNR and a high DCC value. From a chip perspective, the QPD calibration mode divides the pixels on the chip into four parts to obtain the left and right PD points. The SPD calibration mode, however, obscures half of the pixels to obtain the left and right PD points. This results in the PD point density of the QPD calibration mode being twice that of the SPD calibration mode on the chip, leading to a PD value that is approximately twice that of the SPD calibration mode during calculation. Consequently, the final DCC value of the QPD calibration mode is approximately half that of the SPD calibration mode. Although environmental factors may affect the calibration process, the overall multiplier is between 1.93 and 1.96, and the module's dynamic error is about 5%. Therefore, the coefficient matrix is ​​used to calculate the DCC value of the SPD mode.

[0235] Using the coefficient calibration model in step 2, the defocus conversion coefficient (DCC) is predicted pairwise for different object distances under the same calibration mode. The following coefficient conversion algorithm is used to predict the defocus conversion coefficient (DCC) for the same object distance under different calibration modes; the specific formula is as follows:

[0236] DCC spd = k × DCC qpd ;

[0237] DCC qpd Indicates the defocus conversion factor for QPD calibration mode; DCC spd The defocus conversion coefficient represents the SPD calibration mode, and k represents the mode conversion matrix. For example, it includes the mode conversion matrix under the condition of 10cm object distance, the mode conversion matrix under the condition of 80cm object distance, the mode conversion matrix under the condition of 5cm object distance, and the mode conversion matrix under the condition of 6cm object distance.

[0238] The specific formula for predicting the phase difference offset between the same object distance in different calibration modes is as follows:

[0239] ;

[0240] This indicates the phase difference offset of the SPD calibration mode; Indicates the defocus conversion factor of the SPD calibration mode; This indicates the defocus conversion factor in the QPD calibration mode; This indicates the phase difference offset of the QPD calibration mode.

[0241] For predicting the phase difference offset between different object distances under the same calibration mode, since 5cm, 6cm, and 10cm object distances are all macro object distances with similar physical characteristics, the offset of 10cm is used to calculate the offsets of 5cm and 6cm. The specific formula is as follows:

[0242] ;

[0243] This represents the phase difference offset at a 5cm object distance. The defocus conversion factor represents the distance from the object at a 5cm object distance; The defocus conversion factor represents the distance between the object and the object at a distance of 10cm. This indicates the phase difference offset at a distance of 10cm from the object.

[0244] ;

[0245] This represents the phase difference offset at a distance of 6m. This represents the defocus conversion factor at an object distance of 6m. This represents the defocus conversion factor at an object distance of 10m. This represents the phase difference offset at a distance of 10m.

[0246] In this embodiment, the original process of taking 8 images and calculating 16 DCC and offsets is reduced to taking 2 images and calculating 2 DCC and offsets, which greatly improves the PDAF calibration efficiency of the production line.

[0247] In this embodiment, the coefficient calibration model predicts the defocus conversion coefficient (DCC) for pairwise objects at different distances within the same calibration mode. This embodiment selects the DCC at a 10cm and 80cm object distance, and predicts the DCC at a 5cm and 6cm object distance. The specific reasons are as follows: 1. Considering the characteristics of the periscope macro module, the unique periscope object distance of 80cm is used as the measured object distance, i.e., as the input object distance. 2. The detection scheme at the production line inspection station uses 6cm and 10cm object distances for detection. To avoid affecting the reliability of the inspection station, a choice is made between 6cm and 10cm object distances for macro detection. 3. Since the DCC value at 6cm object distance is less stable than that at 10cm object distance, it is not suitable as input data. In summary, the DCC at 10cm and 80cm object distances were ultimately selected.

[0248] In this embodiment, for the third gain calibration model, which uses the offset of one object distance to predict the offsets of the other three object distances for different object distances under the same calibration mode, this embodiment selects the gain matrix for a 10cm object distance to predict the gain matrices for 5cm, 6cm, and 80cm object distances. The specific reasons are as follows: 1. Combining the object distance selection of the coefficient calibration model, a choice is made between 10cm and 80cm object distances. 2. Considering the PDAF detection station on the production line, OK / NG detection is performed using 6cm and 10cm object distances. Therefore, the gain matrix for the 10cm object distance is ultimately used as input to calculate the gain matrices for the other three object distances.

[0249] In one specific embodiment, such as Figure 8 As shown, first, a QPD-10cm-LGM is obtained. The QPD-10cm-LGM is then input into the first gain calibration model to obtain a QPD-10cm-RGM. The QPD-10cm-LGM is then input into the second gain calibration model to obtain an SPD-10cm-LGM. The QPD-10cm-RGM is then input into the second gain calibration model to obtain an SPD-10cm-RGM. The QPD-10cm-LGM is then input into the third gain calibration model to obtain QPD-5cm-LGM, QPD-6cm-LGM, and QPD-80cm-LGM. The QPD-10cm-RGM is then input into the third gain calibration model to obtain QPD-5cm-RGM, QPD-6cm-RGM, and QPD-80cm-RGM. Finally, the SPD-10cm-LGM is input into the third gain calibration model to obtain SPD-5cm-LGM, SPD-6cm-LGM, and SPD-80cm-LGM. Inputting the SPD-10cm-RGM into the third gain calibration model yields the SPD-5cm-RGM, SPD-6cm-RGM, and SPD-80cm-RGM.

[0250] In one embodiment, such as Figure 9As shown, firstly, QPD-10cm-DCC, QPD-10cm-offset, QPD-80cm-DCC, and QPD-80cm-offset are obtained. QPD-10cm-DCC and QPD-80cm-DCC are input into the coefficient calibration model to obtain QPD-5cm-DCC and QPD-6cm-DCC. Based on QPD-10cm-DCC and the 10cm mode transition matrix, SPD-10cm-DCC is determined. Based on QPD-80cm-DCC and the 80cm mode transition matrix, SPD-80cm-DCC is determined. SPD-10cm-DCC and SPD-80cm-DCC are input into the coefficient calibration model to obtain SPD-5cm-DCC and SPD-6cm-DCC. Based on QPD-10cm-offset, QPD-10cm-DCC, and SPD-10cm-DCC, SPD-10cm-offset is determined. Determine the SPD-80cm-offset based on QPD-80cm-DCC, QPD-80cm-offset, and SPD-80cm-DCC. Determine the QPD-5cm-offset based on QPD-10cm-offset, QPD-10cm-DCC, and QPD-5cm-DCC. Determine the QPD-6cm-offset based on QPD-10cm-offset, QPD-10cm-DCC, and QPD-6cm-DCC. Determine the SPD-5cm-offset based on SPD-10cm-offset, SPD-10cm-DCC, and SPD-5cm-DCC. Determine the SPD-6cm-offset based on SPD-10cm-offset, SPD-10cm-DCC, and SPD-6cm-DCC.

[0251] This embodiment proposes a multi-object distance / multi-mode PDAF algorithm merging method. By calculating the gain map value, DCC value, and offset value under multiple calibration modes and multiple object distances based on AI algorithms, it reduces the number of times the camera module needs to re-open the image, the number of images acquired, and the number of equipment required for the project. This not only reduces the time consumption and number of equipment required for PDAF calibration, but also effectively reduces the dynamic error of PDAF caused by module motor movement and image re-opening.

[0252] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0253] Based on the same inventive concept, this application also provides a parameter calibration device for an image acquisition device to implement the parameter calibration method of the image acquisition device described above. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations in one or more embodiments of the parameter calibration device for an image acquisition device provided below can be found in the limitations of the parameter calibration method for the image acquisition device described above, and will not be repeated here.

[0254] In one embodiment, such as Figure 10 As shown, a parameter calibration device for an image acquisition equipment is provided, comprising: an acquisition module 100, a first calibration module 200, a second calibration module 300, and a storage module 400, wherein:

[0255] The acquisition module 100 is used to acquire the first gain matrix and the first defocus conversion coefficient of the image acquisition device in the first calibration mode and at the first object distance, and the second defocus conversion coefficient in the same calibration mode and at the second object distance.

[0256] The first calibration module 200 is used to determine a gain matrix corresponding to the first target calibration mode and / or the first target object distance based on the first gain matrix and the trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance.

[0257] And / or, the second calibration module 300 is used to determine the defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient and the trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance;

[0258] The storage module 400 is used to write the obtained gain matrix and / or defocus conversion coefficient into the memory of the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage.

[0259] The parameter calibration device for the image acquisition equipment also includes a third calibration module.

[0260] The third calibration module is used to obtain the first phase difference offset of the image acquisition device in the first calibration mode and at the first object distance; based on the first defocus conversion coefficient and the first phase difference offset, combined with at least one paired defocus conversion coefficient across modes or across object distances, and the offset mapping relationship, determine the phase difference offset corresponding to the target condition; the target condition is different from the first condition formed by the first calibration mode and the first object distance only in the calibration mode or the object distance.

[0261] The gain mapping relationship is implemented through a gain calibration model group, which includes at least one of a first gain calibration model, a second gain calibration model, and a third gain calibration model, and each model has the following functions: the first gain calibration model is used to perform consistency correction on the input gain matrix under the same calibration mode and the same object distance, and output the consistency-corrected gain matrix; the second gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other calibration mode under the same object distance; the third gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other object distance under the same calibration mode.

[0262] The first gain calibration model, the second gain calibration model, and the third gain calibration model are implemented by different output heads of the same neural network, or by sub-networks that are trained independently.

[0263] The defocus mapping relationship is implemented based on a coefficient calibration model group, which includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and each has the following functions: The coefficient calibration model is used to map the defocus conversion coefficients corresponding to any two object distances to the defocus conversion coefficients corresponding to other two object distances under the same calibration mode; The coefficient conversion algorithm is used to map the known defocus conversion coefficients to the defocus conversion coefficients corresponding to the same object distance and other calibration modes based on the corresponding mode conversion matrix under the same object distance.

[0264] The first calibration module 200 is further configured to determine the gain mapping relationship by: acquiring sample gain matrices covering all calibration modes and all object distances, and after data preprocessing, training each model in the gain calibration model group respectively; the training includes at least: training the first gain calibration model using samples of the same calibration mode and the same object distance; and / or training the second gain calibration model using samples of the same object distance across calibration modes; and / or training the third gain calibration model using samples of the same calibration mode across object distances; to determine the gain mapping relationship.

[0265] The second calibration module 300 is further used to determine the coefficient calibration model in the following way: collecting multiple sets of input-output sample pairs, each set of sample pairs containing: sample defocus conversion coefficients corresponding to any two object distances under the same calibration mode after standardization as input samples; sample defocus conversion coefficients corresponding to other two object distances under the same calibration mode as expected output samples; training the coefficient calibration model based on the input samples and expected output samples to realize the defocus conversion coefficient estimation between different object distances under the same calibration mode in the inference stage.

[0266] The coefficient conversion algorithm is implemented through the mode conversion matrix. The mode conversion matrix is ​​determined by fitting the sample defocus conversion coefficients corresponding to different calibration modes under the same object distance, and is used in the inference stage to convert the defocus conversion coefficient of any calibration mode into the defocus conversion coefficient corresponding to another calibration mode.

[0267] Each module in the parameter calibration device of the aforementioned image acquisition equipment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0268] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a parameter calibration method for an image acquisition device.

[0269] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0270] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the parameter calibration method of any of the image acquisition devices described in the above embodiments.

[0271] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the parameter calibration method of any of the image acquisition devices described in the above embodiments.

[0272] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0273] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0274] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. Those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A parameter calibration method for an image acquisition device, characterized in that, The method includes: Acquire the first gain matrix and first defocus conversion coefficient of the image acquisition device in the first calibration mode and at the first object distance, and the second defocus conversion coefficient in the same calibration mode and at the second object distance; perform at least one of the following two steps: a) Based on the first gain matrix and the trained gain mapping relationship, determine the gain matrix corresponding to the first target calibration mode and / or the first target object distance; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; b) Based on the first defocus conversion coefficient, the second defocus conversion coefficient, and the trained defocus mapping relationship, determine the defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance. The obtained gain matrix and / or defocus conversion coefficient are written into the memory of the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage; The defocus mapping relationship is implemented based on a coefficient calibration model group, which includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and each has the following functions: The coefficient calibration model is used to map the defocus conversion coefficients corresponding to any two object distances to the defocus conversion coefficients corresponding to other two object distances under the same calibration mode; The coefficient conversion algorithm is used to map the known defocus conversion coefficients to the defocus conversion coefficients corresponding to the same object distance and other calibration modes based on the corresponding mode conversion matrix under the same object distance. The gain mapping relationship is implemented through a gain calibration model group, which includes at least one of a first gain calibration model, a second gain calibration model, and a third gain calibration model, and each model has the following functions: the first gain calibration model is used to perform consistency correction on the input gain matrix under the same calibration mode and the same object distance, and output the consistency-corrected gain matrix; the second gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other calibration mode under the same object distance; the third gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other object distance under the same calibration mode.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the first phase difference offset of the image acquisition device when it is in the first calibration mode and at the first object distance; Based on the first defocus conversion coefficient and the first phase difference offset, combined with at least one paired defocus conversion coefficient across modes or across object distances, and the offset mapping relationship, the phase difference offset corresponding to the target condition is determined; the target condition differs from the first condition consisting of the first calibration mode and the first object distance only in the calibration mode or object distance, and the paired defocus conversion coefficient corresponds to the target condition.

3. The method according to claim 1, characterized in that, The first gain calibration model, the second gain calibration model, and the third gain calibration model are implemented by different output heads of the same neural network, or by sub-networks that are trained independently.

4. The method according to claim 1, characterized in that, The gain mapping relationship is determined in the following way: Collect sample gain matrices covering all calibration modes and all object distances, and after data preprocessing, train each model in the gain calibration model group respectively. The training includes at least: The first gain calibration model is trained using samples of the same calibration mode and object distance; and / or The second gain calibration model is trained using samples from the same object distance across calibration patterns; and / or The third gain calibration model was trained using the same calibration pattern across object distance samples; To determine the gain mapping relationship.

5. The method according to claim 1, characterized in that, The coefficient calibration model is determined in the following way: Collect multiple sets of input-output sample pairs, each sample pair containing: The defocus conversion coefficients of any two object distances under the same standardized calibration mode are used as input samples. The defocus conversion coefficients of the samples corresponding to the other two object distances under the same calibration mode are used as the expected output samples. The coefficient calibration model is trained based on the input samples and the expected output samples, so as to realize the defocus conversion coefficient between different object distances under the same calibration mode in the inference stage.

6. The method according to claim 1, characterized in that, The coefficient conversion algorithm is implemented through the mode conversion matrix. The mode conversion matrix is ​​determined by fitting the sample defocus conversion coefficients corresponding to different calibration modes under the same object distance, and is used in the inference stage to convert the defocus conversion coefficient of any calibration mode into the defocus conversion coefficient corresponding to another calibration mode.

7. A parameter calibration device for an image acquisition equipment, characterized in that, The device includes: The acquisition module is used to acquire the first gain matrix and the first defocus conversion coefficient of the image acquisition device in the first calibration mode and at the first object distance, and the second defocus conversion coefficient in the same calibration mode and at the second object distance; The first calibration module is configured to determine a gain matrix corresponding to the first target calibration mode and / or the first target object distance based on the first gain matrix and the trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; and / or The second calibration module is used to determine the defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient and the trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance. A storage module is used to write the obtained gain matrix and / or defocus conversion coefficient into the memory of the image acquisition device for optical shadow correction and phase detection autofocus during the image signal processing stage. The defocus mapping relationship is implemented based on a coefficient calibration model group, which includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and each has the following functions: The coefficient calibration model is used to map the defocus conversion coefficients corresponding to any two object distances to the defocus conversion coefficients corresponding to other two object distances under the same calibration mode; The coefficient conversion algorithm is used to map the known defocus conversion coefficients to the defocus conversion coefficients corresponding to the same object distance and other calibration modes based on the corresponding mode conversion matrix under the same object distance. The gain mapping relationship is implemented through a gain calibration model group, which includes at least one of a first gain calibration model, a second gain calibration model, and a third gain calibration model, and each model has the following functions: the first gain calibration model is used to perform consistency correction on the input gain matrix under the same calibration mode and the same object distance, and output the consistency-corrected gain matrix; the second gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other calibration mode under the same object distance; the third gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other object distance under the same calibration mode.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images