Parameter calibration method and device of image acquisition equipment and computer equipment

Abstract

The invention relates to a parameter calibration method and device of image acquisition equipment and computer equipment. The method comprises at least one of the following two steps: a) determining a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on a first gain matrix and a trained gain mapping relation; and b) determining an out-of-focus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance based on the first out-of-focus conversion coefficient, the second out-of-focus conversion coefficient and the trained out-of-focus mapping relation. And writing the obtained gain matrix and / or defocus conversion coefficient into a memory in the image acquisition equipment for optical shadow correction and phase detection automatic focusing in an image signal processing stage. Through the trained gain mapping relation and the trained defocus mapping relation, corresponding gain matrixes and defocus conversion coefficients under various calibration modes and various object distance conditions can be generated, and the calibration cost and the calibration time consumption are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera modules, and in particular to a parameter calibration method and device for an image acquisition apparatus and a computer device. BACKGROUND

[0002] A camera module is a miniature image acquisition device integrating optical imaging and electronic signal processing. Camera modules are widely used in the fields of smartphones, security monitoring, automotive electronics, drones, medical endoscopes, industrial detection, and AR / VR devices. Taking the application of camera modules in smartphones as an example, with the increasing product diversity and shorter update cycle of the smartphone camera module market, various products such as long-focus, inner-focus, large-aperture, and large-image face have been mass-produced. The assembly process requirements for smartphone camera modules are also becoming higher and higher. During the assembly process of camera modules, PDAF calibration, gain matrix calibration, and phase difference offset calibration are usually required. PDAF calibration is a key step to ensure the normal operation of phase detection autofocus function. PDAF calibration obtains the defocus conversion coefficient (DCC), and the DCC autofocus calibration directly affects the image quality of the camera module.

[0003] In the related art, PDAF calibration and gain matrix calibration are performed on the camera module, especially for PDAF calibration. The camera module is usually calibrated for gain map under a uniform light source, and then the phase difference PD image is obtained to complete the calibration of the defocus conversion coefficient DCC. However, the camera module usually needs to be calibrated for multiple calibration modes and multiple object distances. Calibrating for each calibration mode and each object distance separately will result in high calibration cost and long calibration time of the camera module. SUMMARY

[0004] Therefore, it is necessary to provide a parameter calibration method and device for an image acquisition apparatus and a computer device to solve the above technical problems.

[0005] In a first aspect, the present application provides a parameter calibration method for an image acquisition device, the method comprising: obtaining a first gain matrix and a first defocus conversion coefficient of the image acquisition device in a first calibration mode and at a first object distance, and a second defocus conversion coefficient of the image acquisition device in the same calibration mode and at a second object distance; performing at least one of the following two steps: a) determining a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; b) determining a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient and a trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance; and writing the obtained gain matrix and / or defocus conversion coefficient into a memory in the image acquisition device for optical shading correction and phase detection auto-focusing in an image signal processing stage.

[0006] In one of the embodiments, the method further comprises: obtaining a first phase difference offset of the image acquisition device in the first calibration mode and at the first object distance; and determining a phase difference offset corresponding to a target condition based on the first defocus conversion coefficient and the first phase difference offset, in combination with at least one paired defocus conversion coefficient across modes or object distances and an offset mapping relationship; wherein the target condition is different from a first condition constituted by the first calibration mode and the first object distance only in calibration mode or object distance, and the paired defocus conversion coefficient corresponds to the target condition.

[0007] In one of the embodiments, the gain mapping relationship is implemented by a gain calibration model group, the gain calibration model group comprising at least one of a first gain calibration model, a second gain calibration model and a third gain calibration model, and each model having the following functions: the first gain calibration model is used for consistent correction of an input gain matrix under the same calibration mode and the same object distance condition, and outputs a gain matrix after consistent correction; the second gain calibration model is used for mapping the input gain matrix to a gain matrix corresponding to at least one other calibration mode under the same object distance condition; and the third gain calibration model is used for mapping the input gain matrix to a gain matrix corresponding to at least one other object distance under the same calibration mode condition.

[0008] In one of the embodiments, the first gain calibration model, the second gain calibration model and the third gain calibration model are implemented by different output heads of the same neural network, or by sub-networks trained independently.

[0009] In one of the embodiments, the defocus mapping relationship is implemented based on a coefficient calibration model group, the coefficient calibration model group includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and respectively has the following functions; the coefficient calibration model is used to map the defocus conversion coefficients corresponding to any two object distances in the same calibration mode to the defocus conversion coefficients corresponding to other two object distances in the same calibration mode; and the coefficient conversion algorithm is used to map the known defocus conversion coefficients to the defocus conversion coefficients corresponding to the same object distance and other calibration modes based on the corresponding mode conversion matrix in the same object distance.

[0010] In one of the embodiments, the gain mapping relationship is determined by the following way: a sample gain matrix covering all calibration modes and all object distances is collected, and after data preprocessing, each model in the gain calibration model group is trained respectively; the training at least includes: using the same calibration mode and the same object distance samples to train the first gain calibration model; and / or using the same object distance and cross-calibration mode samples to train the second gain calibration model; and / or using the same calibration mode and cross-object distance samples to train the third gain calibration model; to determine the gain mapping relationship.

[0011] In one of the embodiments, the coefficient calibration model is determined by the following way: a plurality of groups of input-output sample pairs are collected, each group of sample pairs includes: the normalized sample defocus conversion coefficients corresponding to any two object distances in the same calibration mode as input samples; the sample defocus conversion coefficients corresponding to other two object distances in the same calibration mode as expected output samples; and the coefficient calibration model is trained based on the input samples and the expected output samples to realize the defocus conversion coefficient calculation between different object distances in the same calibration mode in the inference stage.

[0012] In one of the embodiments, the coefficient conversion algorithm is implemented by the mode conversion matrix, the mode conversion matrix is determined by fitting the sample defocus conversion coefficients corresponding to different calibration modes in the same object distance as training data, and is used to convert the defocus conversion coefficients of any calibration mode to the defocus conversion coefficients corresponding to another calibration mode in the inference stage.

[0013] In a second aspect, the present application also provides a parameter calibration device for an image acquisition device, the device comprising: an obtaining module configured to obtain a first gain matrix of the image acquisition device in a first calibration mode and at a first object distance, and a first defocus conversion coefficient, and a second defocus conversion coefficient in the same calibration mode and at a second object distance; a first calibration module configured to determine a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship, wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; and / or a second calibration module configured to determine a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient and a trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance; and a storage module configured to write the obtained gain matrix and / or defocus conversion coefficient into a memory in the image acquisition device for optical shading correction and phase detection auto-focusing in an image signal processing stage.

[0014] In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements any of the parameter calibration methods of the image acquisition device in the first aspect when executing the computer program.

[0015] In a fourth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement any of the parameter calibration methods of the image acquisition device in the first aspect.

[0016] The parameter calibration method and device of the image acquisition device and the computer device first acquire a first gain matrix and a first defocus conversion coefficient of the image acquisition device in a first calibration mode and at a first object distance, and a second defocus conversion coefficient in the same calibration mode and at a second object distance, and at least one of the following two steps is performed: a) determining a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance. b) determining a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient, and a trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from the first object distance and the second object distance. The obtained gain matrix and / or defocus conversion coefficient are written into a memory in the image acquisition device for optical shading correction and phase detection automatic focusing in the image signal processing stage. Under the condition that only the first gain matrix, the first defocus conversion coefficient, and the second defocus conversion coefficient are acquired, the corresponding gain matrix and defocus conversion coefficient under multiple calibration modes and multiple object distances can be generated through the trained gain mapping relationship and the trained defocus mapping relationship, thereby reducing the calibration cost and the calibration time. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of a parameter calibration method of an image acquisition device in an embodiment;

[0018] Figure 2 A flowchart of a training method of a calibration module in an embodiment;

[0019] Figure 3 A flowchart of a training method of a coefficient calibration model in an embodiment;

[0020] Figure 4 A flowchart of a gain matrix calibration method in an embodiment;

[0021] Figure 5 A flowchart of a defocus conversion coefficient calibration method in an embodiment;

[0022] Figure 6 A flowchart of a phase difference offset calibration method in an embodiment;

[0023] Figure 7 A schematic diagram of an FCN model in an embodiment;

[0024] Figure 8 A schematic diagram of a specific gain matrix calibration in an embodiment;

[0025] Figure 9 A diagram for calibration of a specific defocus conversion coefficient and phase difference offset in an embodiment;

[0026] Figure 10 A structure block diagram of a parameter calibration device of an image acquisition device in an embodiment;

[0027] Figure 11 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0028] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0029] A camera module is a kind of miniature image acquisition device integrating optical imaging and electronic signal processing. The camera module is widely used in the fields of smart phones, security monitoring, automotive electronics, unmanned aerial vehicles, medical endoscopes, industrial detection, and AR / VR devices. Taking the application of the camera module in the mobile phone as an example, with the increasing product diversity and shorter update cycle of the mobile phone camera module market, various products such as long focal length, internal focusing, large aperture, and large image surface have been mass-produced. The assembly process requirements of the mobile phone camera module are also getting higher and higher. During the assembly process of the camera module, it is usually necessary to calibrate the PDAF, gain matrix gain map, and phase difference offset offset of the camera module, wherein the PDAF calibration is a key step to ensure the normal working of the phase detection auto-focusing function. The PDAF calibration obtains the defocus conversion coefficient (DCC), and the defocus conversion coefficient DCC auto-focusing calibration directly affects the image quality of the camera module.

[0030] In the related art, for defocus conversion coefficient DCC, the DCC calibration algorithm commonly used by camera module manufacturers is the traditional PDAF algorithm provided by third-party platforms such as High Pass, MTK and HiSilicon. This algorithm usually needs to first calibrate the gain map Gain Map of the camera module under a uniform light source, and then output the phase difference PD map, thereby completing the calibration of the defocus conversion coefficient DCC. However, the camera module usually needs to calibrate the defocus conversion coefficient DCC for multiple calibration modes and multiple object distances respectively. Using the above-mentioned traditional PDAF algorithm for calibration of multiple calibration modes and multiple object distances respectively is costly and time-consuming. Specifically, the cost mainly includes three aspects. First, the phase difference PD map output and the new map opening are time-consuming. At least 11 images are needed when outputting the phase difference PD map, and different calibration modes and different object distances need to be output respectively. The map needs to be opened again when the calibration mode or the object distance is switched. Second, the calculation steps of the traditional PDAF algorithm are complicated, and multiple images need to be processed, which is time-consuming. Third, the same calibration system cannot simultaneously support large and small object distance calibration boards. Multiple calibration systems are needed for calibration of multiple object distances, which is costly. Similarly, for gain matrix gain map calibration, gain matrix calibration needs to be performed for multiple calibration modes and multiple object distances respectively. For phase difference offset calibration, phase difference offset calibration needs to be performed for multiple calibration modes and multiple object distances respectively. Calibration for each calibration mode and each object distance condition respectively will result in high cost and long time consumption of the camera module calibration.

[0031] The specific calibration process of the PDAF traditional algorithm under any calibration mode and any object distance condition is as follows. First, the phase difference PD image is obtained by the camera module, the phase difference PD image is separated to determine the left PD image and the right PD image. The gain correction is performed on the phase difference PD image by the gain matrix gain map, and the gain matrix gain map represents a mapping table for dynamically adjusting the light sensitivity of the phase detection pixel Shield Pixel. For example, the left PD image is gain corrected by the left gain matrix, and the right PD image is gain corrected by the right gain matrix. The left PD image and the right PD image after gain correction are subjected to image down-sampling, that is, vertical binning, to improve the calculation efficiency, and the phase difference PD is calculated. The defocus conversion coefficient DCC is fitted. The focal position is calculated, and the specific formula is as follows: TargetPosition = CurrentPosition + PD × DCC, wherein TargetPosition represents the target motor position, that is, the focusing position; CurrentPosition represents the current motor position; PD represents the phase difference, which is used to measure the sharpness of the image and has a direction; the defocus conversion coefficient DCC represents the ratio of defocus to PD, and the unit is DAC / pd. The camera module factory will calibrate the DCC for each camera module and write it into the module eeprom. It represents the proportional coefficient for converting pixel-level phase difference into motor driving value. Defocus represents the physical distance that the lens needs to move from the current position to the in-focus position. The vertical binning refers to an image processing technique, which is usually used to reduce the amount of image data and improve the subsequent processing efficiency.

[0032] It can be understood that, on the basis of the above formula, the PD needs to be compensated when calculating the target motor position, and the specific formula is as follows: TargetPosition = CurrentPosition + DCC × (PD + offset), wherein the phase difference offset represents the offset of the phase difference PD point, which is used to compensate the PD value to achieve the purpose of accurate focusing.

[0033] After the gain matrix gain map, the defocus conversion coefficient DCC and the phase difference offset offset are obtained through calibration, the camera module will acquire the PD image in the actual use of the auto-focusing process, and split to obtain the left PD image and the right PD image. The left PD image is corrected by the left gain matrix gain map, and the right PD image is corrected by the right gain matrix gain map. The PD value of the current image is calculated in real time through the left PD image and the right PD image after gain correction, and the PD value is multiplied by the DCC value to determine the focusing position, that is, the physical distance that the defocus lens needs to move from the current position to the focusing position. In the above process, the PD value can be compensated by the offset, thereby completing the auto-focusing. Taking the application of the camera module in the mobile phone as an example, the gain matrix gain map, the defocus conversion coefficient DCC and the phase difference offset offset of the camera module in the mobile phone are calibrated. When the mobile phone is normally used for shooting, the PD image is acquired in the process of auto-focusing, the PD image is corrected by the gain matrix gain map, then the PD value is calculated, the PD value is multiplied by the defocus conversion coefficient DCC to obtain the displacement amount defocus of the camera module. The PD value is compensated by the offset, thereby completing the PDAF focusing.

[0034] In one embodiment, as shown in Figure 1 A parameter calibration method of an image acquisition device is provided, comprising the following steps:

[0035] Step 101, acquiring a first gain matrix and a first defocus conversion coefficient of the image acquisition device in a first calibration mode and at a first object distance, and a second defocus conversion coefficient in the same calibration mode and at a second object distance.

[0036] The image acquisition device can be a camera module, or a mobile phone, a tablet and the like including the camera module. The image acquisition device is an image acquisition device that needs to be calibrated for the gain matrix gain map, the defocus conversion coefficient DCC and the phase difference offset offset. If the image acquisition device is a camera module, the camera module can be composed of lenses, image sensors, motors and the like. The camera module realizes image acquisition function through optical imaging and electronic signal processing. The camera module can be a camera module assembled in a mobile phone, a camera module assembled in a security monitoring device, or a camera module assembled in a vehicle, and the application scenarios of the camera module are not limited in the embodiment.

[0037] When the method provided by the embodiment is used for calibration, first, a first gain matrix and a first defocus conversion coefficient in a first calibration mode and at a first object distance are calibrated by using a traditional calibration technology, and a second defocus conversion coefficient in the same calibration mode and at a second object distance is also calibrated.

[0038] The calibration mode represents different calibration technologies when the image acquisition device is calibrated. For example, the plurality of calibration modes includes a QPD calibration mode and an SPD calibration mode. The embodiment does not make a specific limitation on the plurality of calibration modes, and only needs to be able to calibrate the parameters of the image acquisition device. The QPD calibration mode is a four-phase detection autofocus technology, which realizes 2x2 phase detection autofocus PDAF on the entire sensor image array, has a coverage rate of 100%, high sensitivity, small defocus error of QPD, and small required DCC value. The SPD calibration mode is a split-pixel detection technology, which divides the imaging pixels into two (left / right or up / down), each sub-pixel independently photosensitive, only about 50% of the light is used for phase detection, low sensitivity, large defocus error of SPD, and large required DCC value. The accuracy of DCC is inversely proportional to the signal-to-noise ratio of phase detection. The signal-to-noise ratio of DCC calibrated by the QPD calibration mode is high, and the DCC value is small. The signal-to-noise ratio of DCC calibrated by the SPD calibration mode is low, and the DCC value is large. The calibration mode can also be other phase detection autofocus technologies.

[0039] The object distance represents the distance from the object to the optical center (or main plane) of the lens. For example, the plurality of object distances can be 5 cm, 6 cm, 10 cm, and 80 cm. The embodiment does not make a specific limitation on the plurality of object distances, and only needs to be calibrated. The working distance of the camera card is regarded as the object distance, and the value is equal to the distance from the surface of the camera card to the entrance pupil of the lens.

[0040] First, the calibration environment is set to the first calibration mode and the first object distance, and the image acquisition device is calibrated by using the traditional calibration method under this condition to obtain the first gain matrix and the first defocus conversion coefficient. Then, the calibration environment is adjusted to the first calibration mode and the second object distance, and the image acquisition device is calibrated by using the traditional calibration method under this condition to obtain the second defocus conversion coefficient.

[0041] After the first gain matrix, the first defocus conversion coefficient, and the second defocus conversion coefficient are obtained, at least one of the following steps 102 and 103 is executed, that is, only step 102 can be executed, only step 103 can be executed, or steps 102 and 103 can be executed simultaneously.

[0042] In step 102, a gain matrix corresponding to a first target calibration mode and / or a first target object distance is determined based on the first gain matrix and a trained gain mapping relationship.

[0043] The first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance. That is, the first target calibration mode can be any calibration mode other than the first calibration mode, and the first target object distance can be any object distance other than the first object distance. The gain matrix corresponding to the first target calibration mode and / or the first target object distance is determined, that is, the gain matrix corresponding to any calibration mode other than the first calibration mode and any object distance, or the gain matrix corresponding to any object distance other than the first object distance and any calibration mode can be determined.

[0044] The trained gain mapping relationship can be implemented by a trained neural network model and / or a function, describes the mapping relationship of the gain matrix changing with the calibration mode and the object distance, and is used to derive the gain matrix under unmeasured conditions according to the existing gain matrix. Specifically, by pre-acquiring sample left gain matrices and sample right gain matrices under all calibration modes and all object distance conditions as a training data set, the neural network model is trained using the training data set, and thus the trained gain mapping relationship is obtained.

[0045] In this application, after the first gain matrix is obtained, the gain matrix corresponding to multiple calibration modes and multiple object distance conditions can be determined based on the first gain matrix and the trained gain mapping relationship. The first gain matrix can be a left gain matrix of the image acquisition device under the first calibration mode and the first object distance condition, or a right gain matrix of the image acquisition device under the first calibration mode and the first object distance condition. That is, when the gain matrix of the image acquisition device is calibrated using the traditional calibration technology, only the left gain matrix or the right gain matrix needs to be calibrated. The left gain matrix represents a matrix for gain correction of a left PD image, and the right gain matrix represents a matrix for gain correction of a right PD image.

[0046] In step 103, a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance is determined based on the first defocus conversion coefficient, the second defocus conversion coefficient, and a trained defocus mapping relationship.

[0047] The second target calibration mode is different from the first calibration mode, or the second target distance is different from the first distance and the second distance. That is, the second target calibration mode can be any calibration mode other than the first calibration mode. The second target distance can be any distance other than the first distance and the second distance. The defocus conversion coefficients corresponding to the second target calibration mode and / or the second target distance are determined, that is, the defocus conversion coefficients corresponding to any calibration mode other than the first calibration mode and any distance, or the defocus conversion coefficients corresponding to any distance other than the first distance and the second distance and any calibration mode can be determined.

[0048] The trained defocus mapping relationship can be implemented by a trained neural network model and / or a function, which is used to describe the mapping relationship of the defocus conversion coefficients changing with the calibration mode and / or the distance, and can be used to derive the defocus conversion coefficient DCC value under other conditions according to one or two known defocus conversion coefficient DCC values. Specifically, by using all the sample defocus conversion coefficients under all the calibration modes and all the distance conditions as a training data set, the neural network model is trained using the training data set, thereby obtaining the trained defocus mapping relationship. Or based on the sample defocus conversion coefficients under all the calibration modes and all the distance conditions, a function relationship between the sample defocus conversion coefficients under different calibration modes and different distance conditions is determined, thereby obtaining the trained defocus mapping relationship.

[0049] In the present application, after obtaining the first defocus conversion coefficient and the second defocus conversion coefficient, the corresponding defocus conversion coefficients under multiple calibration modes and multiple distance conditions can be determined based on the first defocus conversion coefficient, the second defocus conversion coefficient and the trained defocus mapping relationship.

[0050] In step 104, the obtained gain matrix and / or defocus conversion coefficient is written into the memory in the image acquisition device for optical shading correction and phase detection autofocus in the image signal processing stage.

[0051] After obtaining the gain matrix under all the calibration modes and all the distance conditions, the gain matrix under all the calibration modes and all the distance conditions is written into the memory in the image acquisition device. After obtaining the defocus conversion coefficient under all the calibration modes and all the distance conditions, the defocus conversion coefficient under all the calibration modes and all the distance conditions is written into the memory in the image acquisition device. In actual use of the image acquisition device, according to the currently used conditions, the gain matrix and the defocus conversion coefficient under the corresponding calibration mode and the corresponding distance condition are selected for optical shading correction and phase detection autofocus. Part of the parameters can be directly obtained by using the traditional method, and part of the parameters can be obtained by using the mapping relationship in the above steps, which is not limited in the present application.

[0052] The embodiment first obtains a first gain matrix and a first defocus conversion coefficient of an image acquisition device in a first calibration mode and at a first object distance, and a second defocus conversion coefficient in the same calibration mode and at a second object distance, and at least one of the following two steps is performed: a) determining a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance. b) determining a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient and a trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from the first object distance and the second object distance. The obtained gain matrix and / or defocus conversion coefficient is written into a memory in the image acquisition device for optical shading correction and phase detection autofocus in the image signal processing stage. Under the condition of only obtaining the first gain matrix, the first defocus conversion coefficient and the second defocus conversion coefficient, the corresponding gain matrix and defocus conversion coefficient under multiple calibration modes and multiple object distances can be generated through the trained gain mapping relationship and the trained defocus mapping relationship, thereby reducing the calibration cost and time consumption.

[0053] In one of the embodiments, the method further comprises the following steps:

[0054] Step 1a, obtaining a first phase difference offset of the image acquisition device in the first calibration mode and at the first object distance.

[0055] Through the conventional calibration technology, in addition to obtaining other parameters, the first phase difference offset can also be obtained when calibrating in the first calibration mode and at the first object distance. That is, the first phase difference offset of the device in the first calibration mode and at the first object distance is obtained.

[0056] Step 1b, determining a phase difference offset corresponding to a target condition according to the first defocus conversion coefficient and the first phase difference offset, combining at least one paired defocus conversion coefficient across modes or object distances, and an offset mapping relationship.

[0057] The target condition is different from the first condition constituted by the first calibration mode and the first object distance only in calibration mode or object distance. The first condition is the condition corresponding to the first defocus conversion coefficient and the first phase difference offset, wherein the paired defocus conversion coefficient also corresponds to the first condition, that is, the phase difference offset corresponding to the paired defocus conversion coefficient and the target condition is also corresponding.

[0058] The pair of cross-mode or cross-DOF parallax conversion coefficients corresponding to the target condition, i.e., the parallax conversion coefficients corresponding to the target condition. The phase difference offset corresponding to the target condition can be obtained by the pair of corresponding parallax conversion coefficients and phase difference offsets obtained, and the parallax conversion coefficient corresponding to the target condition. By changing the pair of cross-mode or cross-DOF parallax conversion coefficients of different conditions, and combining the offset mapping relationship, the phase difference offset under the required calibration mode and the required object distance condition can be obtained.

[0059] The offset mapping relationship is a function of calculating the phase difference offset under different calibration modes and different object distance conditions, and is used to generate the phase difference offset under the unmeasured condition in combination with the known phase difference offset offset and other parameters. The offset algorithm can be obtained based on physical optical characteristics analysis or historical data statistics induction. For example, according to the first parallax conversion coefficient and the first phase difference offset, and in combination with the second calibration mode and the parallax conversion coefficient under the first object distance condition, the phase difference offset under the second calibration mode and the first object distance condition can be obtained by the offset mapping relationship. For example, according to the first parallax conversion model and the first phase difference offset, and in combination with the first calibration mode and the parallax conversion coefficient under the third object distance condition, the phase difference offset under the first calibration mode and the third object distance condition can be obtained by the offset mapping relationship.

[0060] In the embodiment, only the first gain matrix, the first parallax conversion coefficient, the first phase difference offset, the second parallax conversion coefficient, and the second phase difference offset are obtained, and the corresponding gain matrix, parallax conversion coefficient, and phase difference offset under multiple calibration modes and multiple object distance conditions can be generated by the trained gain mapping relationship, the trained parallax mapping relationship, and the offset mapping relationship, thereby reducing the calibration cost and the calibration time. In addition, by replacing the traditional calibration mode with the model group and the algorithm, the number of image acquisition and the consumption of computing resources can be reduced, and the delay caused by frequent re-opening of the image and multi-image processing can be avoided.

[0061] In one of the embodiments, the above-mentioned gain mapping relationship can be realized by a gain calibration model group, which can include at least one of a first gain calibration model, a second gain calibration model, and a third gain calibration model, and each model has the following functions:

[0062] The first gain calibration model is configured to correct the input gain matrix in consistency under the same calibration mode and the same object distance condition, and output the gain matrix after the consistency correction. The consistency correction refers to calculating the right gain matrix by the left gain matrix, or calculating the left gain matrix by the right gain matrix, so that the gain values of the same pixel position in the left and right channels remain predetermined consistency. For example, the first gain calibration model is configured to predict the right gain matrix by the left gain matrix, or predict the left gain matrix by the right gain matrix, under the same calibration mode and the same object distance condition. For example, the right gain matrix under the first calibration mode and the first object distance condition is predicted by the left gain matrix under the first calibration mode and the first object distance condition.

[0063] The second gain calibration model is configured to map the input gain matrix to the gain matrix corresponding to at least one other calibration mode under the same object distance condition. For example, the second gain calibration model is configured to predict the left gain matrix (or the right gain matrix) of the remaining calibration mode by the left gain matrix (or the right gain matrix) of any calibration mode under the condition of different calibration modes and the same object distance. For example, the left gain matrix (or the right gain matrix) of the remaining calibration mode and the first object distance condition is predicted by the left gain matrix (or the right gain matrix) under the first calibration mode and the first object distance condition.

[0064] The third gain calibration model is configured to map the input gain matrix to the gain matrix corresponding to at least one other object distance under the same calibration mode condition. For example, the third gain calibration model is configured to predict the left gain matrix of the remaining object distance by the left gain matrix of any object distance under the condition of the same calibration mode and different object distances, or predict the right gain matrix of the remaining object distance by the right gain matrix of any object distance. For example, the left gain matrix of the remaining object distance and the first calibration mode is predicted by the left gain matrix under the first calibration mode and the first object distance condition.

[0065] In this application, through the trained gain mapping relationship, other gain matrices that have not been actually measured can be obtained based on the obtained part of the gain matrices. Compared with the traditional calibration method, this method greatly reduces the number of output graphs and the calibration time, and improves the calibration efficiency.

[0066] In one of the embodiments, before calibrating the camera module, the gain calibration module needs to be trained in advance to establish the gain mapping relationship. In this application, the sample gain matrix covering all calibration modes and all object distances can be collected, and after data preprocessing, each model in the gain calibration model group is trained respectively; the training at least includes: using the same calibration mode and object distance sample to train the first gain calibration model; and / or using the same object distance sample across the calibration mode to train the second gain calibration model; and / or using the same calibration mode sample across the object distance to train the third gain calibration model; to determine the gain mapping relationship.

[0067] Specifically, as shown in Figure 2 The training method of the gain calibration module can include the following steps:

[0068] Step 201, obtaining a first initial data set.

[0069] The sample gain matrix covering all calibration modes and all object distances is obtained, and the first initial data set includes sample left gain matrix and sample right gain matrix under all calibration modes and all object distances. The first initial data set can be historical calibration data on the production line. This application is exemplified in the following case, and the multiple calibration modes include QPD calibration mode and SPD calibration mode. The multiple object distances are: 5cm, 6cm, 10cm and 80cm. Among them, the sample left gain matrix and the sample right gain matrix can be a 12 × 16 matrix.

[0070] Then a group of data in the first initial data set includes: sample left gain matrix and sample right gain matrix under QPD calibration mode and 5cm object distance condition; sample left gain matrix and sample right gain matrix under SPD calibration mode and 5cm object distance condition; sample left gain matrix and sample right gain matrix under QPD calibration mode and 6cm object distance condition; sample left gain matrix and sample right gain matrix under SPD calibration mode and 6cm object distance condition; sample left gain matrix and sample right gain matrix under QPD calibration mode and 10cm object distance condition; sample left gain matrix and sample right gain matrix under SPD calibration mode and 10cm object distance condition; sample left gain matrix and sample right gain matrix under QPD calibration mode and 80cm object distance condition; sample left gain matrix and sample right gain matrix under SPD calibration mode and 80cm object distance condition. It can be understood that the first initial data set includes multiple groups of the above data.

[0071] Step 202, data preprocessing is performed on all sample left gain matrix and sample right gain matrix in the first initial data set to obtain a preprocessed data set.

[0072] After obtaining the first initial data set, in order to reduce the error caused by the fluctuation of the data itself, make the prediction result of the gain matrix fluctuate within a certain range, and inhibit the appearance of abnormal values, therefore, it is necessary to perform data preprocessing on the sample left gain matrix and the sample right gain matrix. The data preprocessing can include standardization, denoising or normalization processing, etc. For example, a reference quantity is set for the sample left gain matrix and the sample right gain matrix of different calibration modes respectively, and the sample left gain matrix and the sample right gain matrix corresponding to the calibration mode are preprocessed by the reference quantity.

[0073] For the sample left gain matrix and the sample right gain matrix under the condition of the QPD calibration mode, the numerical value of the sample left gain matrix and the sample right gain matrix under the condition of the QPD calibration mode is about 2. Therefore, the first reference value corresponding to the QPD calibration mode can be set as 2. When the sample left gain matrix and the sample right gain matrix under the condition of the QPD calibration mode are preprocessed, each value in the sample left gain matrix is subtracted by the first reference value to obtain the preprocessed sample left gain matrix; each value in the sample right gain matrix is subtracted by the first reference value to obtain the preprocessed sample right gain matrix.

[0074] For the sample left gain matrix and the sample right gain matrix under the condition of the SPD calibration mode, the numerical value of the sample left gain matrix and the sample right gain matrix under the condition of the SPD calibration mode is about 1. Therefore, the second reference value corresponding to the SPD calibration mode can be set as 1. When the sample left gain matrix and the sample right gain matrix under the condition of the SPD calibration mode are preprocessed, each value in the sample left gain matrix is subtracted by the second reference value to obtain the preprocessed sample left gain matrix; each value in the sample right gain matrix is subtracted by the second reference value to obtain the preprocessed sample right gain matrix.

[0075] After data preprocessing, the preprocessed data set is obtained based on all the preprocessed sample left gain matrices and all the preprocessed sample right gain matrices.

[0076] In step 203, the first training data set, the second training data set and the third training data set are constructed according to the preprocessed data set.

[0077] Since the gain calibration model group includes: the first gain calibration model corresponding to the left and right directions, the second gain calibration model corresponding to different calibration modes, and the third gain calibration model corresponding to different object distances. Therefore, it is necessary to construct the first training data set for the first gain calibration model corresponding to the left and right directions, the second training data set for the second gain calibration model corresponding to different calibration modes, and the third training data set for the third gain calibration model corresponding to different object distances according to the preprocessed data set.

[0078] Taking a set of data from the preprocessed dataset, including: the sample left and right gain matrices under QPD calibration mode and a 5cm object distance condition; the sample left and right gain matrices under SPD calibration mode and a 5cm object distance condition; the sample left and right gain matrices under QPD calibration mode and a 6cm object distance condition; the sample left and right gain matrices under SPD calibration mode and a 6cm object distance condition; the sample left and right gain matrices under QPD calibration mode and a 10cm object distance condition; the sample left and right gain matrices under SPD calibration mode and a 10cm object distance condition; the sample left and right gain matrices under QPD calibration mode and a 80cm object distance condition; and the sample left and right gain matrices under SPD calibration mode and a 80cm object distance condition, as an example, the following explanation will be provided:

[0079] The first training dataset includes multiple sets of sample left and right gain matrices under the same calibration mode and object distance conditions. Specifically, the first training dataset is a training set composed of sample left and right gain matrices with the same calibration mode and object distance conditions selected from the preprocessed dataset. The first training dataset is used to model a stable correspondence between the left and right gain matrices and train a first gain calibration model describing the response relationship between the left and right pixel channels. For example, based on a set of data from the aforementioned preprocessed dataset, for instance, the sample left and right gain matrices under the QPD calibration mode and 10cm object distance condition are used as a set of training data in the first training dataset. The sample left and right gain matrices under the SPD calibration mode and 10cm object distance condition are also used as a set of training data in the first training dataset.

[0080] The second training dataset includes multiple sets of left gain matrices and right gain matrices for samples under different calibration modes and the same object distance condition. Specifically, the second training dataset is a training set composed of multiple sets of left gain matrices and right gain matrices for samples under different calibration modes and the same object distance condition, selected from the preprocessed dataset. The second training dataset is used to reflect the influence of calibration mode changes on the gain matrix. For example, based on a set of data from the aforementioned preprocessed dataset, such as the left gain matrices of samples under QPD calibration mode and 10cm object distance condition, and the left gain matrices of samples under SPD calibration mode and 10cm object distance condition, are used as a set of training data in the second training dataset. Similarly, the right gain matrices of samples under QPD calibration mode and 10cm object distance condition, and the right gain matrices of samples under SPD calibration mode and 10cm object distance condition are used as a set of training data in the second training dataset.

[0081] The third training dataset includes multiple sets of left gain matrices and right gain matrices for samples under the same calibration mode and different object distances. Specifically, the third training dataset is a training set composed of multiple sets of left gain matrices and right gain matrices for samples under the same calibration mode and different object distances, selected from the preprocessed dataset. The third training dataset is used to reflect the influence of object distance changes on the gain matrix. For example, based on a set of data from the aforementioned preprocessed dataset, such as the left gain matrices of samples under the QPD calibration mode and a 10cm object distance, the left gain matrices of samples under the QPD calibration mode and a 5cm object distance, the left gain matrices of samples under the QPD calibration mode and a 6cm object distance, and the left gain matrices of samples under the QPD calibration mode and a 80cm object distance, the third training dataset is used as a set of training data. The right gain matrix of the sample under the SPD calibration mode and the 10cm object distance condition, the right gain matrix of the sample under the SPD calibration mode and the 5cm object distance condition, the right gain matrix of the sample under the SPD calibration mode and the 6cm object distance condition, and the right gain matrix of the sample under the SPD calibration mode and the 80cm object distance condition are used as a set of training data in the third training dataset.

[0082] Step 204: Train the first initial model based on the first training dataset to obtain the first gain calibration model corresponding to the left and right directions.

[0083] The first gain calibration model is trained using samples with the same calibration mode and object distance. Specifically, the first initial model is the initial machine learning model used to train the first gain calibration model. The first initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, in order to balance efficiency and memory constraints in practical application scenarios, the first initial model is an FCN model.

[0084] When training the first initial model using the first training dataset, an example is taken: a set of training data including the QPD calibration mode and the sample left and right gain matrices under a 10cm object distance condition. The sample left gain matrix under the QPD calibration mode and 10cm object distance condition is used as the model input, and the sample right gain matrix under the QPD calibration mode and 10cm object distance condition is used as the model output true value; or the sample right gain matrix under the QPD calibration mode and 10cm object distance condition is used as the model input, and the sample left gain matrix under the QPD calibration mode and 10cm object distance condition is used as the model output true value, to train the first initial model. If the requirement is to derive the right gain matrix based on the left gain matrix, then the sample left gain matrix is ​​used as the input when training the model; if the requirement is to derive the left gain matrix based on the right gain matrix, then the sample right gain matrix is ​​used as the input when training the model. That is, the trained model corresponds one-to-one with the inference requirement.

[0085] The loss function of the first initial model is as follows:

[0086] ;

[0087] in, This represents the model's predicted value; y i represents the true value; n represents the number of samples.

[0088] The first training dataset is input into the first initial model for training. The model parameters are optimized through the loss function until the convergence condition is met, and the first gain calibration model is obtained.

[0089] Step 205: Train the second initial model based on the second training dataset to obtain the second gain calibration model corresponding to different calibration modes.

[0090] The second gain calibration model is trained using samples from the same object distance across calibration modes. Specifically, the second initial model is the initial machine learning model used to train the second gain calibration model. The second initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, to balance efficiency and memory constraints in practical applications, the second initial model is an FCN model.

[0091] When training the second initial model using the second training dataset, an example is taken: a set of training data including the left gain matrix of samples under the QPD calibration pattern and the 10cm object distance condition, and the left gain matrix of samples under the SPD calibration pattern and the 10cm object distance condition. The QPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model input, and the SPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model output true values; or, the SPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model input, and the QPD calibration pattern and the left gain matrix of samples under the 10cm object distance condition are used as the model output true values, to train the second initial model. If the requirement is to derive the SPD calibration pattern based on the QPD calibration pattern, then the QPD calibration pattern is used as the input when training the model; if the requirement is to derive the QPD calibration pattern based on the SPD calibration pattern, then the SPD calibration pattern is used as the input when training the model. That is, the trained model corresponds one-to-one with the inference requirement.

[0092] The loss function for the second initial model is as follows:

[0093] ;

[0094] in, This represents the model's predicted value; y i represents the true value; n represents the number of samples.

[0095] The second training dataset is input into the second initial model for training. The model parameters are optimized through the loss function until the convergence condition is met, thus obtaining the second gain calibration model.

[0096] Step 206: Train the third initial model based on the third training dataset to obtain the third gain calibration model corresponding to different object distances.

[0097] The third gain calibration model is trained using samples across object distances with the same calibration pattern. Specifically, the third initial model is the initial machine learning model used to train the third gain calibration model. The third initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, to balance efficiency and memory constraints in practical applications, the third initial model is an FCN model. Since the third gain calibration model predicts the gain matrices of multiple other object distances by inputting the gain matrix of any one object distance, the FCN model needs to be adjusted to meet the requirements of practical applications. In this embodiment, the FCN model uses a fully connected layer added to both the input and output, and a concat structure is used to enhance information extraction capabilities in order to improve the FCN model's information acquisition ability.

[0098] When training the third initial model based on the third training dataset, an example is taken: the left gain matrix of samples under the QPD calibration mode and a 10cm object distance condition, the left gain matrix of samples under the QPD calibration mode and a 5cm object distance condition, the left gain matrix of samples under the QPD calibration mode and a 6cm object distance condition, and the left gain matrix of samples under the QPD calibration mode and a 80cm object distance condition. The left gain matrix of samples under the QPD calibration mode and a 10cm object distance condition is used as the model input, and the left gain matrices of samples under the QPD calibration mode and a 5cm object distance condition, the QPD calibration mode and a 6cm object distance condition, and the QPD calibration mode and a 80cm object distance condition are used as the model's true values ​​to train the third initial model.

[0099] The loss function for the third initial model is as follows:

[0100] ;

[0101] in, This represents the model's predicted value; y i represents the true value; n represents the number of samples.

[0102] The third training dataset is input into the third initial model for training. The model parameters are optimized through the loss function until the convergence condition is met, thus obtaining the third gain calibration model.

[0103] The first, second, and third gain calibration models can be implemented by different output heads of the same neural network, or by independently trained sub-networks. Specifically, the first, second, and third gain calibration models can use the same neural network, with different output heads in this network outputting corresponding calibration parameters. Each of the first, second, and third gain calibration models can correspond to a sub-network, and each sub-network consists of a neural network.

[0104] This embodiment constructs a first initial dataset by collecting sample gain matrices covering all calibration modes and object distance conditions, and preprocesses them to form a high-quality preprocessed dataset. Based on this, a first training dataset, a second training dataset, and a third training dataset are decoupled, corresponding to three modeling dimensions: left-right channel relationship, cross-calibration mode change, and cross-object distance change, respectively. Then, a first gain calibration model, a second gain calibration model, and a third gain calibration model are trained to form a multi-dimensional collaborative gain matrix prediction system. In actual calibration, it is not necessary to repeatedly perform image acquisition and gain calculation for each calibration mode and object distance combination. Only a small amount of measured data is needed to derive the full-condition gain matrix through the gain calibration model group. This significantly reduces the number of PD image acquisitions, the frequency of environment switching, and the computational load, effectively reducing the time cost and equipment occupation cost of the calibration process, while improving parameter consistency and reproducibility.

[0105] In one embodiment, the defocus mapping relationship is implemented based on a coefficient calibration model group, which includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and each has the following functions;

[0106] The coefficient calibration model is used to predict the defocus conversion coefficients for other object distances under the same calibration mode and different object distances, using the defocus conversion coefficients for any two object distances. In other words, this model is used to map the defocus conversion coefficients corresponding to any two object distances to the defocus conversion coefficients corresponding to other two object distances under the same calibration mode. For example, the coefficient calibration model is used to predict the defocus conversion coefficients for other object distances under the same calibration mode and different object distances, using the defocus conversion coefficients for any two object distances. For example, using the first defocus conversion coefficient under the first calibration mode and the first object distance condition, and the second defocus conversion coefficient under the first calibration mode and the second object distance condition, the defocus conversion coefficients under the first calibration mode and the other object distance conditions are predicted.

[0107] The coefficient conversion algorithm is used to predict the defocus conversion coefficients of other calibration modes under different calibration modes and the same object distance, using the defocus conversion coefficient of any calibration mode. In other words, this algorithm maps known defocus conversion coefficients to defocus conversion coefficients corresponding to other calibration modes at the same object distance, based on the corresponding mode conversion matrix. For example, the coefficient conversion algorithm is used to predict the defocus conversion coefficients of other calibration modes under different calibration modes and the same object distance, using the defocus conversion coefficient of any calibration mode and a known mode conversion matrix. For instance, using the first defocus conversion coefficient under the first calibration mode and the first object distance, combined with the mode conversion matrix under the first object distance, the algorithm predicts the defocus conversion coefficients of other calibration modes and the first object distance.

[0108] In one embodiment, a set of coefficient calibration models needs to be trained before calibrating the camera module. For example... Figure 3 As shown, the training method for the coefficient calibration model may include the following steps:

[0109] Step 301: Collect multiple sets of input-output sample pairs. Each sample pair includes: the defocus conversion coefficients of samples corresponding to any two object distances under the same calibration mode after standardization as input samples; and the defocus conversion coefficients of samples corresponding to other two object distances under the same calibration mode as expected output samples.

[0110] Specifically, a second initial dataset can be obtained first. This second initial dataset includes the defocus conversion coefficients of samples under all calibration modes and object distance conditions. The second initial dataset can be historical calibration data from the production line. Taking the calibration modes including QPD and SPD calibration modes, and object distances including 5cm, 6cm, 10cm, and 80cm as an example, the data in the second initial dataset includes: defocus conversion coefficients of samples under QPD calibration mode and 5cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 5cm object distance; defocus conversion coefficients of samples under QPD calibration mode and 6cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 6cm object distance; defocus conversion coefficients of samples under QPD calibration mode and 10cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 10cm object distance; defocus conversion coefficients of samples under QPD calibration mode and 80cm object distance; defocus conversion coefficients of samples under SPD calibration mode and 80cm object distance. Understandably, the second initial dataset includes multiple sets of the aforementioned data.

[0111] For example, the second initial dataset is configured in groups, each group containing: initial input samples and expected output samples; the initial input samples include: QPD calibration mode and sample defocus conversion coefficients under 10cm object distance conditions and QPD calibration mode and sample defocus conversion coefficients under 80cm object distance conditions; the expected output samples include: QPD calibration mode and sample defocus conversion coefficients under 5cm object distance conditions and QPD calibration mode and sample defocus conversion coefficients under 6cm object distance conditions.

[0112] For example, the second initial dataset is configured in groups, each group containing: initial input samples and expected output samples; the initial input samples include: the defocus conversion coefficient of the sample under SPD calibration mode and 10cm object distance condition, and the defocus conversion coefficient of the sample under SPD calibration mode and 80cm object distance condition; the expected output samples include: the defocus conversion coefficient of the sample under SPD calibration mode and 5cm object distance condition, and the defocus conversion coefficient of the sample under SPD calibration mode and 6cm object distance condition.

[0113] Based on the above, according to actual usage requirements, for the same calibration mode, the defocus conversion coefficients corresponding to any two object distances can be used as initial input samples, and the defocus conversion coefficients corresponding to other two object distances under the same calibration mode can be used as expected output samples.

[0114] Furthermore, the defocus conversion coefficients of the initial input samples in the second initial dataset can be standardized to obtain standard conversion coefficients. These standard conversion coefficients are the standardized sample defocus conversion coefficients and can be used as input samples. During the standardization process, the mean and standard deviation of the sample defocus conversion coefficients used as initial input samples are calculated respectively. Based on the mean and standard deviation, the sample defocus conversion coefficients for the corresponding calibration mode and object distance are standardized to obtain the standard conversion coefficients for the corresponding calibration mode and object distance.

[0115] Taking the standardization of the defocus conversion coefficient of the sample under QPD calibration mode and 10cm object distance as an example, the specific formula for standardization is as follows:

[0116] DCC after = (DCC before – DCC mean ) / DCC std ;

[0117] Among them, DCC after Standard conversion factor; DCC before DCC is the sample defocus conversion factor. mean The average values ​​are for QPD calibration mode and under a 10cm object distance condition; DCC std This represents the standard deviation under QPD calibration mode and a 10cm object distance condition. For each value in the matrix corresponding to the sample defocus conversion coefficient, the standard conversion coefficient is calculated using the above formula. Based on the above formula, the sample defocus conversion coefficients of all initial input samples are standardized to obtain the standard conversion coefficients for the input samples.

[0118] Step 302: The coefficient calibration model is trained based on the input samples and the expected output samples to calculate the defocus conversion coefficient between different object distances under the same calibration mode in the inference stage.

[0119] Specifically, the fourth initial model is trained based on the input samples and the expected output samples to obtain the coefficient calibration model. The fourth initial model is the initial machine learning model used to train the coefficient calibration model. The fourth initial model can include one or more of the following: linear regression model, multilayer perceptron model, convolutional neural network model, etc. Specifically, defocus conversion coefficient calibration can actually be viewed as time-series extrapolation; therefore, the fourth initial model is a Transformer model. However, the multi-head attention mechanism used by the Transformer model is time-consuming and memory-intensive, and its core self-attention value calculation is cumbersome, easily leading to overfitting on small datasets. Therefore, this embodiment optimizes the Transformer model as follows: First, the number of attention heads is reduced to 2 attention heads. Second, a channel attention mechanism is added to the existing Transformer model to improve its ability to extract channel information. Third, the sigmoid function of the channel attention mechanism is replaced with upsampling to make the model more suitable for the application scenario of this embodiment.

[0120] When training the fourth initial model based on input samples and expected output samples, the following example illustrates the process: Input samples include the QPD calibration mode and standard conversion coefficients under a 10cm object distance condition, as well as the QPD calibration mode and standard conversion coefficients under an 80cm object distance condition; expected output samples include the QPD calibration mode and sample defocus conversion coefficients under a 5cm object distance condition, as well as the QPD calibration mode and sample defocus conversion coefficients under a 6cm object distance condition. The fourth initial model is trained using these as inputs, and the QPD calibration mode and sample defocus conversion coefficients under a 5cm object distance condition and the QPD calibration mode and sample defocus conversion coefficients under a 6cm object distance condition as expected outputs. Corresponding to the standardization process for the initial input samples, before calculating the loss function using the model's initial predicted values ​​and expected output samples, the initial predicted values ​​are destandardized to obtain the model's predicted values.

[0121] The loss function for the fourth initial model is as follows:

[0122] ;

[0123] ;

[0124] ;

[0125] loss = λ1 × loss1 +λ2 × loss2 +λ3 × loss3;

[0126] in, This represents the model's predicted value; y i y represents the true value; abs represents the absolute value operation; n represents the number of samples. λ1, λ2, and λ3 represent the coefficients corresponding to each loss function, with preferred values ​​of λ1=0.3, λ2=0.5, and λ3=0.2. 19 y 20 y 27 y 28 These represent the true values ​​at indices 19, 20, 27, and 28 in the one-dimensional vector of DCC, respectively, under the measured object distance; corresponding to... , , , The target distance calculated by the model is the predicted value at the same index. The DCC one-dimensional vector is obtained by sampling along the straight line (horizontal centerline or radial line) passing through the optical center of the DCC matrix and arranging them in distance order. Index 20 is aligned with the center of the field of view, so the four points mentioned here are all located in the central region.

[0127] This embodiment uses model training to determine the nonlinear mapping relationship of the defocus conversion coefficient under the same calibration mode but different object distances. In subsequent calibration, only a small number of measured defocus conversion coefficient DCC values ​​are needed to derive the defocus conversion coefficient under other unmeasured conditions, which significantly reduces the number of image acquisitions and computational load, and reduces calibration cost and cycle while ensuring accuracy.

[0128] In one embodiment, the coefficient conversion algorithm is implemented through the mode conversion matrix. The mode conversion matrix is ​​determined by fitting sample defocus conversion coefficients corresponding to different calibration modes as training data at the same object distance. This matrix is ​​then used in the inference stage to convert the defocus conversion coefficients of any calibration mode into the defocus conversion coefficients corresponding to another calibration mode. Therefore, before calibrating the camera module, it is necessary to obtain mode conversion matrices corresponding to different object distances. These mode conversion matrices are used to convert the defocus conversion coefficients of one calibration mode into the defocus conversion coefficients of another calibration mode under the same object distance condition.

[0129] For example, let's take the example of obtaining the mode conversion matrix between the first and second calibration modes under a first object distance condition: Taking a first object distance of 10cm, and the first and second calibration modes being QPD and SPD calibration modes respectively, at this object distance, the sample defocus conversion coefficients for each camera module under QPD mode and SPD mode are extracted from historical production line data. For the same module, the sample defocus conversion coefficient under SPD mode is divided element-wise by the sample defocus conversion coefficient under QPD mode to obtain the individual mode conversion matrix for that module. Then, the average value is taken for the corresponding positions of the individual matrices of all modules to generate the final mode conversion matrix at a 10cm object distance. It is understandable that the median or other reasonable calculation methods can be used instead of the average value to calculate the mode conversion matrix.

[0130] Understandably, the mode conversion matrix between the first and second calibration modes under an object distance of 80cm, the mode conversion matrix between the first and second calibration modes under an object distance of 5cm, and the mode conversion matrix between the first and second calibration modes under an object distance of 6cm can also be obtained using the above method.

[0131] This embodiment obtains the mode conversion matrix corresponding to a certain object distance based on the defocus conversion coefficient in the first calibration mode and the defocus conversion coefficient in the second calibration mode. Based on the difference in defocus conversion coefficients of different calibration modes under the same object distance, it establishes the mode conversion rules, thereby obtaining the unmeasured mode parameters, saving calibration time and improving parameter calibration efficiency.

[0132] The following section details how to use the above mapping relationship and some measured parameters to obtain other unmeasured parameters.

[0133] In one embodiment, the gain matrix corresponding to multiple calibration modes and multiple object distance conditions is determined, specifically as follows: Figure 4 As shown, a gain matrix calibration method is provided, including the following steps:

[0134] This embodiment uses the gain calibration model, which includes three gain calibration models, as an example. The first calibration mode is the QPD calibration mode, the second calibration mode is the SPD calibration mode, the first object distance is 10cm, the second object distance is 80cm, the third object distance is 5cm, and the fourth object distance is 6cm as examples.

[0135] Step 401: Input the first gain matrix into the first gain calibration model to obtain the first calibration mode and the second gain matrix corresponding to the first object distance condition.

[0136] The first gain matrix can be the left gain matrix of the image acquisition device under the first calibration mode and the first object distance condition, or it can be the right gain matrix of the image acquisition device under the first calibration mode and the first object distance condition.

[0137] Taking the first gain matrix as the left gain matrix under the first calibration mode and the first object distance condition as an example, the first gain matrix is ​​the left gain matrix under the QPD calibration mode and the 10cm object distance condition. The first gain matrix is ​​input into the first gain calibration model, and the second gain matrix corresponding to the first calibration mode and the first object distance condition is output through the first gain calibration model, which is the right gain matrix under the QPD calibration mode and the 10cm object distance condition.

[0138] Step 402: Input the first gain matrix and the second gain matrix into the second gain calibration model to obtain the third gain matrix corresponding to the second calibration mode and the first object distance condition, and the fourth gain matrix corresponding to the second calibration model and the first object distance condition.

[0139] The first gain matrix, i.e. the left gain matrix under the QPD calibration mode and the 10cm object distance condition, is input into the second gain calibration model. The second gain calibration model outputs the third gain matrix, i.e. the left gain matrix under the SPD calibration mode and the 10cm object distance condition.

[0140] The second gain matrix, i.e. the right gain matrix under QPD calibration mode and 10cm object distance condition, is input into the second gain calibration model. The second gain calibration model outputs the fourth gain matrix, i.e. the right gain matrix under SPD calibration mode and 10cm object distance condition.

[0141] Step 403: Input the first gain matrix, the second gain matrix, the third gain matrix and the fourth gain matrix into the third gain calibration model to obtain multiple calibration modes and the corresponding gain matrices under multiple object distance conditions other than the first object distance.

[0142] The first gain matrix, i.e. the left gain matrix under the QPD calibration mode and the 10cm object distance condition, is input into the third gain calibration model. The third gain calibration model outputs the left gain matrix under the QPD calibration mode and the 5cm object distance condition, the left gain matrix under the QPD calibration mode and the 6cm object distance condition, and the left gain matrix under the QPD calibration mode and the 80cm object distance condition.

[0143] The second gain matrix, namely the right gain matrix under the QPD calibration mode and the 10cm object distance condition, is input into the third gain calibration model. The third gain calibration model outputs the right gain matrix under the QPD calibration mode and the 5cm object distance condition, the right gain matrix under the QPD calibration mode and the 6cm object distance condition, and the right gain matrix under the QPD calibration mode and the 80cm object distance condition.

[0144] Input the third gain matrix, namely the left gain matrix under the SPD calibration mode and the 10cm object distance condition, into the third gain calibration model. The third gain calibration model outputs the left gain matrix under the SPD calibration mode and the 5cm object distance condition, the left gain matrix under the SPD calibration mode and the 6cm object distance condition, and the left gain matrix under the SPD calibration mode and the 80cm object distance condition.

[0145] The fourth gain matrix, namely the right gain matrix under the SPD calibration mode and the 10cm object distance condition, is input into the third gain calibration model. The third gain calibration model outputs the right gain matrix under the SPD calibration mode and the 5cm object distance condition, the right gain matrix under the SPD calibration mode and the 6cm object distance condition, and the right gain matrix under the SPD calibration mode and the 80cm object distance condition.

[0146] Based on the above, the left and right gain matrices under all conditions are obtained. The above embodiment is only an example; in practice, the gain matrices corresponding to different object distances can be derived first, then the gain matrices corresponding to different calibration modes can be derived, and finally the left and right gain matrices can be derived, etc. This application does not impose any limitations. That is to say, the order and number of times the first, second, and third gain calibration models are used can be flexibly adjusted according to the actual calibration parameters obtained, as long as the left and right gain matrices under all conditions can be obtained by using the first, second, and third gain calibration models. Based on the same input, the accuracy of the results obtained by using the above three models, from high to low, is: third gain calibration model, second gain calibration model, and first gain calibration model.

[0147] This embodiment achieves gain transfer in the left and right directions by using the first gain calibration model, the second gain calibration model to achieve gain transfer across calibration modes, and the third gain calibration model to achieve gain transfer across object distances. Thus, starting from a portion of the measured gain matrix, a complete set of gain matrices covering multiple calibration modes and multiple object distance conditions is systematically generated.

[0148] In one embodiment, the defocus conversion coefficients corresponding to various calibration modes and various object distance conditions are determined, specifically as follows: Figure 5As shown, a method for calibrating the defocus conversion coefficient is provided, including the following steps:

[0149] This embodiment uses the coefficient mapping relationship implemented through a coefficient calibration model and a coefficient conversion algorithm as an example. The first calibration mode is QPD calibration mode, the second calibration mode is SPD calibration mode, the first object distance is 10cm, the second object distance is 80cm, the third object distance is 5cm, and the fourth object distance is 6cm as examples for illustration.

[0150] Step 501: Input the first defocus conversion coefficient and the second defocus conversion coefficient into the coefficient calibration model to obtain the third defocus conversion coefficient corresponding to the first calibration mode and the third object distance condition, and the fourth defocus conversion coefficient corresponding to the first calibration mode and the fourth object distance condition.

[0151] The first defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 10cm object distance condition. The second defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 80cm object distance condition. The first and second defocus conversion coefficients are input into the coefficient calibration model, which outputs the third and fourth defocus conversion coefficients. Specifically, the third defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 5cm object distance condition; the fourth defocus conversion coefficient is the defocus conversion coefficient under QPD calibration mode and a 6cm object distance condition. It is understandable that before inputting the first and second defocus conversion coefficients into the coefficient calibration model, they need to be standardized. The standardization method is the same as described in step 301 above and will not be repeated here. Similarly, the original output of the coefficient calibration model is de-standardized to obtain the third and fourth defocus conversion coefficients.

[0152] Step 502: Obtain the first mode conversion matrix corresponding to the first object distance and the second mode conversion matrix corresponding to the second object distance.

[0153] The first mode conversion matrix is ​​the mode conversion matrix between the first calibration mode and the second calibration mode under an object distance of 10cm. The second mode conversion matrix is ​​the mode conversion matrix between the first calibration mode and the second calibration mode under an object distance of 80cm.

[0154] Step 503: Based on the first defocus conversion coefficient, the first mode conversion matrix, and the coefficient conversion algorithm, obtain the second calibration mode and the fifth defocus conversion coefficient corresponding to the first object distance condition.

[0155] After obtaining the first mode conversion matrix, the fifth defocus conversion coefficient is obtained based on the first defocus conversion coefficient, the first mode conversion matrix, and the coefficient conversion algorithm. The specific formula for the coefficient conversion algorithm is as follows:

[0156] DCC spd = k1 × DCC qpd ;

[0157] Among them, DCC qpd This represents the first defocus conversion factor, which is also the defocus conversion factor under QPD calibration mode and a 10cm object distance condition; DCC spd represents the fifth defocus conversion factor, which is the defocus conversion factor under SPD calibration mode and 10cm object distance conditions; k1 represents the first mode conversion matrix. The matrix size of the mode conversion matrix is ​​the same as the matrix size of the matrix corresponding to the defocus conversion factor.

[0158] Step 504: Based on the second defocus conversion coefficient, the second mode conversion matrix, and the coefficient conversion algorithm, obtain the sixth defocus conversion coefficient corresponding to the second calibration mode and the second object distance condition.

[0159] After obtaining the second mode conversion matrix, the sixth defocus conversion coefficient is obtained based on the second defocus conversion coefficient, the second mode conversion matrix, and the coefficient conversion algorithm. The specific formula for the coefficient conversion algorithm is as follows:

[0160] DCC spd = k2 × DCC qpd ;

[0161] Among them, DCC qpd This indicates the second defocus conversion factor, which is the defocus conversion factor under QPD calibration mode and 80cm object distance conditions; DCC spd represents the sixth defocus conversion factor, which is the defocus conversion factor under SPD calibration mode and 80cm object distance conditions; k2 represents the second mode conversion matrix. The matrix size of the mode conversion matrix is ​​the same as the matrix size of the matrix corresponding to the defocus conversion factor.

[0162] Step 505: Input the fifth and sixth defocus conversion coefficients into the coefficient calibration model to obtain the seventh defocus conversion coefficient corresponding to the second calibration mode and the third object distance condition, and the eighth defocus conversion coefficient corresponding to the second calibration mode and the fourth object distance condition.

[0163] After obtaining the fifth and sixth defocus conversion coefficients, they are input into the coefficient calibration model. The model then outputs the seventh and eighth defocus conversion coefficients. The seventh defocus conversion coefficient corresponds to the defocus conversion coefficient under the SPD calibration mode and a 5cm object distance condition; the eighth defocus conversion coefficient corresponds to the defocus conversion coefficient under the SPD calibration mode and a 6cm object distance condition. Understandably, before inputting the fifth and sixth defocus conversion coefficients into the coefficient calibration model, they need to be standardized. The standardization method is the same as described in step 301 above and will not be repeated here. Similarly, the original output of the coefficient calibration model is destandardized to obtain the seventh and eighth defocus conversion coefficients.

[0164] Optionally, when determining the seventh and eighth defocus conversion factors:

[0165] The third mode conversion matrix between the first and second calibration modes under a 5cm object distance condition can be obtained. Using the third defocus conversion coefficient, the third mode conversion matrix, and the coefficient conversion algorithm corresponding to the first calibration mode and the third object distance condition, the seventh defocus conversion coefficient corresponding to the second calibration mode and the third object distance condition can be obtained.

[0166] The fourth mode conversion matrix between the first and second calibration modes under a 6cm object distance condition can be obtained. Using the fourth defocus conversion coefficient, the fourth mode conversion matrix, and the coefficient conversion algorithm corresponding to the first calibration mode and the fourth object distance condition, the eighth defocus conversion coefficient corresponding to the second calibration mode and the fourth object distance condition can be obtained.

[0167] This application does not specifically limit the method for determining the seventh and eighth defocus conversion coefficients. Since the model yields better accuracy and takes less time than the coefficient matrix, the coefficient calibration model is preferred for determining the seventh and eighth defocus conversion coefficients.

[0168] Based on the above, the defocus conversion coefficient under all conditions is obtained.

[0169] This embodiment achieves cross-object distance extension of DCC under the same calibration mode by using DCC values ​​at two known object distances as input to call the coefficient calibration model and outputting DCC values ​​at other object distances under the same mode. By substituting the first defocus conversion coefficient and the first mode conversion matrix into the coefficient conversion algorithm to perform inter-mode conversion calculations and outputting DCC values ​​in the target mode, DCC migration from a known mode to a new calibration mode is achieved. Using the two DCC values ​​derived across modes as a new starting point, the coefficient calibration model is called again to perform cross-object distance extension, generating a DCC set under the complete object distance sequence, completing a systematic coverage from a small number of measured parameters to full-dimensional calibration. This avoids the redundant operations of repeatedly performing PD image acquisition, multi-image processing, and DCC fitting for each mode-object distance combination in traditional methods, significantly reducing the number of image acquisitions and computational load, and greatly reducing resource consumption while ensuring accuracy, thereby reducing calibration costs and time.

[0170] In one embodiment, the phase difference offset corresponding to multiple calibration modes and multiple object distance conditions is determined, specifically as follows: Figure 6 As shown, a phase difference offset calibration method is provided, which specifically includes the following steps:

[0171] Step 601: Based on the first phase difference offset, the first defocus conversion coefficient, the fifth defocus conversion coefficient, and the first offset conversion algorithm, obtain the third phase difference offset corresponding to the second calibration mode and the first object distance condition.

[0172] The offset mapping relationship is achieved through a first offset conversion algorithm between different modes and a second offset conversion algorithm between different object distances.

[0173] The following explanation uses the following calibration modes as examples: QPD calibration mode as the first calibration mode, SPD calibration mode as the second calibration mode, and object distances of 10cm, 80cm, 5cm, and 6cm as the third and fourth calibration modes.

[0174] The first phase difference offset represents the phase difference offset in QPD calibration mode and under a 10cm object distance condition. The first defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 10cm object distance condition. The fifth defocus conversion factor represents the defocus conversion factor in SPD calibration mode and under a 10cm object distance condition. The third phase difference offset represents the phase difference offset in SPD calibration mode and under a 10cm object distance condition.

[0175] The specific formula for the first offset conversion algorithm to calculate the third phase difference offset is as follows:

[0176] ;

[0177] in, Indicates the third phase difference offset; Indicates the fifth defocus conversion factor; Indicates the first defocus conversion factor; This represents the first phase difference offset.

[0178] Step 602: Based on the second phase difference offset, the second defocus conversion coefficient, the sixth defocus conversion coefficient, and the first offset conversion algorithm, obtain the fourth phase difference offset corresponding to the second calibration mode and the second object distance condition.

[0179] The second phase difference offset can be obtained using traditional methods while acquiring the second defocus conversion coefficient. Alternatively, it can be obtained using a conversion algorithm based on the first phase difference offset, the first defocus conversion coefficient, the second defocus conversion coefficient, and the second offset.

[0180] The second phase difference offset represents the phase difference offset in QPD calibration mode and under an 80cm object distance condition. The second defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under an 80cm object distance condition. The sixth defocus conversion factor represents the defocus conversion factor in SPD calibration mode and under an 80cm object distance condition. The fourth phase difference offset represents the phase difference offset in SPD calibration mode and under an 80cm object distance condition.

[0181] The specific formula for the first offset conversion algorithm to calculate the fourth phase difference offset is as follows:

[0182] ;

[0183] in, This indicates the fourth phase difference offset; Indicates the sixth defocus conversion factor; Indicates the second defocus conversion factor; This indicates the second phase difference offset.

[0184] Step 603: Based on the first phase difference offset, the first defocus conversion coefficient, the third defocus conversion coefficient, and the second offset conversion algorithm, obtain the fifth phase difference offset corresponding to the first calibration mode and the third object distance condition.

[0185] The first phase difference offset represents the phase difference offset in QPD calibration mode and under a 10cm object distance condition. The first defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 10cm object distance condition. The third defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 5cm object distance condition. The fifth phase difference offset represents the phase difference offset in QPD calibration mode and under a 5cm object distance condition.

[0186] The specific formula for the second offset conversion algorithm to calculate the fifth phase difference offset is as follows:

[0187] ;

[0188] in, This indicates the fifth phase difference offset; Indicates the third defocus conversion factor; Indicates the first defocus conversion factor; This represents the first phase difference offset.

[0189] Step 604: Based on the first phase difference offset, the first defocus conversion coefficient, the fourth defocus conversion coefficient, and the second offset conversion algorithm, obtain the sixth phase difference offset corresponding to the first calibration mode and the fourth object distance condition.

[0190] The first phase difference offset represents the phase difference offset in QPD calibration mode and under a 10cm object distance condition. The first defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 10cm object distance condition. The fourth defocus conversion factor represents the defocus conversion factor in QPD calibration mode and under a 6cm object distance condition. The sixth phase difference offset represents the phase difference offset in QPD calibration mode and under a 6cm object distance condition.

[0191] The specific formula for the second offset conversion algorithm to calculate the sixth phase difference offset is as follows:

[0192] ;

[0193] in, This indicates the sixth phase difference offset; Indicates the fourth defocus conversion factor; Indicates the first defocus conversion factor; This represents the first phase difference offset.

[0194] Step 605: Based on the third phase difference offset, the fifth defocus conversion coefficient, the seventh defocus conversion coefficient, and the second offset conversion algorithm, obtain the seventh phase difference offset corresponding to the second calibration mode and the third object distance condition.

[0195] The third phase difference offset represents the phase difference offset in SPD calibration mode and under a 10cm object distance condition. The fifth defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 10cm object distance condition. The seventh defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 5cm object distance condition. The seventh phase difference offset represents the phase difference offset in SPD calibration mode and under a 5cm object distance condition.

[0196] The specific formula for the second offset conversion algorithm to calculate the seventh phase difference offset is as follows:

[0197] ;

[0198] in, This indicates the seventh phase difference offset; Indicates the seventh defocus conversion factor; Indicates the fifth defocus conversion factor; This indicates the offset of the third phase difference.

[0199] Step 606: Based on the third phase difference offset, the fifth defocus conversion coefficient, the eighth defocus conversion coefficient, and the second offset conversion algorithm, obtain the eighth phase difference offset corresponding to the second calibration mode and the fourth object distance condition.

[0200] The third phase difference offset represents the phase difference offset in SPD calibration mode and under a 10cm object distance condition. The fifth defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 10cm object distance condition. The eighth defocus conversion coefficient represents the defocus conversion coefficient in SPD calibration mode and under a 6cm object distance condition. The eighth phase difference offset represents the phase difference offset in SPD calibration mode and under a 6cm object distance condition.

[0201] The specific formula for the second offset conversion algorithm to calculate the eighth phase difference offset is as follows:

[0202] ;

[0203] in, This indicates the eighth phase difference offset; This represents the eighth defocus conversion factor; Indicates the fifth defocus conversion factor; This indicates the offset of the third phase difference.

[0204] Steps 605 and 606 above obtain the seventh phase difference offset and the eighth phase difference offset through the second offset conversion algorithm. In actual use, the seventh phase difference offset and the eighth phase difference offset can also be calculated by the first offset conversion algorithm. This embodiment does not make specific limitations. It is only necessary to combine the first offset conversion algorithm and the second offset conversion algorithm to obtain the phase difference offset under all object distance conditions in the multi-calibration mode.

[0205] This embodiment constructs a dual-path offset conversion mechanism. The first offset conversion algorithm is used for offset migration across calibration modes, and the second offset conversion algorithm is used for offset migration across object distances. The algorithm-driven parameter migration mechanism replaces the traditional method of repeatedly performing phase difference offset calibration for each combination of calibration mode and object distance. This significantly reduces calibration resource consumption, calibration cost, and time consumption while ensuring focusing accuracy.

[0206] In one specific embodiment, a multi-object distance / multi-mode PDAF algorithm merging method includes the following steps:

[0207] The two calibration modes are QPD calibration mode and SPD calibration mode. The four object distances are 10cm, 80cm, 5cm and 6cm.

[0208] Step 1: Train the first gain calibration model corresponding to the left and right directions, the second gain calibration model corresponding to different calibration modes, and the third gain calibration model corresponding to different object distances. The gain matrix (Gain map) is mainly used for preprocessing the image for PDAF calculation. Therefore, the gain matrix includes the left gain matrix (LGM) and the right gain matrix (RGM). The gain matrix is ​​a 12×16 matrix. Based on the above, each calibration mode and each object distance corresponds to a left gain map and a right gain map.

[0209] Step 1.1: The first gain calibration model corresponding to the left and right directions shows opposite trends in the left and right gain maps, and the sum of corresponding positions is approximately 4. Therefore, an AI algorithm is used to calculate it, as follows:

[0210] a. Data preprocessing: Since the gain matrix value is approximately 2 in QPD calibration mode and approximately 1 in SPD calibration mode, based on the following formula, 'a' equals 2 in QPD calibration mode and 'a' equals 1 in SPD calibration mode. All gain matrices are preprocessed using the following formula.

[0211] GMafter = GMbefore - a;

[0212] Where GMafter represents the gain matrix after preprocessing; GMbefore represents the gain matrix before preprocessing. Data preprocessing can reduce errors caused by fluctuations in the data itself, making the calibration results fluctuate within a certain range and suppressing the occurrence of outliers.

[0213] b. Training the model: Left Gain map and Right Gain map. The input is a 1×12×16 Left Gain map, and the output is a 1×12×16 Right Gain map, with stable relationships between the data. Considering the efficiency of production line applications and memory limitations, the FCN algorithm is chosen as the prediction model.

[0214] c. Loss function design, the specific loss function is as follows:

[0215] ;

[0216] Step 1.2: The second gain calibration model corresponds to different calibration modes. The gain matrix value of the QPD calibration mode is around 2, and the gain matrix value of the SPD calibration mode is around 1, with a stable trend. The model input is the Left Gain map and Right Gain map of the first calibration mode, and the output is the Left Gain map and Right Gain map of the second calibration mode. The specific method is the same as in Step 1.1.

[0217] Step 1.3: Third gain calibration model for different object distances. Considering the object distance selection issue in DCC merging, QPD-10CM-LGM, QPD-10CM-RGM, SPD-10CM-LGM, and SPD-10CM-RGM were ultimately selected as measured inputs to infer the gain maps for the other three object distances. The steps are as follows:

[0218] a. Data preprocessing is the same as in step 1.1.

[0219] b. Training the model: Taking QPD-10CM-LGM as an example, the input of the third gain calibration model is a 1×12×16 QPD-10CM-LGM, but it needs to output the LGMs of the other three object distances simultaneously. The FCN (Fully Convolutional Network) model cannot meet this requirement, so the FCN model needs to be optimized. The optimized FCN model process is as follows: Figure 7 As shown, the core of the new model remains the FCN model, but fully connected layers are added at the beginning and end of the FCN model to improve the model's ability to acquire information. Furthermore, the input and output of this sub-model are concatenated in the concatenation layer to further enhance the model's ability to acquire information.

[0220] c. Loss function design, the specific loss function is as follows:

[0221] ;

[0222] In the third gain calibration model, the output of each sub-model is output at the end. Therefore, it is necessary to calculate the loss for each sub-model's output, denoted as loss1, loss2, and loss3 respectively. The final loss formula is as follows: loss = loss1 + loss2 + loss3. This embodiment can reduce the original process of sampling 8 images and calculating 16 gain maps to sampling 1 image and calculating 1 gain map, greatly improving the efficiency of PDAF calibration on the production line.

[0223] Step 2: Train the coefficient calibration model for the DCC (Discrete Conversion Coefficient). Based on the efficiency, accuracy, and profitability of the production line, a two-to-two algorithm is ultimately used for object distance merging. Considering the module's characteristics, the testing scheme for the inspection station, and data stability, DCC values ​​of 10cm and 80cm object distances are selected as the actual measured input data to predict DCC values ​​of 5cm and 6cm object distances. Details are as follows:

[0224] Step 2.1, data preprocessing, can reduce the impact of data fluctuations on accuracy and limit the results to a certain range, thus reducing the risk of model inference anomalies. The standardized formula is as follows:

[0225] DCC after = (DCC before – DCC mean ) / DCC std ;

[0226] Among them, DCC after For standardized DCC; DCC before DCC before standardization; DCC mean The average value of DCC; DCC std This represents the standard deviation of DCC.

[0227] Step 2.2, Model Training: Since there is currently no commercially available method for extrapolating PDAF data using AI models, this algorithm requirement is shifted to time-series extrapolation, and the Transformer algorithm is ultimately chosen for extrapolation. However, the multi-head attention mechanism used by the Transformer model is time-consuming and memory-intensive, and its core self-attention value calculation is cumbersome, making it prone to overfitting on small datasets. Therefore, the following optimizations are made to the Transformer algorithm: 1. The multi-head attention mechanism is reduced to a fixed number of 2 heads. 2. To improve the attention mechanism's ability to extract channel information, a channel attention mechanism is added. 3. The sigmoid function of the channel attention mechanism is replaced with upsampling, making the model more suitable for the task in this embodiment.

[0228] Step 2.3, Loss Function Design: The loss function is as follows:

[0229] ;

[0230] ;

[0231] ;

[0232] loss = λ1 × loss1 +λ2 × loss2 +λ3 × loss3;

[0233] in, This represents the model's predicted value; y i `abs` represents the true value; `abs` represents the absolute value operation; `n` represents the number of samples. `loss1` represents the mean squared error; `loss2` represents the DCC numerical precision error; `loss3` represents the center position weighted error. `λ1`, `λ2`, and `λ3` represent the coefficients corresponding to each loss function, with preferred values ​​of `λ1` = 0.3, `λ2` = 0.5, and `λ3` = 0.2. 19 y 20 y 27 y 28 These represent the true values ​​at indices 19, 20, 27, and 28 in the one-dimensional vector of DCC, respectively, under the measured object distance; corresponding to... , , , The target distance calculated by the model is the predicted value at the same index. The DCC one-dimensional vector is obtained by sampling along the straight line (horizontal centerline or radial line) passing through the optical center of the DCC matrix and arranging them in distance order. Index 20 is aligned with the center of the field of view, so the four points mentioned here are all located in the central region.

[0234] Step 3: Predict the defocus conversion coefficient (DCC) and phase difference offset for various calibration modes and object distances. Mathematically, the accuracy of the defocus conversion coefficient (DCC) is inversely proportional to the signal-to-noise ratio (SNR) of phase detection. The QPD calibration mode has a high SNR and a low DCC value; the SPD calibration mode has a low SNR and a high DCC value. From a chip perspective, the QPD calibration mode divides the pixels on the chip into four parts to obtain the left and right PD points. The SPD calibration mode, however, obscures half of the pixels to obtain the left and right PD points. This results in the PD point density of the QPD calibration mode being twice that of the SPD calibration mode on the chip, leading to a PD value that is approximately twice that of the SPD calibration mode during calculation. Consequently, the final DCC value of the QPD calibration mode is approximately half that of the SPD calibration mode. Although environmental factors may affect the calibration process, the overall multiplier is between 1.93 and 1.96, and the module's dynamic error is about 5%. Therefore, the coefficient matrix is ​​used to calculate the DCC value of the SPD mode.

[0235] Using the coefficient calibration model in step 2, the defocus conversion coefficient (DCC) is predicted pairwise for different object distances under the same calibration mode. The following coefficient conversion algorithm is used to predict the defocus conversion coefficient (DCC) for the same object distance under different calibration modes; the specific formula is as follows:

[0236] DCC spd = k × DCC qpd ;

[0237] DCC qpd Indicates the defocus conversion factor for QPD calibration mode; DCC spd The defocus conversion coefficient represents the SPD calibration mode, and k represents the mode conversion matrix. For example, it includes the mode conversion matrix under the condition of 10cm object distance, the mode conversion matrix under the condition of 80cm object distance, the mode conversion matrix under the condition of 5cm object distance, and the mode conversion matrix under the condition of 6cm object distance.

[0238] The specific formula for predicting the phase difference offset between the same object distance in different calibration modes is as follows:

[0239] ;

[0240] This indicates the phase difference offset of the SPD calibration mode; Indicates the defocus conversion factor of the SPD calibration mode; This indicates the defocus conversion factor in the QPD calibration mode; This indicates the phase difference offset of the QPD calibration mode.

[0241] For predicting the phase difference offset between different object distances under the same calibration mode, since 5cm, 6cm, and 10cm object distances are all macro object distances with similar physical characteristics, the offset of 10cm is used to calculate the offsets of 5cm and 6cm. The specific formula is as follows:

[0242] ;

[0243] This represents the phase difference offset at a 5cm object distance. The defocus conversion factor represents the distance from the object at a 5cm object distance; This represents the defocus conversion factor at an object distance of 10cm. This represents the phase difference offset at a distance of 10cm from the object.

[0244] ;

[0245] This represents the phase difference offset at a distance of 6m. This represents the defocus conversion factor at an object distance of 6m. This represents the defocus conversion factor at an object distance of 10m. This represents the phase difference offset at a distance of 10m.

[0246] In this embodiment, the original process of taking 8 images and calculating 16 DCC and offsets is reduced to taking 2 images and calculating 2 DCC and offsets, which greatly improves the PDAF calibration efficiency of the production line.

[0247] In this embodiment, the coefficient calibration model predicts the defocus conversion coefficient (DCC) for pairwise objects at different distances within the same calibration mode. This embodiment selects the DCC at a 10cm and 80cm object distance, and predicts the DCC at a 5cm and 6cm object distance. The specific reasons are as follows: 1. Considering the characteristics of the periscope macro module, the unique periscope object distance of 80cm is used as the measured object distance, i.e., as the input object distance. 2. The detection scheme at the production line inspection station uses 6cm and 10cm object distances for detection. To avoid affecting the reliability of the inspection station, a choice is made between 6cm and 10cm object distances for macro detection. 3. Since the DCC value at 6cm object distance is less stable than that at 10cm object distance, it is not suitable as input data. In summary, the DCC at 10cm and 80cm object distances were ultimately selected.

[0248] In this embodiment, for the third gain calibration model, which uses the offset of one object distance to predict the offsets of the other three object distances for different object distances under the same calibration mode, this embodiment selects the gain matrix for a 10cm object distance to predict the gain matrices for 5cm, 6cm, and 80cm object distances. The specific reasons are as follows: 1. Combining the object distance selection of the coefficient calibration model, a choice is made between 10cm and 80cm object distances. 2. Considering the PDAF detection station on the production line, OK / NG detection is performed using 6cm and 10cm object distances. Therefore, the gain matrix for the 10cm object distance is ultimately used as input to calculate the gain matrices for the other three object distances.

[0249] In one specific embodiment, such as Figure 8 As shown, first, a QPD-10cm-LGM is obtained. The QPD-10cm-LGM is then input into the first gain calibration model to obtain a QPD-10cm-RGM. The QPD-10cm-LGM is then input into the second gain calibration model to obtain an SPD-10cm-LGM. The QPD-10cm-RGM is then input into the second gain calibration model to obtain an SPD-10cm-RGM. The QPD-10cm-LGM is then input into the third gain calibration model to obtain QPD-5cm-LGM, QPD-6cm-LGM, and QPD-80cm-LGM. The QPD-10cm-RGM is then input into the third gain calibration model to obtain QPD-5cm-RGM, QPD-6cm-RGM, and QPD-80cm-RGM. Finally, the SPD-10cm-LGM is input into the third gain calibration model to obtain SPD-5cm-LGM, SPD-6cm-LGM, and SPD-80cm-LGM. Inputting the SPD-10cm-RGM into the third gain calibration model yields the SPD-5cm-RGM, SPD-6cm-RGM, and SPD-80cm-RGM.

[0250] In one embodiment, such as Figure 9As shown, firstly, QPD-10cm-DCC, QPD-10cm-offset, QPD-80cm-DCC, and QPD-80cm-offset are obtained. QPD-10cm-DCC and QPD-80cm-DCC are input into the coefficient calibration model to obtain QPD-5cm-DCC and QPD-6cm-DCC. Based on QPD-10cm-DCC and the 10cm mode transition matrix, SPD-10cm-DCC is determined. Based on QPD-80cm-DCC and the 80cm mode transition matrix, SPD-80cm-DCC is determined. SPD-10cm-DCC and SPD-80cm-DCC are input into the coefficient calibration model to obtain SPD-5cm-DCC and SPD-6cm-DCC. Based on QPD-10cm-offset, QPD-10cm-DCC, and SPD-10cm-DCC, SPD-10cm-offset is determined. Determine the SPD-80cm-offset based on QPD-80cm-DCC, QPD-80cm-offset, and SPD-80cm-DCC. Determine the QPD-5cm-offset based on QPD-10cm-offset, QPD-10cm-DCC, and QPD-5cm-DCC. Determine the QPD-6cm-offset based on QPD-10cm-offset, QPD-10cm-DCC, and QPD-6cm-DCC. Determine the SPD-5cm-offset based on SPD-10cm-offset, SPD-10cm-DCC, and SPD-5cm-DCC. Determine the SPD-6cm-offset based on SPD-10cm-offset, SPD-10cm-DCC, and SPD-6cm-DCC.

[0251] This embodiment proposes a multi-object distance / multi-mode PDAF algorithm merging method. By calculating the gain map value, DCC value, and offset value under multiple calibration modes and multiple object distances based on AI algorithms, it reduces the number of times the camera module needs to re-open the image, the number of images acquired, and the number of equipment required for the project. This not only reduces the time consumption and number of equipment required for PDAF calibration, but also effectively reduces the dynamic error of PDAF caused by module motor movement and image re-opening.

[0252] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0253] Based on the same inventive concept, this application also provides a parameter calibration device for an image acquisition device to implement the parameter calibration method of the image acquisition device described above. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations in one or more embodiments of the parameter calibration device for an image acquisition device provided below can be found in the limitations of the parameter calibration method for the image acquisition device described above, and will not be repeated here.

[0254] In one embodiment, such as Figure 10 As shown, a parameter calibration device for an image acquisition equipment is provided, comprising: an acquisition module 100, a first calibration module 200, a second calibration module 300, and a storage module 400, wherein:

[0255] The acquisition module 100 is used to acquire the first gain matrix and the first defocus conversion coefficient of the image acquisition device in the first calibration mode and at the first object distance, and the second defocus conversion coefficient in the same calibration mode and at the second object distance.

[0256] The first calibration module 200 is used to determine a gain matrix corresponding to the first target calibration mode and / or the first target object distance based on the first gain matrix and the trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance.

[0257] And / or, the second calibration module 300 is used to determine the defocus conversion coefficient corresponding to the second target calibration mode and / or the second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient and the trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from both the first object distance and the second object distance;

[0258] The storage module 400 is used to write the obtained gain matrix and / or defocus conversion coefficient into the memory of the image acquisition device for use in optical shadow correction and phase detection autofocus during the image signal processing stage.

[0259] The parameter calibration device for the image acquisition equipment also includes a third calibration module.

[0260] The third calibration module is used to obtain the first phase difference offset of the image acquisition device in the first calibration mode and at the first object distance; based on the first defocus conversion coefficient and the first phase difference offset, combined with at least one paired defocus conversion coefficient across modes or across object distances, and the offset mapping relationship, determine the phase difference offset corresponding to the target condition; the target condition is different from the first condition formed by the first calibration mode and the first object distance only in the calibration mode or the object distance.

[0261] The gain mapping relationship is implemented through a gain calibration model group, which includes at least one of a first gain calibration model, a second gain calibration model, and a third gain calibration model, and each model has the following functions: the first gain calibration model is used to perform consistency correction on the input gain matrix under the same calibration mode and the same object distance, and output the consistency-corrected gain matrix; the second gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other calibration mode under the same object distance; the third gain calibration model is used to map the input gain matrix to the gain matrix corresponding to at least one other object distance under the same calibration mode.

[0262] The first gain calibration model, the second gain calibration model, and the third gain calibration model are implemented by different output heads of the same neural network, or by sub-networks that are trained independently.

[0263] The defocus mapping relationship is implemented based on a coefficient calibration model group, which includes at least one of a coefficient calibration model and a coefficient conversion algorithm, and each has the following functions: The coefficient calibration model is used to map the defocus conversion coefficients corresponding to any two object distances to the defocus conversion coefficients corresponding to other two object distances under the same calibration mode; The coefficient conversion algorithm is used to map the known defocus conversion coefficients to the defocus conversion coefficients corresponding to the same object distance and other calibration modes based on the corresponding mode conversion matrix under the same object distance.

[0264] The first calibration module 200 is further configured to determine the gain mapping relationship by: acquiring sample gain matrices covering all calibration modes and all object distances, and after data preprocessing, training each model in the gain calibration model group respectively; the training includes at least: training the first gain calibration model using samples of the same calibration mode and the same object distance; and / or training the second gain calibration model using samples of the same object distance across calibration modes; and / or training the third gain calibration model using samples of the same calibration mode across object distances; to determine the gain mapping relationship.

[0265] The second calibration module 300 is further used to determine the coefficient calibration model in the following way: collecting multiple sets of input-output sample pairs, each set of sample pairs containing: sample defocus conversion coefficients corresponding to any two object distances under the same calibration mode after standardization as input samples; sample defocus conversion coefficients corresponding to other two object distances under the same calibration mode as expected output samples; training the coefficient calibration model based on the input samples and expected output samples to realize the defocus conversion coefficient estimation between different object distances under the same calibration mode in the inference stage.

[0266] The coefficient conversion algorithm is implemented through the mode conversion matrix. The mode conversion matrix is ​​determined by fitting the sample defocus conversion coefficients corresponding to different calibration modes under the same object distance, and is used in the inference stage to convert the defocus conversion coefficient of any calibration mode into the defocus conversion coefficient corresponding to another calibration mode.

[0267] Each module in the parameter calibration device of the aforementioned image acquisition equipment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0268] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a parameter calibration method for an image acquisition device.

[0269] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0270] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the parameter calibration method of any of the image acquisition devices described in the above embodiments.

[0271] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the parameter calibration method of any of the image acquisition devices described in the above embodiments.

[0272] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0273] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0274] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. Those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for calibrating parameters of an image acquisition device, characterized in that, The method comprises: obtaining a first gain matrix and a first defocus conversion coefficient of an image acquisition device in a first calibration mode and at a first object distance, and a second defocus conversion coefficient in the same calibration mode and at a second object distance; at least one of the following two steps is performed: a) determining a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship; wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; b) determining a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient and a trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from the first object distance and the second object distance; writing the obtained gain matrix and / or defocus conversion coefficient into a memory in the image acquisition device for optical shading correction and phase detection automatic focusing in the image signal processing stage.

2. The method of claim 1, wherein, The method further comprises: obtaining a first phase difference offset of the image acquisition device in the first calibration mode and at the first object distance; determining a phase difference offset corresponding to a target condition according to the first defocus conversion coefficient and the first phase difference offset, in combination with at least one paired defocus conversion coefficient across modes or object distances, and an offset mapping relationship; the target condition is different from the first condition constituted by the first calibration mode and the first object distance only in calibration mode or object distance, and the paired defocus conversion coefficient corresponds to the target condition.

3. The method of claim 1, wherein, The gain mapping relationship is realized by a gain calibration model group, the gain calibration model group comprises at least one of a first gain calibration model, a second gain calibration model and a third gain calibration model, and each model has the following functions: The first gain calibration model is used for consistent correction of an input gain matrix under the same calibration mode and the same object distance condition, and outputs a gain matrix after consistent correction; The second gain calibration model is used for mapping an input gain matrix into a gain matrix corresponding to at least one other calibration mode under the same object distance condition; The third gain calibration model is used for mapping an input gain matrix into a gain matrix corresponding to at least one other object distance under the same calibration mode condition.

4. The method of claim 3, wherein, The first gain calibration model, the second gain calibration model and the third gain calibration model are realized by different output heads of the same neural network, or are realized by sub-networks trained independently.

5. The method of claim 1, wherein, The defocus mapping relationship is realized based on a coefficient calibration model group, the coefficient calibration model group comprises at least one of a coefficient calibration model and a coefficient conversion algorithm, and has the following functions: The coefficient calibration model is used for mapping defocus conversion coefficients corresponding to any two object distances into defocus conversion coefficients corresponding to other two object distances under the same calibration mode; The coefficient conversion algorithm is used to map the known defocus conversion coefficients to the defocus conversion coefficients corresponding to the same object distance and other calibration modes based on the corresponding mode conversion matrix under the same object distance.

6. The method of claim 3, wherein, The gain mapping relationship is determined by: Collecting sample gain matrices covering all calibration modes and all object distances, and training each model in the gain calibration model group after data preprocessing; The training at least includes: Training the first gain calibration model using the same calibration mode and object distance samples; and / or Training the second gain calibration model using the same object distance and across calibration mode samples; and / or Training the third gain calibration model using the same calibration mode and across object distance samples; To determine the gain mapping relationship.

7. The method of claim 5, wherein, The coefficient calibration model is determined by: Collecting multiple groups of input-output sample pairs, each group of sample pairs containing: The normalized sample defocus conversion coefficients corresponding to any two object distances under the same calibration mode as input samples; Sample defocus conversion coefficients corresponding to another two object distances under the same calibration mode as expected output samples; The coefficient calibration model is trained based on the input samples and expected output samples to calculate the defocus conversion coefficients between different object distances under the same calibration mode in the inference stage.

8. The method of claim 5, wherein, The coefficient conversion algorithm is implemented by the mode conversion matrix, which is determined by fitting the sample defocus conversion coefficients corresponding to different calibration modes under the same object distance as training data, and is used to convert the defocus conversion coefficients of any calibration mode to the defocus conversion coefficients corresponding to another calibration mode in the inference stage.

9. A parameter calibration device of an image acquisition apparatus, characterized in that, The device includes: An acquisition module configured to acquire a first gain matrix and a first defocus conversion coefficient of an image acquisition device under a first calibration mode and at a first object distance, and a second defocus conversion coefficient under the same calibration mode and at a second object distance; A first calibration module configured to determine a gain matrix corresponding to a first target calibration mode and / or a first target object distance based on the first gain matrix and a trained gain mapping relationship, wherein the first target calibration mode is different from the first calibration mode, or the first target object distance is different from the first object distance; and / or A second calibration module configured to determine a defocus conversion coefficient corresponding to a second target calibration mode and / or a second target object distance based on the first defocus conversion coefficient, the second defocus conversion coefficient, and a trained defocus mapping relationship, wherein the second target calibration mode is different from the first calibration mode, or the second target object distance is different from the first object distance and the second object distance; A storage module configured to write the obtained gain matrix and / or defocus conversion coefficient into a memory in the image acquisition device for optical shading correction and phase detection autofocus in the image signal processing stage. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 8.

Images