Pixel correction method and device, computer device and storage medium

Abstract

The application relates to a pixel correction method and device, computer equipment and a storage medium. The method is used for pixel correction of an image sensor, the image sensor comprises a plurality of repeating units, any repeating unit comprises a plurality of pixel points; the method comprises the following steps: acquiring correction data comprising a plurality of sets of correction coefficients, the number of sets of correction coefficients corresponds to the number of pixel points contained in the repeating unit, and a set of correction coefficients is used for correcting the pixel points at the same relative position in each repeating unit; determining a first correction coefficient corresponding to a first pixel point from the plurality of sets of correction coefficients based on the position of the first pixel point in the corresponding repeating unit; calculating a compensation value corresponding to the first pixel point based on the coordinates of the first pixel point on the image sensor and the first correction coefficient; and correcting the first pixel point based on the compensation value corresponding to the first pixel point, so as to reduce the storage space occupation, improve the resource utilization rate and solve the problem of degradation of the correction effect caused by lens offset.

Description

Technical Field

[0001] This application relates to the field of equipment calibration technology, and in particular to a pixel correction method, apparatus, computer device, and storage medium. Background Technology

[0002] As the pixel size of image sensors continues to increase, the impact of pixel response inconsistencies on imaging uniformity becomes increasingly significant. Therefore, pixel correction has become one of the important technologies for improving imaging uniformity. Traditional techniques offer a pixel correction method that independently stores corresponding correction coefficients for each block in the sensor array, thereby completing pixel-level correction of the sensor image data. However, this method requires storing a large amount of correction data, which becomes costly and power-consuming given the ever-increasing pixel size.

[0003] This shows that traditional techniques still suffer from high storage overhead and low resource utilization in pixel correction. Summary of the Invention

[0004] Therefore, it is necessary to provide a pixel correction method, apparatus, computer equipment, and storage medium that can reduce storage space occupation, improve resource utilization, and solve the problem of correction effect degradation caused by lens shift, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a pixel correction method for performing pixel correction on an image sensor, the image sensor including a plurality of repeating units, each of the repeating units including a plurality of pixels; the method includes:

[0006] Obtain correction data, which includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. One set of correction coefficients is used to correct pixels at the same relative position in each repeating unit.

[0007] For the first pixel in the repeating unit, based on the position of the first pixel in the corresponding repeating unit, the first correction coefficient corresponding to the first pixel is determined from the multiple sets of correction coefficients;

[0008] Based on the coordinates of the first pixel on the image sensor and the first correction coefficient, calculate the compensation value corresponding to the first pixel;

[0009] The first pixel is corrected based on the compensation value corresponding to the first pixel.

[0010] In one embodiment, a repeating unit includes a plurality of pixel blocks, each pixel block corresponding to a color channel, and a set of correction coefficients is used to correct pixels at the same relative position in each pixel block of the image sensor.

[0011] In one embodiment, the image sensor is disposed on a camera module, the camera module further includes a lens assembly, and the correction data further includes a pre-trained model configured to obtain the correspondence between the relative position between the lens assembly and the image sensor and the correction coefficient offset.

[0012] The first correction factor is the correction factor when the lens assembly or the image sensor is located at the reference position;

[0013] The step of calculating the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient includes:

[0014] Obtain the relative position information between the lens assembly and the image sensor;

[0015] The first correction coefficient offset is obtained based on the relative position information and the pre-trained model;

[0016] Based on the first correction coefficient offset and the first correction coefficient, the target correction coefficient is determined;

[0017] The compensation value corresponding to the first pixel is calculated based on the coordinates of the first pixel on the image sensor and the target correction coefficient.

[0018] In one embodiment, before obtaining the coefficient offset based on the relative position information and the pre-trained model, the method further includes:

[0019] A training dataset is obtained, comprising multiple sets of training data corresponding to multiple camera modules. Each set of training data includes coefficient offsets and position offsets of the corresponding camera module at multiple preset positions. The coefficient offset is the difference between the correction coefficient of the camera module at the preset position and the reference correction coefficient. The position offset is the offset of the preset position relative to the reference position, and the reference correction coefficient is the correction coefficient of the camera module at the reference position. The multiple camera modules are of the same model.

[0020] The prediction model is trained based on the training dataset to obtain the pre-trained model.

[0021] In one embodiment, the process of obtaining the training dataset includes:

[0022] Based on a plurality of preset position offsets, the correction coefficient of the camera module at the corresponding preset position is obtained;

[0023] Based on the difference between the correction coefficient and the reference correction coefficient, the coefficient offset of the camera module at the corresponding preset position is determined;

[0024] Based on multiple pairs of position offsets and coefficient offsets of the camera module, the training data corresponding to the camera module is determined;

[0025] Based on the training data corresponding to the multiple camera modules, multiple sets of training data are obtained as training datasets.

[0026] In one embodiment, each set of training data further includes feature parameters of the corresponding camera module, which are obtained by dimensionality reduction of the benchmark correction coefficients.

[0027] In one embodiment, the plurality of preset positions includes eight extreme positions within the working range corresponding to a focusing plane.

[0028] In one embodiment, the plurality of preset positions include eight extreme positions within the working range corresponding to each of at least two focusing planes.

[0029] In one embodiment, calculating the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient includes:

[0030] Construct a polynomial model using the first correction coefficient;

[0031] The coordinates of the first pixel on the image sensor are input into the polynomial model to obtain the compensation value corresponding to the first pixel.

[0032] In one embodiment, acquiring the correction data includes:

[0033] Based on the sample image output by the image sensor under preset acquisition conditions, the pixel compensation matrix corresponding to the image sensor is calculated, and the pixel compensation matrix includes the compensation value corresponding to each pixel of the image sensor.

[0034] Based on the number of pixels contained in the repeating unit, the pixel compensation matrix is ​​divided into multiple pixel compensation sub-matrices; each pixel compensation sub-matrice corresponds to a relative position in each repeating unit.

[0035] The correction data is obtained by performing polynomial fitting on each of the pixel compensation sub-matrices.

[0036] In one embodiment, calculating the pixel compensation matrix corresponding to the image sensor includes:

[0037] Based on the array structure of the image sensor, the sample image is separated into multiple sample sub-blocks; each sample sub-block includes multiple pixels.

[0038] Based on the initial value of the pixel, the average value of multiple pixels in each sample sub-block is calculated respectively;

[0039] Based on the initial value of the pixel and the average value of the corresponding sample sub-block, calculate the compensation value for each pixel;

[0040] Based on the compensation values ​​of multiple pixels, the pixel compensation matrix corresponding to the image sensor is obtained.

[0041] In one embodiment, calculating the pixel compensation matrix corresponding to the image sensor includes:

[0042] Based on a preset block size, the sample image is separated into multiple sample image blocks; each sample image block includes multiple sample sub-blocks; each sample sub-block includes multiple pixels.

[0043] Based on the initial value of the pixel, the average pixel value of each color channel within the sample image block is calculated as the channel average value;

[0044] Based on the initial value of the pixel, the average value of pixels at the same relative position in all sample sub-blocks within the sample image block is calculated as the sub-position average value;

[0045] Based on the channel average value and the sub-position average value, calculate the shared compensation value for the same relative position within all the sample sub-blocks;

[0046] Based on the shared compensation values ​​of multiple sample image blocks, the pixel compensation matrix corresponding to the image sensor is obtained.

[0047] Secondly, this application provides a pixel correction apparatus for performing pixel correction on an image sensor, the image sensor including a plurality of repeating units, each of the repeating units including a plurality of pixels; the apparatus includes:

[0048] The data acquisition module is used to acquire correction data, which includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. One set of correction coefficients is used to correct pixels at the same relative position in each repeating unit.

[0049] The coefficient determination module is used to determine, for the first pixel in the repeating unit, the first correction coefficient corresponding to the first pixel from the multiple sets of correction coefficients based on the position of the first pixel in the corresponding repeating unit;

[0050] The compensation calculation module is used to calculate the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient.

[0051] The pixel correction module is used to correct the first pixel based on the compensation value corresponding to the first pixel.

[0052] Thirdly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0053] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0054] The aforementioned pixel correction method, apparatus, computer device, and storage medium acquire correction data, which includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. One set of correction coefficients is used to correct pixels at the same relative position in each repeating unit. For a first pixel in the repeating unit, a first correction coefficient corresponding to the first pixel is determined from the multiple sets of correction coefficients based on the position of the first pixel in the corresponding repeating unit. Based on the coordinates of the first pixel on the image sensor and the first correction coefficient, a compensation value corresponding to the first pixel is calculated. Based on the coordinates of the first pixel on the image sensor and the first correction coefficient, a compensation value corresponding to the first pixel is calculated. The compensation value is used to correct the first pixel. The concept of repeating unit is introduced, which changes the pixel correction from the traditional independent storage mode of pixel by pixel to a group storage mode based on relative position. This can significantly reduce the storage space occupied by the correction data and the hardware cost. At the same time, based on the deep learning model combined with lens offset and specific correction coefficients, it dynamically adapts to the light field changes caused by lens displacement. This can overcome the problem of the degradation of the effect of traditional static correction methods in optical image stabilization scenarios. Thus, while ensuring high imaging uniformity, it significantly improves resource utilization and the adaptability of image sensors in complex optical environments and imaging quality, achieving the technical effects of reducing storage space occupation, improving resource utilization and preventing the correction effect degradation caused by lens offset. Attached Figure Description

[0055] Figure 1 This is an application environment diagram of the pixel correction method in one embodiment;

[0056] Figure 2 This is a flowchart illustrating a pixel correction method in one embodiment;

[0057] Figure 3 This is a schematic diagram of an embodiment where the basic array unit is an RGGB structure;

[0058] Figure 4 This is a flowchart illustrating a method for acquiring calibration data in one embodiment;

[0059] Figure 5 This is a flowchart illustrating a model training method in one embodiment;

[0060] Figure 6 This is a schematic diagram of the algorithm flow of a pixel correction method in one embodiment;

[0061] Figure 7 This is a schematic diagram of a 4×4 pixel block in one embodiment;

[0062] Figure 8 This is a schematic diagram of the entire process of an image sensor pixel correction method in one embodiment;

[0063] Figure 9 This is a schematic diagram of multiple offset acquisition positions in one embodiment;

[0064] Figure 10 This is a schematic diagram of the structure of a lightweight fully connected neural network in one embodiment;

[0065] Figure 11 This is a schematic diagram of the AI ​​model process for training the correspondence between lens offset and coefficient deviation in one embodiment.

[0066] Figure 12 This is a schematic diagram comparing the effects of different correction methods in one embodiment;

[0067] Figure 13 This is a structural block diagram of a pixel correction device in one embodiment;

[0068] Figure 14 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0070] The pixel correction method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with camera module 104. Camera module 104 includes an image sensor. Terminal 102 acquires correction data, which includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. One set of correction coefficients is used to correct pixels at the same relative position in each repeating unit. For a first pixel in the repeating unit, based on the position of the first pixel in the corresponding repeating unit, a first correction coefficient corresponding to the first pixel is determined from the multiple sets of correction coefficients. Based on the coordinates of the first pixel on the image sensor and the first correction coefficient, a compensation value corresponding to the first pixel is calculated. The first pixel is corrected based on the compensation value corresponding to the first pixel. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc.

[0071] In one embodiment, such as Figure 2 As shown, a pixel correction method is provided. Taking the application of this method to pixel correction of an image sensor as an example, the image sensor includes multiple repeating units, and each repeating unit includes multiple pixels; the method includes:

[0072] Step S110: Obtain calibration data.

[0073] The correction data includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. Each set of correction coefficients is used to correct pixels at the same relative position in each repeating unit.

[0074] An image sensor can be a photoelectric conversion device with a specific pixel arrangement. It is understood that the output data of an image sensor may contain non-uniform pixel response characteristics due to manufacturing processes and optical microlens deviations. For example, depending on the type of image sensor, it may include, but is not limited to, Bayer array sensors, four-color array sensors, and layered stacked sensors.

[0075] For example, an image sensor employs a Color Filter Array (CFA) overlaid on a photosensitive pixel array to achieve selective transmission and acquisition of light signals of different wavelengths. The CFA consists of periodically arranged basic array units, each covering multiple photosensitive pixels and multiple color channels. A basic array unit includes at least a red (R) channel, a green (G) channel, and a blue (B) channel. In some examples, the basic array unit may also include a white (W) channel.

[0076] Furthermore, the basic array unit is a 2×2 array structure, meaning each basic array unit corresponds to four color channels. For example, the basic array units are RGGB, BGGR, GRBG, GBRG, or RGBW (white), respectively.

[0077] For example, each color channel in a basic array cell can cover n×n pixels. For instance, one color channel in a basic array cell can cover 2×2, 3×3, or 4×4 pixels.

[0078] Figure 3 This is an example of a basic array cell structure called RGGB, such as Figure 3 As shown, the first row of the basic array unit contains the red and green channels, and the second row contains the green and blue channels. Each color channel in this basic array unit covers 4×4=16 pixels, meaning one basic array unit corresponds to 64 pixels. Figure 3The red channel covers pixels R0 to R15. The aforementioned repeating unit can be a structural unit with periodic arrangement characteristics in an image sensor array. Utilizing its spatial periodicity, the vast global pixel set can be divided into several subsets with the same internal structure, thereby simplifying the mapping relationship of the correction data. For example, one repeating unit corresponds to one basic array unit; for instance, one repeating unit may include... Figure 3 The diagram shows 64 pixels. As another example, one repeating unit corresponds to one color channel; for instance, a repeating unit may include, for example,... Figure 3 The red channel shown corresponds to 16 pixels (i.e., R0~R15).

[0079] Correction data can be a set of parameters used to correct pixel response inconsistencies, providing the reference parameters required for correction, thus replacing the traditional method of storing correction data independently for each pixel. For example, correction data can be obtained by acquiring a reference image under standard lighting conditions and calculating the deviation of each pixel from the ideal response. In an exemplary embodiment, correction data may include, but is not limited to, gain coefficients, offsets, lookup table indices, or polynomial fitting parameters.

[0080] For example, the correction data can be stored in non-volatile memory, and the aforementioned multiple sets of correction coefficients can be obtained by reading data from a specified location. It is understood that by establishing a mapping relationship between relative positions and sets of correction coefficients, the amount of data that needs to be stored can be significantly reduced, thereby avoiding the huge storage overhead of storing correction data separately for each physical pixel.

[0081] The aforementioned sets of correction coefficients correspond one-to-one with the multiple pixels of the repeating unit. Each set of correction coefficients is used to correct pixels at the same relative position within each repeating unit; that is, different pixels within a repeating unit correspond to different correction coefficients. For example, if each repeating unit includes 64 pixels, the aforementioned correction data includes 64 sets of correction coefficients. The first pixel in each repeating unit corresponds to the first set of correction coefficients, the second pixel in each repeating unit corresponds to the second set of correction coefficients, and so on, with the 64th pixel in each repeating unit corresponding to the 64th set of correction coefficients.

[0082] Step S120: For the first pixel in the repeating unit, based on the position of the first pixel in the corresponding repeating unit, determine the first correction coefficient corresponding to the first pixel from multiple sets of correction coefficients.

[0083] The first pixel can be any pixel in the repeating unit or the target pixel currently being processed. For example, in actual execution, the first pixel can be selected from the raw data stream output by the image sensor according to the raster scan order or parallel processing logic. In a specific embodiment, the first pixel can be a single photosensitive unit under any color channel.

[0084] The first correction coefficient can be a correction parameter matched to the relative position of the first pixel within the repeating unit, to achieve differential correction based on position type, ensuring that pixels at the same relative position use the same correction logic. For example, the first correction coefficient can be retrieved or calculated from multiple sets of correction coefficients based on the local coordinates of the first pixel within its repeating unit.

[0085] The first correction coefficient is determined, for example, by identifying the local row and column index of the first pixel within its corresponding repeating unit, and selecting a matching correction coefficient from multiple sets of correction coefficients based on this local index. It is understood that by reusing the correction coefficients, the impact of image sensor resolution on the amount of stored correction coefficient data can be significantly reduced. With the repeating unit structure remaining unchanged, the required number of correction coefficient sets can also remain stable, thereby significantly improving the utilization rate of storage resources.

[0086] Step S130: Calculate the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient.

[0087] The compensation value can be a numerical increment or product factor used to adjust the original signal intensity of the first pixel, and can characterize the non-uniformity of pixel imaging caused by inconsistent response of the photosensitive unit of the first pixel. For example, the compensation value may include, but is not limited to, additive compensation value, multiplicative gain value, etc., which are not limited in this embodiment.

[0088] For example, the compensation value can be calculated by obtaining the absolute or relative coordinates of the first pixel within the entire image sensor array, combining them with a predetermined first correction coefficient, and then substituting them into a preset compensation calculation model. Furthermore, one or more of lens displacement variables and illumination distribution functions can be introduced to dynamically adjust the coefficient weights. Calculating the compensation value in this way not only considers the inherent inconsistencies in pixel response but also, by introducing a coordinate-related dynamic adjustment mechanism, can adapt to changes in illumination distribution caused by minute lens displacements due to optical image stabilization, preventing degradation of the correction effect.

[0089] In one possible implementation, the coordinates of the first pixel on the image sensor can be normalized coordinates. That is, the coordinates of each pixel in the image sensor are normalized, and then the compensation value corresponding to each pixel is calculated using the normalized coordinates.

[0090] Step S140: Correct the first pixel based on the compensation value corresponding to the first pixel.

[0091] To correct the first pixel, for example, the calculated compensation value can be applied to the original output value of the first pixel, and addition, gain multiplication or other operations can be performed to generate the corrected pixel value.

[0092] This embodiment provides a pixel correction method. The correction data includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. One set of correction coefficients is used to correct pixels at the same relative position in each repeating unit. That is, pixels at the same relative position in multiple repeating units reuse one set of correction data, without storing the compensation value corresponding to each pixel. In other words, pixel correction is changed from the traditional pixel-by-pixel independent storage mode to a grouped storage mode based on relative position, which can significantly reduce the storage space occupation and hardware cost of correction data.

[0093] In one embodiment, a repeating unit includes multiple pixel blocks, each pixel block corresponding to a color channel. A set of correction coefficients is used to correct pixels at the same relative position within each pixel block of the image sensor. A pixel block can be a sub-region within the repeating unit, divided according to the color filter array arrangement rules, and each pixel block includes multiple pixels. For example, it can correspond to a set or logical grouping of single-color photosensitive units, and can serve as the granularity unit for allocating correction coefficients, ensuring that pixels with the same relative position within the same color channel share the correction logic.

[0094] For example, pixel blocks can be obtained by dividing the physical pixels within a repeating unit into different logical groups according to the color filter mode of the image sensor. For example, a pixel block can be a logical set of red pixels, green pixels, or blue pixels in a Bayer array; in a specific embodiment, a pixel block can be a 2×2 cluster of pixels of the same color.

[0095] Color channels represent specific wavelength ranges. Since different colored photosensitive materials or filters have different sensitivities to light and noise characteristics, independent correction can be considered. Each pixel block corresponds to a color channel, thus allowing the differentiation of the response characteristics of different color components.

[0096] A set of correction coefficients is used to correct pixels at the same relative position in each pixel block of the image sensor. That is, the multiple sets of correction coefficients can be divided according to different pixel blocks. Each pixel block has the same number of correction coefficient sets, and the number of correction coefficient sets for each pixel block is the same as the number of pixels in that pixel block. Each pixel in a pixel block has a different correction coefficient. When determining the first correction coefficient corresponding to the first pixel, the pixel block in which the first pixel is located can be determined first, and then the first correction coefficient can be determined based on the relative position of the first pixel within that pixel block.

[0097] For example, a repeating unit consists of 4 pixel blocks, each corresponding to one of the 4 color channels. Each pixel block contains 16 pixels. The correction data includes 64 sets of correction coefficients, with 16 sets corresponding to each pixel block. The first set of correction coefficients corresponds to the first pixel in the pixel block, the second set corresponds to the second pixel, and so on, up to the 16th set, which corresponds to the 16th pixel in the pixel block.

[0098] For example, during the correction process, the color channel to which the first pixel belongs and its relative position within the corresponding pixel block can be identified. Then, a set of correction coefficients that match the position and are specific to that color channel can be selected from the correction data. For instance, if the first pixel is in the red channel and is located at the top left corner of the pixel block, the correction coefficient corresponding to the top left corner of the red channel can be used. It is understandable that by determining the correction coefficients based on both color channel and relative position, the unique non-uniformity of each channel can be compensated more accurately, avoiding color deviation or detail loss caused by cross-channel correction, thereby further improving the accuracy of color reproduction and the overall uniformity of the image.

[0099] This embodiment provides a pixel correction method that, through pixel blocks and color channels, refines the correction granularity from simple geometric position to a granularity combining geometric position and color attributes. By independently configuring correction coefficients for pixels at specific relative positions in each color channel, it can effectively solve the problem of color inhomogeneity caused by differences in the response characteristics of different color pixels while maintaining low storage costs and high resource utilization. Especially when dealing with complex lighting or lens shift scenes, the dynamic compensation of each channel can more accurately correct the brightness deviation of each color channel, preventing color cast or false color phenomena. Thus, while maintaining low hardware costs, it achieves the technical effect of improving the image quality and color consistency of color images.

[0100] In one embodiment, an image sensor is disposed on a camera module, the camera module further includes a lens assembly, and the correction data further includes a pre-trained model configured to obtain the correspondence between the relative position between the lens assembly and the image sensor and the correction coefficient offset.

[0101] The first correction factor is the correction factor when the lens assembly or image sensor is located at the reference position;

[0102] Based on the coordinates of the first pixel on the image sensor and the first correction coefficient, the compensation value corresponding to the first pixel is calculated, including:

[0103] Acquire the relative position information between the lens assembly and the image sensor;

[0104] The offset of the first correction coefficient is obtained based on the relative position information and the pre-trained model;

[0105] The target correction coefficient is determined based on the first correction coefficient offset and the first correction coefficient.

[0106] The compensation value corresponding to the first pixel is calculated based on the coordinates of the first pixel on the image sensor and the target correction coefficient.

[0107] The pre-trained model can be a mathematical mapping model established through offline training. It can characterize the nonlinear correspondence between the relative position between the lens assembly and the image sensor and the deviation of the correction coefficient, thus enabling real-time prediction of the dynamic adjustment amount of the correction parameters. For example, the pre-trained model can be obtained by acquiring standard whiteboard images under various lens displacement states, analyzing the pixel response change patterns, and training it using algorithms such as regression analysis. In an exemplary embodiment, the pre-trained model can include, but is not limited to, lightweight mathematical models such as multinomial fitting models, support vector machines, and neural networks.

[0108] In this embodiment, the first correction factor can be the correction factor when the lens assembly or image sensor is located at a reference position. The reference position can be an initial reference attitude in which the lens assembly or image sensor is set to the center focal length and has no image stabilization offset.

[0109] Relative position information can be a parameter describing the current spatial state of the lens assembly relative to the image sensor. In one exemplary embodiment, the relative position information may include the actual amount of displacement of the lens due to focusing or image stabilization. In another exemplary embodiment, the relative position information may include the actual amount of displacement of the image sensor due to focusing or image stabilization. Exemplarily, the relative position information can be determined by reading the lens drive current through a Hall sensor, an encoder, or based on feedback data from a displacement sensor. In this embodiment, the relative position information may include, but is not limited to, one or more of the following: X-axis displacement, Y-axis displacement, Z-axis displacement, tilt angle, etc.

[0110] It is understood that in this application, the camera module can achieve focusing or image stabilization functions by moving the lens assembly or image sensor. During focusing or image stabilization, the relative position between the lens assembly and the image sensor changes due to the movement of the lens assembly or image sensor. The relative position information in this application can be the displacement information of the lens assembly or the position information of the image sensor.

[0111] The first correction coefficient offset can be the offset value of the correction coefficient caused by the position of the lens assembly deviating from the reference position, representing the dynamic change in pixel response characteristics caused by the change in the optical path. For example, the first correction coefficient offset can be obtained by inputting relative position information into a pre-trained model, and the model outputting the corresponding coefficient deviation value. In a specific embodiment, the first correction coefficient offset may include, but is not limited to, an additive offset value, a multiplicative scaling factor, etc.

[0112] The offset of the first correction coefficient is obtained based on the relative position information and the pre-trained model. The obtained relative position information can be used as an input variable to be input into the pre-trained model for inference or table lookup, and the offset of the first correction coefficient corresponding to the position can be output.

[0113] The target correction coefficient can be determined by offset compensation of the first correction coefficient by the offset of the first correction coefficient, thereby providing an accurate correction coefficient that can adapt to the specific position of the current lens and ensure uniform correction under different optical conditions.

[0114] This embodiment provides a pixel correction method that obtains the relative position information between the lens assembly and the image sensor, obtains the first correction coefficient offset based on the relative position information and a pre-trained model, determines the target correction coefficient based on the first correction coefficient offset and the first correction coefficient, and calculates the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the target correction coefficient. A dynamic correction mechanism based on a pre-trained model is introduced. By monitoring the relative position between the lens assembly and the image sensor in real time and using the pre-trained model to predict the offset of the correction coefficient, the method accurately compensates for changes in illumination distribution caused by lens displacement due to optical image stabilization or autofocus. This not only retains the advantage of low storage overhead but also significantly improves the correction accuracy and robustness of the system in dynamic optical scenes, effectively overcoming the defect of image uniformity degradation in traditional static correction methods when the lens moves. This achieves the technical effect of high-precision, low-power, and highly adaptable pixel correction.

[0115] In one embodiment, before obtaining the coefficient offset based on relative position information and the pre-trained model, the method further includes:

[0116] Obtain the training dataset, which includes multiple sets of training data corresponding to multiple camera modules. Each set of training data includes the coefficient offset and position offset of the corresponding camera module at multiple preset positions. The coefficient offset is the difference between the correction coefficient of the camera module at the preset position and the reference correction coefficient; the position offset is the offset of the preset position relative to the reference position, and the reference correction coefficient is the correction coefficient of the camera module at the reference position; wherein, multiple camera modules are of the same model;

[0117] The prediction model is trained based on the training dataset to obtain a pre-trained model.

[0118] The training dataset is a collection of samples used to build and validate a pre-trained model. It contains paired data of input features and target labels. In this embodiment, it can provide learning materials for training the prediction model, enabling the model to capture the intrinsic relationship between lens displacement and changes in correction coefficients. For example, the training dataset can be obtained by collecting, cleaning, and formatting measured data from multiple camera modules of the same model under different physical conditions.

[0119] In this embodiment, multiple camera modules are of the same model. Camera modules of the same model refer to those containing identical hardware and undergoing the same assembly process. For example, camera modules of the same model have identical optical systems. For instance, they use lenses, infrared cut-off filters, and bases of the same specifications to ensure that optical transfer function, field of view, focal length, and other optical parameters have a unified design goal. Furthermore, the image sensors in camera modules of the same model are identical, meaning they use the same type of CMOS image sensor or other types of image sensor, with the same hardware and electrical parameters such as pixel size and full-well charge. Also, the actuators and driving methods for focusing, image stabilization, and other functions in camera modules of the same model are consistent. For example, to achieve focusing, image stabilization, and other functions, camera modules of the same model should use the same type of voice coil motor and other actuators, and the various parameters of the actuators should be consistent. Finally, the production process and calibration procedures for camera modules of the same model are consistent. Camera modules of the same model should have the same manufacturing process and calibration procedures, such as consistent mounting accuracy and white balance calibration procedures. For example, the software programs of camera modules of the same model should also be consistent. If the camera module needs to incorporate certain algorithms or other software programs, then the software programs of camera modules of the same model should be generated and stored using a unified process or specification.

[0120] For example, the camera module can obtain relevant operational data by moving the lens assembly to a specific position through a precision mechanical device in a controlled laboratory environment and collecting corresponding image response data.

[0121] Preset positions can be selected spatial coordinate points within the lens assembly's range of motion. These points can serve as sampling points, characterizing different states of the lens throughout the entire motion space and ensuring comprehensive coverage of the model training data. For example, depending on focusing and image stabilization requirements, preset positions can include one or more of the following: close-focus position, telephoto position, center balance position, and extreme image stabilization positions in various directions. Furthermore, preset positions can be achieved by positioning the lens to these discrete points using a high-precision shift stage or internal driver to cover key nodes in the focusing range and / or image stabilization range.

[0122] The coefficient offset can reflect the deviation of correction parameters caused by changes in pixel response characteristics with lens position, and can be used as a target variable for model training. In this embodiment, the coefficient offset can be the difference between the correction coefficient of the camera module at a preset position and the reference correction coefficient.

[0123] Position offset can be a spatial displacement vector describing the current preset position of the lens assembly relative to a reference position, representing the physical cause of the change in the correction coefficient. In this embodiment, the position offset is the offset of the preset position relative to the reference position. For example, depending on the displacement dimension, the position offset may include one or more linear displacements along the X-axis, Y-axis, and Z-axis.

[0124] The reference correction coefficient can be the initial correction parameter obtained by calibrating the camera module at the reference position. It can be used to provide a stable reference system, so that the coefficient offset has a clear physical meaning and comparability.

[0125] Pre-trained models can be mathematical mapping functions or neural networks that have been trained on a large number of samples and possess predictive capabilities. They can be used to establish nonlinear mapping relationships from position offsets to coefficient offsets, enabling dynamic correction parameter prediction without real-time calibration. For example, depending on the algorithm architecture, pre-trained models can employ one or more lightweight models such as multinomial fitting models, multilayer perceptrons (MLP), random forests, and support vector regression (SVR).

[0126] The prediction model is trained based on the training dataset. For example, the positional offsets in the training dataset can be used as input features and the coefficient offsets as output labels, which are then input into the initialized prediction model. Furthermore, the internal parameters of the model can be adjusted through backpropagation or other optimization algorithms to make the predicted offsets output by the model as close as possible to the true coefficient offsets.

[0127] This embodiment provides a pixel correction method that obtains a training dataset containing coefficient offsets and position offsets of multiple sets of camera modules of the same model at different preset positions. The prediction model is then trained offline using this dataset to obtain a pre-trained model. This model can accurately predict high-precision coefficient offsets based on real-time position offsets without requiring complex online full-range calibration for each individual device. This significantly reduces production calibration costs and time, and ensures that correction parameters are quickly and accurately adjusted when the lens shifts due to focusing or image stabilization, effectively maintaining image uniformity. This achieves the technical effect of low-cost, high-efficiency, and high-precision adaptive pixel correction.

[0128] In one embodiment, the process of obtaining the training dataset includes:

[0129] Based on multiple preset position offsets, the correction coefficient of the camera module at the corresponding preset position is obtained;

[0130] Based on the difference between the correction coefficient and the reference correction coefficient, the coefficient offset of the camera module at the corresponding preset position is determined;

[0131] Based on multiple pairs of position offsets and coefficient offsets of the camera module, the training data corresponding to the camera module is determined;

[0132] Multiple sets of training data are obtained based on the training data corresponding to multiple camera modules, which serve as the training dataset.

[0133] In this process, based on multiple preset position offsets, the correction coefficients of the camera module at the corresponding preset positions are obtained. For example, the lens assembly of the camera module can be controlled to move sequentially to the physical positions corresponding to the multiple preset position offsets, and a standard image can be acquired at each stable position and the current correction coefficient can be calculated, thereby establishing a measured correspondence between discrete position points and specific correction parameters.

[0134] Based on the difference between the correction coefficient and the reference correction coefficient, the coefficient offset of the camera module at the corresponding preset position is determined. For example, the coefficient offset specific to that position can be obtained by reading the correction coefficient at each preset position and subtracting the reference correction coefficient at the reference position, or by calculating the ratio of the correction coefficient at the reference position to the reference correction coefficient at the reference position. This can eliminate the inherent static non-uniform background of the pixel.

[0135] Based on multiple pairs of position offsets and coefficient offsets of the camera module, the training data corresponding to the camera module is determined. For example, the position offset of each preset position can be used as the input feature, and the calculated coefficient offset can be used as the output label to form multiple pairs of training samples for the camera module, so as to complete the conversion from the original measurement data to a format usable for machine learning.

[0136] Based on the training data corresponding to multiple camera modules, multiple sets of training data are obtained as training datasets. For example, the above process can be repeated to collect data from multiple camera modules of the same model, and the training data of all modules can be aggregated and merged to obtain a large-scale dataset for model training.

[0137] This embodiment provides a pixel correction method that obtains the correction coefficients of the camera module at corresponding preset positions based on multiple preset position offsets, determines the coefficient offset of the camera module at the corresponding preset positions based on the difference between the correction coefficients and the reference correction coefficients, determines the training data corresponding to the camera module based on multiple pairs of position offsets and coefficient offsets, and obtains multiple sets of training data as training datasets based on the training data corresponding to multiple camera modules. This not only ensures that the model can accurately capture the nonlinear mapping law between lens displacement and correction parameter changes, but also effectively suppresses the noise interference caused by individual manufacturing tolerances through the fusion of multi-module data. Thus, while reducing storage costs, it achieves high-precision, adaptive dynamic pixel correction, thereby improving the uniformity and stability of the imaging system in complex motion scenes.

[0138] In one embodiment, each set of training data also includes the feature parameters of the corresponding camera module, which are obtained by dimensionality reduction of the baseline correction coefficients.

[0139] Feature parameters can be low-dimensional data vectors used to characterize individual differences in camera modules. They can help the model distinguish the differences in manufacturing tolerances and optical properties of different camera modules, thereby improving the model's prediction accuracy for specific modules.

[0140] Dimensionality reduction can be achieved by taking the high-dimensional baseline correction coefficients of each camera module as input and using statistical methods, machine learning algorithms, etc., to map the baseline correction coefficients to a low-dimensional space, extracting the key feature components that best reflect the differences between modules, and generating corresponding feature parameters. Furthermore, dimensionality reduction can be implemented using algorithms such as Principal Component Analysis (PCA) and Singular Value Decomposition (SVD).

[0141] This embodiment provides a pixel correction method that enhances the pre-trained model's ability to perceive individual differences in camera modules by introducing feature parameters obtained from dimensionality reduction of the benchmark correction coefficients into the training data. The dimensionality-reduced feature parameters serve as the module's identity identifier or state supplement, enabling the model to dynamically adjust the prediction strategy according to the specific characteristics of the module. This further improves the accuracy of coefficient offset prediction, ensuring high-precision dynamic correction even on mass-produced modules with large manufacturing tolerances. At the same time, the dimensionality reduction technique controls the data scale, avoiding the computational burden caused by introducing additional parameters, thereby achieving the technical effect of improving accuracy and efficiency.

[0142] In one embodiment, the multiple preset positions include eight extreme positions within the working range corresponding to a focusing plane.

[0143] The focal plane can be the geometric plane in an optical system where light rays converge to form a sharp image. For example, the focal plane can be determined by adjusting the distance between the lens assembly and the image sensor, and thus defining the focal plane as the plane where the object is imaged most clearly at a given object distance.

[0144] The working range can be the complete physical travel range within a single plane that the lens assembly of the camera module is allowed to move, corresponding to the boundaries of lens movement. For example, the working range can be determined by the mechanical limits and electrical control range of the voice coil motor (VCM) or other drive mechanism.

[0145] Extreme positions can be spatial coordinates at the boundary extremes within the working range, corresponding to the maximum output of the driving capability. They can be used to capture changes in the optical characteristics of the lens under extreme conditions. Understandably, extreme positions are often accompanied by maximum aberrations and / or illumination inhomogeneity, thus providing important reference value for robust training of the model.

[0146] Determining multiple preset positions includes eight extreme positions within the working range corresponding to the focusing plane. For example, eight representative boundary points can be selected as preset positions within the working range corresponding to the focusing plane. For example, the eight extreme positions can include the maximum offset of the X-axis and Y-axis in two directions, and the combination of the maximum offset of the X-axis and Y-axis in their respective two directions.

[0147] By focusing on extreme positions for data acquisition and model training, the most significant optical distortions and correction coefficient deviations produced by the lens at maximum displacement can be captured. Since the performance of linear or nonlinear systems at extreme points often determines the overall dynamic range, covering these extreme positions can ensure that the pre-trained model has higher interpolation accuracy and generalization ability when dealing with arbitrary intermediate positions, while reducing redundant sampling of non-critical intermediate points and optimizing training efficiency.

[0148] This embodiment provides a pixel correction method that sets eight extreme positions within the focusing working range as key preset positions. This enables the pre-trained model to more accurately handle the correction requirements caused by large lens displacements. Compared to a large number of uniformly distributed sampling points, selecting extreme positions not only reduces the complexity of data acquisition and processing but also enhances the stability of the model under boundary conditions. This ensures that the camera module can still obtain high-precision correction coefficients through the prediction model in extreme usage scenarios such as fast focusing or severe image stabilization, thereby achieving the technical effect of reducing training computation while maintaining image quality.

[0149] In one embodiment, the plurality of preset positions include eight extreme positions within the working range corresponding to each of at least two focal planes.

[0150] It is understandable that there are at least two focusing planes, corresponding to different positions of the lens assembly on the optical axis, or different positions of the image sensor on the optical axis. For example, the at least two focusing planes include a near-focus plane, a middle-focus plane, and a telephoto plane. On the near-focus plane, the distance between the lens assembly and the image sensor is the shortest; on the telephoto plane, the distance between the lens assembly and the image sensor is the farthest; and on the middle-focus plane, the distance between the lens assembly and the image sensor is between that of the near-focus plane and the telephoto plane.

[0151] In this embodiment, the focal plane corresponds to a specific position of the lens assembly in the optical axis direction, which can be used to characterize the optical state of the camera module at different shooting distances. Differences in the optical path structure under different focal planes will lead to changes in pixel response characteristics.

[0152] In this embodiment, the multiple preset positions include at least two extreme positions of the focusing plane, which can reflect the coefficient offset caused by the lens assembly at different positions in the optical axis direction.

[0153] This embodiment provides a pixel correction method that, by selecting eight extreme positions under at least two focal planes, enables the pre-trained model to not only more accurately handle the correction requirements caused by large lens displacements on the same focal plane, but also to cope with coefficient shifts caused by different optical axis positions of the lens assembly. This ensures that the camera module can still obtain high-precision correction coefficients through the prediction model in more extreme usage scenarios, thereby achieving the technical effect of reducing training computation while maintaining image quality.

[0154] In one embodiment, the compensation value corresponding to the first pixel is calculated based on the coordinates of the first pixel on the image sensor and the first correction coefficient, including:

[0155] Construct a polynomial model using the first correction coefficient;

[0156] The coordinates of the first pixel on the image sensor are input into the polynomial model to obtain the compensation value corresponding to the first pixel.

[0157] The polynomial model can be a mathematical function expression used to describe the nonlinear mapping relationship between input and output variables. In this embodiment, the polynomial model can be used as an intermediate carrier for calculating the compensation value, transforming the discrete first correction coefficients into a continuous spatial distribution function, thereby enabling the dynamic generation of accurate compensation amounts based on arbitrary coordinate points. For example, the specific order of the polynomial model can be selected according to the complexity of lens distortion or illumination inhomogeneity.

[0158] A polynomial model can be constructed using the first correction coefficient. For example, the first correction coefficient can be used as a constant term of the polynomial or a coefficient of a specific order, and filled into a preset polynomial structure to construct a function expression with the image sensor coordinates as the independent variable, which serves as the polynomial model.

[0159] This embodiment provides a pixel correction method that replaces the traditional lookup table correction with a function-based analytical correction by constructing a polynomial model using the first correction coefficient. This retains the advantage of grouped storage to reduce space occupation and gives the correction process spatial continuity. It also achieves high-precision pixel-level correction with minimal computational cost. Combined with a pre-trained model, it can significantly improve the adaptability of the correction algorithm to dynamic changes in the optical system with almost no increase in storage burden, achieving a triple optimization of storage efficiency, computational efficiency, and correction effect.

[0160] In one embodiment, acquiring correction data includes:

[0161] Based on the sample images output by the image sensor under preset acquisition conditions, the pixel compensation matrix corresponding to the image sensor is calculated. The pixel compensation matrix includes the compensation value corresponding to each pixel of the image sensor.

[0162] Based on the number of pixels contained in the repeating unit, the pixel compensation matrix is ​​divided into multiple pixel compensation sub-matrices; each pixel compensation sub-matrice corresponds to a relative position in each repeating unit.

[0163] Corrected data is obtained by fitting multiple pixel compensation sub-matrices.

[0164] The preset acquisition conditions can be a standardized set of environmental parameters used to stimulate the uniform response characteristics of the sensor. For example, the preset acquisition conditions may include baseline acquisition conditions. For example, the parameters of the preset acquisition conditions may include one or more acquisition condition parameters such as environmental parameters, light source parameters, lens parameters, and temperature parameters. In some exemplary embodiments, the environmental parameters of the baseline acquisition conditions may be a standard environment, the light source conditions may be uniform surface light source illumination, the lens parameters may involve fixing the lens at a central focal position, and the temperature parameters may involve maintaining a constant temperature.

[0165] Sample images can be raw digital image data output by an image sensor under preset acquisition conditions. It is understood that a sample image can be a single image or multiple images. In an exemplary example, under baseline acquisition conditions, the sample image can be one or more RAW images captured by the sensor under standard conditions and uniform illumination.

[0166] The pixel compensation matrix is ​​a global two-dimensional array that matches the dimension of the sample image data, where each element corresponds to the compensation value of a single pixel. For example, this embodiment calculates the pixel compensation matrix in the following manner:

[0167] Method 1: Calculate the average value of all pixels within a single sample sub-block, and then calculate the compensation value of the corresponding pixel based on the average value and the original value of each pixel. The sample sub-block can adopt a 2×2, 4×4 or other N×N structure.

[0168] Method 2: Divide the sample image into large regions with a preset size, calculate the average value of each color channel and sub-position within each large region, and then calculate the common compensation value of the corresponding sub-position within the region based on the average value of the corresponding channel and the average value of the sub-position. The large region can be 128×128, 256×256, etc.

[0169] Based on any of the above methods, the compensation values ​​for each pixel in the entire image can be obtained, forming a pixel compensation matrix. Subsequently, based on the pixel coordinates and corresponding compensation values, various fitting methods such as polynomial fitting and function fitting can be used to construct a mapping relationship. Specific fitting algorithms include, but are not limited to, the least squares method, and this embodiment does not limit this.

[0170] A pixel compensation submatrix can be a data set with specific spatial sparsity extracted from the pixel compensation matrix. It is used to classify and reorganize global pixel compensation data according to their relative positions, ensuring high statistical similarity among the data within each submatrix. For example, the pixel compensation matrix can be separated by color channel, then further separated based on the specific position of the pixels within the unit. All elements belonging to the same preset pixel type are extracted and rearranged in order to form independent submatrices. This allows the spatial periodicity of the sensor array to be utilized to provide data support for fitting model coefficients based on the mapping relationship between pixel coordinates and compensation values, thereby achieving compression and regularization of large-scale correction data and reducing data redundancy. In an exemplary embodiment, the pixel array uses 2×2 as the smallest subunit. Four types of 2×2 subunits are spliced ​​together to form a 4×4 complete periodic unit, with each row arranged as RRGrGr, RRGrGr, GbGbBB, and GbGbBB. This periodic unit contains 16 independent local pixel positions, which are split to generate 16 pixel compensation submatrices. Each submatrix stores the compensation parameters for all pixels at the corresponding local coordinates.

[0171] The calibration data is obtained by fitting multiple pixel compensation sub-matrices. For example, by performing a fitting operation on the data in each pixel compensation sub-matrix, parameters representing the common features of the group of pixels can be extracted, such as mean, slope, curvature coefficient or piecewise linear nodes. Based on one or more of the above parameters, multiple sets of calibration coefficients are constructed. This achieves the removal of random noise and minor individual differences in the data through mathematical abstraction, while retaining the main systematic deviation features. A very small number of parameters are used to replace a large number of original compensation values, which significantly reduces storage requirements and improves the robustness of the calibration results.

[0172] This embodiment provides a pixel correction method that acquires sample images under preset acquisition conditions and calculates a full-size pixel compensation matrix to retain fine-grained information. Based on the repeating unit structure, the matrix is ​​separated into pixel compensation sub-matrices with statistical similarity to achieve data dimension transformation. Finally, a fitting algorithm is used to extract parameterized correction coefficients from the sub-matrices to remove noise and compress data. This method can reduce storage overhead while further eliminating measurement noise and random errors, making the generated correction coefficients more representative and stable. It can reduce data storage by several orders of magnitude while ensuring correction accuracy, thereby achieving the technical effect of optimizing hardware resource usage.

[0173] In one embodiment, the pixel compensation matrix corresponding to the image sensor is calculated, including:

[0174] Based on the resolution of the sample image, the sample image is divided into multiple sample sub-blocks; each sample sub-block includes multiple pixels.

[0175] Based on the initial value of each pixel, calculate the average value of all pixels in each sample sub-block;

[0176] The compensation value for each pixel is calculated based on the initial value of the pixel and the average value of the corresponding sample sub-block.

[0177] The pixel compensation matrix corresponding to the image sensor is obtained based on the compensation values ​​of multiple pixels.

[0178] The sample image can be the raw data matrix output by the target image sensor under preset acquisition conditions. It is understood that the sample image can be a single image or multiple images. In an exemplary embodiment, under baseline acquisition conditions, the sample image can be one or more RAW images captured by the sensor under uniform illumination in a standard environment.

[0179] Sample sub-blocks can be local rectangular regions cut from a two-dimensional pixel matrix based on the resolution of the sample image and a preset segmentation strategy. This allows for the capture of local illumination features or sensor response trends in different regions of the image, avoiding the masking of local differences by global averaging. For example, sample sub-blocks can be obtained by determining the segmentation boundaries based on the total resolution of the sample image and a preset sub-block size or number, and then cutting the complete sample image data stream into multiple independent sample sub-block data sets.

[0180] The initial value of a pixel can be the raw digital signal intensity of a single pixel in a sample image before any correction processing, and can be used to represent the actual response of that pixel to the current lighting conditions.

[0181] Calculate the average value of all pixels within each sample sub-block. For example, this can be achieved by summing the initial values ​​of all pixels within each sample sub-block and dividing by the total number of pixels in that sub-block. This can smooth out random noise in individual pixels and extract low-frequency background information that reflects lens vignetting, shadows, or differences in local sensor sensitivity.

[0182] Pixel compensation values ​​can be correction values ​​used to correct differences in pixel response. For example, they can be obtained by comparing the initial value of each pixel with the average value of its sample sub-block through simple operations such as subtraction or division.

[0183] The pixel compensation matrix can be a two-dimensional array with the same size as the sample image, where each element corresponds to the correction value of a pixel. For example, after calculating the compensation values ​​for all pixels, the array can be reassembled according to the spatial coordinates of the pixels in the original image to form a two-dimensional matrix structure with the same resolution as the original image.

[0184] In one specific embodiment, the sample image can be divided into blocks using N×N minimum repeating units as sample sub-blocks. The average value of all pixels within each sample sub-block is calculated. Based on this average value and the original values ​​of each pixel within the sub-block, a compensation value is calculated pixel by pixel. After traversing the entire image, a complete full-image pixel compensation matrix is ​​generated. The sample sub-blocks can adopt a 2×2 structure (for 50M), a 4×4 structure (for 200M), or other N×N structures.

[0185] In another specific embodiment, the sample image can be divided into blocks using preset large-size regions such as 128×128 and 256×256. Within each large region, the average value of the channel and the average value of the sub-position are calculated according to the color channel and sub-position. The common compensation value of the corresponding sub-position within the region is calculated based on the average value of the channel and the average value of the sub-position. After traversing block by block, a complete full-image pixel compensation matrix is ​​generated.

[0186] This embodiment provides a pixel correction method that separates a sample image into multiple sample sub-blocks, calculates the average value of the initial pixel values ​​in each sample sub-block to extract a local illumination reference, and calculates a compensation value for each pixel based on the initial pixel value and the local average value to eliminate common deviations and retain individual differences. Finally, a pixel compensation matrix is ​​assembled based on the compensation values ​​of multiple pixels to generate a complete correction mapping table. This method can accurately fit the spatial non-uniform illumination distribution caused by lens optical characteristics, while effectively separating the inconsistencies in the pixel's own response. Thus, while maintaining computational complexity, it improves the adaptability of the correction model to local illumination changes.

[0187] In one embodiment, the pixel compensation matrix corresponding to the image sensor is calculated, including:

[0188] Based on a preset block size, the sample image is separated into multiple sample image blocks; each sample image block includes multiple sample sub-blocks; each sample sub-block includes multiple pixels.

[0189] Based on the initial value of the pixel, the average pixel value of each color channel in the sample image block is calculated as the channel average value;

[0190] Based on the initial value of the pixel, the average value of the pixels at the same relative position in all sample sub-blocks within the sample image block is calculated as the sub-position average value;

[0191] Based on the channel average value and the sub-position average value, calculate the shared compensation value for the same relative position within all sample sub-blocks;

[0192] The pixel compensation matrix corresponding to the image sensor is obtained based on the shared compensation value of multiple sample image blocks.

[0193] In this embodiment, the sample image blocks can be independent processing regions divided according to preset geometric dimensions. This can be used to decompose a globally complex non-uniform distribution into locally approximately uniform units, which facilitates parallel processing and / or local statistical feature extraction.

[0194] A sample sub-block can be a structural unit that is further subdivided within a sample image block. Its structure can correspond to the repeating unit mentioned above. Random noise interference can be eliminated by extracting the local statistical regularity of pixels at the same relative position.

[0195] Based on a preset block size, the sample image is separated into multiple sample image blocks. For example, the sample image data can be read, and the image can be divided into multiple non-overlapping or partially overlapping sample image blocks according to the preset block size parameters using a sliding window or grid segmentation algorithm. The structure of the sample sub-blocks within each block can then be further identified.

[0196] The channel average can be the arithmetic mean of the initial values ​​of all pixels in a specific color channel within a sample image block. It can characterize the overall brightness level of that block under that color channel, thereby eliminating global illumination differences between blocks. For example, the channel average can be obtained by iterating through all pixels belonging to the same color channel within the block, summing them, and then dividing by the total number of pixels.

[0197] The sub-location average can be the average of pixels at the same relative position in all sample sub-blocks within a sample image block, reflecting the typical response characteristics of pixels at a specific relative position. For example, the sub-location average can be calculated by aggregating the pixel values ​​with the same coordinate offset in all sub-blocks within a block.

[0198] Based on the initial value of the pixel, the average value of the pixels at the same relative position in all sample sub-blocks within the sample image block is calculated as the sub-position average value. For example, within each sample image block, the pixels at the same relative position in all sample sub-blocks can be located, the initial values ​​of these pixels can be collected, and the average value can be calculated.

[0199] The shared compensation value can be a standardized correction parameter generated for a pixel at a specific relative position within a sample sub-block, which can be used to quantify the systematic response error of the pixel at that relative position.

[0200] Based on the channel average and sub-position average, a shared compensation value for the same relative position within all sample sub-blocks is calculated. For example, the channel average can be used as a global reference, compared to the sub-position average, and a scaling factor or difference can be calculated between the two to obtain the shared compensation value for that relative position. In an exemplary embodiment, if the sub-position average is lower than the channel average, the compensation value is biased towards signal enhancement; conversely, it is suppressed. This allows for the generation of standardized correction parameters for specific relative positions, ensuring consistent correction logic for pixels of the same type across different blocks, while adapting to local illumination variations.

[0201] Based on the shared compensation values ​​of multiple sample image blocks, the pixel compensation matrix corresponding to the image sensor is obtained. For example, by traversing all sample image blocks, the calculated shared compensation values ​​of each relative position are filled into the corresponding matrix units according to their absolute coordinate positions in the original image, thus forming a pixel compensation matrix covering the entire sensor.

[0202] This embodiment provides a pixel correction method that divides a sample image into multiple blocks and further subdivides them into sample sub-blocks. The average value of each channel and the average value of each sub-position are calculated, and then a common compensation value is derived and a pixel compensation matrix is ​​constructed. By using local statistical averaging, the interference of random noise on the calculation of correction parameters can be effectively suppressed, and the robustness of the correction data can be improved. At the same time, the block-based calculation strategy can reduce the amount of data processed in a single operation and improve the calculation efficiency of parameter generation. While achieving high-precision imaging uniformity correction, it can also achieve efficient utilization of computing and storage resources.

[0203] Please see Figure 4 , Figure 4 This is a flowchart illustrating a correction data acquisition method provided in an embodiment of this application, as shown below. Figure 4 As shown, the method includes, but is not limited to, the following steps:

[0204] S210: Based on the sample image output by the image sensor under preset acquisition conditions, calculate the pixel compensation matrix corresponding to the image sensor.

[0205] The pixel compensation matrix includes the compensation value corresponding to each pixel of the image sensor.

[0206] The preset acquisition conditions can be a standardized set of environmental parameters used to stimulate the uniform response characteristics of the sensor. For example, the parameters of the preset acquisition conditions may include one or more acquisition condition parameters such as environmental parameters, light source parameters, lens parameters, and temperature parameters. In some exemplary embodiments, the environmental parameters of the preset acquisition conditions may be a standard environment, the light source conditions may be uniform surface light source illumination, the lens parameters may involve fixing the lens assembly at the mid-focus position, and the temperature parameters may be maintaining a constant temperature.

[0207] The sample image can be a RAW image (i.e., a shading image) captured by the camera module under preset acquisition conditions and under uniform lighting. Each pixel in the sample image corresponds to a pixel value, which is the initial value of the pixel.

[0208] Furthermore, the specific implementation methods for calculating the pixel compensation matrix include:

[0209] Implementation Method 1: Based on the array structure of the image sensor, the sample image is divided into multiple sample sub-blocks; each sample sub-block includes multiple pixels; based on the initial value of the pixels, the average value of all pixels in each sample sub-block is calculated; based on the initial value of the pixels and the average value of the corresponding sample sub-block, the compensation value of each pixel is calculated; based on the compensation values ​​of multiple pixels, the pixel compensation matrix corresponding to the image sensor is obtained.

[0210] For example, based on the array structure of the image sensor, the sample image is divided into multiple N×N sample sub-blocks. For each sample sub-block, the average value of the N×N pixels within that sub-block is calculated. Then, based on this average value and the initial value of each pixel, the compensation value of each pixel is calculated.

[0211] Specifically, the compensation value of a pixel satisfies the following formula:

[0212]

[0213] Where si represents the compensation value of a pixel, mean represents the average value of N×N pixels in the sample sub-block, and xi represents the initial value of a pixel.

[0214] The compensation value of each pixel in the sample image is calculated based on the above method. The compensation values ​​of each pixel are used to form a pixel compensation matrix. The size of the pixel compensation matrix is ​​equal to the resolution of the image sensor.

[0215] For example, N can be 2, 3, or 4, that is, the above sample sub-block can be a 2×2 pixel block, a 3×3 pixel block, or a 4×4 pixel block.

[0216] In this implementation, the array structure of the image sensor is used to divide the sample into sub-blocks, and the compensation value of each pixel is calculated for each sample sub-block to refine the compensation and ensure local uniformity.

[0217] Implementation Method 2: Based on a preset block size, the sample image is divided into multiple sample image blocks; each sample image block includes multiple sample sub-blocks; each sample sub-block includes multiple pixels; based on the initial values ​​of the pixels, the average pixel value of each color channel within the sample image block is calculated as the channel average value; based on the initial values ​​of the pixels, the average pixel value at the same relative position within all sample sub-blocks within the sample image block is calculated as the sub-position average value; based on the channel average value and the sub-position average value, a common compensation value is calculated, which is the compensation value at the same relative position within each sample sub-block of the sample image block; based on the common compensation value of multiple sample image blocks, the pixel compensation matrix corresponding to the image sensor is obtained.

[0218] For example, the sample image is divided into blocks corresponding to four color channels, and each color channel corresponds to multiple N×N sample sub-blocks. The channel average value is the pixel average value in these multiple N×N sample sub-blocks, and the sub-position average value is the pixel average value at the i-th position in the multiple sample sub-blocks. The shared compensation value corresponds to the pixel at the i-th position in the multiple sample sub-blocks, that is, the pixel at the i-th position in the multiple sample sub-blocks corresponds to the same compensation value.

[0219] For example, the above-mentioned shared compensation value satisfies the following formula:

[0220]

[0221] Where s represents the shared compensation value, R represents the channel average value, and Ri represents the sub-position average value.

[0222] For example, the above sample image blocks can be 128×128 pixel blocks, 256×256 pixel blocks, or 512×512 pixel blocks, and the sample sub-blocks can be 2×2 pixel blocks, 3×3 pixel blocks, or 4×4 pixel blocks.

[0223] In this implementation, the sample image is divided into large sample image blocks. Within each sample image block, the same relative position in multiple sample sub-blocks corresponding to the same color channel uses the same compensation value, which improves global uniformity and facilitates higher fitting accuracy in subsequent processes. Furthermore, since multiple pixels share the same compensation value, there is no need to calculate the compensation value for each pixel individually, improving computational efficiency and saving computational resources.

[0224] S220, based on the number of pixels contained in the repeating unit, separates the pixel compensation matrix into multiple pixel compensation sub-matrices.

[0225] The number of pixel compensation sub-matrices is the same as the number of pixels in each repeating unit, and each pixel compensation sub-matrix corresponds to a relative position in each repeating unit. Each pixel compensation sub-matrix includes the compensation value at a relative position in each repeating unit. For example, the i-th pixel compensation sub-matrix corresponds to the i-th pixel in each repeating unit, and the j-th element in the i-th pixel compensation sub-matrix is ​​the i-th pixel in the j-th repeating unit.

[0226] In one possible implementation, the pixel compensation matrix can be separated into color compensation matrices for each color channel according to the color channel, and then the color compensation matrix can be separated into multiple pixel compensation sub-matrices according to the same relative position of multiple sample sub-blocks corresponding to each color channel.

[0227] For example, if a sample image corresponds to 4 color channels and each sample sub-block includes 4×4 pixels, the pixel compensation matrix can be separated into 4×16=64 pixel compensation sub-matrices.

[0228] S230, based on fitting multiple pixel compensation sub-matrices, obtains the correction data.

[0229] The correction data includes multiple sets of correction coefficients, each set of correction coefficients corresponding to a pixel compensation sub-matrix.

[0230] For example, for each pixel compensation submatrix, a set of correction coefficients is obtained by using the pixel coordinates (x, y) as variables and the compensation value corresponding to the pixel as the fitting target for polynomial fitting. These correction coefficients can characterize the mapping relationship between the coordinates of each pixel and the compensation value in the pixel compensation submatrix.

[0231] In some implementations, the coordinates of the aforementioned pixels can be normalized coordinates. For example, taking the center pixel of the sample image as (0,0), the top-left pixel as (-1,1), the top-right pixel as (1,1), the bottom-left pixel as (-1,-1), and the bottom-right pixel as (1,-1), normalize the coordinates of each pixel in the sample image to obtain normalized coordinates. By normalizing the coordinates of each pixel, the polynomial converges faster during the subsequent fitting process, optimizing the fitting effect.

[0232] For example, the fitting model for polynomial fitting can be a bivariate seventh-order polynomial, and the correction coefficients obtained after polynomial fitting can be a 36-dimensional coefficient vector. For instance, this polynomial can be expressed as:

[0233]

[0234] When the pixel compensation matrix is ​​separated into 64 pixel compensation sub-matrices, the correction data includes 64 × 36 data points. If each data point occupies 1.5 bytes, the correction data for a 200M image using this method occupies 64 × 36 × 1.5 = 3456 bytes, while the total data volume of the traditional correction method is 32 × 24 × 64 = 49152 bytes; for a 50M image, the total data volume is 16 × 36 × 1.5 = 864 bytes, while the total data volume of the traditional correction method is 16 × 12 × 16 = 3072 bytes.

[0235] Therefore, the correction method provided in this application can reduce the amount of stored correction data by an order of magnitude while ensuring high-precision mapping capability, which significantly alleviates the pressure on storage media capacity and cost.

[0236] Please see Figure 5 , Figure 5 This is a flowchart illustrating a model training method provided in an embodiment of this application. Figure 5 As shown, the method includes, but is not limited to, the following steps:

[0237] S310, Obtain the training dataset.

[0238] The training dataset includes multiple sets of training data, each corresponding to a different camera module. Each set of training data includes the coefficient offset and position offset of the corresponding camera module at multiple preset positions. The coefficient offset is the difference between the correction coefficient of the camera module at the preset position and the reference correction coefficient. The position offset is the offset of the preset position relative to the reference position, and the reference correction coefficient is the correction coefficient of the camera module at the reference position. All the camera modules are of the same model.

[0239] In one possible implementation, the process of obtaining the training dataset includes: obtaining the correction coefficients of the camera module at the corresponding preset positions based on multiple preset position offsets; determining the coefficient offset of the camera module at the corresponding preset positions based on the difference between the correction coefficients and the reference correction coefficients; determining the training data corresponding to the camera module based on multiple pairs of position offsets and coefficient offsets of the camera module; and obtaining multiple sets of training data based on the training data corresponding to multiple camera modules, which serve as the training dataset.

[0240] Specifically, for each camera module, it is positioned at multiple preset locations, and a RAW image under uniform lighting is captured at each preset location to obtain multiple RAW images.

[0241] For each RAW image, based on such Figure 4The method for acquiring correction data shown calculates multiple sets of correction coefficients for each RAW image, i.e., multiple sets of correction coefficients for each preset position. The difference between the multiple sets of correction coefficients for each preset position and the baseline correction coefficient is calculated to obtain the coefficient offset of the camera module at the corresponding preset position. The position offset and coefficient offset constitute a set of training data, where the position offset is the input data and the coefficient offset is the output data.

[0242] In one implementation, each set of training data also includes the feature parameters of the corresponding camera module, which are obtained by dimensionality reduction of the benchmark correction coefficients.

[0243] For example, the camera module corresponds to multiple sets of reference correction coefficients, and this feature parameter can be obtained by averaging the values ​​of each set of reference correction coefficients. For instance, the camera module corresponds to 64 sets of reference correction coefficients, each set of reference correction coefficients being 36-dimensional. Averaging the values ​​of each set of reference correction coefficients yields 64 data points, which are used to characterize the features of the camera module.

[0244] In one possible example, the multiple preset positions include eight extreme positions within the working range corresponding to a focal plane.

[0245] In another possible example, the multiple preset positions include eight extreme positions within the working range corresponding to each of the at least two focal planes.

[0246] S320 trains the prediction model based on the training dataset to obtain a pre-trained model.

[0247] In this embodiment, based on a deep learning model combined with lens offset and specific correction coefficients, the light field changes caused by lens displacement are dynamically adapted. This can overcome the problem of performance degradation of traditional static correction methods in optical image stabilization scenarios. Thus, while ensuring high imaging uniformity, it significantly improves resource utilization and the adaptability and imaging quality of image sensors in complex optical environments, achieving the technical effects of reducing storage space occupation, improving resource utilization, and preventing lens offset from causing correction effect degradation.

[0248] To more clearly illustrate the technical solution of this application, a detailed embodiment is also provided.

[0249] In image sensors based on high-pixel, 4×4 pattern architectures, inconsistencies in response exist between photosensitive units due to factors such as process deviations, material dispersion, and packaging stress. This is known as pixel response non-uniformity, which severely affects image uniformity. Without correction, image quality uniformity will be compromised, resulting in visually visible defects in low-light conditions and large areas of solid color. To address this non-uniformity at the module level, a mechanism for correcting spatial non-uniformity of the pixel array, namely QSC (Quadrature Spatial Correction), is employed.

[0250] Traditional QSC calibration schemes typically employ a block-based approach, dividing the entire sensor array into multiple blocks and storing the independent calibration coefficients for each block in EEPROM. While this method provides accurate compensation, the number of physical blocks required for traditional block-based calibration increases significantly as image sensor pixel sizes evolve from 50M to 200M. This method requires storing calibration coefficients independently for each block, and the storage overhead increases linearly with the number of blocks. For example, for a 50M pixel sensor, dividing it into 192 regions requires storing only about 3kB of data; however, for a 200M pixel sensor, dividing it into 768 blocks requires storing approximately 49.15 kB of calibration data. This need to store large amounts of discrete data not only increases the reliance on large-capacity off-chip EEPROMs but also directly leads to increased hardware costs, making it difficult to meet the low-cost, low-power requirements of mobile devices.

[0251] To overcome the aforementioned problem of excessive storage overhead, this embodiment proposes a correction scheme based on fitting an approximate polynomial. This embodiment abandons the approach of storing massive amounts of independent block coefficients, instead constructing a mapping relationship between pixel coordinates and their required compensation values. Through a fitting algorithm, a continuous function mapping relationship from pixel coordinates to their required correction values ​​is established, and this mapping relationship is represented as one or more sets of finite high-order polynomial coefficients. This method requires only a small amount of model parameters to calculate the compensation value in real time during the correction phase by substituting the corresponding pixel coordinates into this polynomial function. This achieves high compensation accuracy while reducing external storage requirements by an order of magnitude, thereby optimizing cost and power consumption while maintaining high-precision compensation capabilities.

[0252] In one embodiment, such as Figure 6 As shown, an image sensor pixel correction method is provided, which includes the following steps:

[0253] I. Calculation of benchmark correction coefficients.

[0254] During the mass production calibration phase, N modules of the same model and batch are sampled. Under standard conditions where the lens is in the mid-focus position and there is no optical shift, a set of reference correction coefficients is calculated for each camera module. These coefficients are then burned into the EEPROM of the corresponding module and used as the parameter reference for all subsequent real-time correction operations. The specific implementation process is as follows:

[0255] (a) Baseline data collection.

[0256] In a standard environment, the module under test is illuminated by a uniform surface light source, and the lens is fixed at the mid-focus position. This results in a RAW image (raw image without QSC shading) under uniform illumination, which is the reference image data, captured at the reference acquisition position.

[0257] (ii) Calculate pixel compensation gain.

[0258] For RAW images captured by the 200M chip, the sensor array structure is used to divide them into continuous 4×4 pixel blocks (2x2 for the 50M chip), which are preset repeating units. Larger blocks, such as 128×128, 256×256, 512×512, etc., can also be used. This embodiment only uses 4×4 pixel blocks as an example.

[0259] In one exemplary embodiment, such as Figure 7 As shown, the compensation value calculation can be performed by calculating the mean of all pixels within each 4x4 block, i.e., each sample sub-image data. For each pixel within the block with an initial value xi, its compensation value si = (mean - xi) / mean is calculated. By traversing the entire image, a pixel-level compensation value matrix with the same size as the sensor resolution can be obtained, i.e., the pixel compensation matrix.

[0260] In another exemplary embodiment, the complete image can first be divided into several 256×256 local regions according to a preset large area size. Within a single 256×256 region, pixel positions are divided into 4×4 basic sub-units, including 16 fixed sub-position pixel classes from R0 to R15, with Gr, Gb, and B color channels arranged synchronously. For each independent 256×256 region, the overall channel mean and the mean pixel value of each sub-position are calculated. The compensation coefficient of each sub-position within the region is obtained by calculating the formula s=(channel mean − sub-position mean) / channel mean (e.g., s=(R mean − R0 mean) / R mean). Within the same 256×256 region, pixels with the same sub-position share the same set of compensation values. After traversing the entire image region by region, a global compensation value matrix consistent with the full resolution size of the sensor is finally generated.

[0261] Currently, mainstream high-pixel imaging chips generally adopt a partitioned multiplexing architecture. Among them, 50M pixel specifications are mostly equipped with a 2×2 array architecture, and 200M pixel specifications are mostly equipped with a 4×4 array architecture. The N×N general array architecture that may be iterated in the future can all refer to the logic of this solution to complete the unified calculation of the full-image compensation value matrix.

[0262] Method 1: Using an N×N array as the smallest correction unit, the pixel mean is calculated for each unit, and the compensation parameters are solved pixel by pixel using the correction calculation formula. After traversing the entire domain, a complete full-image pixel compensation matrix is ​​generated.

[0263] Method 2: Divide the entire image into large blocks of 256×256 size (or larger). Within a single block, calculate the compensation coefficients corresponding to each fixed sub-position. Pixels arranged at the same sub-position share the same set of compensation coefficients. Traverse the entire image area block by block and finally output the full image correction compensation data.

[0264] After calculating the full-image compensation value matrix, according to the color channels and the N×N sub-position arrangement rules, the corresponding number of compensation matrices are extracted, resulting in a total of 4×N×N sets of compensation matrices. Polynomial fitting operations are performed on each set of compensation matrices to complete the modeling of global non-uniformity correction parameters.

[0265] Understandably, the first implementation method provides more refined compensation and better local uniformity, while the second implementation method offers higher stability and higher fitting accuracy.

[0266] (iii) Separation of compensation value matrix.

[0267] Based on the specific Bayer array pattern of the chip, the above full-image compensation value matrix is ​​separated by channel and position. First, it is separated by color channel, and then within each color channel, it is further separated according to the specific position of the pixel in the 4×4 merging unit. Finally, 4 (number of color channels) × 16 (number of relative positions) = 64 independent compensation value submatrices are obtained, namely pixel compensation submatrices.

[0268] (iv) Polynomial coefficient fitting.

[0269] Generate a bivariate high-order polynomial. For the 64 separated sub-matrices, calculate the original pixel coordinates, and fit the model using the compensation value s as the fitting target and its corresponding pixel coordinates (x, y) as independent variables. The fitting model uses a bivariate seventh-order polynomial:

[0270] s = A0 + A1x + A2y + A3x 2 +…+A 35 y 7

[0271] By fitting, a unique set of values ​​from A0 to A1 is obtained for each submatrix. 35 A total of 36-dimensional coefficient vectors are used to establish a mapping function from two-dimensional spatial coordinates to its exclusive compensation value for each type of sub-pixel. For 64 sets of sub-matrices, 64 sets of polynomial coefficients are recorded to obtain complete polynomial coefficient data (64x36) for a module, which is used to construct the pixel compensation mapping model.

[0272] Taking a 200M image as an example, the total data volume based on the approximate polynomial method in this embodiment is 64 × 36 × 1.5 = 3456 bytes, where 1.5 is the number of bytes occupied by data storage (2 bytes can also be used for storage), while the total data volume of the traditional block method is 32 × 24 × 64 = 49152 bytes; and in a 50M image, the total data volume of this embodiment is 16 × 36 × 1.5 = 864 bytes, while the total data volume of the traditional block method is 16 × 12 × 16 = 3072 bytes.

[0273] As can be seen, this embodiment abandons the traditional method of storing correction data in blocks and adopts a method of mathematically modeling the response characteristics of the entire sensor image. During the calibration stage, under ideal optical path conditions at the reference position (lens focal length and xy plane centered), the sensor response data is collected, the compensation value matrix of the entire image is calculated, and several sets of high-order polynomial coefficients that can characterize the compensation value matrix of the entire sensor response surface are generated through fitting algorithms. By functional modeling of coordinate-compensation values, while ensuring high-precision mapping capability, only a small number of polynomial coefficients need to be stored. Correction values ​​can be generated for pixels at any coordinate on the sensor by calculation, reducing the amount of correction parameter data that must be stored by an order of magnitude. This fundamentally solves the problem of huge storage overhead and significantly alleviates the pressure on storage medium capacity and cost.

[0274] The traditional block-based method and the approximate polynomial method mentioned above typically use data stored in the EEPROM acquired under ideal optical path conditions (i.e., the optical center of the test lens and the center of the sensor's photosensitive array are strictly aligned). However, in actual camera modules, due to the requirements of functions such as optical image stabilization (OIS) or autofocus (AF), the lens motor drives the lens group to undergo physical displacement, resulting in a non-linear change in the illuminance distribution projected onto the sensor. In this case, if fixed correction coefficients or polynomial coefficients based on the center position calibration are still used for correction, the correction effect may be reduced because these are calculated based on ideal center data.

[0275] To address this issue, a further embodiment provides a method for predicting QSC using AI adaptive coefficients, building upon the approximate polynomial method, and further introduces a lightweight AI prediction model. The core function of this model is to pre-train on data to learn the mapping relationship between the lens motor position and the required adjustment amounts of the polynomial coefficients, i.e., the coefficient correction amounts. During mass production, only a set of coefficients corresponding to the reference center position needs to be fixed in the EEPROM using the approximate polynomial method. During terminal device operation, the actual position code of the motor is acquired in real time and input into the built-in AI prediction model to instantly predict the corresponding coefficient correction amount. This correction amount is dynamically fused with the reference coefficients to generate optimal polynomial correction parameters adapted to the current lens position, realizing a two-level correction parameter generation system of reference modeling + AI dynamic prediction.

[0276] This embodiment's AI-adaptive coefficient prediction method for QSC aims to solve two major problems: the excessive data storage required by traditional block-based methods and the poor correction effect based on reference data when lens shift occurs. It mainly includes three stages: On the production line, a set of reference coefficients is calculated for each module under lens-off conditions as the basis for subsequent correction; on the training side, N typical modules are selected to train an AI prediction model, establishing a mapping relationship between lens shift and corresponding coefficient correction amounts; on the application side, the reference coefficients are read from the EEPROM, and the lens shift is detected in real time. The trained AI prediction model is then loaded to predict the coefficient correction amounts, and finally, high-precision real-time image correction is achieved by combining the reference coefficients. Figure 9 As shown, on the production line, after the module lens captures a shading image at a reference position, the reference coefficients for each module are calculated using the approximate polynomial method (including calculating the full image compensation value, separating the compensation matrix, and fitting the polynomial coefficients). The reference coefficients are then burned into the EEPROM. On the training side, N modules of the same model batch are selected, and training images are captured at different calibration positions. Training samples are constructed using the approximate polynomial method, the model is trained, and a model file is obtained. On the application side, the lens offset is detected, the reference coefficients in the EEPROM are read, and the model prediction coefficients offset are loaded, thereby achieving correction based on the reference coefficients + AI prediction offset.

[0277] During mass production, only a set of reference center position coefficients needs to be burned into the EEPROM. When the terminal device is actually running, the system acquires the lens motor position information in real time, inputs it into the AI ​​prediction model, predicts the corresponding coefficient correction amount, and combines it with the reference coefficients to generate the optimal correction parameters suitable for the current lens position. Therefore, this embodiment's solution, while possessing the core advantage of low storage overhead of the approximate polynomial method, can solve its sensitivity to lens offset, achieving adaptive and highly robust QSC correction.

[0278] The training process for lightweight AI prediction models includes:

[0279] (a) Sampling was conducted to select N modules of the same model and batch.

[0280] (ii) Adjust each sample module to various extreme offset positions (8 extreme directions) in the mid-focus state and the center reference position, and take the corresponding shading map.

[0281] 1. Multi-location calibration data acquisition.

[0282] like Figure 9 As shown, for each module, it is positioned at eight discrete points at the extreme positions within the working range of the XY plane, along with a central reference point. These eight discrete points serve as offset acquisition positions. A RAW image under uniform illumination is acquired at each preset position, ultimately constructing a training dataset containing lens position coordinates and corresponding response images, with a size of N×9 data sets. If AF is introduced, nine more position data sets are added for both near and far focus, resulting in a size of N×27 data sets. The following explanation uses a module with a 200M chip as an example.

[0283] 2. Calculation of the full set of position coefficients.

[0284] For each image captured in the dataset, the coefficient set of a bivariate seventh-order polynomial for each of the 64 sub-matrices was calculated using the approximate polynomial fitting method described above. Each shot position corresponds to a complete set of coefficients (64×36).

[0285] (iii) Using the approximate polynomial method, the coefficients of the nine position images are calculated and compared with the reference coefficients to obtain the coefficient deviation values, and the lens offset and coefficient deviation data pairs are compiled.

[0286] Using the central reference position coefficient set as a benchmark, the difference between each position coefficient set and the benchmark is calculated to obtain the coefficient correction amount ΔC, which is the coefficient residual data. Training samples are then constructed based on this: the inputs are the relative position of the lens (ΔX, ΔY) and the dimensionality reduction features of the benchmark position coefficients (features unique to each module), and the output is the corresponding coefficient correction amount.

[0287] (iv) Train the AI ​​prediction model with the data to establish a mapping relationship between the lens offset and the required coefficient compensation value.

[0288] like Figure 10 As shown, this embodiment uses a lightweight neural network as the mapping model, trained with the goal of minimizing prediction error. Regularization is used to prevent overfitting, and accuracy is evaluated on a test set to ultimately obtain the corresponding model file for this batch of modules.

[0289] In this embodiment, the lightweight neural network employs a fully connected feedforward neural network, composed of multiple stacked fully connected layers. Neurons in each layer are fully connected (i.e., every neuron in each layer is connected to all neurons in the layer above). Complex nonlinear mapping relationships are fitted using a nonlinear activation function, such as... Figure 8 As shown, the network contains multiple hidden fully connected layers for layer-by-layer transformation and abstraction of features, realizing a high-dimensional nonlinear mapping from input to output.

[0290] The lightweight neural network in this embodiment supports two input configurations, corresponding to 2D and 3D camera position shift scenarios, respectively. In the 2D camera position shift scenario, the input dimension of the lightweight neural network is 2 + 64 dimensions, corresponding to xy coordinates + 64-dimensional features. In the 3D camera position shift scenario, the input dimension of the lightweight neural network is 3 + 64 dimensions, corresponding to xyz coordinates + 64-dimensional features. The output dimension of the lightweight neural network is 64 × 36 dimensions, corresponding to the coefficient correction vector required for the task, used to complete the nonlinear regression from camera position shift to coefficient correction.

[0291] For 2D and 3D lens offset scenarios, the need to add the Z-axis dimension is determined by testing and verifying the correction results of the same module at near and far focal positions. Using a reference coefficient calibrated at the mid-focal position (Z-axis midpoint), QSC correction is performed on images taken by the same module at near and far focal positions to obtain zoom correction results. If the correction effect still meets the preset tolerance conditions, i.e., remains within the system tolerance range, it proves that the optical system of this module is relatively insensitive to changes in the focal plane (Z-axis displacement). For example, when testing modules corresponding to two chips with a 200M specification, the motor code position at near and far focal points is approximately 1000 to 1300 units away from the mid-focal point. Within this range, using the mid-focal calibration data effectively accommodates QSC correction scenarios at both near and far focal points, and the correction effect does not show significant attenuation. This indicates that the sensor response non-uniformity surface deformation introduced by Z-axis displacement is small and can be considered a secondary factor, which can be simplified or ignored in the AI ​​prediction model and actual correction process.

[0292] Understandably, from the perspectives of implementation complexity, model computation, and power consumption, building an AI prediction model that only predicts two-dimensional (XY) displacement requires less training data, has a simpler structure, and faster inference speed. Assuming the Z-axis influence is verified to be controllable, a 2D lens shift scenario can be used for training and correction, meaning multiple offset acquisition positions include multiple offset acquisition positions on a single focal plane. If the correction effect shows significant attenuation, a 3D lens shift scenario should be used for training and correction, meaning multiple offset acquisition positions include multiple offset acquisition positions on at least two focal planes.

[0293] The lightweight fully connected neural network in this embodiment, by performing a nonlinear regression from finite-dimensional lens position offset to a fixed-dimensional coefficient correction vector, can fully fit the mapping relationship, thus eliminating the need for CNNs or Transformers designed for image, sequence, or other data, and avoiding architectural redundancy. The inventors' research revealed that introducing a Transformer actually slightly reduces correction accuracy. Therefore, considering the power consumption and memory requirements of terminal devices, the lightweight fully connected neural network has significant advantages in terms of fewer parameters, computational efficiency, and ease of hardware deployment. Furthermore, under training conditions based on finite calibration data, the fully connected neural network is less prone to overfitting, providing more stable and predictable generalization performance, ensuring high robustness and reliability of the correction system in practical applications.

[0294] During the calibration application phase, optical calibration coefficients are dynamically calculated and applied based on the lens position. This primarily involves acquiring position feedback signals from the lens motor driver. This position information is input into the AI ​​prediction model, which immediately outputs the corresponding polynomial coefficient correction. Subsequently, these correction values ​​are fused in real-time with the reference center position coefficients read from memory to synthesize the optimal calibration parameters suitable for the current actual lens position. Figure 11 As shown, the specific process is as follows:

[0295] (1) Loading the benchmark coefficients and obtaining the dimensionality reduction features of the benchmark coefficients.

[0296] The system reads the reference correction coefficients calibrated at the factory from the EEPROM as the basis for subsequent dynamic compensation calculations.

[0297] (2) Position offset detection to obtain the lens offset.

[0298] Real-time detection of the physical offset of the lens assembly relative to its standard calibration position, and precise quantification of its offset direction and displacement.

[0299] (3) AI prediction model compensation calculation.

[0300] The detected offset parameters are input into the trained AI prediction model, which then calculates and outputs the optimal coefficient deviation compensation value for the current offset.

[0301] (4) Dynamic coefficient synthesis and loading are performed based on the predicted coefficient deviation to obtain the final correction coefficient.

[0302] The baseline coefficients are superimposed with the deviation compensation values ​​calculated by the AI ​​prediction model to generate the final correction coefficients. For each pixel, based on the final correction coefficients corresponding to each channel and sub-position combination and the spatial coordinates of each pixel, the pixel compensation value is calculated point by point and applied to the original data of that pixel to obtain the corrected image, thus completing the adaptive QSC correction. The specific process is as follows: For various channel sub-position combinations such as R0, the 36 final correction coefficients matching R0 are extracted first. The original coordinates of all pixels belonging to the channel sub-position are substituted into a binary seventh-order polynomial, and the compensation value corresponding to each pixel coordinate of R0 is obtained by combining the 36 final correction coefficients corresponding to R0. Finally, based on the operation relationship that the corrected pixel value is equal to the original value of the current pixel multiplied by one plus the compensation value, the data correction is completed pixel by pixel. The same applies to other channels and sub-positions to complete the adaptive correction of the non-uniformity of pixel response of the whole image.

[0303] like Figure 12 As shown, the uniformity of the correction strategies—no correction, reference correction, and reference + AI correction—is compared when lens shift occurs. Reference correction uses only the reference coefficients of the test module itself. Reference + AI correction, on the other hand, uses an AI prediction model trained by N training modules to predict coefficient deviations based on real-time lens shift, and then fuses these predictions with the reference coefficients for correction. It is evident that the image center region is more sensitive to shift, and the reference coefficients struggle to adapt to its dynamic changes, leading to a decrease in compensation effectiveness. Using a combination of reference coefficients and AI prediction coefficients achieves better correction results.

[0304] The solution in this embodiment can perform QSC correction not only on 200M images with a 4×4 architecture, but also on 50M chips with a 2×2 architecture. Compared with the traditional block-based method, the method in this embodiment can significantly reduce the amount of data stored in the EEPROM and provide better compensation when the lens shifts. It is understood that the method in this embodiment is not limited to a specific sensor model or resolution. This method is also applicable to other high-pixel sensors that use similar color filter array structures (such as 2×2, 4×4, and other array structures). It is only necessary to adjust the division method and related parameters of the compensation matrix according to the repeating unit structure of the pixel array, and adapt it to the multinomial fitting dimension and the input-output structure of the AI ​​prediction model.

[0305] This embodiment provides a pixel correction method that can resolve two inherent contradictions faced by existing image sensors, especially high-pixel N×N (2×2, 4×4) architecture chips, when implementing correction: the contradiction between high-precision correction requirements and limited storage resources, and the compatibility contradiction between static correction models with fixed parameters and dynamic optical systems supporting OIS / AF.

[0306] To address the issues of high hardware cost and high static power consumption caused by the heavy reliance on external large-capacity EEPROMs due to the storage of block correction coefficients in traditional block correction methods, this embodiment reduces the storage requirements of correction data by an order of magnitude (from approximately 49kB to approximately 3.5kB) through a parametric modeling method using approximate polynomial fitting. By establishing a binary high-order polynomial mapping relationship between pixel coordinates and compensation values, the capacity requirements and procurement costs of external storage chips can be significantly reduced, the overall power consumption of the system can be lowered, and a more compact packaging design can be achieved.

[0307] To address the issues of traditional block correction methods and approximate polynomial correction methods based on fixed-position calibration, which suffer from correction model failure and decreased image quality uniformity due to lens optical image stabilization (OIS) or autofocus (AF) displacement in practical use, this invention introduces an adaptive prediction mechanism. This mechanism enables the correction parameters to adapt to the actual physical position of the lens in real time, thereby ensuring high-precision and consistent correction results under various shooting conditions (especially when the lens is shifted). This improves the robustness and accuracy of the correction system in dynamic usage scenarios, ultimately enhancing the reliability and stability of the final image quality.

[0308] Specifically, in response to the problem that the compensation coefficient calculated based on the reference position significantly decreases when the lens physically shifts, this embodiment utilizes an AI prediction model. By training and learning the complex mapping relationship between the lens shift and the system correction coefficient, high-precision adaptive compensation for lens shift scenarios can be achieved.

[0309] In summary, this embodiment combines high-order polynomial compression storage with AI prediction model-enhanced offset compensation, enabling high-resolution chips to significantly reduce storage requirements and improve system robustness in real-world applications while maintaining correction accuracy.

[0310] It should be understood that, for modules corresponding to 200M specification chips, the order of compensation value calculation and matrix fitting can be as follows: first, calculate the compensation value of all pixels in the image to generate a full-image compensation value matrix; then, divide the full-image compensation value matrix into 64 sub-matrices, and perform polynomial fitting on each sub-matrix to obtain the corresponding correction coefficients. In some embodiments, the order can also be changed. For example, the entire image can be divided into 64 sub-matrices first, and then the compensation value of each pixel in each sub-matrix can be calculated pixel by pixel. It should be noted that the calculation of pixel compensation values ​​still needs to be based on the surrounding pixel data of the original image pixel, rather than on the pixel data in the current sub-matrix. Therefore, at least some steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0311] Based on the same inventive concept, this application also provides an apparatus for implementing the method described above. The solution provided by this apparatus is similar to the solution described in the above method; therefore, specific limitations in one or more apparatus embodiments provided below can be found in the limitations of the method described above, and will not be repeated here.

[0312] In one embodiment, such as Figure 13 As shown, this application provides a pixel correction apparatus for performing pixel correction on an image sensor. The image sensor includes multiple repeating units, each repeating unit including multiple pixels. The apparatus includes:

[0313] The data acquisition module 100 is used to acquire correction data, which includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. Each set of correction coefficients is used to correct pixels at the same relative position in each repeating unit.

[0314] The coefficient determination module 200 is used to determine the first correction coefficient corresponding to the first pixel point from multiple sets of correction coefficients based on the position of the first pixel point in the corresponding repeating unit.

[0315] The compensation calculation module 300 is used to calculate the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient.

[0316] The pixel correction module 400 is used to correct the first pixel based on the compensation value corresponding to the first pixel.

[0317] In one embodiment, a repeating unit includes a plurality of pixel blocks, each pixel block corresponding to a color channel, and a set of correction coefficients is used to correct pixels at the same relative position in each pixel block of the image sensor.

[0318] In one embodiment, the image sensor is disposed on a camera module, the camera module further includes a lens assembly, and the correction data further includes a pre-trained model configured to obtain the correspondence between the relative position between the lens assembly and the image sensor and the correction coefficient offset.

[0319] The first correction factor is the correction factor when the lens assembly or the image sensor is located at the reference position;

[0320] The compensation calculation module 300 is also used for:

[0321] Obtain the relative position information between the lens assembly and the image sensor;

[0322] The first correction coefficient offset is obtained based on the relative position information and the pre-trained model;

[0323] Based on the first correction coefficient offset and the first correction coefficient, the target correction coefficient is determined;

[0324] The compensation value corresponding to the first pixel is calculated based on the coordinates of the first pixel on the image sensor and the target correction coefficient.

[0325] In one embodiment, the pixel correction module further includes a model training module for:

[0326] A training dataset is obtained, comprising multiple sets of training data corresponding to multiple camera modules. Each set of training data includes coefficient offsets and position offsets of the corresponding camera module at multiple preset positions. The coefficient offset is the difference between the correction coefficient of the camera module at the preset position and the reference correction coefficient. The position offset is the offset of the preset position relative to the reference position, and the reference correction coefficient is the correction coefficient of the camera module at the reference position. The multiple camera modules are of the same model.

[0327] The prediction model is trained based on the training dataset to obtain the pre-trained model.

[0328] In one embodiment, the process of obtaining the training dataset includes:

[0329] Based on a plurality of preset position offsets, the correction coefficient of the camera module at the corresponding preset position is obtained;

[0330] Based on the difference between the correction coefficient and the reference correction coefficient, the coefficient offset of the camera module at the corresponding preset position is determined;

[0331] Based on multiple pairs of position offsets and coefficient offsets of the camera module, the training data corresponding to the camera module is determined;

[0332] Based on the training data corresponding to the multiple camera modules, multiple sets of training data are obtained as training datasets.

[0333] In one embodiment, each set of training data further includes feature parameters of the corresponding camera module, which are obtained by dimensionality reduction of the benchmark correction coefficients.

[0334] In one embodiment, the plurality of preset positions includes eight extreme positions within the working range corresponding to a focusing plane.

[0335] In one embodiment, the plurality of preset positions include eight extreme positions within the working range corresponding to each of at least two focusing planes.

[0336] In one embodiment, the compensation calculation module 300 is further configured to:

[0337] Construct a polynomial model using the first correction coefficient;

[0338] The coordinates of the first pixel on the image sensor are input into the polynomial model to obtain the compensation value corresponding to the first pixel.

[0339] In one embodiment, the data acquisition module 100 is further configured to:

[0340] Based on the sample image output by the image sensor under preset acquisition conditions, the pixel compensation matrix corresponding to the image sensor is calculated, and the pixel compensation matrix includes the compensation value corresponding to each pixel of the image sensor.

[0341] Based on the number of pixels contained in the repeating unit, the pixel compensation matrix is ​​divided into multiple pixel compensation sub-matrices; each pixel compensation sub-matrice corresponds to a relative position in each repeating unit.

[0342] The correction data is obtained by performing polynomial fitting on each of the pixel compensation sub-matrices.

[0343] In one embodiment, the data acquisition module 100 is further configured to:

[0344] Based on the array structure of the image sensor, the sample image is separated into multiple sample sub-blocks; each sample sub-block includes multiple pixels.

[0345] Based on the initial value of the pixel, the average value of multiple pixels in each sample sub-block is calculated respectively;

[0346] Based on the initial value of the pixel and the average value of the corresponding sample sub-block, calculate the compensation value for each pixel;

[0347] Based on the compensation values ​​of multiple pixels, the pixel compensation matrix corresponding to the image sensor is obtained.

[0348] In one embodiment, the data acquisition module 100 is further configured to:

[0349] Based on a preset block size, the sample image is separated into multiple sample image blocks; each sample image block includes multiple sample sub-blocks; each sample sub-block includes multiple pixels.

[0350] Based on the initial value of the pixel, the average pixel value of each color channel within the sample image block is calculated as the channel average value;

[0351] Based on the initial value of the pixel, the average value of pixels at the same relative position in all sample sub-blocks within the sample image block is calculated as the sub-position average value;

[0352] Based on the channel average value and the sub-position average value, calculate the shared compensation value for the same relative position within all the sample sub-blocks;

[0353] Based on the shared compensation values ​​of multiple sample image blocks, the pixel compensation matrix corresponding to the image sensor is obtained.

[0354] Each module in the aforementioned pixel correction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0355] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 14As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a pixel correction method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0356] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0357] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described... Figure 2 or Figure 4 or Figure 5 The method shown.

[0358] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program implementing the above-described functionality when executed by a processor. Figure 2 or Figure 4 or Figure 5 The method shown.

[0359] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0360] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0361] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0362] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A pixel correction method, characterized in that, The method is used to perform pixel correction on an image sensor, the image sensor comprising a plurality of repeating units, each of the repeating units comprising a plurality of pixels; the method includes: Obtain correction data, which includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. One set of correction coefficients is used to correct pixels at the same relative position in each repeating unit. For the first pixel in the repeating unit, based on the position of the first pixel in the corresponding repeating unit, the first correction coefficient corresponding to the first pixel is determined from the multiple sets of correction coefficients; Based on the coordinates of the first pixel on the image sensor and the first correction coefficient, calculate the compensation value corresponding to the first pixel; The first pixel is corrected based on the compensation value corresponding to the first pixel.

2. The method according to claim 1, characterized in that, A repeating unit includes multiple pixel blocks, each pixel block corresponding to a color channel, and a set of correction coefficients is used to correct pixels at the same relative position in each pixel block of the image sensor.

3. The method according to claim 1, characterized in that, The image sensor is mounted on the camera module, which also includes a lens assembly. The correction data includes a pre-trained model configured to obtain the correspondence between the relative position between the lens assembly and the image sensor and the correction coefficient offset. The first correction factor is the correction factor when the lens assembly or the image sensor is located at the reference position; The step of calculating the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient includes: Obtain the relative position information between the lens assembly and the image sensor; The first correction coefficient offset is obtained based on the relative position information and the pre-trained model; Based on the first correction coefficient offset and the first correction coefficient, the target correction coefficient is determined; The compensation value corresponding to the first pixel is calculated based on the coordinates of the first pixel on the image sensor and the target correction coefficient.

4. The method according to claim 3, characterized in that, Before obtaining the coefficient offset based on the relative position information and the pre-trained model, the method further includes: A training dataset is obtained, comprising multiple sets of training data corresponding to multiple camera modules. Each set of training data includes coefficient offsets and position offsets of the corresponding camera module at multiple preset positions. The coefficient offset is the difference between the correction coefficient of the camera module at the preset position and the reference correction coefficient. The position offset is the offset of the preset position relative to the reference position, and the reference correction coefficient is the correction coefficient of the camera module at the reference position. The multiple camera modules are of the same model. The prediction model is trained based on the training dataset to obtain the pre-trained model.

5. The method according to claim 4, characterized in that, The process of obtaining the training dataset includes: Based on a plurality of preset position offsets, the correction coefficient of the camera module at the corresponding preset position is obtained; Based on the difference between the correction coefficient and the reference correction coefficient, the coefficient offset of the camera module at the corresponding preset position is determined; Based on multiple pairs of position offsets and coefficient offsets of the camera module, the training data corresponding to the camera module is determined; Based on the training data corresponding to the multiple camera modules, multiple sets of training data are obtained as the training dataset.

6. The method according to claim 4, characterized in that, Each set of training data also includes the feature parameters of the corresponding camera module, which are obtained by dimensionality reduction of the benchmark correction coefficients.

7. The method according to claim 4, characterized in that, The multiple preset positions include eight extreme positions within the working range corresponding to a focusing plane.

8. The method according to claim 4, characterized in that, The multiple preset positions include eight extreme positions within the working range corresponding to each of the at least two focusing planes.

9. The method according to claim 1, characterized in that, The step of calculating the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient includes: Construct a polynomial model using the first correction coefficient; The coordinates of the first pixel on the image sensor are input into the polynomial model to obtain the compensation value corresponding to the first pixel.

10. The method according to claim 1, characterized in that, The acquisition of correction data includes: Based on the sample image output by the image sensor under preset acquisition conditions, the pixel compensation matrix corresponding to the image sensor is calculated, and the pixel compensation matrix includes the compensation value corresponding to each pixel of the image sensor. Based on the number of pixels contained in the repeating unit, the pixel compensation matrix is ​​divided into multiple pixel compensation sub-matrices; each pixel compensation sub-matrice corresponds to a relative position in each repeating unit. The correction data is obtained by performing polynomial fitting on each of the pixel compensation sub-matrices.

11. The method according to claim 10, characterized in that, The calculation of the pixel compensation matrix corresponding to the image sensor includes: Based on the array structure of the image sensor, the sample image is separated into multiple sample sub-blocks; each sample sub-block includes multiple pixels. Based on the initial value of the pixel, calculate the average value of multiple pixels within each sample sub-block; Based on the initial value of the pixel and the average value of the corresponding sample sub-block, calculate the compensation value for each pixel; Based on the compensation values ​​of multiple pixels, the pixel compensation matrix corresponding to the image sensor is obtained.

12. The method according to claim 10, characterized in that, The calculation of the pixel compensation matrix corresponding to the image sensor includes: Based on a preset block size, the sample image is separated into multiple sample image blocks; each sample image block includes multiple sample sub-blocks; each sample sub-block includes multiple pixels. Based on the initial value of the pixel, the average pixel value of each color channel within the sample image block is calculated as the channel average value; Based on the initial value of the pixel, the average value of pixels at the same relative position in all sample sub-blocks within the sample image block is calculated as the sub-position average value; Based on the channel average value and the sub-position average value, calculate the shared compensation value for the same relative position within all the sample sub-blocks; Based on the shared compensation values ​​of multiple sample image blocks, the pixel compensation matrix corresponding to the image sensor is obtained.

13. A pixel correction device, characterized in that, The apparatus is used for pixel correction of an image sensor, the image sensor including a plurality of repeating units, each of the repeating units including a plurality of pixels; the apparatus includes: The data acquisition module is used to acquire correction data, which includes multiple sets of correction coefficients. The number of sets of correction coefficients corresponds to the number of pixels contained in the repeating unit. One set of correction coefficients is used to correct pixels at the same relative position in each repeating unit. The coefficient determination module is used to determine, for the first pixel in the repeating unit, the first correction coefficient corresponding to the first pixel from the multiple sets of correction coefficients based on the position of the first pixel in the corresponding repeating unit; The compensation calculation module is used to calculate the compensation value corresponding to the first pixel based on the coordinates of the first pixel on the image sensor and the first correction coefficient. The pixel correction module is used to correct the first pixel based on the compensation value corresponding to the first pixel.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

Images