Image processing method and device

Abstract

The invention relates to an image processing method and device, and the method comprises the steps: obtaining a to-be-processed image, and calculating a first gain value of a pixel in the to-be-processed image; performing lens shadow correction processing on pixels in the to-be-processed image based on the first gain value to generate a first corrected image; and calculating a second gain value of a pixel in the to-be-processed image, taking the second gain value as a modulation parameter, and performing radial differentiated image enhancement processing on the first corrected image to obtain a second corrected image. According to the embodiment of the invention, the two-dimensional gain mapping table is constructed by adopting the unequally-spaced uniform sampling grids, and the storage requirement on the external expanded edge region is directly eliminated from the data source, so that the storage overhead is remarkably reduced. And by introducing boundary three-point cubic interpolation logic, on one hand, the high-precision characteristic of bicubic interpolation is maintained, and on the other hand, boundary correction errors possibly introduced due to misalignment of a gain value of an external expansion region are completely avoided.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to image processing methods and apparatus. Background Technology

[0002] In the field of digital imaging, due to the inherent physical characteristics of optical lenses and their interaction with image sensors, incident light undergoes a gradual attenuation as it propagates from the center of the image towards the corners. This phenomenon is known as lens shading (LS) or vignetting. This results in an image that is brighter at the center and darker around the edges, with varying degrees of attenuation across different color channels, severely impacting image uniformity and color fidelity. As the pixel count of image sensors used in mobile devices continues to increase while lens sizes become increasingly smaller, the lens shading effect becomes more pronounced. Therefore, efficient and high-precision lens shading correction of acquired raw images has become a fundamental and critical requirement in image processing pipelines.

[0003] To correct lens shading, related technologies generally employ a method based on a pre-calibrated two-dimensional gain mapping table combined with interpolation algorithms. Although nonlinear interpolation algorithms such as bicubic can improve the grid-like artifacts caused by bilinear interpolation, in actual hardware implementation, these technologies need to expand the gain mapping table to process image edge pixels. This not only increases storage overhead but may also introduce boundary errors due to inaccurate gain values ​​in the expanded region. Summary of the Invention

[0004] This application provides an image processing method and apparatus.

[0005] According to a first aspect of the embodiments of this application, an image processing method is provided, the method comprising: The image to be processed is acquired, and the first gain value of the pixels in the image to be processed is calculated; wherein the first gain value is calculated based on a pre-calibrated two-dimensional gain mapping table of non-uniformly spaced uniform sampling grid points and a cubic interpolation algorithm; Based on the first gain value, lens shading correction processing is performed on the pixels in the image to be processed to generate a first corrected image; Calculate the second gain value of the pixels in the image to be processed; wherein the second gain value is based on a one-dimensional gain mapping table and obtained by querying the distance from the pixel to the optical center of the image, the optical center of the image corresponds to the projection of the optical center of the lens onto the image plane, and the one-dimensional gain mapping table is used to characterize the mapping relationship between the gain value and the distance from the pixel to the optical center of the image; Using the second gain value as a modulation parameter, radial differential image enhancement processing is performed on the first corrected image to obtain the second corrected image.

[0006] According to a second aspect of the embodiments of this application, an image processing apparatus is provided, the apparatus comprising: The image acquisition module is used to acquire the image to be processed. The first gain value calculation module is used to calculate the first gain value of the pixels in the image to be processed; wherein, the first gain value is calculated based on a pre-calibrated two-dimensional gain mapping table of non-uniformly spaced uniform sampling grid points and a cubic interpolation algorithm; The first image correction module is used to perform lens shading correction processing on the pixels in the image to be processed based on the first gain value, and generate a first corrected image; The second gain value calculation module is used to calculate the second gain value of the pixels in the image to be processed; wherein, the second gain value is based on a one-dimensional gain mapping table and obtained by querying the distance from the pixel to the optical center of the image, the optical center of the image corresponds to the projection of the optical center of the lens onto the image plane, and the one-dimensional gain mapping table is used to characterize the mapping relationship between the gain value and the distance from the pixel to the optical center of the image; The second image correction module is used to perform radially differentiated image enhancement processing on the first corrected image by using the second gain value as a modulation parameter to obtain the second corrected image.

[0007] According to a third aspect of the embodiments of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0008] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described above in this application.

[0009] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described above in this application.

[0010] The image processing method and apparatus provided in this application acquire an image to be processed and calculate a first gain value of the pixels in the image to be processed; based on the first gain value, lens shading correction processing is performed on the pixels in the image to be processed to generate a first corrected image; by calculating a second gain value of the pixels in the image to be processed, and using the second gain value as a modulation parameter, radial differential image enhancement processing is performed on the first corrected image to obtain a second corrected image. The embodiments construct a two-dimensional gain mapping table using a non-uniformly spaced uniform sampling grid without edge expansion, directly eliminating the storage requirement for the external edge expansion region from the data source, thereby significantly reducing storage overhead. Furthermore, by introducing boundary cubic interpolation logic, the edge interpolation problem, which traditionally relies on physical edge expansion data, is transformed into an internal logical operation that dynamically calculates virtual points using mathematical functions based on the gain of internal effective grid points. This maintains the high accuracy of bicubic interpolation while completely avoiding boundary correction errors that may be introduced due to inaccurate gain values ​​in the external expansion region, thus achieving a balance between storage efficiency and correction accuracy while ensuring the quality of image edge correction. This effectively solves the storage overhead and boundary error problems caused by the expansion of the gain mapping table in related technologies. Attached Figure Description

[0011] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 A flowchart of an image processing method provided as an exemplary embodiment of this application; Figure 2 A schematic diagram of the overall system architecture and data flow provided for an exemplary embodiment of this application; Figure 3 A schematic diagram of the overall architecture and data flow of an image processing system provided as an exemplary embodiment of this application; Figure 4 A schematic diagram of an interpolation unit in a single direction at an image edge, provided as an exemplary embodiment of this application; Figure 5 A schematic block diagram of the functional modules of an image processing apparatus provided in an exemplary embodiment of this application; Figure 6 A structural block diagram of an electronic device provided in an exemplary embodiment of this application; Figure 7 A structural block diagram of a computer system provided for an exemplary embodiment of this application. Detailed Implementation

[0012] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0013] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0015] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more". The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0016] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0017] Due to the structural characteristics of optical lenses and image sensors, according to the cosine attenuation law, light incident on an imaging system exhibits near-axially symmetrical attenuation from the image center to the corners. This phenomenon is known as lens shading or vignetting. In the past, lens shading was primarily caused by inherent properties of optical lenses (such as off-axis illumination and vignetting). However, with the increasing use of megapixel or even multi-megapixel CMOS (Complementary Metal-Oxide-Semiconductor) image sensors in smartphones, and the increasingly smaller lenses mounted on them, the shading effect has become more pronounced. Furthermore, because pixels of different colors on a color filter array do not possess completely uniform attenuation curves, image processing pipelines typically require preprocessing of the acquired raw images to correct corner shading using an LSC module in order to obtain a globally uniform image with brightness and color independent of the scene.

[0018] To correct lens shading, a gain map of the same size, independent of space and color (though it may be scene-dependent, such as the color temperature of the light source), needs to be applied to the image. A common approach is to interpolate and complete the two-dimensional gain table (Gain Table) after downsampling the Gain Map point-by-point. The two-dimensional gain table can be obtained through offline calibration or real-time software calculation. Mesh grid-based methods are a simple and effective implementation of LSC (Lens Shading Correction). This involves calibrating the gain at sparsely sampled grid points in the image and then interpolating at the smallest grid cell where the pixel is located. LSC based on bilinear interpolation can produce grid-like artifacts at the corners of the image when lens shading is severe. To address this issue without increasing memory consumption, newer nonlinear interpolation algorithms, such as bicubic interpolation, can be used. In addition to satisfying the internal correspondences in this complete lens shading correction process, its interactive effects on the entire image processing pipeline and imaging system must also be considered. Firstly, when using bicubic interpolation to process image edges, it is necessary to provide enough four interpolation units in the horizontal or vertical direction. Generally, a two-dimensional gain mapping table needs to be configured after expanding the edge beyond the original image. The number of rows and columns expanded is one to two, depending on the application. Secondly, when the image scene has temporal field of view changes for the same scene, if there is a process that supports the cropping and scaling before the LSC module in the image pipeline, it is generally necessary to perform adaptive re-interpolation adaptation on the two-dimensional gain mapping table generated at the size of the source sensor output image. Thirdly, due to lens shading and its correction, radial processing is usually required at the four corners of the image with relatively low signal-to-noise ratio to obtain more satisfactory local and global image quality.

[0019] To address the aforementioned technical problems, this application aims to provide an efficient and universal lens shading correction interpolation method and construct a unified image pipeline radial processing framework. To achieve this, this application first proposes a strategy based on relative position sampling. By introducing the concept of virtual grid points, the edge expansion processing, which traditionally requires external processing within the two-dimensional gain mapping table, is moved to the hardware interpolation logic. This significantly reduces the configuration size and storage overhead of the gain mapping table while maintaining correction accuracy.

[0020] Furthermore, this application embodiment proposes to convert the two-dimensional gain mapping table for lens shading correction into a radial one-dimensional gain mapping table, or to support manual adjustment and configuration of this one-dimensional table. This one-dimensional gain mapping table can capture the gain attenuation characteristics related to the optical center distance in the image, thereby providing radial modulation reference information for subsequent modules in the image processing pipeline (such as denoising, sharpening, or image enhancement modules). This assists these modules in achieving differentiated processing adapted to the characteristics of the image region, ultimately improving the overall image quality.

[0021] Therefore, in order to provide an efficient, accurate, and hardware-friendly solution for lens shading correction and image radial processing, this application first provides an image processing method, such as... Figure 1 As shown, the method may include the following steps: In step S110, the image to be processed is acquired, and the first gain value of the pixels in the image to be processed is calculated.

[0022] The first gain value is calculated based on a pre-calibrated two-dimensional gain mapping table of non-uniformly spaced uniform sampling grid points and a cubic interpolation algorithm.

[0023] In this step, the first gain value can be calculated using a cubic interpolation algorithm based on a pre-calibrated, non-uniformly spaced, uniformly sampled grid table without margin expansion. "Non-uniformly spaced sampling" is an innovative sampling strategy that uses a fixed number of grid points on the image and integer parameters and fixed-point calculations to ensure uniform grid point distribution across the entire image, but the spacing between adjacent grid points is not constant. Compared to traditional fixed-interval sampling, this method provides higher effective sampling density and adaptability with the same number of grid points, making it particularly suitable for scenarios such as image scaling. "No margin expansion" means that the two-dimensional gain map table does not need to fill in additional gain values ​​at the image boundaries, significantly reducing storage space usage.

[0024] In this embodiment, the cubic interpolation algorithm used to calculate the first gain value includes special optimizations for image edge processing, namely "boundary three-point cubic interpolation logic". When the pixel to be processed is located at the edge of the image and cannot provide four complete reference grid points, this logic introduces a virtual grid point and expresses the gain value of this virtual point as a function of the gains of three actual effective grid points (e.g., d = a - 3(b - c)). Thus, high-quality cubic interpolation can be completed using only three effective points, effectively avoiding image edge correction errors.

[0025] In step S120, lens shading correction processing is performed on the pixels in the image to be processed based on the first gain value to generate a first corrected image.

[0026] This step is the application phase. The first gain value calculated for each pixel in step S110 is multiplied by the original value of the corresponding pixel in the image to be processed (e.g., sensor data in the RAW domain) or other forms of gain application operation are performed. Through this processing, the brightness attenuation and color inconsistency problems in the center and periphery of the image caused by lens shading can be directly compensated, thereby outputting an intermediate image that is more uniform in brightness and color, namely the first corrected image.

[0027] In step S130, the second gain value of the pixels in the image to be processed is calculated.

[0028] The second gain value is obtained based on a one-dimensional gain mapping table and by querying the distance from the pixel to the optical center of the image. The optical center of the image corresponds to the projection of the optical center of the lens onto the image plane. The one-dimensional gain mapping table is used to characterize the mapping relationship between the gain value and the distance from the pixel to the optical center of the image.

[0029] In this embodiment, the second gain value can be obtained based on a one-dimensional gain mapping table and by querying the distance from the pixel to the optical center of the image (corresponding to the projection of the lens optical center onto the image plane). The core of this one-dimensional gain mapping table is that it establishes a mapping relationship between the "gain value" and the "distance from the pixel to the optical center," which stems from the optical principle that lens shadows attenuate radially.

[0030] In this embodiment, the one-dimensional gain mapping table can be converted from the aforementioned two-dimensional gain mapping table by software. The conversion process typically includes: for each sampling distance in the one-dimensional table, calculating the coordinates of the corresponding points in the original image in multiple directions (e.g., 8 or 12 directions) emanating from the optical center; then interpolating the gain of these points from the two-dimensional gain mapping table; and finally calculating the average value as the one-dimensional gain value for that distance. Furthermore, it also supports manual configuration adjustments, providing high flexibility.

[0031] During the query process, hardware optimization can be used. For example, the initial grid point index of the image to be processed relative to the optical center can be pre-initialized in software. Then, based on the relative position of the pixel point to the optical center, the index can be incremented or decremented iteratively to quickly locate the grid interval corresponding to the current pixel in the one-dimensional gain mapping table. Finally, the accurate second gain value can be obtained through interpolation calculation.

[0032] In step S140, the second gain value is used as a modulation parameter to perform radial differential image enhancement processing on the first corrected image to obtain the second corrected image.

[0033] In this embodiment, this step enables the reuse and expansion of lens shading correction information across a broader image processing pipeline. The second gain value calculated here is no longer directly used for pixel value correction, but is instead passed as a "modulation parameter" or "reference information" to other image enhancement modules in the image processing pipeline, such as a denoising module, a sharpening module, or an overall image enhancement module.

[0034] The second gain values ​​received by these modules reflect the original degree of shadow attenuation in different regions of the image (based on their distance from the optical center). Based on this information, the modules can dynamically and differentially adjust their processing intensity. For example, in the originally darker corner areas of the image (corresponding to higher LSC gain), the denoising intensity can be reduced to minimize detail loss, or the sharpening intensity can be moderately increased to enhance visual appeal. In this way, "radial differentiation" processing of the first corrected image is achieved, ultimately outputting a second corrected image that is not only uniformly bright but also radially optimized in overall image quality, significantly improving both local and global visual effects.

[0035] The image processing method provided in this embodiment accurately completes lens shadow correction through non-equal interval sampling and boundary interpolation, and transforms the correction information into radial modulation parameters, thus establishing the connection between lens shadow correction and subsequent image enhancement processing, and providing a unified and efficient solution for pipelined image radial processing.

[0036] Based on the above embodiments, in another embodiment provided in this application, the method may further include the step of generating the one-dimensional gain mapping table, and the method may further include the following steps: In step S150, for each sampling distance in the one-dimensional gain mapping table, the coordinates of the corresponding points in the original image in multiple directions originating from the optical center are calculated.

[0037] This step is the preparatory stage for constructing a one-dimensional gain mapping table. First, the required sampling distance sequence for the one-dimensional gain mapping table is determined. This sequence defines the radial distance from the image's optical center (i.e., the projection of the lens's optical center onto the sensor plane). For each sampling distance in the sequence, the specific coordinates of multiple sampling points corresponding to that distance on the original image plane need to be calculated. These sampling points are typically evenly distributed along a circle centered on the optical center, with the number of directions being 8, 12, or other reasonable configurations to ensure that the gain from different directions at that distance is adequately represented.

[0038] In step S160, the gain values ​​of corresponding points in each direction are obtained by interpolation from the two-dimensional gain mapping table based on the coordinates.

[0039] After obtaining the coordinates of the sampling points in each direction, this step performs gain value mapping. Since these coordinate points may not directly fall on the non-uniformly spaced grid points pre-marked in the two-dimensional gain mapping table, interpolation calculations are needed using the established two-dimensional gain mapping table. Specifically, the same cubic interpolation algorithm (including its boundary handling logic) as used to calculate the first gain value is employed to interpolate the theoretical gain value for each corresponding position in each direction from the two-dimensional gain mapping table based on these coordinates. This process ensures the accuracy and consistency of the data source for the conversion from a two-dimensional table to a one-dimensional table.

[0040] In step S170, the gain values ​​in each direction are averaged, and the average value is used as the one-dimensional gain value for the corresponding sampling distance to obtain a one-dimensional gain mapping table.

[0041] This step completes the aggregation of one-dimensional gain data. For the same sampling distance, the gain values ​​calculated in step S160 in all directions (e.g., 8 or 12 directions) are averaged arithmetically or in other forms. This average comprehensively reflects the overall level of lens shadow gain in different orientations at that radial distance, thus effectively "compressing" or "projecting" the two-dimensional gain information into one-dimensional information that is only related to distance. Finally, an average gain value is assigned to each sampling distance, thereby completely constructing the mapping relationship from "radial distance" to "average gain," i.e., the one-dimensional gain mapping table.

[0042] The steps described in this embodiment systematically and automatically derive a one-dimensional gain mapping table suitable for radial processing, with a smaller data volume and hardware-friendly architecture, from an accurate but large-scale two-dimensional gain mapping table. This lays a solid foundation for achieving efficient pipelined radial image processing. This method ensures data accuracy while significantly optimizing storage and access efficiency.

[0043] Based on the above embodiments, in another embodiment provided in this application, the process of calculating the second gain value of the pixels in the image to be processed is further refined. By employing a hardware-optimized index iteration method to quickly locate the position of the pixel in the one-dimensional gain mapping table, the above step S130 may further include the following steps: In step S131, the initial grid point indices of the image to be processed relative to the optical center of the image in the horizontal and vertical directions are initialized.

[0044] In this embodiment, this step is a preprocessing stage for efficient querying. Before starting pixel-by-pixel processing of the image to be processed, an initial grid point index is pre-calculated and set by software. Here, "grid point index" specifically refers to the index of the one-dimensional gain map, which essentially sorts and stores gain values ​​according to their radial distance from the optical center. The initialization process determines an initial index value, which typically corresponds to the grid interval of the one-dimensional gain map where pixels in a specific region of the image (e.g., the starting corner or center region) are most likely to fall radially. This step provides an accurate starting point for the rapid indexing iteration of all subsequent pixels.

[0045] In step S132, based on the relative positional relationship between the pixels in the image to be processed and the optical center of the image in the horizontal and vertical directions, the initial grid point index is iteratively incremented or decremented to locate the grid interval corresponding to the pixel in the one-dimensional gain mapping table.

[0046] This step abandons the traditional, computationally intensive binary search method. Image processing is performed pixel-by-pixel in a specific order (e.g., raster scan order). When processing one pixel and preparing to process the next adjacent pixel, the system logically determines whether the radial distance of the current pixel should increase or decrease based on the relative positions of these two pixels with respect to the optical center in the horizontal (X-axis) and vertical (Y-axis) directions. Specifically: If the next pixel is farther from the optical center than the current pixel, then the index of the currently maintained grid point is incremented.

[0047] If the next pixel is closer to the optical center than the current pixel, then the currently maintained grid point index is decremented.

[0048] The iterative approach described above allows for rapid and low-power tracking and updating of the current pixel's position within the grid interval of the one-dimensional gain map. This method is suitable for hardware pipeline implementation and can significantly reduce computational latency and logical complexity.

[0049] In step S133, the second gain value is calculated by interpolation based on the located grid interval.

[0050] In this embodiment, after quickly locating the grid interval where the current pixel is located through step S132, this step performs the final gain value calculation. The gain value at the boundary of the grid interval (e.g., the gain at the start and end of the interval) is read from the one-dimensional gain map table, and then the final second gain value is calculated using an interpolation algorithm (e.g., simple linear interpolation, or cubic interpolation consistent with the aforementioned embodiment) based on the relative position of the current pixel within the interval according to its precise radial distance.

[0051] This embodiment provides an indexing and locating method that far surpasses the efficiency of traditional binary search by combining initialization with iteration, greatly optimizing the query process for the second gain value, making it particularly suitable for deployment in hardware image processing pipelines with high real-time requirements.

[0052] Based on the above embodiments, in another embodiment provided in this application, in order to further improve the flexibility and debugging friendliness of the one-dimensional gain mapping table in practical applications, the method may further include the following steps before performing step S130: In step S134, the one-dimensional gain mapping table is modulated, and this modulation is used to adjust the gain curve by configuring the guard radius and guard gain parameters.

[0053] In this embodiment, this step introduces an intuitive and effective parametric modulation mechanism, enabling engineers to fine-tune the one-dimensional gain curve automatically generated based on optical principles according to specific scenarios or subjective image quality preferences.

[0054] Protection Radius: This parameter defines a circular region around the center of the image. Pixels within this region have their radial distance considered zero, or their gain values ​​are subject to special processing logic. The purpose of setting the protection radius is to acknowledge that in areas very close to the optical center, lens shading effects are negligible, and excessive gain correction may introduce noise or cause center overexposure. This parameter ensures that the image quality in the center area is not negatively affected by the correction process.

[0055] Protection Gain: This parameter typically defines the target gain value that should be applied within the protection radius (or at its boundary points). For example, it can be set to 1.0 (i.e., no additional gain is applied), or a value slightly greater than 1.0 to slightly brighten the center, resulting in a more natural overall brightness transition in the image.

[0056] In this embodiment, the modulation process can be understood as follows: based on the two higher-order parameters of the protection radius and protection gain configured by the user, the system internally calculates (for example, by smoothly transitioning or directly replacing the gain values ​​in the intervals with radii smaller than the protection radius in the original one-dimensional gain mapping table) to generate a new, modulated one-dimensional gain mapping table. This process frees technicians from the tedious work of directly modifying a large number of discrete gain values, allowing them to quickly and flexibly "shape" the ideal radial gain curve by controlling a few parameters with clear physical or visual significance.

[0057] This embodiment not only enhances the practicality and adaptability of the one-dimensional gain mapping table algorithm, but also greatly simplifies the later debugging process, providing an important guarantee for obtaining optimal image quality under different product platforms and scenarios.

[0058] Based on the above embodiments, in another embodiment provided in this application, the method for determining the non-uniformly spaced sampling grid points is specifically described. Its core lies in efficiently generating a uniformly distributed but variable-interval sampling grid through integer and fixed-point calculations. The aforementioned non-uniformly spaced sampling grid points are determined in the following way: (1) Configure a preset number of grid points and initialize the grid size parameters in each direction by integerization.

[0059] In this embodiment, firstly, a preset number of grid points (e.g., 23 points horizontally and 17 points vertically) can be configured for the width and height directions of the image, based on the required precision and hardware resources. Then, integer operations are used to initialize the basic grid size parameters for each direction. Specifically, the total size of the image (such as the number of pixels in the image width or height) is divided by the corresponding number of grid points minus one (because the grid points are distributed on the boundaries) to obtain a baseline grid spacing. The term "integer operation" here means that integer operations are primarily used throughout the calculation process, which brings significant efficiency advantages to the subsequent hardware implementation, avoiding complex floating-point arithmetic units.

[0060] (2) Introduce the fixed-point parameter of the reciprocal of the segment number, calculate and generate the coordinate table of each grid point, and obtain the grid points with uniform distribution and non-equal grid spacing.

[0061] In this embodiment, this step is crucial for achieving "non-uniformly spaced distribution". To achieve a uniform distribution of grid points under integer operations, while allowing for subtle, non-fixed variations in grid spacing, the concept of a "fixed-point parameter of the reciprocal of the number of segments" is introduced.

[0062] Segment number: can be understood as the granularity of further subdividing the baseline grid spacing.

[0063] Fixed-point conversion parameter: Converts the reciprocal of the segment number (a decimal less than 1) into a fixed-point number (e.g., a Q-format fixed-point number) so that the decimal can be used in calculations in the integer arithmetic unit.

[0064] When calculating the coordinates of each grid point, iterative calculations can be performed using the baseline grid spacing and the aforementioned fixed-point parameters. In this way, the calculated grid point coordinates can macroscopically and uniformly cover the entire image area. However, due to the truncation or rounding characteristics of integer operations and the fine-tuning of the fixed-point parameters, the actual pixel spacing between adjacent grid points will dynamically change within a very small range (e.g., between grid_size and grid_size+1), thus achieving the characteristic of "non-uniform spacing." Finally, all these calculated coordinates are organized into a grid point coordinate table.

[0065] This embodiment can efficiently generate an optimized sampling mesh that maintains global uniformity while enhancing its adaptability through non-uniform intervals. More importantly, the entire process is based on integer and fixed-point computation, and the generated coordinate table can be directly used in hardware logic without storing massive amounts of floating-point coordinate data. In some implementations, coordinates can even be calculated in real time using the same logic without storing the coordinate table, thus greatly saving storage resources and computational power consumption.

[0066] Based on the above embodiments, in another embodiment provided in this application, an optimized solution is provided for processing image edge regions, specifically for calculating the first gain value based on a two-dimensional gain mapping table and a cubic interpolation algorithm. By introducing the "boundary three-point cubic interpolation logic" of virtual grid points, the traditional method avoids the need for physical expansion of the gain mapping table while ensuring high-precision correction. Therefore, the method may further include the following steps: In step S135, if there are no complete four reference grid points in the edge region of the image to be processed, the gain values ​​of three valid grid points are obtained from the two-dimensional gain mapping table.

[0067] In this embodiment, this step is the triggering of boundary processing and data preparation. When it is necessary to calculate the first gain value of a pixel located at the edge or corner of the image to be processed, the system first identifies that the local region where the pixel is located cannot provide the 16 reference grid points (4 in each direction, 4 in the horizontal and 4 in the vertical) typically required for bicubic interpolation. Specifically, in the current interpolation direction (horizontal or vertical), only 3 valid, calibrated grid points may be found. At this time, the system accurately reads the gain values ​​stored at these 3 available valid grid points from the unexpanded two-dimensional gain mapping table as the core interpolation input data.

[0068] In step S136, virtual grid points are introduced, and the gain value of the virtual grid points is expressed as a function of the gain of the three effective grid points.

[0069] In this embodiment, in order to construct the four reference points necessary for cubic interpolation even with only three physical grid points, a "virtual grid point" is introduced. The gain value of this virtual point is not pre-stored, but is dynamically calculated and generated from the gain values ​​of the three valid grid points obtained in step S135 using a defined mathematical function.

[0070] For example, a typical functional relationship can be: Virtual point gain d = a - 3 (b - c), where a, b, and c represent the gain values ​​of three effective grid points arranged from the image boundary inwards. This formula is based on reasonable assumptions and mathematical derivations about the gain variation trend at the boundary, and aims to predict the virtual gain value outside the boundary with high fidelity using known internal gain points, thereby forming a complete interpolation unit.

[0071] In step S137, the first gain value is obtained by three interpolation logic calculations based on the gain values ​​of the three valid grid points and the gain value of the virtual grid points.

[0072] After successfully constructing a complete four-point reference set containing three real points and one virtual point, this step performs a standard bicubic interpolation process (cubic interpolation in a single direction). The gain values ​​of these four points are substituted into the cubic interpolation formula, and the weight coefficients calculated using the relative position of the pixel to be interpolated in the current grid cell are used to finally synthesize the accurate first gain value of the edge pixel.

[0073] This embodiment can successfully solve the problem of high-quality interpolation correction of image edges. The embodiment not only eliminates the storage overhead and calibration complexity caused by physically expanding the gain table, but also effectively prevents image edge correction distortion or artifacts caused by inaccurate external gain values.

[0074] like Figure 2 As shown, Figure 2 This diagram illustrates the overall system architecture and data flow of the image processing method provided in this application embodiment. As shown, the complete process covers all stages from offline calibration and online adaptive processing to hardware pipeline execution, specifically including: (1) Calibration of non-equidistant relative positions without expansion: This is the primary offline preparation stage. The system calibrates the image sensor to obtain non-uniformly spaced sampling grid points based on the concept of relative position and their corresponding two-dimensional gain mapping table. The core innovation of this step is that the generated gain mapping table is borderless, meaning there is no need to fill the image with virtual gain values, which fundamentally reduces the amount of calibration data and the subsequent storage space required.

[0075] (2) Adaptive LSC algorithm processing: The online software scheduling module is triggered when the camera system starts up or the scene changes. This module takes statistical data (such as color temperature and brightness) collected from the image pipeline as input and calls the adaptive LSC algorithm library. Based on the aforementioned calibration data, this algorithm library can dynamically calculate or select the optimal two-dimensional gain mapping table according to the current environment to achieve more accurate lens shading correction.

[0076] (3) Boundary three-point LSC(2D) Bicubic interpolation: This is the core correction step, typically executed in a hardware pipeline. The system utilizes the two-dimensional gain mapping table output from the adaptive stage to perform pixel-by-pixel lens shading correction on the input raw image. For pixels within the image, standard bicubic interpolation is used; while for image edge regions, the boundary tricubic interpolation logic proposed in this application's embodiments is applied. This logic, by introducing virtual grid points, successfully solves the problem of not being able to perform complete bicubic interpolation in edge regions due to the lack of four reference points. Thus, without the need for an external gain table, high-quality, grid-artifact-free correction is achieved across the entire image range, and a first corrected image is output.

[0077] (4) Online software scheduling: Online software scheduling refers to the process in which a software module coordinates and makes decisions during the operation of the camera system (online), dynamically selecting or calculating the most suitable lens shading correction parameters based on the current actual shooting scene and environment.

[0078] Its core purpose is to achieve "adaptive" lens shading correction, so that the correction effect will not deteriorate due to changes in ambient light.

[0079] Specifically, its working mechanism and included elements are as follows: 1) Triggering time: When the camera is started, or when a significant change in the scene is detected during the shooting process (such as moving from indoors to outdoors, or the light source changing from fluorescent lights to sunlight).

[0080] 2) Input Information: This scheduling module receives data from the front-end statistics module of the image processing pipeline, mainly including: Ambient color temperature: Whether the light source is warm (such as tungsten lamp) or cool (such as blue sky).

[0081] Scene brightness: The overall light intensity of the environment.

[0082] Light source type identification, etc.

[0083] 3) Core action: In the embodiment, the scheduling itself does not perform calculations, but acts like a "scheduler" to call a suitable algorithm or parameter set from a pre-prepared "adaptive LSC algorithm library" based on the above input information.

[0084] 4) After scheduling and executing the algorithm, a better two-dimensional gain mapping table will be generated. This newly generated gain mapping table matches the current environment and provides better correction than using a fixed default gain table.

[0085] (5) 2D to 1D parameter transfer and conversion on the Pipeline: This module embodies the system-level innovation of this application's embodiments. While initializing the two-dimensional LSC correction parameters, the online software performs a parameter transfer and transformation process. Specifically, it converts the gain mapping table (2D) used for two-dimensional correction into a radial one-dimensional gain mapping table (1D), or directly loads the manually configured one-dimensional table. This transformation bridges the gap from precise pixel-level correction to macroscopic radial processing.

[0086] (6) Module calls LSC(1D) interpolation gain radial modulation: Finally, in subsequent nodes of the image processing pipeline, such as denoising, sharpening, or enhancement modules, this one-dimensional gain mapping table is invoked. These modules obtain their gain values ​​through interpolation based on the distance from the currently processed pixel to the optical center. This gain value is not used as a direct correction coefficient, but rather as a radial modulation parameter to dynamically adjust the processing intensity of each module when processing the first corrected image (e.g., appropriately reducing denoising intensity at image edges to preserve details), ultimately achieving differentiated optimization processing for different radial regions and outputting a second corrected image with better image quality.

[0087] Based on the above embodiments, in the embodiments provided in this application, the image processing system achieves efficient lens shading correction and radial processing through a combination of software and hardware. The entire system's workflow comprises three main stages: offline preparation, online initialization, and hardware processing.

[0088] During the offline preparation phase, the system obtains two-dimensional gain mapping tables based on non-uniformly spaced sampling grid points without extended edges under different light source conditions through a calibration process. These mapping tables provide the basic data for subsequent real-time processing.

[0089] During the online initialization phase, after the camera starts, the system first collects statistical information from the lens shading correction node in the image pipeline as input. Based on reference information such as the current ambient light source color temperature and brightness, the system can call the adaptive algorithm library to obtain a two-dimensional gain mapping table more suitable for the current scene. Next, the online software completes two key initialization tasks: first, configuring the hardware register parameters for two-dimensional lens shading correction; and second, initializing the one-dimensional gain mapping parameters required by each radial processing module. For the one-dimensional gain mapping table, the system provides two generation methods: converting the two-dimensional gain mapping table through software, or directly loading the manually configured one-dimensional gain mapping table.

[0090] In the hardware processing stage, the image pipeline executes the following processes sequentially: First, the lens shading correction module uses an initialized two-dimensional gain map table and a bicubic interpolation algorithm containing boundary tricubic interpolation logic to calculate the first gain value pixel by pixel and perform precise lens shading correction on the input image, generating the first corrected image. Subsequently, each radial processing module in the pipeline (such as the denoising module and the enhancement module) queries the initialized one-dimensional gain map table based on the distance from the currently processed pixel to the optical center of the image to obtain the second gain value as a modulation parameter. This second gain value is not directly used for pixel value correction, but rather as a radial modulation coefficient to dynamically adjust the processing intensity of each module on the first corrected image, thereby achieving differentiated image optimization based on radial position, and finally outputting a fully processed second corrected image.

[0091] Specifically, such as Figure 3 As shown, Figure 3 This diagram illustrates the overall architecture and data flow of the image processing system provided in this embodiment. The system employs a design that combines software and hardware, as well as offline and online processing, to achieve a complete pipeline from raw image acquisition to high-quality correction and radial optimization. The entire system's workflow can be divided into three stages: offline preparation, online initialization and scheduling, and hardware pipeline processing, which are described in detail below: (1) Offline preparation and online initialization phase.

[0092] The system begins with "Sensor raw and parameter input". This path contains two parallel workflows: Offline calibration: In a factory or laboratory environment, the system uses a uniform light source to illuminate the image sensor and acquire raw RAW data. Based on this data, a two-dimensional gain mapping table without margin expansion and other sampling and configuration parameters are generated using the non-uniformly spaced uniform sampling method described in this application embodiment. These parameters will be stored for subsequent use by the camera.

[0093] Online Software Scheduling: Upon camera startup or operation, the system enters the online phase. The module performing "Image Information Statistics" analyzes statistical data (such as scene color temperature and average brightness) from the image pipeline in real time. This statistical information is transmitted to the module performing "Online Software Scheduling" via the "Information Interaction" channel. This module then triggers the "Adaptive LSC Software Algorithm," using the aforementioned statistical information and offline calibration data as input to dynamically calculate or select the most suitable two-dimensional gain mapping table for the current shooting environment, achieving adaptive lens shading correction.

[0094] (2) Hardware pipeline processing stage.

[0095] After initialization, the image data enters the core hardware processing pipeline: Boundary Three-Point LSC(2D) Bicubic Interpolation: This is the first core processing node on the mainline. It receives the raw image from the sensor and loads a two-dimensional gain mapping table optimized by online software scheduling. For each pixel in the image, it applies the boundary three-point cubic interpolation algorithm described in this invention to calculate an accurate first gain value, which is then directly applied to the correction of the RAW image, outputting a first corrected image with uniform brightness and color.

[0096] 2D to 1D Parameter Transfer and Conversion on the Pipeline: The online software performs parameter conversion while initializing the 2D correction parameters. Specifically, it converts the 2D gain map used for precise pixel correction into a 1D gain map suitable for macroscopic radial processing. This conversion establishes a bridge from "geometric position" to "radial distance".

[0097] The module calls LSC(1D) interpolation gain radial modulation: Subsequent image processing modules (such as denoising, sharpening, enhancement, etc.) can call this one-dimensional gain mapping table. These modules look up the one-dimensional table based on the distance from the currently processed pixel to the optical center and obtain the second gain value through interpolation. This gain value is not used as a direct correction coefficient, but as a radial modulation parameter to dynamically adjust the processing intensity of each module on the first corrected image, thereby achieving radial differentiation optimization of the image.

[0098] Information statistics and processing, and image processing and adjustment: These modules represent the later processing nodes in the pipeline. They can use a one-dimensional gain mapping table for radial modulation, and may also generate new statistical information (such as local image contrast and noise level) to feed back to the system for more advanced image processing and adjustment, ultimately generating and outputting a second corrected image with the best image quality.

[0099] This embodiment demonstrates a highly efficient and unified lens shading correction and radial image processing system. The system is based on offline calibration, dynamically adapts through online software scheduling, and performs lossless edge expansion with high precision on the hardware pipeline using boundary three-point interpolation. Innovatively, it uses 2D-to-1D parameter conversion to empower subsequent processing modules with the correction information, achieving global image quality optimization through radial modulation. The entire architecture ensures end-to-end image quality improvement from source to end.

[0100] Therefore, based on the above embodiments, the main components of the embodiments of this application include, but are not limited to, an integer-based non-uniformly spaced uniform sampling method, a boundary three-point cubic interpolation algorithm, a conversion implementation from a two-dimensional gain mapping table to a one-dimensional gain mapping table, and a design of a one-dimensional LSC interpolation gain modulation based on the conversion to establish the connection between lens shading and pipeline image radial processing. Specifically: (1) Initialize the grid parameters. The system configures a fixed number of grid points in the width and height directions of the image. Based on this number, the basic grid size parameters in each direction are initialized through integer division operations. This design ensures that subsequent calculations can be performed within an efficient integer arithmetic domain.

[0101] Secondly, a fixed-point parameter is introduced to accurately calculate the grid coordinates. To achieve a non-uniformly spaced distribution of grid points, this invention introduces a fraction parameter, which is a fixed-point representation of the reciprocal of the number of grid point segments. Using this fixed-point parameter, combined with the initialized base grid size, the system iteratively generates the integer coordinates of each grid point in the image coordinate system, thus forming a grid point coordinate table. The grid points generated by this method are macroscopically uniformly distributed across the entire image, while microscopically the spacing between adjacent grid points dynamically changes between grid_size and grid_size+1, forming the aforementioned non-uniformly spaced characteristic while maintaining good central symmetry.

[0102] Regarding hardware optimization, the system provides two strategies. First, the hardware interpolation logic can directly read this pre-calculated coordinate table. Second, to save storage space, the hardware can also utilize the same integer-based iterative logic as the coordinate table generation to calculate the coordinates of each grid point in real time, without needing to store the complete coordinate table. Furthermore, the actual grid point spacing (i.e., grid size) can be dynamically obtained by calculating the difference between the coordinates of adjacent indexed grid points. Since this spacing varies only within a finite range of values, this feature can also be used for information encoding to transmit grid distribution information to the hardware in a more compact manner.

[0103] Finally, in the offline calibration stage, the calibration system uses the grid point coordinate table generated by the above method to locate the local area corresponding to each grid point on the uniform reference image output by the image sensor. By calculating the ratio of the brightness of the area to the brightness of the central reference, the gain value of the grid point position is generated, and the construction of the edgeless two-dimensional gain mapping table is finally completed.

[0104] (2) In the bicubic lens shadow interpolation correction process, based on the characteristic of reading the image data stream line by line, the vertical interpolation of the image is performed first, and then the horizontal interpolation is performed and the correction gain is output. A typical complete bicubic interpolation unit requires 4 points in each of the two directions. The detailed explanation of the boundary three-point cubic interpolation is introduced by simplifying the logic of the expansion edge.

[0105] When implementing bicubic interpolation based on the above sampling scheme, an obvious problem arises: how to handle pixels on the edges of the image when complete interpolation units are missing? (See below.) Figure 4 As shown, consider a complete interpolation unit in a single direction. a, b, and c represent the gains at valid grid point locations, indicated by solid-lined circles; d represents the gains at missing virtual grid point locations on the edge, indicated by dashed-lined circles. The middle dashed line is the line of symmetry about the single interpolation grid unit in the demonstration direction.

[0106] Specifically, such as Figure 4 As shown, consider an interpolation unit in a horizontal (or vertical) direction at the edge of an image. The figure shows three solid-lined circles representing three valid grid points obtained through calibration inside the image boundary, with gain values ​​of a, b, and c, respectively. Outside the image boundary, a fourth reference point should exist, but because it is located outside the valid image area, its true gain value cannot be obtained through calibration.

[0107] To address this issue, the embodiment introduces virtual grid points, denoted as dashed circles, with a gain value denoted as d. The gain value of this virtual point is not arbitrarily set, but is calculated using a rigorous derivation formula based on the gains of three valid grid points: d = a - 3×(b - c)(1) The derivation of formula (1) is based on reasonable assumptions about the continuity and smoothness of gain changes at the boundary, and aims to predict the virtual gain value outside the boundary with high fidelity using known internal gain points.

[0108] Figure 4The dashed line in the middle represents the symmetry line of the interpolation unit. Depending on the specific position of the pixel to be processed in this interpolation unit (located to the left or right of the symmetry line), the system will select the corresponding simplified interpolation weight calculation formula (1), substitute the gain values ​​of the four points a, b, c, and d into it, and finally calculate the accurate first gain value of the edge pixel.

[0109] In summary, through Figure 4 The described "boundary tri-point cubic interpolation logic" eliminates the need for edge expansion in the implementation, eliminating the need for physical edge expansion in the top, bottom, left, and right directions during the configuration phase of the two-dimensional gain mapping table. This significantly reduces storage space usage and calibration complexity. Through these virtual points, high-quality cubic interpolation algorithms can still be applied to image edges, effectively avoiding the mesh-like artifacts caused by directly using bilinear interpolation, or the edge correction errors caused by inaccurate expansion in traditional bicubic interpolation. Furthermore, the implementation transforms complex boundary case handling into a unified and predictable computational flow, making it highly suitable for parallel processing and timing optimization in hardware pipelines.

[0110] This application provides a detailed description of the core algorithm for the boundary three-point cubic interpolation logic. Through rigorous mathematical derivation, this algorithm transforms the concept of virtual grid points into a unified calculation formula that can be efficiently implemented in hardware.

[0111] When performing cubic interpolation in a single direction, gain values ​​from four reference points are typically required, and the interpolation result can be expressed as: Res=w0×a+w1×b+w2×c+w3×d (2) Where w0, w1, w2, and w3 are the interpolation weights calculated based on the relative positions of the points to be interpolated.

[0112] When point d is a virtual grid point at the edge of the image, its gain value d = a - 3×(b - c) is substituted into the general formula (2) above, and by combining like terms algebraically, a simplified calculation formula for the first case can be derived (corresponding to...). Figure 4 (The case where the middle pixel is located to the right of the symmetry line): Res=a+(3×(1-w0-w1-w2)+w2)×c+(3×(w0+w1+w2-1)+w1)×b+(-w1-w2)×a(3) Based on the principle of symmetry, when the position of the pixel to be processed in the current grid cell is offset to the left of the symmetry line, the correspondence of the interpolation weights will be reversed. At this time, the role of the virtual point shifts from d to a, and by substituting into the general formula, a simplified calculation formula for the second case is obtained: Res = a + (w1 + 3 × w0) × c + (w2 - 3 × w0) × b + (-w1 - w2) × a (4) In the specific hardware implementation, the system first determines the relative position of the pixel to be interpolated within the grid cell (with the grid cell centerline as the boundary). Based on the determination result, the hardware selectively calls formula (3) or formula (4) for calculation. The common advantage of these two formulas is that they completely eliminate the explicit dependence on the virtual point gain value; all calculations are based on three real and valid grid point gains (a, b, c) and pre-calculated weight coefficients determined by the pixel position. This design transforms the handling of complex boundary cases into a unified and deterministic calculation process, greatly simplifying the hardware logic design, optimizing timing, and ensuring the consistency of interpolation quality across the entire image.

[0113] (3) To achieve efficient radial processing, this application adopts a technical solution of converting a two-dimensional gain mapping table into a one-dimensional form. The one-dimensional gain mapping table supports two generation methods: automatic conversion from a calibrated two-dimensional gain mapping table via software, or direct loading of manually configured table data. The number of one-dimensional grid points, grid_num_1d, is a configurable parameter to adapt to different precision requirements.

[0114] This conversion process can be performed offline by the software or during system initialization, as detailed below: Coordinate calculation: For each preset radial sampling distance in the one-dimensional table, calculate the coordinates of the corresponding point on the original image plane along multiple directions (e.g., 8 or extended to 12 directions, including horizontal and vertical lines passing through the optical center) starting from the optical center.

[0115] Gain interpolation: Based on the coordinates of each direction calculated above, the target gain value at each corresponding position is calculated by interpolation from the source two-dimensional gain mapping table using the boundary three-point bicubic interpolation algorithm provided in the embodiment.

[0116] Gain averaging: The gain values ​​obtained in all directions (e.g., 8 or 12) are summed and the arithmetic mean is calculated. This average value is used as the one-dimensional gain value corresponding to the current radial sampling distance.

[0117] Through the above process, a complete one-dimensional gain mapping table can be constructed. For processing that needs to be performed in the YUV color space, the reference one-dimensional gain mapping table (usually the Y luminance channel) can be obtained by synthesizing the one-dimensional tables of each channel converted from the Bayer Raw domain two-dimensional table using the standard RGB to YUV conversion formula.

[0118] (4) Hardware optimization query and radial modulation application of one-dimensional gain.

[0119] In the hardware pipeline, the process of querying a one-dimensional gain map for each pixel to obtain a second gain value has been deeply optimized.

[0120] In this embodiment, a hardware-friendly index iteration strategy is provided to replace the computationally intensive binary search method. The system first initializes the initial grid point indices of the current image relative to the optical center in the horizontal and vertical directions via software. Subsequently, when processing pixels, the hardware performs simple increment or decrement operations on the currently maintained indexes based on the relative position of the pixel to the optical center in the horizontal and vertical directions (e.g., the change in distance from the optical center as the pixel moves sequentially along the grating), thereby quickly locating the grid interval corresponding to the pixel in the one-dimensional gain mapping table. This method reduces the complexity of point-by-point search to constant level, greatly improving processing efficiency.

[0121] In this embodiment, to enhance flexibility, the system supports modulation of the one-dimensional gain mapping table. This is achieved by introducing a corresponding modulation parameter register, which allows for linear scaling of the gain value, for example, adjusting it within the range of 1 / (2^n) to 2^n. This calculation can be performed through hardware-optimized fixed-point arithmetic. Simultaneously, for ease of debugging, the system adds two intuitive parameters: a guard radius and a guard gain. The guard radius defines a circular region in the center of the image where gain modulation is not applied, while the guard gain sets the reference gain within this region. Engineers can easily "shape" the desired radial gain curve by adjusting these two parameters without directly modifying a large amount of discrete gain data.

[0122] In this embodiment, a second gain value can be interpolated based on the radial distance of the current pixel by querying (and optionally modulating) a one-dimensional gain mapping table. This gain value is passed as a modulation parameter to subsequent image processing modules (such as denoising, sharpening, and enhancement modules). Based on this parameter, each module makes radially differentiated adjustments to its internal processing intensity parameters (such as denoising intensity and enhancement amplitude), thereby achieving precise optimization of different regions of the image. For example, the denoising intensity can be appropriately reduced at the image edges (which typically correspond to higher LSC gain) to retain more details.

[0123] By dividing each function into corresponding functional modules, this application provides an image processing apparatus, which can be a server, a terminal, or a chip applied to a server. Figure 5 This is a schematic block diagram of the functional modules of an image processing apparatus provided for an exemplary embodiment of this application. Figure 5 As shown, the image processing apparatus includes: Image acquisition module 51 is used to acquire the image to be processed; The first gain value calculation module 52 is used to calculate the first gain value of the pixels in the image to be processed; wherein, the first gain value is calculated based on a pre-calibrated two-dimensional gain mapping table of non-uniformly spaced uniform sampling grid points and a cubic interpolation algorithm; The first image correction module 53 is used to perform lens shading correction processing on the pixels in the image to be processed based on the first gain value, and generate a first corrected image. The second gain value calculation module 54 is used to calculate the second gain value of the pixels in the image to be processed; wherein, the second gain value is based on a one-dimensional gain mapping table and obtained by querying the distance from the pixel to the optical center of the image, the optical center of the image corresponds to the projection of the optical center of the lens on the image plane, and the one-dimensional gain mapping table is used to characterize the mapping relationship between the gain value and the distance from the pixel to the optical center of the image; The second image correction module 55 is used to perform radially differentiated image enhancement processing on the first corrected image by using the second gain value as a modulation parameter to obtain the second corrected image.

[0124] In another embodiment provided in this application, the apparatus further includes a one-dimensional gain mapping table acquisition module, which is used for: For each sampling distance in the one-dimensional gain mapping table, calculate the coordinates of the corresponding points in the original image in multiple directions originating from the optical center; Based on the coordinates, the gain values ​​of the corresponding points in each direction are obtained from the two-dimensional gain mapping table by interpolation. The gain values ​​in each direction are averaged, and the average value is used as the one-dimensional gain value for the corresponding sampling distance to obtain a one-dimensional gain mapping table.

[0125] In another embodiment provided in this application, the second gain value calculation module 54 is specifically used for: Initialize the initial grid point indices of the image to be processed relative to the optical center of the image in the horizontal and vertical directions; Based on the relative positional relationship between the pixel and the optical center of the image in the horizontal and vertical directions, the initial grid point index is iterated by incrementing or decrementing to locate the grid interval corresponding to the pixel in the one-dimensional gain mapping table. The second gain value is obtained by interpolation based on the located grid interval.

[0126] In another embodiment provided in this application, the device further includes a modulation module, specifically used for: The one-dimensional gain mapping table is modulated, and the modulation is used to adjust the gain curve by configuring the guard radius and guard gain parameters.

[0127] In another embodiment provided in this application, the device further includes a non-uniformly spaced sampling grid point determination module, specifically used for: Configure a preset number of grid points and initialize the grid size parameters in each direction using integer methods; By introducing a fixed-point parameter based on the reciprocal of the segmentation number, a coordinate table for each grid point is generated, resulting in a uniformly distributed grid with non-uniform grid spacing.

[0128] In another embodiment provided in this application, the first gain value calculation module 52 is further configured to: If there are no four complete reference grid points in the edge region of the image to be processed, obtain the gain values ​​of three valid grid points from the two-dimensional gain mapping table; Virtual grid points are introduced, and the gain value of the virtual grid points is expressed as a function of the gain of the three effective grid points; Based on the gain values ​​of the three effective grid points and the gain values ​​of the virtual grid points, the first gain value is obtained through cubic interpolation logic calculation.

[0129] This application also provides an electronic device, including: at least one processor; a memory for storing executable instructions of the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the method disclosed in the embodiments of this application.

[0130] Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. For example... Figure 6 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can perform the corresponding steps in the methods disclosed in the embodiments of this application.

[0131] The processor 1801 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this application can be implemented by the integrated logic circuitry in the hardware of the processor 1801 or by instructions in software form. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1802, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1801 reads information from the memory 1802 and, in conjunction with its hardware, completes the steps of the above method.

[0132] Furthermore, the various operations / processes according to this application, when implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 7 The computer system 1900 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including those described above. Figure 7 A structural block diagram of a computer system provided for an exemplary embodiment of this application.

[0133] Computer System 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0134] like Figure 7As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computer system 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0135] Multiple components in computer system 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disks and optical disks. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0136] The computing unit 1901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in the embodiments of this application can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in the embodiments of this application by any other suitable means (e.g., by means of firmware).

[0137] This application also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this application.

[0138] The computer-readable storage medium in this application embodiment may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0139] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0140] This application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this application.

[0141] In embodiments of this application, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0143] The modules, components, or units described in the embodiments of this application can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0144] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0145] The above description is merely an embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0146] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: The image to be processed is acquired, and the first gain value of the pixels in the image to be processed is calculated; wherein the first gain value is calculated based on a pre-calibrated two-dimensional gain mapping table of non-uniformly spaced uniform sampling grid points and a cubic interpolation algorithm; Based on the first gain value, lens shading correction processing is performed on the pixels in the image to be processed to generate a first corrected image; Calculate the second gain value of the pixels in the image to be processed; wherein the second gain value is based on a one-dimensional gain mapping table and obtained by querying the distance from the pixel to the optical center of the image, the optical center of the image corresponds to the projection of the optical center of the lens onto the image plane, and the one-dimensional gain mapping table is used to characterize the mapping relationship between the gain value and the distance from the pixel to the optical center of the image; Using the second gain value as a modulation parameter, radial differential image enhancement processing is performed on the first corrected image to obtain the second corrected image.

2. The method according to claim 1, characterized in that, The method further includes: For each sampling distance in the one-dimensional gain mapping table, calculate the coordinates of the corresponding points in the original image in multiple directions originating from the optical center; Based on the coordinates, the gain values ​​of the corresponding points in each direction are obtained from the two-dimensional gain mapping table by interpolation. The gain values ​​in each direction are averaged, and the average value is used as the one-dimensional gain value for the corresponding sampling distance to obtain a one-dimensional gain mapping table.

3. The method according to claim 1, characterized in that, The calculation of the second gain value of the pixels in the image to be processed includes: Initialize the initial grid point indices of the image to be processed relative to the optical center of the image in the horizontal and vertical directions; Based on the relative positional relationship between the pixel and the optical center of the image in the horizontal and vertical directions, the initial grid point index is iterated by incrementing or decrementing to locate the grid interval corresponding to the pixel in the one-dimensional gain mapping table. The second gain value is obtained by interpolation based on the located grid interval.

4. The method according to claim 1, characterized in that, Before calculating the second gain value of the pixels in the image to be processed, the method further includes: The one-dimensional gain mapping table is modulated, and the modulation is used to adjust the gain curve by configuring the guard radius and guard gain parameters.

5. The method according to claim 1, characterized in that, The non-uniformly spaced sampling grid points are determined in the following way: Configure a preset number of grid points and initialize the grid size parameters in each direction using integer methods; By introducing a fixed-point parameter based on the reciprocal of the segmentation number, a coordinate table for each grid point is generated, resulting in a uniformly distributed grid with non-uniform grid spacing.

6. The method according to claim 1, characterized in that, The method further includes: If there are no four complete reference grid points in the edge region of the image to be processed, obtain the gain values ​​of three valid grid points from the two-dimensional gain mapping table; Virtual grid points are introduced, and the gain value of the virtual grid points is expressed as a function of the gain of the three effective grid points; Based on the gain values ​​of the three effective grid points and the gain values ​​of the virtual grid points, the first gain value is obtained through cubic interpolation logic calculation.

7. An image processing apparatus, characterized in that, The device includes: The image acquisition module is used to acquire the image to be processed. The first gain value calculation module is used to calculate the first gain value of the pixels in the image to be processed; wherein, the first gain value is calculated based on a pre-calibrated two-dimensional gain mapping table of non-uniformly spaced uniform sampling grid points and a cubic interpolation algorithm; The first image correction module is used to perform lens shading correction processing on the pixels in the image to be processed based on the first gain value, and generate a first corrected image; The second gain value calculation module is used to calculate the second gain value of the pixels in the image to be processed; wherein, the second gain value is based on a one-dimensional gain mapping table and obtained by querying the distance from the pixel to the optical center of the image, the optical center of the image corresponds to the projection of the optical center of the lens onto the image plane, and the one-dimensional gain mapping table is used to characterize the mapping relationship between the gain value and the distance from the pixel to the optical center of the image; The second image correction module is used to perform radially differentiated image enhancement processing on the first corrected image by using the second gain value as a modulation parameter to obtain the second corrected image.

8. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images