Image processing method, device, chip and module equipment

Abstract

The present application discloses an image processing method, device, chip and module equipment, which relates to the field of electronic technology. The method comprises: obtaining a first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is magnified K times after bicubic interpolation processing of the second initial compensation table, and the second initial compensation table is used by the hardware module to perform color shading correction processing on the image; calling the software module to perform color shading correction processing on the target image based on the first initial compensation table. Based on the method described in the present application, it is beneficial to improve the efficiency of the software module in performing color shading correction processing.

Description

Technical Field

[0001] The present invention relates to the field of electronic technology, and in particular to an image processing method, device, chip and module equipment. Background Art

[0002] Color shading is a common lens shading, which is manifested as inconsistent colors between the center and the surrounding of the image. Due to the different degrees of refraction of light of different spectra by the lens, light of different wavelengths falls on different positions of the image sensor. Under different light source color temperatures, the attenuation trends of R, G, and B channels are different, resulting in color deviations around and in the center of the image. The color uniformity of the image is poor, which seriously affects the image quality. To address this problem, color shading correction is generally performed on the original image based on a compensation table after the original image is captured.

[0003] The modern camera system is a comprehensive hardware and software system with a complex processing flow, not a single software algorithm or direct hardware output. Therefore, the software modules and hardware modules in the existing modern camera system will use the same compensation table to perform bicubic interpolation to correct the color of the image. Since the execution methods of the hardware module and the software code are completely different, the hardware module is completely designed for the algorithm corresponding to the compensation table of color shading correction, but the software module is not designed for the algorithm corresponding to the compensation table of color shading correction. This results in low efficiency and long operation time when the software module is used to perform color correction on the image at the back end of the shooting or preview process. Summary of the invention

[0004] The present application provides an image processing method, device, chip and module equipment, which are beneficial to improving the efficiency of color shading correction processing performed by the software module.

[0005] In a first aspect, the present application provides an image processing method, the method comprising: obtaining a first initial compensation table from a pre-stored initial compensation table, the first initial compensation table being a compensation table that is magnified K times after bicubic interpolation processing of the second initial compensation table, the second initial compensation table being used by a hardware module to perform color shading correction processing on an image, the width of the second initial compensation table being M, the length of the second initial compensation table being N, the second initial compensation table storing M×N compensation coefficients, the width of the first initial compensation table being K×M, the length of the first initial compensation table being K×N, the first initial compensation table storing K×M×K×N compensation coefficients, M, N and K being positive integers greater than 1; calling a software module to perform color shading correction processing on a target image based on the first initial compensation table.

[0006] Based on the method described in the first aspect, since the processing rate of the software module is slow during the process of performing bicubic interpolation processing on the compensation table, and the first initial compensation table is obtained by enlarging the second initial compensation table by performing bicubic interpolation processing in advance, the software module directly obtains the first initial compensation table from the pre-stored initial compensation table without the need to perform bicubic interpolation processing. Therefore, based on the method described in the present application, the efficiency of the software module in performing color shading correction processing can be improved.

[0007] In a possible implementation, the method further includes: S1, using the bicubic interpolation method to enlarge the second initial compensation table by L times to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K; S2, using the bicubic interpolation method to enlarge the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1; S3, using the bilinear interpolation method to enlarge the third initial compensation table by P times to obtain a fifth initial compensation table; S4, calculating the error between the fourth initial compensation table and the fifth initial compensation table; if the error is less than or equal to the first threshold, determining the third initial compensation table as the first initial compensation table, and saving the first initial compensation table; if the error is greater than the first threshold, increasing the value of L, and re-executing steps S1 to S4. Based on this implementation, if the calculated error value is less than or equal to the first threshold, it means that if the software module uses the third initial compensation table currently obtained through the second initial compensation table to perform color shading compensation, the result obtained by the hardware module using the second initial compensation table to perform color shading compensation has a small error, and can be approximately regarded as consistent. Therefore, based on this implementation, it is helpful to ensure the consistency of the color shading correction processing results performed by the hardware module and the software module.

[0008] In a possible implementation, the method further includes: if the value of L is greater than or equal to the second threshold, determining the third initial compensation table obtained by the last execution of step S1 as the first initial compensation table, and saving the first initial compensation table. Based on this implementation, when the value of L is greater than or equal to the second threshold, since the Lens Shade curve is a slowly changing arc, the error between the fifth initial compensation table obtained by the third initial compensation table and the fourth initial compensation table is limited by increasing the value of L, and at the same time, the larger the value of L, the larger the third initial compensation table. Pre-storing an overly large compensation table will also burden the storage space of the electronic device. Therefore, if the value of L is greater than or equal to the second threshold, determining the third initial compensation table as the first initial compensation table, and saving the first initial compensation table is conducive to reducing the burden on the device.

[0009] In a possible implementation, the number of the second initial compensation tables is multiple, and the pre-stored initial compensation tables include multiple first initial compensation tables, one first initial compensation table corresponds to one color temperature, and the color temperatures corresponding to the multiple first initial compensation tables are different from each other; the first initial compensation table is obtained from the pre-stored initial compensation table, and the specific implementation is: based on the color temperature of the target image, two first initial compensation tables are determined from the pre-stored initial compensation table, and the color temperature of the target image is between the two color temperatures corresponding to the two first initial compensation tables; the software module is called to perform color shading correction processing on the target image based on the first initial compensation table, and the specific implementation is: based on the two first initial compensation tables and the target interpolation coefficient, the first target compensation table is determined; the software module is called to perform color shading correction processing on the target image based on the first target compensation table. Based on this implementation, by determining the first initial target table and the target interpolation coefficient that are most suitable for the current light source, the target compensation table corresponding to the color temperature of the current light source is obtained, which is conducive to accurately compensating the color shading of the current light source and effectively improving the color shading correction effect.

[0010] In a possible implementation, before determining the first target compensation table based on the two first initial compensation tables and the target interpolation coefficients, the method further includes: determining a plurality of first optional interpolation coefficients from a first value range according to a first step length, wherein the value of the first step length is less than the difference between the maximum value and the minimum value in the first value range, and the first step length is the difference between two adjacent first optional interpolation coefficients; S5, performing color shading correction on the downsampled target image based on the two first initial compensation tables and the plurality of first optional interpolation coefficients to obtain a plurality of corrected statistical value images; S6, determining a plurality of first optional interpolation coefficients based on the plurality of corrected statistical value images The first error values ​​corresponding to the selected interpolation coefficients are determined; if the preset iteration condition is not met, multiple second optional interpolation coefficients are determined based on the first optional interpolation coefficient corresponding to the smallest first error value among the multiple first error values, and the multiple second optional interpolation coefficients are used as the new multiple first optional interpolation coefficients, and step S5 and step S6 are performed again, and the preset iteration condition is that the smallest first error value among the multiple first error values ​​is less than the third threshold value or the number of iterations exceeds the fourth threshold value; if the preset iteration condition is met, the interpolation coefficient corresponding to the smallest error value among the multiple first error values ​​when step S6 is last executed is determined as the target interpolation coefficient. Based on this implementation, when the smallest first error value among the multiple first error values ​​is less than the third threshold value, it means that the error of the first optional interpolation coefficient corresponding to the current smallest first error value is small, and it can be determined as the target interpolation coefficient, thereby improving the effect of the software module on color shading correction compensation. The number of iterations can also be understood as the number of loops, that is, the number of times step S402 is executed. Since the more the number of loops, the lower the efficiency, the higher the burden on the device, by limiting the number of iterations to exceed the fourth threshold value, it is beneficial to improve the efficiency of color shading correction processing, and at the same time, it can also reduce the burden on the device.

[0011] In a possible implementation, multiple second optional interpolation coefficients are determined based on the first optional interpolation coefficient corresponding to the smallest first error value among multiple first error values. The specific implementation is: determine the second value range according to the first optional interpolation coefficient corresponding to the smallest first error value among multiple first error values; determine multiple second optional interpolation coefficients from the second value range according to the second step length, wherein the second step length is smaller than the difference between the maximum value and the minimum value in the second value range, and the second step length is the difference between two adjacent first optional interpolation coefficients. Based on this implementation, it is beneficial to determine a target interpolation coefficient with a smaller error value, thereby improving the effect of the software module on color shading correction processing of the image.

[0012] In a possible implementation, multiple first error values ​​corresponding to multiple first optional interpolation coefficients are determined based on multiple corrected statistical value images. The specific implementation method is: converting the ratio of the R channel to the G channel of the sub-image block in the statistical value image corrected based on the first optional interpolation coefficient to the logarithmic domain to obtain a first sub-error value, and the corrected statistical value image includes multiple sub-image blocks; converting the ratio of the B channel to the G channel of the sub-image block to the logarithmic domain to obtain a second sub-error value; determining the variance of the first sub-error value and the second sub-error value as the error value of the sub-image block; and determining the first error value corresponding to the first optional interpolation coefficient based on the error values ​​of multiple sub-image blocks of the statistical value image.

[0013] In a second aspect, the present application provides a device, which includes: an acquisition unit and a processing unit; the acquisition unit is used to acquire a first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is magnified K times after bicubic interpolation processing of the second initial compensation table, the second initial compensation table is used by the hardware module to perform color shading correction processing on the image, the width of the second initial compensation table is M, the length of the second initial compensation table is N, the second initial compensation table stores M×N compensation coefficients, the width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, the first initial compensation table stores K×M×K×N compensation coefficients, M, N and K are all positive integers greater than 1; the processing unit is used to call the software module to perform color shading correction processing on the target image based on the first initial compensation table.

[0014] In a third aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the processor is configured to enable the chip to execute the method in the above-mentioned first aspect or any possible implementation manner thereof.

[0015] In a fourth aspect, the present application provides a module device, which includes a communication module, a power module, a storage module and a chip, wherein: the power module is used to provide power to the module device; the storage module is used to store data and instructions; the communication module is used for internal communication within the module device, or for the module device to communicate with external devices; the chip is used to execute the method in the above-mentioned first aspect or any possible implementation thereof.

[0016] In a fifth aspect, an embodiment of the present invention discloses an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method in the above-mentioned first aspect or any possible implementation thereof.

[0017] In a sixth aspect, the present application provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed on a device, the device executes the method in the above-mentioned first aspect or any possible implementation thereof.

[0018] In a seventh aspect, the present application provides a computer program or a computer program product, comprising codes or instructions, which, when executed on a computer, enable the computer to execute the method in the first aspect or any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0021] Figure 2 It is a flowchart of a method for pre-storing a first initial compensation table provided in an embodiment of the present application;

[0022] Figure 3 It is a flowchart of another image processing method provided in an embodiment of the present application;

[0023] Figure 4 It is a flowchart of an iterative search method provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0026] Figure 7 is a schematic diagram of the structure of a device provided in an embodiment of the present application;

[0027] Figure 8 It is a structural schematic diagram of a module device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0030] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present application and in the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0031] Color shading is a common lens shading, which is manifested as inconsistent colors between the center and the surrounding of the image. Due to the different degrees of refraction of light of different spectra by the lens, light of different wavelengths falls on different positions of the image sensor. Under different light source color temperatures, the attenuation trends of R, G, and B channels are different, resulting in color deviations around and in the center of the image. The color uniformity of the image is poor, which seriously affects the image quality. To address this problem, color shading correction is generally performed on the original image based on a compensation table after the original image is captured.

[0032] The modern camera system is a comprehensive hardware and software system with a complex processing flow. It is not a single software algorithm processing or direct hardware output, but includes several hardware pipelines and several software post-processing. According to the scene design of the photo / preview process, the hardware pipeline and software link will also be dynamically set. At the same time, many algorithms for image processing will also exist in the hardware pipeline and software library at the same time. According to the configuration of the photo or preview process, these algorithms may be enabled in the hardware, or may be called in the software link, or may need to be called multiple times, or the order of calling will be adjusted, or multiple channels of data must be processed at the same time, or the same frame of data must be split and processed through heterogeneous modes. Therefore, in order to avoid color shadows in the image, the software modules and hardware modules in the existing modern camera system may use the same compensation table for bicubic interpolation to enlarge the compensation table and then perform color correction on the image. Bicubic interpolation is a commonly used interpolation method in two-dimensional space. In this method, the value of the function f at a point (x, y) can be obtained by weighted average of the nearest sixteen sampling points in a rectangular grid. Two polynomial interpolation cubic functions are required, one for each direction. Since the execution methods of the hardware module and the software code are completely different, the hardware module is completely designed for the algorithm corresponding to the compensation table of color shading correction, but the software module is not designed for the algorithm corresponding to the compensation table of color shading correction. The software module has a slow processing rate during the bicubic interpolation process of the compensation table. This results in low efficiency when color correcting the image through the software module at the back end of the shooting or preview process, and requires a long calculation time.

[0033] In order to improve the efficiency of color shading correction processing by the software module, the present application proposes an image processing method, which includes: obtaining a first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is amplified K times after bicubic interpolation processing of the second initial compensation table, and the second initial compensation table is used by the hardware module to perform color shading correction processing on the image; calling the software module to perform color shading correction processing on the target image based on the first initial compensation table. Since the software module has a slow processing rate in the process of performing bicubic interpolation processing on the compensation table, and the first initial compensation table is obtained by enlarging the second initial compensation table by bicubic interpolation processing in advance, the software module directly obtains the first initial compensation table from the pre-stored initial compensation table without the need for bicubic interpolation processing. Therefore, based on the method described in the present application, the efficiency of color shading correction processing by the software module can be improved.

[0034] The image processing method provided in the embodiment of the present application is further described in detail below:

[0035] See also Figure 1 , Figure 1It is a flowchart of an image processing method provided in an embodiment of the present application. Figure 1 The subject executing the method shown may be an electronic device, or the subject may be a chip in the electronic device. Figure 1 The electronic device is used as an example to illustrate the execution subject of the method. The execution subjects of the image processing methods shown in other figures of the embodiments of the present application are similar, and will not be described in detail below. Figure 1 The image processing method shown includes step 101 and step 102. Wherein:

[0036] 101. Obtain a first initial compensation table from a pre-stored initial compensation table, where the first initial compensation table is a compensation table obtained by performing bicubic interpolation processing on the second initial compensation table and amplifying it by K times, and the second initial compensation table is used by the hardware module to perform color shading correction processing on the image.

[0037] In the embodiment of the present application, the compensation table includes data for color shading correction processing of the image, and the electronic device can perform color shading correction processing on the image through the compensation table. Among them, the second initial compensation table is used for the hardware module to perform color shading correction processing, the width of the second initial compensation table is M, the length of the second initial compensation table is N, and the second initial compensation table stores M×N compensation coefficients, and M and N are positive integers greater than 1. Among them, due to the limited storage space in the hardware module and the fast rate of the hardware module to perform bicubic interpolation and amplification processing on the compensation table, usually, the second initial compensation table stored in the hardware module is small. The software module performs bicubic interpolation processing on the compensation table at a slow processing rate. Therefore, in the method described in the present application, the second initial compensation table for bicubic interpolation processing is pre-enlarged K times to obtain the first initial compensation table, the width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, and the first initial compensation table stores K×M×K×N compensation coefficients, and K is a positive integer greater than 1. Since the processing rate of the software module is slow during the process of performing bicubic interpolation processing on the compensation table, and the first initial compensation table is obtained by enlarging the second initial compensation table by performing bicubic interpolation processing in advance, the software module directly obtains the first initial compensation table from the pre-stored initial compensation table without performing bicubic interpolation processing. Therefore, based on the method described in the present application, the efficiency of the software module in performing color shading correction processing can be improved.

[0038] In one possible implementation, the electronic device will use different interpolation algorithms when using two sizes of compensation tables for color shading compensation. For example, when the hardware module uses the second initial compensation table, bicubic interpolation is used for compensation, and when the software module uses the first initial compensation table, bilinear interpolation is used for compensation. The bilinear interpolation used is an extension of the linear interpolation function with two variables. The core idea is to perform linear interpolation once in two directions. Because the bilinear interpolation algorithm has a simple process and is more efficient than the bicubic interpolation algorithm, it can improve the efficiency of the software module in color shading correction processing. However, since the bicubic interpolation algorithm and the bilinear interpolation algorithm are different, the results obtained by performing color shading correction processing through the hardware module and the software module may be different. In order to ensure the consistency of the color shading correction processing results of the hardware module and the software module, the method may also include a method for pre-storing the first initial compensation table. The specific process is as follows: Figure 2 As shown:

[0039] S201, using the bicubic interpolation method to amplify the second initial compensation table by L times to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K.

[0040] S202, using the bicubic interpolation method to amplify the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1.

[0041] S203 , amplify the third initial compensation table by P times using the bilinear interpolation method to obtain a fifth initial compensation table.

[0042] S204: Calculate the error between the fourth initial compensation table and the fifth initial compensation table.

[0043] The width of the second initial compensation table is M, the length is N, and M×N compensation coefficients are stored. Therefore, the width of the third initial compensation table obtained by enlarging the second initial compensation table by L times using the bicubic interpolation method is L×M, the length is L×N, and a total of L×M×L×N compensation coefficients are stored. The width of the fifth initial compensation table obtained by enlarging the third initial compensation table by P times using the bilinear interpolation method is L×P×M, the length is L×P×M, and a total of L×P×M×L×P×N compensation coefficients are stored. The width of the fourth initial compensation table obtained by enlarging the second initial compensation table by P×L times using the bicubic interpolation method is L×P×M, the length is L×P×M, and a total of L×P×M×L×P×N compensation coefficients are stored. It can be seen that the size of the fifth initial compensation table is the same as the size of the fourth initial compensation table. Therefore, the error between the size of the fifth initial compensation table and the size of the fourth initial compensation table can be calculated. Optionally, the error between the fifth initial compensation table and the fourth initial compensation table can be calculated by formula (1):

[0044] E=max(|T 5 (i,j)-T 4 (i,j)|)i∈[1,x]j∈[1,y](1)

[0045] Wherein, E represents the error between the fifth initial compensation table and the fourth initial compensation table, the fifth initial compensation table and the fourth initial compensation table respectively store x×y compensation coefficients, x represents the width of the fourth initial compensation table and the fifth initial compensation table, y represents the length of the fourth initial compensation table and the fifth initial compensation table, x and y are both integers greater than 1, T 4 represents the fourth initial compensation table, T 5 represents the fifth initial compensation table, T 4 (i,j) represents the compensation coefficient in the i-th row and j-th column of the fourth initial compensation table, T 5 (i,j) represents the compensation coefficient of the i-th row and j-th column in the fifth initial compensation table, the value of i is an integer between 1 and x (including 1 and x), the value range of j is an integer between 1 and y (including 1 and y), the function symbol || represents taking the absolute value, the function max() represents finding the maximum value, which is used to return the maximum value in the specified expression, therefore, E is equal to the maximum value of the absolute value of the difference between the compensation coefficients at the same position in the fifth initial compensation table and the fourth initial compensation table.

[0046] It should be noted that the error between the fifth initial compensation table and the fourth initial compensation table can also be calculated by other methods. For example, the error between the fifth initial compensation table and the fourth initial compensation table is equal to the average value of the absolute value of the difference between the compensation coefficients at the same position in the fifth initial compensation table and the fourth initial compensation table. The embodiment of the present application is not limited to this.

[0047] S205: Determine whether the error is less than or equal to the first threshold, or whether the value of L is greater than or equal to the second threshold. If the error is less than or equal to the first threshold, or the value of L is greater than or equal to the second threshold, execute step S206; if the error is greater than the first threshold and the value of L is less than the second threshold, execute step S207.

[0048] S206: Determine the third initial compensation table as the first initial compensation table, and save the first initial compensation table.

[0049] S207. Increase the value of L and re-execute steps S201 to S204.

[0050] When the calculated error value is less than or equal to the first threshold, it means that if the software module uses the third initial compensation table to perform color shading compensation, the error of the result obtained by the hardware module using the second initial compensation table to perform color shading compensation is small, and they can be approximately considered to be consistent. Therefore, based on this implementation, it is helpful to ensure the consistency of the color shading correction processing results of the hardware module and the software module.

[0051] When the calculated error value is greater than the first threshold, it means that if the software module uses the third initial compensation table to perform color shadow compensation, the result obtained by the hardware module using the second initial compensation table to perform color shadow compensation has a large error, so it is necessary to increase the value of L to improve the accuracy of the third initial compensation table and reduce the error between the fifth initial compensation table and the fourth initial compensation table. Wherein, increasing the value of L can be to increase the unit value on the basis of the original value of L, and the size of the unit value can be any value, for example, L is originally 8, the unit value is 2, and the value of L obtained after increasing the unit value is 10. Or increasing the value of L can also be to multiply the unit value on the original value of L, and the size of the unit value can be any value, for example, L is originally 8, the unit value is 2, and the value of L obtained after increasing the unit value is 16. Wherein, when step 206 is finally executed, the size of the value of L at this time is the same as K in the above step 101.

[0052] When the value of L is greater than or equal to the second threshold, since the Lens Shade curve is a slowly changing arc, the error between the fifth initial compensation table and the fourth initial compensation table can be reduced to a limited extent by increasing the value of L. At the same time, the larger the value of L, the larger the third initial compensation table. Pre-storing an overly large compensation table will also burden the storage space of the electronic device. Therefore, if the value of L is greater than or equal to the second threshold, the third initial compensation table is determined to be the first initial compensation table, and the first initial compensation table is saved, which is conducive to reducing the burden on the device.

[0053] 102. Call a software module to perform color shading correction processing on the target image based on the first initial compensation table.

[0054] In the embodiment of the present application, when the electronic device uses two sizes of compensation tables for color shading compensation, different interpolation algorithms will be used. When the hardware module uses the second initial compensation table, bicubic interpolation is used for compensation, and when the software module uses the first initial compensation table, bilinear interpolation is used for compensation. Since the bilinear interpolation algorithm has a simple process and is more efficient than the bicubic interpolation algorithm, the efficiency of the color shading correction processing of the software module can be improved.

[0055] Usually, when people perform color shading correction, they will take multiple original images under multiple standard light sources in advance with a mobile electronic device, and calibrate the lens shading compensation table under each light source based on the original images, and store it in the device. When shooting images in actual scenes, the stored compensation table is called according to the current scene light source to perform color shading compensation on the current scene light source image. Since the light source in the actual environment is relatively complex, the color temperature of the photos taken will be different depending on the location and scene when the photos are taken. Directly calling the pre-calibrated compensation table cannot adapt well to complex light source scenes, resulting in unsatisfactory color shading correction effect.

[0056] In order to improve the effect of color shading correction and compensation by the software module, the present application embodiment proposes an image processing method, see Figure 3 , Figure 3 is a flow chart of an image processing method provided in an embodiment of the present application, wherein the method is Figure 1 A specific implementation of the method shown. Figure 3 The subject executing the method shown may be an electronic device, or the subject may be a chip in the electronic device. Figure 3 The electronic device is used as an example to illustrate the execution subject of the method. The execution subjects of the image processing methods shown in other figures of the embodiments of the present application are similar. Figure 3 The image processing method shown includes steps 301 to 303. Among them:

[0057] 301. Determine two first initial compensation tables from pre-stored initial compensation tables based on the color temperature of a target image, wherein the color temperature of the target image is between the two color temperatures corresponding to the two first initial compensation tables.

[0058] In the embodiment of the present application, color temperature is a unit of measurement indicating the color components contained in light. The color temperature of a light source is determined by comparing its color with a theoretical thermal black body radiator. Optionally, the electronic device can determine the color temperature of a target image through an R channel, a B channel, and a G channel in the target image.

[0059] Among them, the electronic device predetermines multiple second initial compensation tables, one second initial compensation table corresponds to one color temperature, and the color temperatures corresponding to the multiple second initial compensation tables are different. Optionally, the specific implementation method of the electronic device predetermining the second initial compensation table is: the electronic device is in a dark room environment, the parameters of the electronic device are adjusted to the shooting requirements, the camera of the electronic device is covered with a frosted glass sheet, facing the center of the light box, and the original image for calibration is taken under light sources of different color temperatures, and one original image corresponds to one color temperature light source; then the electronic device determines multiple second initial compensation tables based on the multiple original images, and the color temperatures corresponding to the multiple second initial compensation tables.

[0060] Based on the above method, in order to improve the efficiency of the software module in color shading correction compensation, the electronic device will obtain multiple first initial compensation tables based on multiple second initial compensation tables. The specific implementation method can refer to the description in the above step 101, and the embodiment of the present application will not be repeated here. The electronic device stores the determined multiple first initial compensation tables in the pre-stored initial compensation table. Since one second initial compensation table corresponds to one color temperature, the color temperature corresponding to each first initial compensation table can also be determined, and the color temperatures corresponding to the multiple first initial compensation tables are different from each other. The electronic device can determine two first initial compensation tables from the pre-stored initial compensation table according to the color temperature of the target image, and the size of the color temperature of the target image is between the sizes of the two color temperatures corresponding to the two first initial compensation tables. Exemplarily, it is assumed that the pre-stored initial compensation table contains 4 first initial compensation tables, namely compensation table 1, compensation table 2, compensation table 3 and compensation table 4, wherein the color temperature corresponding to compensation table 1 is 3000K, the color temperature corresponding to compensation table 2 is 4000K, the color temperature corresponding to compensation table 3 is 5000K, and the color temperature corresponding to compensation table 4 is 6000K. Assuming that the color temperature of the target image is 4500K, since 4500K is between 4000K and 5000K, the electronic device determines two first initial compensation tables from the pre-stored initial compensation tables, namely compensation table 2 and compensation table 3.

[0061] 302. Determine a first target compensation table based on the two first initial compensation tables and the target interpolation coefficient.

[0062] In the embodiment of the present application, the first target compensation table can be determined by formula (2):

[0063] Table 1 =Table a ×ratio+Table b ×(1-ratio)(2)

[0064] Among them, Table 1 Table is the first target compensation table. a and Table b They refer to two first initial compensation tables respectively, and ratio is the target interpolation coefficient.

[0065] In one possible implementation, the target interpolation coefficients are determined in the following two ways:

[0066] Method 1: Determine based on the color temperature corresponding to the two first initial compensation tables and the color temperature of the target image. Specifically, it can be expressed by formula (3):

[0067]

[0068] Among them, ratio represents the target interpolation coefficient, colora and color b are the color temperatures corresponding to the two first initial compensation tables, color a Is the compensation table a Corresponding color temperature, color b Is the compensation table b Corresponding color temperature, color c is the color temperature of the target image. For example, assuming that the color temperature of the target image is 4300K, the electronic device determines two first initial compensation tables from the preset initial compensation table, respectively: Table a and Table b , Table a The corresponding color temperature is 4000K, Table b The corresponding color temperature is 5000 K. Based on the above formula (3), the target interpolation coefficient ratio can be determined to be 0.7.

[0069] Method 2: Use iterative search method to determine the target interpolation coefficient with the smallest compensation effect error. For specific implementation methods, please refer to Figure 4 As shown, it includes steps S401 to S406.

[0070] S401. Determine a plurality of first optional interpolation coefficients from a first value range according to a first step length, wherein the value of the first step length is smaller than the difference between the maximum value and the minimum value in the first value range, and the first step length is the difference between two adjacent first optional interpolation coefficients.

[0071] Among them, the first value range can be expressed as [x, y], x and y can be any values, x is less than y, the first step length is expressed as s, and the first step length is the difference between two adjacent first optional interpolation coefficients. Therefore, multiple first optional interpolation coefficients can be specifically expressed as {x, x+s, x+2×s,…, y}. Exemplarily, assuming that the first step length is 0.2, the first value range is 0 to 1, and the first step length is the difference between two adjacent first optional interpolation coefficients, it can be determined that the multiple first optional interpolation coefficients are 0, 0.2, 0.4, 0.6, 0.8, and 1 respectively.

[0072] S402 : performing color shading correction on the downsampled target image based on two first initial compensation tables and a plurality of first optional interpolation coefficients to obtain a plurality of corrected statistical value images.

[0073] Optionally, step S402 can be expressed by formula (4):

[0074] lscm_apply_table=LSCM×table a ×ratio s+LSCM×table b ×(1-ratio s )(4)

[0075] Among them, lscm_apply_table represents the corrected statistical value image, LSCM represents the downsampled target image, sampling is used to reduce the data size corresponding to the target image, and iterative search method is used to determine the target interpolation coefficient. It takes multiple iterations. At this time, the downsampled target image is used to reduce the amount of calculation of electronic equipment and improve efficiency. a and Table b Refers to the two first initial compensation tables, ratio s Indicates the first optional interpolation coefficient

[0076] S403: Determine a plurality of first error values ​​corresponding to a plurality of first optional interpolation coefficients based on a plurality of corrected statistical value images.

[0077] Optionally, a specific implementation method of determining the first error value includes the following steps S403a to S403d:

[0078] S403a, converting the ratio of the R channel to the G channel of the sub-image block in the statistical value image corrected based on the first optional interpolation coefficient into a logarithmic domain to obtain a first sub-error value;

[0079] S403b, converting the ratio of the B channel to the G channel of the sub-image block into a logarithmic domain to obtain a second sub-error value;

[0080] S403c, determining the variance of the first sub-error value and the second sub-error value as the error value of the sub-image block;

[0081] Specifically, after obtaining the statistical value image lscm_apply_table, the statistical value image lscm_apply_table is divided into P×Q sub-image blocks, and then the error value of each sub-image block is calculated respectively, where P and Q are both positive integers. Among them, for the i-th (i is a positive integer from 1 to P×Q) sub-image block, the ratio of the R channel to the G channel of the sub-image block is converted to the logarithmic domain to obtain the first sub-error value error_r, and the ratio of the B channel to the G channel of the sub-image block is converted to the logarithmic domain to obtain the second sub-error value error_b, and then the sum of the variances of error_r and error_b is calculated to obtain the error value of the i-th sub-image block, which is recorded as error_value. Based on this method, the corresponding P×Q error values ​​can be obtained based on the P×Q sub-image blocks. Generally, the color of the captured image is balanced, that is, the values ​​of the R channel, the B channel and the G channel are close. Therefore, by calculating the error value based on this implementation method, large color differences in the images captured by the user can be avoided.

[0082] S403d. Determine a first error value corresponding to a first optional interpolation coefficient based on error values ​​of multiple sub-image blocks of the statistical value image.

[0083] Optionally, the specific implementation method of step S403d is: calculating the average value of the error values ​​of multiple sub-image blocks; calculating the weights of the multiple sub-image blocks based on the average value of the error values ​​of the multiple sub-image blocks; and determining the first error value based on the weights of the multiple sub-image blocks and the error values ​​of the multiple sub-image blocks.

[0084] Further optionally, the weights of multiple sub-image blocks can be calculated by formula (5):

[0085]

[0086] Among them, error weight(i) Represents the weight of the i-th sub-image block, the statistical value image lscm_apply_table is divided into P×Q sub-image blocks, i is a positive integer from 1 to P×Q, error_value(i) represents the error value of the i-th sub-image block, and avg_error_value represents the average error value of multiple sub-image blocks.

[0087] Further optionally, the first error value can be calculated by formula (6):

[0088]

[0089] Among them, error represents the first error value, error weight(i) represents the weight of the i-th sub-image block, and error_value(i) represents the error value of the i-th sub-image block.

[0090] Based on this implementation method, different weights are assigned to the error values ​​of different sub-image blocks, and the importance of the error value of each sub-image block to the error is fully considered, so that the error calculation is more accurate, thereby ensuring that the optimal interpolation coefficient calculation of the compensation table is more accurate, so that the final target compensation table is the compensation table that best suits the current light source color temperature.

[0091] S404: Determine whether a preset iteration condition is met. If the preset iteration condition is not met, execute step S406; if the preset iteration condition is met, execute step S405.

[0092] Among them, the preset iteration condition is that the smallest first error value among the multiple first error values ​​is less than the third threshold or the number of iterations exceeds the fourth threshold. When the smallest first error value among the multiple first error values ​​is less than the third threshold, it means that the error of the first optional interpolation coefficient corresponding to the current smallest first error value is small, and it can be determined as the target interpolation coefficient, thereby improving the effect of the software module in color shading correction compensation. The number of iterations can also be understood as the number of loops, that is, the number of times step S402 is executed. Since the more the number of loops, the lower the efficiency, the higher the burden on the device, by limiting the number of iterations to exceed the fourth threshold, it is beneficial to improve the efficiency of color shading correction processing, and at the same time, it can also reduce the burden on the device.

[0093] It should be noted that the preset iteration condition may also be other conditions, which is not limited in the embodiments of the present application.

[0094] S405: If the preset iteration condition is met, the interpolation coefficient corresponding to the minimum error value among the multiple first error values ​​when step S403 is performed for the last time is determined as the target interpolation coefficient.

[0095] S406. Determine multiple second optional interpolation coefficients based on the first optional interpolation coefficient corresponding to the smallest first error value among the multiple first error values, and use the multiple second optional interpolation coefficients as new multiple first optional interpolation coefficients, and execute step S405 and step S406 again.

[0096] Optionally, multiple second optional interpolation coefficients are determined based on the first optional interpolation coefficient corresponding to the smallest first error value among multiple first error values. The specific implementation method is: determine the second value range according to the first optional interpolation coefficient corresponding to the smallest first error value among the multiple first error values; determine multiple second optional interpolation coefficients from the second value range according to the second step size, wherein the second step size is smaller than the difference between the maximum value and the minimum value in the second value range, and the second step size is the difference between two adjacent first optional interpolation coefficients.

[0097] For example, the first step is The first value range is [0,1], where the first optional interpolation coefficient includes Assume that the first optional interpolation coefficient corresponding to the minimum first error value obtained after the first calculation is the middle point in the first value range, for example Then confirm The first two optional interpolation coefficients adjacent to each other are and At this time, the second value range can be determined as Assume that the second step length is Then, according to the second step length, a plurality of second optional interpolation coefficients are determined from the second value range as follows: Assuming that the first optional interpolation coefficient corresponding to the minimum first error value obtained after the first calculation is the maximum value or the minimum value in the first value range, for example, 0, then determine a first optional interpolation coefficient close to 0, that is, At this time, the second value range can be determined as Assume that the second step length is Then, according to the second step length, a plurality of second optional interpolation coefficients are determined from the second value range as follows:

[0098] Based on the iterative search method described above, it is helpful to determine a target interpolation coefficient with a smaller error value, thereby improving the effect of the software module on color shading correction processing of the image.

[0099] In a possible implementation, the target interpolation coefficient may also be determined by a hardware module, and the software module may directly use the target interpolation coefficient determined by the hardware module to determine the first target compensation table. Optionally, the hardware module in the electronic device may determine the target interpolation coefficient by the color temperature corresponding to the two second initial compensation tables and the color temperature of the target image. The specific implementation is the same as the method one described above, and the embodiments of the present application are not described here. Optionally, the hardware module in the electronic device may also use an iterative search method to determine the target interpolation coefficient with the smallest compensation effect error. The specific implementation is the same as the method two described above, and the embodiments of the present application are not described here.

[0100] 303. Call a software module to perform color shading correction processing on the target image based on the first target compensation table.

[0101] In the embodiment of the present application, since the first target compensation table is obtained based on the two first initial compensation tables and the target interpolation coefficient, the size is the same as the first initial compensation table, and the size of the first target compensation table is larger than the second initial compensation table. When the electronic device uses two sizes of compensation tables for color shading compensation, different interpolation algorithms will be used. When the hardware module uses the compensation table, bicubic interpolation is used for compensation, and when the software module uses the compensation table, bilinear interpolation is used for compensation. Since the bilinear interpolation algorithm has a simple process and is more efficient than the bicubic interpolation algorithm, based on this implementation method, the efficiency of the software module in color shading correction processing can also be improved.

[0102] Similarly, the hardware module can also determine two second initial compensation tables based on the color temperature of the target image, and determine the target compensation table based on the interpolation coefficient and the two second initial compensation tables to perform color shading correction on the target image. The specific implementation method is the same as that of the above-mentioned software module, and will not be repeated in the embodiments of the present application.

[0103] In another possible implementation, the method may also determine the two first initial compensation tables based on the minimum error value without determining the two first initial compensation tables from the pre-stored initial compensation tables based on the color temperature of the target image. The specific implementation is as follows: multiple compensation table combinations are determined based on the pre-stored initial compensation tables, each compensation table combination includes two first initial compensation tables; the error values ​​of each compensation table combination for color shading correction under different interpolation coefficients are determined respectively, and the target compensation table combination and the target interpolation coefficient corresponding to the minimum error value are determined; the target compensation table is determined according to the two first initial compensation tables in the target compensation table combination and the target interpolation coefficient, and the software module is called to perform color shading correction processing based on the target compensation table.

[0104] Wherein, multiple compensation table combinations are determined based on the pre-stored initial compensation table, and the specific implementation method is to combine multiple first initial compensation tables in the pre-stored initial compensation table in pairs to determine multiple compensation table combinations. Optionally, the color temperature values ​​corresponding to the two first initial compensation tables in a compensation table combination are adjacent. Exemplarily, the pre-stored initial compensation table contains 4 first initial compensation tables, namely compensation table 1, compensation table 2, compensation table 3 and compensation table 4, wherein the color temperature corresponding to compensation table 1 is 3000K, the color temperature corresponding to compensation table 2 is 4000K, the color temperature corresponding to compensation table 3 is 5000K, and the color temperature corresponding to compensation table 4 is 6000K. It is assumed that the color temperature values ​​corresponding to the two first initial compensation tables in a compensation table combination are adjacent, and the following three compensation table combinations can be obtained: compensation table combination 1 includes compensation table 1 and compensation table 2, compensation table combination 2 includes compensation table 2 and compensation table 3, and compensation table combination 4 includes compensation table 3 and compensation table 4.

[0105] It should be noted that the error value of each compensation table combination for color shading correction under different interpolation coefficients is determined, and the target compensation table combination and the target interpolation coefficient corresponding to the minimum error value are determined. The specific implementation method is the same as the above Figure 4 The implementation method of determining the target interpolation coefficient by using the iterative search method described in is the same, and the embodiment of the present application will not be repeated here. The target compensation table is determined according to the two first initial compensation tables in the target compensation table combination and the target interpolation coefficient. The specific implementation method is the same as the method described in the above step 303, and the embodiment of the present application will not be repeated here.

[0106] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 4 As shown, the electronic device includes: an image capturing module, an image signal processing hardware module, an image algorithm processing software module and an output module.

[0107] The image capture module includes an image sensor for capturing images, and specifically may be a camera phone, a video camera, a digital camera, or other forms of camera devices.

[0108] The image signal processing hardware module includes a black level correction module, a lens shading correction 1 module, an automatic white balance module, a de-mosaic module and other image processing modules (not shown in the figure). The lens shading correction 1 module is mainly used to determine a second target compensation table suitable for the current light source based on the target interpolation coefficient and two second initial compensation tables, thereby performing color shading correction processing on the target image.

[0109] The image algorithm processing software module includes a lens shading correction 2 module, other image algorithm modules and other image processing modules (not shown in the figure). The lens shading correction 2 module is mainly used to determine a first target compensation table suitable for the current light source based on the target interpolation coefficient and two first initial compensation tables, thereby performing color shading correction processing on the target image.

[0110] The output module includes a display and a memory. The display is used to display images, and may be a light emitting diode (LED) display or a liquid crystal display (LCD), an organic electroluminescence display (OLED), etc. The memory is used to store data, and may be, but not limited to, the following types: random access memory (RAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electric erasable programmable read only memory (EEPROM), etc.

[0111] The target image data output by the image sensor is input to the image signal hardware processing module, processed by modules such as black level correction, lens shading correction, and automatic white balance, and output to the image algorithm processing software module. In the lens shading correction 1 module, a second target compensation table suitable for the current light source is determined based on two second initial compensation tables and target interpolation coefficients, and image shading correction processing is performed based on the second target compensation table. In the image algorithm processing software module, after other image processing algorithms, lens shading correction 2, and other image algorithm processing, it is output to the display device for display. In the lens shading correction 2 module, a first target compensation table suitable for the current light source is determined based on two first initial compensation tables and target interpolation coefficients, and image shading correction processing is performed based on the second target compensation table.

[0112] Among them, the first initial compensation table in the image algorithm processing software module is obtained through the second initial compensation table in the image signal processing hardware module. Since the software module has a slow processing rate in the process of performing bicubic interpolation processing on the compensation table, and the first initial compensation table is obtained by amplifying the second initial compensation table by bicubic interpolation processing in advance, the software module directly obtains the first initial compensation table from the pre-stored initial compensation table without the need for bicubic interpolation processing. Therefore, based on the method described in this application, the efficiency of the software module in color shading correction processing can be improved. Based on the first initial target table and the target interpolation coefficient that are most suitable for the current light source, the target compensation table corresponding to the color temperature of the current light source is obtained, which is conducive to accurately compensating the color shading of the current light source and effectively improving the color shading correction effect.

[0113] It should be noted that Figure 5 This is only a schematic diagram of the structure of an electronic device provided in the embodiment of the present application. In actual situations, the structure of the electronic device may not include an image signal processing hardware module or an image algorithm processing software module, or the image signal processing hardware module and the image algorithm processing software module may be output in parallel or in series-parallel combination, or the structure of the electronic device may also include other image processing modules, which are not limited in the embodiment of the present application. Figure 5 For details not mentioned in the article and the specific implementation methods of each step, please refer to Figure 1 and Figure 3 The illustrated embodiments and the aforementioned contents will not be described in detail here.

[0114] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a device provided in an embodiment of the present invention. The device may be an electronic device or a device having electronic device functions (such as a chip). The device may execute the steps related to the electronic device in the aforementioned method embodiment. Specifically, Figure 6As shown, the device 600 may include an acquisition unit 601 and a processing unit 602 .

[0115] The acquisition unit 601 is used to acquire a first initial compensation table from a pre-stored initial compensation table. The first initial compensation table is a compensation table obtained by magnifying the second initial compensation table by K times after bicubic interpolation processing. The second initial compensation table is used by the hardware module to perform color shading correction processing on the image. The width of the second initial compensation table is M, the length of the second initial compensation table is N, and the second initial compensation table stores M×N compensation coefficients. The width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, and the first initial compensation table stores K×M×K×N compensation coefficients, where M, N and K are all positive integers greater than 1. The processing unit 602 is used to call the software module to perform color shading correction processing on the target image based on the first initial compensation table.

[0116] In a possible implementation, the processing unit 602 is further used to: S1, use the bicubic interpolation method to enlarge the second initial compensation table by L times to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K; S2, use the bicubic interpolation method to enlarge the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1; S3, use the bilinear interpolation method to enlarge the third initial compensation table by P times to obtain a fifth initial compensation table; S4, calculate the error between the fourth initial compensation table and the fifth initial compensation table; if the error is less than or equal to a first threshold, determine that the third initial compensation table is the first initial compensation table, and save the first initial compensation table; if the error is greater than the first threshold, increase the value of L, and re-execute steps S1 to S4.

[0117] In a possible implementation, the processing unit 602 is further configured to: if the value of L is greater than or equal to a second threshold, determine that the third initial compensation table obtained by the last execution of step S1 is the first initial compensation table, and save the first initial compensation table.

[0118] In a possible implementation, there are multiple second initial compensation tables, and the pre-stored initial compensation tables include multiple first initial compensation tables, one first initial compensation table corresponds to one color temperature, and the color temperatures corresponding to the multiple first initial compensation tables are different from each other; the acquisition unit 601 acquires the first initial compensation table from the pre-stored initial compensation table, and is specifically used to: determine two first initial compensation tables from the pre-stored initial compensation table based on the color temperature of the target image, and the color temperature of the target image is between the two color temperatures corresponding to the two first initial compensation tables; the processing unit 602 calls the software module to perform color shading correction processing on the target image based on the first initial compensation table, and is specifically used to: determine the first target compensation table based on the two first initial compensation tables and the target interpolation coefficient; and call the software module to perform color shading correction processing on the target image based on the first target compensation table.

[0119] In a possible implementation, the processing unit 602 is further used to: determine a plurality of first optional interpolation coefficients from a first value range according to a first step length, wherein the value of the first step length is less than the difference between the maximum value and the minimum value in the first value range, and the first step length is the difference between two adjacent first optional interpolation coefficients; S5, perform color shading correction on the downsampled target image based on two first initial compensation tables and the plurality of first optional interpolation coefficients to obtain a plurality of corrected statistical value images; S6, determine a plurality of first error values ​​corresponding to the plurality of first optional interpolation coefficients based on the plurality of corrected statistical value images; if a preset iteration condition is not satisfied, determine a plurality of second optional interpolation coefficients based on the first optional interpolation coefficient corresponding to the minimum first error value among the plurality of first error values, and use the plurality of second optional interpolation coefficients as the plurality of new first optional interpolation coefficients, and execute step S5 and step S6 again, wherein the preset iteration condition is that the minimum first error value among the plurality of first error values ​​is less than a third threshold value or the number of iterations exceeds a fourth threshold value; if the preset iteration condition is satisfied, determine the interpolation coefficient corresponding to the minimum error value among the plurality of first error values ​​when step S6 is executed for the last time as the target interpolation coefficient.

[0120] In a possible implementation, the processing unit 602 determines multiple second optional interpolation coefficients based on the first optional interpolation coefficient corresponding to the smallest first error value among multiple first error values, and is specifically used to: determine the second value range according to the first optional interpolation coefficient corresponding to the smallest first error value among the multiple first error values; determine multiple second optional interpolation coefficients from the second value range according to the second step size, wherein the second step size is smaller than the difference between the maximum value and the minimum value in the second value range, and the second step size is the difference between two adjacent first optional interpolation coefficients.

[0121] In a possible implementation, the processing unit 602 determines multiple first error values ​​corresponding to multiple first optional interpolation coefficients based on multiple corrected statistical value images, and is specifically used to: convert the ratio of the R channel to the G channel of the sub-image block in the statistical value image corrected based on the first optional interpolation coefficient to a logarithmic domain to obtain a first sub-error value, wherein the corrected statistical value image includes multiple sub-image blocks; convert the ratio of the B channel to the G channel of the sub-image block to a logarithmic domain to obtain a second sub-error value; determine the variance of the first sub-error value and the second sub-error value as the error value of the sub-image block; and determine the first error value corresponding to the first optional interpolation coefficient based on the error values ​​of multiple sub-image blocks of the statistical value image.

[0122] The present application also provides a chip that can execute the steps of the electronic device in the above method embodiment. The chip includes a processor and a communication interface.

[0123] The chip is used to: obtain a first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is amplified K times after bicubic interpolation processing is performed on the second initial compensation table, the second initial compensation table is used by a hardware module to perform color shading correction processing on an image, the width of the second initial compensation table is M, the length of the second initial compensation table is N, the second initial compensation table stores M×N compensation coefficients, the width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, the first initial compensation table stores K×M×K×N compensation coefficients, and M, N and K are all positive integers greater than 1; and call a software module to perform color shading correction processing on a target image based on the first initial compensation table.

[0124] In a possible implementation, the chip is also used for: S1, using the bicubic interpolation method to amplify the second initial compensation table by L times to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K; S2, using the bicubic interpolation method to amplify the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1; S3, using the bilinear interpolation method to amplify the third initial compensation table by P times to obtain a fifth initial compensation table; S4, calculating the error between the fourth initial compensation table and the fifth initial compensation table; if the error is less than or equal to a first threshold, determining that the third initial compensation table is the first initial compensation table, and saving the first initial compensation table; if the error is greater than the first threshold, increasing the value of L, and re-executing steps S1 to S4.

[0125] In a possible implementation, the chip is further configured to: if the value of L is greater than or equal to a second threshold, determine that the third initial compensation table obtained by the last execution of step S1 is the first initial compensation table, and save the first initial compensation table.

[0126] In a possible implementation, there are multiple second initial compensation tables, and the pre-stored initial compensation tables include multiple first initial compensation tables, one first initial compensation table corresponds to one color temperature, and the color temperatures corresponding to the multiple first initial compensation tables are different from each other; the chip obtains the first initial compensation table from the pre-stored initial compensation table, specifically for: determining two first initial compensation tables from the pre-stored initial compensation table based on the color temperature of the target image, the color temperature of the target image is between the two color temperatures corresponding to the two first initial compensation tables; the chip calls the software module to perform color shading correction processing on the target image based on the first initial compensation table, specifically for: determining the first target compensation table based on the two first initial compensation tables and the target interpolation coefficient; calling the software module to perform color shading correction processing on the target image based on the first target compensation table.

[0127] In a possible implementation, the chip is further used to: determine multiple first optional interpolation coefficients from a first value range according to a first step length, wherein the value of the first step length is less than the difference between the maximum value and the minimum value in the first value range, and the first step length is the difference between two adjacent first optional interpolation coefficients; S5, perform color shading correction on the downsampled target image based on two first initial compensation tables and multiple first optional interpolation coefficients to obtain multiple corrected statistical value images; S6, determine multiple first error values ​​corresponding to the multiple first optional interpolation coefficients based on the multiple corrected statistical value images; if the preset iteration condition is not met, determine multiple second optional interpolation coefficients based on the first optional interpolation coefficient corresponding to the minimum first error value among the multiple first error values, and use the multiple second optional interpolation coefficients as new multiple first optional interpolation coefficients, and execute step S5 and step S6 again, the preset iteration condition is that the minimum first error value among the multiple first error values ​​is less than the third threshold or the number of iterations exceeds the fourth threshold; if the preset iteration condition is met, the interpolation coefficient corresponding to the minimum error value among the multiple first error values ​​when step S6 is executed for the last time is determined as the target interpolation coefficient.

[0128] In a possible implementation, the chip determines multiple second optional interpolation coefficients based on the first optional interpolation coefficient corresponding to the smallest first error value among multiple first error values, and is specifically used to: determine the second value range according to the first optional interpolation coefficient corresponding to the smallest first error value among the multiple first error values; determine multiple second optional interpolation coefficients from the second value range according to the second step size, wherein the second step size is smaller than the difference between the maximum value and the minimum value in the second value range, and the second step size is the difference between two adjacent first optional interpolation coefficients.

[0129] In a possible implementation, the chip determines multiple first error values ​​corresponding to multiple first optional interpolation coefficients based on multiple corrected statistical value images, and is specifically used to: convert the ratio of the R channel to the G channel of the sub-image block in the statistical value image corrected based on the first optional interpolation coefficient to a logarithmic domain to obtain a first sub-error value, the corrected statistical value image includes multiple sub-image blocks; convert the ratio of the B channel to the G channel of the sub-image block to the logarithmic domain to obtain a second sub-error value; determine the variance of the first sub-error value and the second sub-error value as the error value of the sub-image block; and determine the first error value corresponding to the first optional interpolation coefficient based on the error values ​​of multiple sub-image blocks of the statistical value image.

[0130] See also Figure 7 , Figure 7 700 may include a memory 701 and a processor 702. Optionally, the electronic device may further include a communication interface 703. The memory 701, the processor 702 and the communication interface 703 are connected via one or more communication buses. The communication interface 703 is controlled by the processor 702 for sending and receiving information. Optionally, the electronic device may further include a camera 704 and / or a display 705, the camera 704 is used to capture images, and the display 705 is used to display images. Optionally, the electronic device may be an electronic device that is integrated with a camera module, has a shooting function and needs to perform color shading correction, such as a mobile phone, an industrial camera, a web camera, a tablet computer camera, a game console camera, a video-enabled smart watch, a security camera, a camera module, etc.

[0131] The memory 701 may include a read-only memory and a random access memory, and provides instructions and data to the processor 702. A portion of the memory 701 may also include a nonvolatile random access memory.

[0132] The communication interface 703 is used to receive or send data.

[0133] The processor 702 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or alternatively, the processor 702 may be any conventional processor, etc. Among them:

[0134] The memory 701 is used to store program instructions.

[0135] The processor 702 is used to call the program instructions stored in the memory 701.

[0136] The processor 702 calls the program instructions stored in the memory 701 to enable the electronic device 700 to execute the method executed by the electronic device in the above method embodiment.

[0137] like Figure 8 As shown, Figure 8 800 can execute the steps of the electronic device in the above method embodiment, and the module device 800 includes: a communication module 801, a power module 802, a storage module 803 and a chip 804.

[0138] Among them, the power module 802 is used to provide power to the module device; the storage module 803 is used to store data and instructions; the communication module 801 is used for internal communication of the module device, or for the module device to communicate with external devices; the chip 804 is used to execute the method executed by the electronic device in the above method embodiment.

[0139] It should be noted that Figure 7 and Figure 8 The contents not mentioned in the corresponding embodiments and the specific implementation methods of each step can be found in Figure 1 and Figure 3 The illustrated embodiments and the aforementioned contents will not be described in detail here.

[0140] An embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is executed on a processor, the method flow of the above method embodiment is implemented.

[0141] The embodiment of the present application also provides a computer program product. When the computer program product runs on a processor, the method flow of the above method embodiment is implemented.

[0142] Regarding the various modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, or hardware modules / units, or they can be partially software modules / units and partially hardware modules / units. For example, for various devices and products applied to or integrated in a chip, the various modules / units included therein can all be implemented in the form of hardware such as circuits, or at least some of the modules / units can be implemented in the form of software programs, which run on the integrated processor inside the chip, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in a chip module, the various modules / units included therein can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same part of the chip module (such as a chip, circuit module, etc.) or in different components, or at least some of the modules / units can be implemented in the form of software programs. The software programs run on the integrated processor inside the chip, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits. It can be implemented in the form of a software program, which runs on a processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated in the terminal, the modules / units contained therein can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, chip, circuit module, etc.) or in different components in the terminal, or, at least some modules / units can be implemented in the form of a software program, which runs on a processor integrated inside the terminal, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits.

[0143] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain operations can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0144] The descriptions of the various embodiments provided in this application can refer to each other, and the descriptions of the various embodiments have different focuses. For parts that are not described in detail in a certain embodiment, refer to the relevant descriptions of other embodiments. For the convenience and simplicity of description, for example, the functions of the various devices and equipment provided in the embodiments of this application and the operations performed can refer to the relevant descriptions of the method embodiments of this application, and the various method embodiments and the various device embodiments can also refer to, combine or quote each other.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: S1. Amplify the second initial compensation table by L times using a bicubic interpolation method to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K; S2, using a bicubic interpolation method to amplify the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1; S3, using a bilinear interpolation method to amplify the third initial compensation table by P times to obtain a fifth initial compensation table; S4, calculating the error between the fourth initial compensation table and the fifth initial compensation table; If the error is less than or equal to a first threshold, determining the third initial compensation table as the first initial compensation table, and saving the first initial compensation table; If the error is greater than the first threshold, increase the value of L and re-execute steps S1 to S4; If the value of L is greater than or equal to the second threshold, determining that the third initial compensation table obtained by the last execution of step S1 is the first initial compensation table, and saving the first initial compensation table; Obtaining the first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is K times larger than the second initial compensation table after bicubic interpolation processing, the second initial compensation table is used by the hardware module to perform color shading correction processing on the image, the width of the second initial compensation table is M, the length of the second initial compensation table is N, the second initial compensation table stores M×N compensation coefficients, the width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, the first initial compensation table stores K×M×K×N compensation coefficients, and the M, the N, and the K are all positive integers greater than 1; The software module is called to perform color shading correction processing on the target image based on the first initial compensation table.

2. The method according to claim 1, characterized in that: The number of the second initial compensation tables is multiple, the pre-stored initial compensation tables include multiple first initial compensation tables, one first initial compensation table corresponds to one color temperature, and the color temperatures corresponding to the multiple first initial compensation tables are different from each other; The acquiring the first initial compensation table from the pre-stored initial compensation table comprises: determining two first initial compensation tables from the pre-stored initial compensation table based on the color temperature of the target image, wherein the color temperature of the target image is between the two color temperatures corresponding to the two first initial compensation tables; Calling a software module to perform color shading correction processing on the target image based on the first initial compensation table includes: Determining a first target compensation table based on the two first initial compensation tables and the target interpolation coefficients; The software module is called to perform color shading correction processing on the target image based on the first target compensation table.

3. The method according to claim 2, characterized in that Before determining the first target compensation table based on the two first initial compensation tables and the target interpolation coefficient, the method further includes: According to the first step length, a plurality of first optional interpolation coefficients are determined from a first value range, wherein the value of the first step length is less than the difference between the maximum value and the minimum value in the first value range, and the first step length is the difference between two adjacent first optional interpolation coefficients; S5, performing color shading correction on the downsampled target image based on the two first initial compensation tables and the plurality of first optional interpolation coefficients to obtain a plurality of corrected statistical value images; S6. Determine a plurality of first error values ​​corresponding to the plurality of first optional interpolation coefficients based on the plurality of corrected statistical value images; If the preset iteration condition is not met, a plurality of second optional interpolation coefficients are determined based on the first optional interpolation coefficient corresponding to the smallest first error value among the plurality of first error values, and the plurality of second optional interpolation coefficients are used as new plurality of first optional interpolation coefficients, and step S5 and step S6 are performed again, wherein the preset iteration condition is that the smallest first error value among the plurality of first error values ​​is less than the third threshold value or the number of iterations exceeds the fourth threshold value; If the preset iteration condition is met, the interpolation coefficient corresponding to the minimum error value among the multiple first error values ​​when step S6 is performed for the last time is determined as the target interpolation coefficient.

4. The method according to claim 3, characterized in that The determining of a plurality of second optional interpolation coefficients based on a first optional interpolation coefficient corresponding to a minimum first error value among the plurality of first error values ​​comprises: Determine a second value range according to a first optional interpolation coefficient corresponding to the smallest first error value among the multiple first error values; According to a second step length, a plurality of second optional interpolation coefficients are determined from the second value range, wherein the second step length is smaller than a difference between a maximum value and a minimum value in the second value range, and the second step length is a difference between two adjacent first optional interpolation coefficients.

5. The method according to claim 3 or 4, characterized in that: The determining, based on the plurality of corrected statistical value images, a plurality of first error values ​​corresponding to the plurality of first optional interpolation coefficients comprises: Converting a ratio of an R channel to a G channel of a sub-image block in a statistical value image corrected based on a first optional interpolation coefficient to a logarithmic domain to obtain a first sub-error value, wherein the corrected statistical value image includes a plurality of sub-image blocks; Converting the ratio of the B channel to the G channel of the sub-image block to a logarithmic domain to obtain a second sub-error value; Determine a variance of the first sub-error value and the second sub-error value as an error value of the sub-image block; Based on the error values ​​of the plurality of sub-image blocks of the statistical value image, a first error value corresponding to the first optional interpolation coefficient is determined.

6. An image processing device, characterized in that: The device comprises: an acquisition unit and a processing unit; The processing unit is used for: S1. Amplify the second initial compensation table by L times using a bicubic interpolation method to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K; S2, using a bicubic interpolation method to amplify the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1; S3, using a bilinear interpolation method to amplify the third initial compensation table by P times to obtain a fifth initial compensation table; S4, calculating the error between the fourth initial compensation table and the fifth initial compensation table; If the error is less than or equal to a first threshold, determining the third initial compensation table as the first initial compensation table, and saving the first initial compensation table; If the error is greater than the first threshold, increase the value of L and re-execute steps S1 to S4; If the value of L is greater than or equal to the second threshold, determining that the third initial compensation table obtained by the last execution of step S1 is the first initial compensation table, and saving the first initial compensation table; The acquisition unit is used to acquire a first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is amplified K times after bicubic interpolation processing is performed on the second initial compensation table, the second initial compensation table is used by the hardware module to perform color shading correction processing on the image, the width of the second initial compensation table is M, the length of the second initial compensation table is N, the second initial compensation table stores M×N compensation coefficients, the width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, the first initial compensation table stores K×M×K×N compensation coefficients, and the M, the N, and the K are all positive integers greater than 1; The processing unit is further used to call a software module to perform color shading correction processing on the target image based on the first initial compensation table.

7. A chip, characterized in that: Comprising a processor and a communication interface, the chip is used for: S1. Amplify the second initial compensation table by L times using a bicubic interpolation method to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K; S2, using a bicubic interpolation method to amplify the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1; S3, using a bilinear interpolation method to amplify the third initial compensation table by P times to obtain a fifth initial compensation table; S4, calculating the error between the fourth initial compensation table and the fifth initial compensation table; If the error is less than or equal to a first threshold, determining the third initial compensation table as the first initial compensation table, and saving the first initial compensation table; If the error is greater than the first threshold, increase the value of L and re-execute steps S1 to S4; If the value of L is greater than or equal to the second threshold, determining that the third initial compensation table obtained by the last execution of step S1 is the first initial compensation table, and saving the first initial compensation table; Obtaining the first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is K times larger than the second initial compensation table after bicubic interpolation processing, the second initial compensation table is used by the hardware module to perform color shading correction processing on the image, the width of the second initial compensation table is M, the length of the second initial compensation table is N, the second initial compensation table stores M×N compensation coefficients, the width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, the first initial compensation table stores K×M×K×N compensation coefficients, and the M, the N, and the K are all positive integers greater than 1; The software module is called to perform color shading correction processing on the target image based on the first initial compensation table.

8. An image processing module device, characterized in that: The module device includes a communication module, a power module, a storage module and a chip, wherein: The power module is used to provide electrical energy to the module device; The storage module is used to store data and instructions; The communication module is used for internal communication of the module device, or for communication between the module device and an external device; The chip is used for: S1. Amplify the second initial compensation table by L times using a bicubic interpolation method to obtain a third initial compensation table, where L is a positive integer greater than 1 and less than or equal to K; S2, using a bicubic interpolation method to amplify the second initial compensation table by P×L times to obtain a fourth initial compensation table, where P is a positive integer greater than 1; S3, using a bilinear interpolation method to amplify the third initial compensation table by P times to obtain a fifth initial compensation table; S4, calculating the error between the fourth initial compensation table and the fifth initial compensation table; If the error is less than or equal to a first threshold, determining the third initial compensation table as the first initial compensation table, and saving the first initial compensation table; If the error is greater than the first threshold, increase the value of L and re-execute steps S1 to S4; If the value of L is greater than or equal to the second threshold, determining that the third initial compensation table obtained by the last execution of step S1 is the first initial compensation table, and saving the first initial compensation table; Obtaining the first initial compensation table from a pre-stored initial compensation table, the first initial compensation table is a compensation table that is K times larger than the second initial compensation table after bicubic interpolation processing, the second initial compensation table is used by the hardware module to perform color shading correction processing on the image, the width of the second initial compensation table is M, the length of the second initial compensation table is N, the second initial compensation table stores M×N compensation coefficients, the width of the first initial compensation table is K×M, the length of the first initial compensation table is K×N, the first initial compensation table stores K×M×K×N compensation coefficients, and the M, the N, and the K are all positive integers greater than 1; The software module is called to perform color shading correction processing on the target image based on the first initial compensation table.

9. An image processing electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions so that the electronic device executes the method as claimed in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed on a device, the device executes the method according to any one of claims 1 to 5.

Images