Image processing method and system, electronic equipment, medium and program product

Abstract

The invention provides an image processing method and system, electronic equipment, a medium and a program product. The image processing method comprises the following steps: respectively acquiring first pixel information of a first image and second pixel information of a second image; determining deviation information of the first image and the second image according to the first pixel information and the second pixel information; determining a smoothing processing coefficient according to the deviation information in response to the deviation information meeting a preset condition; the smoothing processing coefficient is used for updating a first compensation table of the first image. According to the invention, the stability of LSC processing on the image under various shooting conditions is realized. Therefore, the image processing accuracy is improved, and the visual experience of the user is optimized.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, system, electronic device, medium, and program product. Background Art

[0002] Lens Shading Correction (LSC) is a common image processing technique. The output of the LSC module serves as input to the automatic white balance correction module. If multiple images of the same scene are captured with varying angles, the output of the LSC module may differ to a certain extent. This may lead to differences in the input statistical values of the automatic white balance image and the automatic white balance calculation results, resulting in inconsistent colors in the final output image. To address this issue, LSC is typically performed on the original image based on a compensation table after it is captured. Therefore, the key to LSC processing lies in optimizing the compensation table to improve the color consistency of the final output image. Summary of the Invention

[0003] The technical problem to be solved by the present disclosure is to overcome the defect in the prior art that the compensation table cannot well show the consistency of the colors of multiple frames of adjacent frames, and to provide an image processing method, system, electronic device, medium and program product.

[0004] The present disclosure solves the above technical problems through the following technical solutions:

[0005] The present disclosure provides an image processing method, the image processing method comprising:

[0006] respectively acquiring first pixel information of the first image and second pixel information of the second image;

[0007] determining deviation information between the first image and the second image according to the first pixel information and the second pixel information;

[0008] In response to the deviation information satisfying a preset condition, a smoothing coefficient is determined according to the deviation information; the smoothing coefficient is used to update a first compensation table of the first image.

[0009] Optionally, the first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values corresponding to the first pixel values; and determining the deviation information between the first image and the second image based on the first pixel information and the second pixel information includes:

[0010] Obtaining first average values of a plurality of first pixel values and second average values of corresponding second pixel values, respectively, and determining a pixel mean difference according to a difference between the first average values and the second average values;

[0011] and / or,

[0012] A first eigenvector is determined according to the first pixel value, a second eigenvector is determined according to the corresponding second pixel value, and cosine similarity is determined according to the first eigenvector and the second eigenvector.

[0013] Optionally, in response to the deviation information satisfying a preset condition, determining a smoothing coefficient according to the deviation information includes:

[0014] In response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, the smoothing coefficient is determined according to the pixel mean difference and / or the cosine similarity.

[0015] Optionally, determining the smoothing coefficient according to the pixel mean difference and / or the cosine similarity includes:

[0016] Determining the smoothing coefficient according to the pixel mean difference and the difference threshold;

[0017] or,

[0018] Determining the smoothing coefficient according to the cosine similarity and a similarity threshold;

[0019] or,

[0020] A first smoothing coefficient is determined according to the pixel mean difference and a difference threshold; a second smoothing coefficient is determined according to the cosine similarity and a similarity threshold; and the smoothing coefficient is determined according to the first smoothing coefficient and the second smoothing coefficient.

[0021] Optionally, the image processing method further includes:

[0022] updating the first compensation table according to the smoothing coefficient, the first compensation table for the first image, and the second compensation table for the second image; and performing lens shading correction on the first image based on the updated first compensation table;

[0023] and / or,

[0024] In response to the deviation information not satisfying a preset condition, performing lens shading correction on the first image based on a first compensation table of the first image.

[0025] Optionally, before the step of respectively acquiring the first pixel information of the first image and the second pixel information of the second image, the method further includes:

[0026] Acquire a plurality of preset compensation tables and combine two different preset compensation tables into a plurality of compensation table combinations; calculate errors of the compensation table combinations when performing lens shading correction on a target image; and determine a first compensation table based on a target compensation table combination corresponding to a minimum value of the errors and a corresponding interpolation coefficient;

[0027] and / or,

[0028] In response to the voting mechanism being turned on, the number of times the first compensation table that has been used is counted starting from the Nth frame image is determined according to a preset number of frames, where N is a positive integer; the first compensation table whose number of uses meets the preset quantity condition is used as the first compensation table for the first image.

[0029] The present disclosure further provides an image processing system, comprising:

[0030] an acquisition module, configured to respectively acquire first pixel information of the first image and second pixel information of the second image;

[0031] a first determining module, configured to determine deviation information between the first image and the second image based on the first pixel information and the second pixel information;

[0032] The second determining module is configured to determine a smoothing coefficient according to the deviation information in response to the deviation information satisfying a preset condition; the smoothing coefficient is used to update a first compensation table for the first image.

[0033] Optionally, the first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values corresponding to the first pixel values; and the first determining module is specifically configured to:

[0034] Obtaining first average values of a plurality of first pixel values and second average values of corresponding second pixel values, respectively, and determining a pixel mean difference according to a difference between the first average values and the second average values;

[0035] and / or,

[0036] A first eigenvector is determined according to the first pixel value, a second eigenvector is determined according to the corresponding second pixel value, and cosine similarity is determined according to the first eigenvector and the second eigenvector.

[0037] Optionally, the second determining module is specifically configured to:

[0038] In response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, the smoothing coefficient is determined according to the pixel mean difference and / or the cosine similarity.

[0039] Optionally, the second determining module is specifically configured to:

[0040] Determining the smoothing coefficient according to the pixel mean difference and the difference threshold;

[0041] or,

[0042] Determining the smoothing coefficient according to the cosine similarity and a similarity threshold;

[0043] or,

[0044] A first smoothing coefficient is determined according to the pixel mean difference and a difference threshold; a second smoothing coefficient is determined according to the cosine similarity and a similarity threshold; and the smoothing coefficient is determined according to the first smoothing coefficient and the second smoothing coefficient.

[0045] Optionally, the image processing system further includes:

[0046] an updating module, configured to update the first compensation table according to the smoothing coefficient, the first compensation table for the first image, and the second compensation table for the second image; and perform lens shading correction on the first image based on the updated first compensation table;

[0047] and / or,

[0048] A correction module is configured to perform lens shading correction on the first image based on a first compensation table of the first image in response to the deviation information not satisfying a preset condition.

[0049] Optionally, the image processing system further includes:

[0050] A compensation table generation module is configured to obtain a plurality of preset compensation tables and combine two different preset compensation tables into a plurality of compensation table combinations; calculate errors of each compensation table combination when performing lens shading correction on a target image; and determine a first compensation table based on a target compensation table combination corresponding to a minimum value of the errors and a corresponding interpolation coefficient;

[0051] and / or,

[0052] The compensation table generation module is further configured to determine, in response to the voting mechanism being enabled, the number of times the first compensation table that has been used starting from the Nth frame image according to a preset number of frames, where N is a positive integer; and use the first compensation table whose number of uses meets the preset quantity condition as the first compensation table for the first image.

[0053] The present disclosure also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and configured to run on the processor, wherein the processor implements any of the above-described image processing methods when executing the computer program.

[0054] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned image processing methods when executed by a processor.

[0055] The present disclosure also provides a computer program product, comprising a computer program, which implements any of the above-described image processing methods when executed by a processor.

[0056] The present disclosure also provides a chip having a computer program stored thereon, and when the computer program is executed by the chip, the image processing method as described in any one of the above items is implemented.

[0057] The present disclosure also provides a chip module, which is applied to electronic equipment, including a transceiver component and a chip, wherein the chip is used to implement any of the image processing methods described above.

[0058] On the basis of conforming to the common sense in this field, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0059] The positive progress of this disclosure lies in determining the deviation between the first and second images using first pixel information from the first image and second pixel information from the second image. When the deviation meets a preset condition, a smoothing coefficient is determined based on the first and second pixel information to dynamically adjust the compensation table, achieving stable LSC processing of images under various shooting conditions. This not only improves image processing accuracy but also optimizes the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flowchart of an existing image signal processing;

[0061] Figure 2 A flowchart of an image processing method provided by an exemplary embodiment of the present disclosure;

[0062] Figure 3 A flowchart of another image processing method provided by an exemplary embodiment of the present disclosure;

[0063] Figure 4 A flowchart of another image processing method provided by an exemplary embodiment of the present disclosure;

[0064] Figure 5A schematic diagram of a module of an image processing system provided by an exemplary embodiment of the present disclosure;

[0065] Figure 6 The present invention provides a structural diagram of an electronic device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present disclosure is further illustrated below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0067] In the embodiments of the present disclosure, prefixes such as "first" and "second" are used only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. In the embodiments of the present disclosure, the use of prefixes such as ordinal numbers to distinguish description objects does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitations should be constituted due to the use of such prefixes. In addition, in the description of this embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0068] Before describing the embodiments, the original intention of the present invention is first described. Figure 1 As can be seen, existing image signal processing generally requires processing through an image capture module, an image signal processing module, and an output module. The image capture module includes an image sensor, typically a CMOS sensor, which converts received light signals into electrical signals through a photoelectric conversion process. The image signal processing module specifically includes modules such as black level correction, lens shading correction, automatic white balance, and demosaicing. The output module includes a display and memory. The processed image signal can be displayed directly to the user through the display or stored in the memory for subsequent use. Lens shading includes luma shading (brightness shading) and color shading (color shading). Color shading is characterized by the color difference between the center and the surrounding area of the image. The lens shading correction module corrects the color shading in the image. Furthermore, the output of the lens shading correction module serves as input data for the automatic white balance correction module. If N images of the same scene are captured with some deviation in angle, the output of the lens shading correction module will vary to a certain extent. This may lead to differences in the input statistical values of the automatic white balance image and the automatic white balance calculation results, resulting in inconsistent color in the final output image. The disclosed technical solution is aimed at solving the above technical problems.

[0069] Example 1

[0070] Figure 2This is a flowchart of an image processing method provided by an exemplary embodiment of the present disclosure. As can be seen from the figure, the image processing method includes:

[0071] Step 101: Acquire first pixel information of a first image and second pixel information of a second image respectively.

[0072] In this step, the first image and the second image can be two consecutive frames in an image sequence (such as a video), or two images separated by a certain number of frames. The purpose of obtaining these two frames of image is to compare the differences between them and thus determine the subsequent processing flow. The pixel information includes but is not limited to the brightness value of each pixel (in a grayscale image) or the RGB color channel value (in a color image). This information is crucial for tasks such as calculating the difference between images, performing image registration, or color correction.

[0073] Let's take a concrete example: suppose we are processing two frames from a video sequence, with the first image being the current frame and the second being the reference frame. Read the first image, scan each pixel, and record the RGB values. For example, the pixel corresponding to the same position in the second image might be recorded as R = 210, G = 160, and B = 110. Similarly, read the second image, scan each pixel, and record their RGB values. For example, a pixel in the image might be recorded as R = 200, G = 150, and B = 100. This data can be stored in a two-dimensional array, where each element is a structure or class containing an RGB value.

[0074] Step 102: Determine deviation information between the first image and the second image based on the first pixel information and the second pixel information.

[0075] Optionally, the first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values corresponding to the first pixel values. The goal of step 102 is to determine the deviation information between the two images based on the pixel information of the two images. In image processing, this step is directly related to the accuracy and efficiency of subsequent processing. Optionally, step 102 specifically includes at least one of the following methods:

[0076] Method 1-1: Obtain the first average of several first pixel values and the second average of the corresponding second pixel values, and determine the pixel mean difference based on the difference between the first average and the second average. Specifically, it is necessary to obtain the first average of the first pixel value of each pixel point in the first image and the second average of the second pixel value of the corresponding pixel point in the second image, and calculate the difference between the first average and the second average. This difference represents the average difference between the two images at the pixel level, that is, the pixel mean difference, which is an important indicator for evaluating image similarity. If this mean difference is small, it means that the two images are relatively close in grayscale or color.

[0077] Method 1-2: Determine a first eigenvector based on the first pixel value, and determine a second eigenvector based on the corresponding second pixel value, and determine cosine similarity based on the first eigenvector and the second eigenvector. Specifically, a eigenvector is generated based on all pixel values of the first image, and similarly, a corresponding eigenvector is generated for the second image. Typically, this process involves arranging a row or column of pixel values in sequence to form a long vector. After obtaining two eigenvectors, calculate the cosine similarity between them. Cosine similarity determines the degree of similarity between two vectors by measuring the cosine value of the inner angle of the two vectors. The closer the value is to 1, the more similar the two vectors are.

[0078] For the pixel mean difference and cosine similarity mentioned above, here is a specific example: suppose there are two images of the same scene but taken at different times, both of size 3×3 pixels. To simplify the demonstration, only grayscale images are considered.

[0079] 1. Regarding calculating pixel mean difference, assume that the pixel values of the first image (from left to right, from top to bottom) are: [40, 50, 60, 71, 80, 92, 103, 110, 120], and the pixel values of the second image (from left to right, from top to bottom) are: [42, 52, 62, 72, 82, 92, 102, 112, 122].

[0080] Then calculate the sum of the pixel values of the first image as:

[0081] 40+50+60+71+80+92+103+110+120=72640+50+60+71+80+92+103+110+120=726.

[0082] Then the first average value of the first pixel values is: 726 / 9≈80.67.

[0083] Calculate the sum of the pixel values of the second image as:

[0084] 42+52+62+72+82+92+102+112+122=72842+52+62+72+82+92+102+112+122=728.

[0085] Then the second average value of the second pixel values is: 728 / 9≈80.89.

[0086] Finally, the difference between the two average values is calculated: 80.89-80.67=0.22.

[0087] It should be understood that if the final difference is negative, the absolute value of the difference can be taken. If the image is a three-channel RGB image, the calculation needs to be performed on the three channels separately according to the above principle.

[0088] 2. Regarding the calculation of cosine similarity, assume that the eigenvector of the first image is: [40, 50, 60, 70, 0, 90, 100, 110, 120], and the eigenvector of the second image is: [42, 52, 62, 72, 82, 92, 102, 112, 122]. Calculate the cosine similarity of these two vectors and calculate the dot product of the two eigenvectors as: (40*42+50*52+60*62+70*72+80*82+90*92+100*102+110*112+120*122)=113360. Then, calculate the modulus of the two eigenvectors, which are 394.35 and 420.37 respectively, and divide the dot product by the product of the modulus of the two vectors. Therefore, the cosine similarity is: 113360 / (394.35*420.37)≈0.669.

[0089] Step 103 : In response to the deviation information satisfying a preset condition, a smoothing coefficient is determined according to the deviation information; the smoothing coefficient is used to update a first compensation table of the first image.

[0090] This step involves determining a smoothing coefficient based on the deviation information between the two images, thereby updating the first compensation table for the first image. This step is crucial for achieving coherence and a natural transition in the image sequence. Optionally, step 103 specifically includes: in response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, determining a smoothing coefficient based on the pixel mean difference and / or the cosine similarity. Conversely, in response to the pixel mean difference being greater than the difference threshold and / or the cosine similarity being less than the similarity threshold, it indicates that the difference between the first and second pixel information is significant, and smoothing is unnecessary. LSC processing can be performed according to the current first compensation table for the first image. The difference threshold and similarity threshold can be set as needed. It should be understood that as the conditions for triggering the determination of the smoothing coefficient, at least one of the conditions of the pixel mean difference being less than or equal to the difference threshold and the cosine similarity being greater than or equal to the similarity threshold is sufficient. Furthermore, when these conditions are met, the smoothing coefficient can be determined based on at least one of the pixel mean difference and the cosine similarity.

[0091] Optionally, the smoothing coefficient is determined according to the pixel mean difference and / or cosine similarity, specifically including the following three processing methods:

[0092] Method 2-1: Determine the smoothing coefficient based on the pixel mean difference and the difference threshold. Optionally, this can be achieved using the following formula:

[0093]

[0094] Among them, r is the smoothing coefficient, a m is the pixel mean difference, a t is the difference threshold.

[0095] Method 2-2: Determine the smoothing coefficient based on the cosine similarity and the similarity threshold. Optionally, this can be achieved using the following formula:

[0096]

[0097] Among them, r is the smoothing coefficient, b m is the cosine similarity, b t is the similarity threshold.

[0098] Method 2-3: determining a first smoothing coefficient based on pixel mean difference and a difference threshold; determining a second smoothing coefficient based on cosine similarity and a similarity threshold; and determining a smoothing coefficient based on the first smoothing coefficient and the second smoothing coefficient.

[0099] Optionally, the first smoothing coefficient is determined according to the pixel mean difference and the difference threshold in Mode 2-3, and is implemented by the following formula:

[0100]

[0101] Among them, r1 is the first smoothing coefficient, a m is the pixel mean difference, a t is the difference threshold;

[0102] and / or,

[0103] The second smoothing coefficient is determined based on the cosine similarity and the similarity threshold in method 2-3, and is implemented by the following formula:

[0104]

[0105] Among them, r2 is the second smoothing coefficient, b m is the cosine similarity, b t is the similarity threshold;

[0106] and / or,

[0107] The method 2-3 determines the smoothing coefficient based on the first smoothing coefficient and the second smoothing coefficient, which is implemented by the following formula:

[0108] r=k1*r1+k2*r2;

[0109] Wherein, r1 is the first smoothing coefficient, r2 is the second smoothing coefficient, r is the smoothing coefficient, k1 is the first weight coefficient, k2 is the second weight coefficient. The first weight coefficient and the second weight coefficient can be set according to actual needs.

[0110] Here is a specific example: suppose that when processing two frames of images (the first image and the second image) in a video sequence, the difference in the pixel mean between the two frames is 5, and the difference threshold is set to 10. The cosine similarity between the two frames is 0.95, and the similarity threshold is set to 0.8. The three ways to calculate the smoothing coefficient are:

[0111] Mode 2-1: The smoothing coefficient calculated based on the pixel mean difference and the difference threshold is: 1-(5 / 10)=0.5.

[0112] Mode 2-2: The smoothing coefficient calculated based on the cosine similarity and the similarity threshold is: (0.95-0.8) / (1-0.8)=0.75.

[0113] Mode 2-3: The first smoothing coefficient calculated according to the pixel mean difference and the difference threshold is: 1-(5 / 10)=0.5.

[0114] The second smoothing coefficient calculated according to the cosine similarity and the similarity threshold is: (0.95-0.8) / (1-0.8)=0.75.

[0115] Assuming that the first weight coefficient and the second weight coefficient are 0.4 and 0.6 respectively, the final smoothing coefficient is: 0.4*0.5+0.6*0.75=0.65.

[0116] In summary, step 103 reasonably determines the smoothing coefficient by evaluating the deviation information, thereby effectively adjusting the compensation table in the image sequence to achieve a more coherent and natural visual effect. The compensation table can well show consistency in the colors of multiple frames of adjacent frames.

[0117] Figure 3 A flowchart of another image processing method provided by an exemplary embodiment of the present disclosure. Optionally, based on steps 101 to 103 above, the image processing method may further include at least one of steps 104 and 105, wherein step 102 may be followed by step 105, and step 103 may be followed by step 104:

[0118] Step 104: Update the first compensation table according to the smoothing coefficient, the first compensation table for the first image, and the second compensation table for the second image, and perform lens shading correction on the first image based on the updated first compensation table. Optionally, step 104 is implemented by the following formula:

[0119] L = L1*r + L2*(1-r), or L = L2*r + L1*(1-r);

[0120] Wherein, L1 is the first compensation table, L2 is the second compensation table, L is the updated first compensation table, and r is the smoothing coefficient.

[0121] The purpose of this step is to use the smoothing coefficient to update the first compensation table of the first image so as to make it more adaptable to the current image environment, thereby improving the coherence and naturalness of the image sequence.

[0122] Step 105 : In response to the deviation information not satisfying a preset condition, performing lens shading correction on the first image based on a first compensation table of the first image.

[0123] The purpose of this step is to correct for large discrepancies between the two images when the deviation information does not meet the preset conditions, indicating that the difference between the two frames is significant, and directly updating the compensation table may not produce the desired effect. In this case, the system will select the original compensation table for the first image to perform lens shading correction to ensure image quality. This step typically involves using the compensation table for the first image to perform lens shading correction on the current image, without relying on information related to the second image. This approach ensures effective lens shading correction even when image content changes significantly or ambient lighting conditions fluctuate significantly, avoiding poor correction results due to large differences between images. For example, if a video suddenly transitions from an indoor scene to an outdoor scene, the significant changes in lighting conditions and shadow characteristics may make the smoothing coefficients and compensation table based on the indoor scene ineffective. In this case, the system will recognize that the deviation information does not meet the preset conditions (for example, if the pixel mean difference suddenly increases) and select the compensation table for the current image (outdoor scene) to perform lens shading correction to ensure the accuracy of the correction effect.

[0124] Through these steps, the image processing method can flexibly respond to different types of scene changes and ensure the best image processing effect under different conditions.

[0125] Optionally, step 101, before the step of respectively acquiring the first pixel information of the first image and the second pixel information of the second image, includes at least one of the following preprocessing steps:

[0126] Preprocessing step 1: obtain several preset compensation tables, and combine two different preset compensation tables into several compensation table combinations; calculate the errors of the compensation table combinations in lens shading correction of the target image; determine the first compensation table according to the target compensation table combination corresponding to the minimum value of the error and the corresponding interpolation coefficient.

[0127] The purpose of this step is to select the optimal compensation table combination from multiple preset compensation tables for subsequent lens shading correction, thereby ensuring optimal correction results. Specifically, the system obtains multiple preset compensation tables, which are typically pre-generated based on different scene or lighting conditions. The system then combines these tables into multiple compensation table combinations. For each combination, the system calculates the error after lens shading correction of the target image using that combination. This typically involves applying the combination to a series of test images and evaluating the difference between the corrected results and the standard image. Based on the minimum error, the system selects the optimal compensation table combination. This combination and its corresponding interpolation coefficients are used to determine the final first compensation table. For a specific example, suppose there are four preset compensation tables, A, B, C, and D. The system generates combinations such as AB, AC, AD, BC, BD, and CD. Each combination is used to perform lens shading correction on a series of test images, and the mean squared error between the corrected images and the standard image is calculated. Assuming that combination BD has the smallest error, it is selected as the optimal combination, and its corresponding interpolation coefficients are used to determine the first compensation table.

[0128] Preprocessing step 2: In response to the voting mechanism being turned on, the number of times the first compensation table that has been used is counted starting from the Nth frame image is determined based on a preset number of frames, where N is a positive integer; the first compensation table whose number of uses meets the preset number condition is used as the first compensation table for the first image.

[0129] The purpose of this step, when the voting mechanism is enabled, is to analyze historical data to select the first compensation table that best suits the current image sequence, ensuring consistent lens shading correction. Specifically, based on a preset frame number, N, the system counts the number of times the first compensation table has been used, starting with the Nth frame. The system obtains historical statistics of previously used first compensation tables. Based on these statistics, the system selects the first compensation table that meets a preset number of usage conditions as the current first compensation table. For example, suppose the voting mechanism is enabled at frame 10 (N=10). The system counts the number of times the first compensation table has been used, starting with frame 10. Historical statistics show that compensation table a has been used 30 times, compensation table b has been used 25 times, and compensation table c has been used only 5 times. If the preset number of usage conditions is the compensation table with the most usage, compensation table a will be selected as the first compensation table for the first image. This preset number of usage conditions can also be a threshold, exceeding which satisfies the preset number of usage conditions.

[0130] These two preprocessing steps provide a flexible compensation table selection mechanism for the image processing method, which can optimize the lens shading correction effect according to different scenes and historical data, thereby obtaining a better visual experience when processing images.

[0131] Based on the above steps, taking the camera function of a mobile phone as an example, the complete process of the image processing method is described as follows. For details, see Figure 4 . In a darkroom environment, set the mobile phone parameters to the shooting requirements. Cover the mobile phone camera with a frosted glass and aim it at the center of the light box. Take the original images for calibration under the uniform low-color-temperature light source A, medium-color-temperature light source B, and high-color-temperature light source C respectively. Calibrate the original images to obtain the compensation tables of the original images under each light source, denoted as table_A, table_B, and table_C respectively.

[0132] Step 201: Initialize multiple combinations of compensation tables. Select several fixed pairs of compensation tables, denoted as (table_A, table_B), (table_B, table_C), or combine them according to other principles to obtain N combinations of compensation tables.

[0133] Step 202: Calculate the optimal interpolation coefficients and errors of the N combinations of compensation tables respectively.

[0134] Step 203: Determine the target combination of compensation tables corresponding to the minimum value of the error and the corresponding interpolation coefficients.

[0135] Step 204: Determine whether to enable the voting mechanism. If it is enabled, go to Step 205; if it is not enabled, go to Step 206.

[0136] Step 205: Determine the target combination of compensation tables and the corresponding interpolation coefficients through the voting mechanism. The specific method is as follows:

[0137] (1) Find the minimum value of the error in the N combinations of compensation tables, denoted as min_error, and save the corresponding compensation table serial number i to min_index. min_index[j] = i when error_i < min_error, i ∈ 0 to (N - 1), and j = cnt % (M + 1), j ∈ 0 to M, cnt ∈ R. error_i is the i-th combination of compensation tables. N, M, and R are positive integers.

[0138] (2) For the first 0 to K frames, use the smoothing processing coefficient corresponding to min_error and the compensation table interpolated from the compensation table to obtain the compensation table of the current light source. K is a positive integer. <00003​​​​(4) Starting from the (M+1)th frame, calculate the number of times the current light source compensation table appears in the record, recorded as cnt_index. cnt_index[i]+=1, when i=min_index[j], where i∈0~N, j∈0~M.

[0141] (5) The voting threshold is set to vote_th. When cnt_index[i]>vote_th, the compensation table and smoothing coefficient corresponding to i are used to interpolate the compensation table of the current light source.

[0142] Step 206 : Determine a first compensation table according to the target compensation table combination determined in step 203 or step 205 and the corresponding interpolation coefficients.

[0143] Step 207 : Determine whether the smoothing function is enabled. If the smoothing function is enabled, proceed to step 208 ; if the smoothing function is not enabled, proceed to step 211 .

[0144] Step 208: Calculate the deviation between the first and second images. Based on the statistical differences between adjacent frames, determine whether the first compensation table requires smoothing and the degree of smoothing. Methods for calculating statistical differences between adjacent frames include the mean difference of the three R, G, and B channels and cosine similarity. For details, see Method 1-2 of Step 102.

[0145] Step 209: Determine whether the deviation information meets the preset conditions. If so, proceed to step 210; if not, proceed to step 211. For details, refer to method 2-3 of step 103.

[0146] Step 210 : Smoothing the compensation table. The smoothing process is to update the first compensation table according to the smoothing coefficient. For details, see step 104 .

[0147] Step 211: Output a first compensation table for performing LSC processing on the first image.

[0148] Step 212: LSC processing: performing LSC processing on the first image using the first compensation table.

[0149] The image processing method of this embodiment achieves stability in performing LSC processing on images under various shooting conditions, which not only improves the accuracy of image processing but also optimizes the user's visual experience.

[0150] Example 2

[0151] Corresponding to the aforementioned image processing method embodiment, the present disclosure also provides an embodiment of an image processing system.

[0152] Figure 5This is a module diagram of an image processing system provided by an exemplary embodiment of the present disclosure, the image processing system comprising:

[0153] An acquisition module 21 is configured to acquire first pixel information of the first image and second pixel information of the second image respectively;

[0154] A first determining module 22, configured to determine deviation information between the first image and the second image based on the first pixel information and the second pixel information;

[0155] The second determining module 23 is configured to determine a smoothing coefficient according to the deviation information in response to the deviation information satisfying a preset condition; the smoothing coefficient is used to update the first compensation table of the first image.

[0156] Optionally, the first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values corresponding to the first pixel values; the first determining module 22 is specifically configured to:

[0157] Obtaining first average values of a plurality of first pixel values and second average values of corresponding second pixel values, respectively, and determining a pixel mean difference according to a difference between the first average value and the second average value;

[0158] and / or,

[0159] A first eigenvector is determined according to the first pixel value, a second eigenvector is determined according to the corresponding second pixel value, and a cosine similarity is determined according to the first eigenvector and the second eigenvector.

[0160] Optionally, the second determining module 22 is specifically configured to:

[0161] In response to the pixel mean difference being less than or equal to the difference threshold and / or the cosine similarity being greater than or equal to the similarity threshold, a smoothing coefficient is determined according to the pixel mean difference and / or the cosine similarity.

[0162] Optionally, the second determining module 22 is specifically configured to:

[0163] Determine the smoothing coefficient according to the pixel mean difference and the difference threshold;

[0164] or,

[0165] Determine a smoothing coefficient based on cosine similarity and a similarity threshold;

[0166] or,

[0167] A first smoothing coefficient is determined according to the pixel mean difference and the difference threshold; a second smoothing coefficient is determined according to the cosine similarity and the similarity threshold; and a smoothing coefficient is determined according to the first smoothing coefficient and the second smoothing coefficient.

[0168] Optionally, the second determining module 22 is configured to implement the following formula:

[0169]

[0170] Among them, r is the smoothing coefficient, a m is the pixel mean difference, a t is the difference threshold;

[0171] or,

[0172] The second determining module 22 is used to implement the following formula:

[0173]

[0174] Among them, r is the smoothing coefficient, b m is the cosine similarity, b t is the similarity threshold.

[0175] Optionally, the second determining module 22 is configured to implement the following formula:

[0176]

[0177] Among them, r1 is the first smoothing coefficient, a m is the pixel mean difference, a t is the difference threshold;

[0178] and / or,

[0179] The second determining module 22 is used to implement the following formula:

[0180]

[0181] Among them, r2 is the second smoothing coefficient, b m is the cosine similarity, b t is the similarity threshold;

[0182] and / or,

[0183] The second determining module 22 is used to implement the following formula:

[0184] r=k1*r1+k2*r2;

[0185] Wherein, r1 is the first smoothing coefficient, r2 is the second smoothing coefficient, r is the smoothing coefficient, k1 is the first weight coefficient, and k2 is the second weight coefficient.

[0186] Optionally, the image processing system further includes:

[0187] An updating module, configured to update the first compensation table according to the smoothing coefficient, the first compensation table for the first image, and the second compensation table for the second image; and perform lens shading correction on the first image based on the updated first compensation table;

[0188] and / or,

[0189] The correction module is configured to perform lens shading correction on the first image based on a first compensation table of the first image in response to the deviation information not satisfying a preset condition.

[0190] Optionally, the update module is used to implement the following formula:

[0191] L = L1*r + L2*(1-r), or L = L2*r + L1*(1-r);

[0192] Wherein, L1 is the first compensation table, L2 is the second compensation table, L is the updated first compensation table, and r is the smoothing coefficient.

[0193] Optionally, the image processing system further includes:

[0194] The compensation table generation module is configured to obtain a plurality of preset compensation tables and combine two different preset compensation tables into a plurality of compensation table combinations; calculate the errors of the compensation table combinations when performing lens shading correction on a target image; and determine a first compensation table based on the target compensation table combination corresponding to the minimum error value and the corresponding interpolation coefficient;

[0195] and / or,

[0196] The compensation table generation module is also used to determine the number of times the first compensation table that has been used has been counted starting from the Nth frame image in response to the voting mechanism being turned on, based on a preset number of frames, where N is a positive integer; and use the first compensation table whose number of uses meets the preset number condition as the first compensation table for the first image.

[0197] The image processing system of this embodiment achieves stability in performing LSC processing on images under various shooting conditions, which not only improves the accuracy of image processing but also optimizes the user's visual experience.

[0198] Since the system embodiments generally correspond to the method embodiments, reference will be made to the description of the method embodiments for relevant details. The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components of the units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the disclosed solution.

[0199] Example 3

[0200] Figure 6 This is a structural diagram of an electronic device showing an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the image processing method described in any of the above embodiments. Figure 6 The electronic device 90 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0201] like Figure 6 As shown, the electronic device 90 may be a general-purpose computing device, such as a server device. Components of the electronic device 90 may include, but are not limited to, the at least one processor 91, the at least one memory 92, and a bus 93 connecting different system components (including the memory 92 and the processor 91).

[0202] The bus 93 includes a data bus, an address bus, and a control bus.

[0203] The memory 92 may include a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0204] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0205] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92, such as the image processing method provided in any of the above embodiments.

[0206] The electronic device 90 can also communicate with one or more external devices 94 (e.g., a keyboard, pointing device, etc.). Such communication can occur via an input / output (I / O) interface 95. Furthermore, the electronic device 90 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 96. As shown, the network adapter 96 communicates with other modules of the electronic device 90 via a bus 93. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the electronic device 90, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0207] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0208] Example 4

[0209] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method provided by any of the above embodiments.

[0210] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0211] Example 5

[0212] An embodiment of the present disclosure further provides a computer program product, comprising a computer program, which implements any of the above-mentioned image processing methods when executed by a processor.

[0213] The program code for executing the computer program product of the present disclosure may be written in any combination of one or more programming languages, and the program code may be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0214] Example 6

[0215] An embodiment of the present disclosure further provides a chip having a computer program stored thereon, and when the computer program is executed by the chip, any of the above-mentioned image processing methods is implemented.

[0216] Example 7

[0217] An embodiment of the present disclosure further provides a chip module, which is applied to an electronic device and includes a transceiver component and a chip, wherein the chip is used to implement any of the above-mentioned image processing methods.

[0218] While specific embodiments of the present disclosure have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of protection of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, and such changes and modifications are intended to fall within the scope of protection of the present disclosure.

Claims

1. An image processing method, characterized in that: The image processing method comprises: respectively acquiring first pixel information of the first image and second pixel information of the second image; determining deviation information between the first image and the second image according to the first pixel information and the second pixel information; In response to the deviation information satisfying a preset condition, a smoothing coefficient is determined according to the deviation information; the smoothing coefficient is used to update a first compensation table of the first image.

2. The image processing method according to claim 1, wherein: The first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values corresponding to the first pixel values; and determining the deviation information between the first image and the second image based on the first pixel information and the second pixel information includes: Obtaining first average values of a plurality of first pixel values and second average values of corresponding second pixel values, respectively, and determining a pixel mean difference according to a difference between the first average values and the second average values; and / or, A first eigenvector is determined according to the first pixel value, a second eigenvector is determined according to the corresponding second pixel value, and cosine similarity is determined according to the first eigenvector and the second eigenvector.

3. The image processing method according to claim 2, wherein: In response to the deviation information satisfying a preset condition, determining a smoothing coefficient according to the deviation information includes: In response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, the smoothing coefficient is determined according to the pixel mean difference and / or the cosine similarity.

4. The image processing method according to claim 3, wherein: The determining the smoothing coefficient according to the pixel mean difference and / or the cosine similarity includes: Determining the smoothing coefficient according to the pixel mean difference and the difference threshold; or, Determining the smoothing coefficient according to the cosine similarity and a similarity threshold; or, A first smoothing coefficient is determined according to the pixel mean difference and a difference threshold; a second smoothing coefficient is determined according to the cosine similarity and a similarity threshold; and the smoothing coefficient is determined according to the first smoothing coefficient and the second smoothing coefficient.

5. The image processing method according to claim 1, wherein: The image processing method further includes: updating the first compensation table according to the smoothing coefficient, the first compensation table for the first image, and the second compensation table for the second image; and performing lens shading correction on the first image based on the updated first compensation table; and / or, In response to the deviation information not satisfying a preset condition, performing lens shading correction on the first image based on a first compensation table of the first image.

6. The image processing method according to claim 1, wherein: The step of respectively acquiring the first pixel information of the first image and the second pixel information of the second image includes: Acquire a plurality of preset compensation tables and combine two different preset compensation tables into a plurality of compensation table combinations; calculate errors of the compensation table combinations when performing lens shading correction on a target image; and determine a first compensation table based on a target compensation table combination corresponding to a minimum value of the errors and a corresponding interpolation coefficient; and / or, In response to the voting mechanism being turned on, the number of times the first compensation table that has been used is counted starting from the Nth frame image is determined according to a preset number of frames, where N is a positive integer; the first compensation table whose number of uses meets the preset quantity condition is used as the first compensation table for the first image.

7. An image processing system, characterized in that: The image processing system comprises: an acquisition module, configured to respectively acquire first pixel information of the first image and second pixel information of the second image; a first determining module, configured to determine deviation information between the first image and the second image based on the first pixel information and the second pixel information; The second determining module is configured to determine a smoothing coefficient according to the deviation information in response to the deviation information satisfying a preset condition; the smoothing coefficient is used to update a first compensation table for the first image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and configured to run on the processor, wherein: When the processor executes the computer program, the image processing method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 6 is implemented.

Images