Image processing method, system, electronic device, medium, and program product

Abstract

The present disclosure provides an image processing method, system, electronic device, medium and program product. The image processing method comprises: acquiring first pixel information of a first image and second pixel information of a second image respectively; determining deviation information of the first image and the second image according to the first pixel information and the second pixel information; in response to the deviation information satisfying a preset condition, determining a smoothing processing coefficient according to the deviation information; and the smoothing processing coefficient is used to update a first compensation table of the first image. The present disclosure realizes the stability of LSC processing of images under various shooting conditions. This not only improves the accuracy of image processing, but also optimizes the visual experience of users.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to an image processing method, system, electronic device, medium, and program product. Background Technology

[0002] Lens Shading Correction (LSC) is a common image processing technique. The output of the LSC module serves as input data for the automatic white balance correction module. If N images of the same scene are taken with varying angles, the output of the lens shading correction module will differ, potentially leading to discrepancies in the input statistical values ​​for automatic white balance and consequently, inconsistencies in the automatic white balance calculations. To address this issue, LSC is typically performed on the original image after it has been captured, based on a compensation table. Therefore, the key to LSC processing lies in optimizing the compensation table to improve the color consistency of the final output image. Summary of the Invention

[0003] The technical problem to be solved by this disclosure is to overcome the defect in the prior art that the compensation table cannot well represent the consistency of color in multiple frames of adjacent images, and to provide an image processing method, system, electronic device, medium and program product.

[0004] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0005] This disclosure provides an image processing method, the image processing method comprising:

[0006] The first pixel information of the first image and the second pixel information of the second image are obtained respectively;

[0007] The deviation information between the first image and the second image is determined based on the first pixel information and the second pixel information;

[0008] In response to the deviation information satisfying a preset condition, a smoothing coefficient is determined based on the deviation information; the smoothing coefficient is used to update the first compensation table of the first image.

[0009] Optionally, the first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values ​​corresponding to the first pixel values; determining the deviation information between the first image and the second image based on the first pixel information and the second pixel information includes:

[0010] The first average value of several first pixel values ​​and the second average value of corresponding second pixel values ​​are obtained respectively, and the pixel mean difference is determined based on the difference between the first average value and the second average value.

[0011] And / or,

[0012] A first feature vector is determined based on the first pixel value, and a second feature vector is determined based on the corresponding second pixel value. Cosine similarity is then determined based on the first feature vector and the second feature vector.

[0013] Optionally, the step of determining a smoothing coefficient based on the deviation information in response to the deviation information satisfying a preset condition includes:

[0014] In response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, the smoothing coefficient is determined based on the pixel mean difference and / or the cosine similarity.

[0015] Optionally, determining the smoothing coefficient based on the pixel mean difference and / or the cosine similarity includes:

[0016] The smoothing coefficient is determined based on the pixel mean difference and the difference threshold.

[0017] or,

[0018] The smoothing coefficients are determined based on the cosine similarity and the similarity threshold.

[0019] or,

[0020] A first smoothing coefficient is determined based on the pixel mean difference and the difference threshold; a second smoothing coefficient is determined based on the cosine similarity and the similarity threshold; and the smoothing coefficient is determined based on the first smoothing coefficient and the second smoothing coefficient.

[0021] Optionally, the image processing method further includes:

[0022] The first compensation table is updated based on the smoothing coefficient, the first compensation table of the first image, and the second compensation table of the second image; lens shading correction is performed on the first image based on the updated first compensation table;

[0023] And / or,

[0024] If the deviation information does not meet the preset conditions, then lens shading correction is performed on the first image based on the first compensation table of the first image.

[0025] Optionally, the steps of obtaining the first pixel information of the first image and the second pixel information of the second image respectively are preceded by:

[0026] Obtain several preset compensation tables, and combine two different preset compensation tables into several compensation table combinations; calculate the error of the compensation table combination in lens shading correction of the target image; determine the first compensation table based on the target compensation table combination corresponding to the minimum value of the error and the corresponding interpolation coefficient;

[0027] And / or,

[0028] In response to the activation of the voting mechanism, the number of times the first compensation table has been used, starting from the Nth frame, is counted according to a preset number of frames, where N is a positive integer; the first compensation table whose number of uses meets a preset quantity condition is used as the first compensation table for the first image.

[0029] This disclosure also provides an image processing system, the image processing system comprising:

[0030] The acquisition module is used to acquire the first pixel information of the first image and the second pixel information of the second image, respectively;

[0031] The first determining module is used to determine the deviation information between the first image and the second image based on the first pixel information and the second pixel information;

[0032] The second determining module is used to determine a smoothing coefficient based on the deviation information in response to the deviation information meeting a preset condition; the smoothing coefficient is used to update the first compensation table of the first image.

[0033] Optionally, the first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values ​​corresponding to the first pixel values; the first determining module is specifically used for:

[0034] The first average value of several first pixel values ​​and the second average value of corresponding second pixel values ​​are obtained respectively, and the pixel mean difference is determined based on the difference between the first average value and the second average value.

[0035] And / or,

[0036] A first feature vector is determined based on the first pixel value, and a second feature vector is determined based on the corresponding second pixel value. Cosine similarity is then determined based on the first feature vector and the second feature vector.

[0037] Optionally, the second determining module is specifically used for:

[0038] In response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, the smoothing coefficient is determined based on the pixel mean difference and / or the cosine similarity.

[0039] Optionally, the second determining module is specifically used for:

[0040] The smoothing coefficient is determined based on the pixel mean difference and the difference threshold.

[0041] or,

[0042] The smoothing coefficients are determined based on the cosine similarity and the similarity threshold.

[0043] or,

[0044] A first smoothing coefficient is determined based on the pixel mean difference and the difference threshold; a second smoothing coefficient is determined based on the cosine similarity and the similarity threshold; and the smoothing coefficient is determined based on the first smoothing coefficient and the second smoothing coefficient.

[0045] Optionally, the image processing system further includes:

[0046] The update module is configured to update the first compensation table according to the smoothing coefficient, the first compensation table of the first image, and the second compensation table of the second image; and perform lens shading correction on the first image based on the updated first compensation table.

[0047] And / or,

[0048] The correction module is used to perform lens shading correction on the first image based on the first compensation table of the first image in response to the deviation information not meeting the preset conditions.

[0049] Optionally, the image processing system further includes:

[0050] The compensation table generation module is used to obtain several preset compensation tables and combine two different preset compensation tables into several compensation table combinations; calculate the error of the compensation table combination in lens shading correction of the target image; and determine the first compensation table based on the target compensation table combination corresponding to the minimum value of the error and the corresponding interpolation coefficient.

[0051] And / or,

[0052] The compensation table generation module is further configured to, in response to the activation of the voting mechanism, determine the number of times the first compensation table has been used since the Nth frame image based on a preset number of frames, where N is a positive integer; and use the first compensation table whose number of uses meets a preset quantity condition as the first compensation table for the first image.

[0053] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the image processing method described in any of the preceding claims.

[0054] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method described in any of the preceding claims.

[0055] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method as described in any of the preceding claims.

[0056] This disclosure also provides a chip on which a computer program is stored, which, when executed by the chip, implements the image processing method as described in any of the preceding claims.

[0057] This disclosure also provides a chip module for use in an electronic device, including a transceiver component and a chip, said chip being used to implement the image processing method as described in any of the preceding claims.

[0058] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.

[0059] The positive advancements of this disclosure lie in the following: by using the first pixel information of the first image and the second pixel information of the second image, the deviation information between the first and second images is determined. When the deviation information meets preset conditions, a smoothing coefficient is determined based on the first and second pixel information to dynamically adjust the compensation table, thus achieving stability in LSC processing of images under various shooting conditions. This not only improves the accuracy of image processing but also optimizes the user's visual experience. Attached Figure Description

[0060] Figure 1 Here is a flowchart of an existing image signal processing method;

[0061] Figure 2 A flowchart of an image processing method provided as an exemplary embodiment of this disclosure;

[0062] Figure 3 A flowchart of yet another image processing method provided as an exemplary embodiment of the present disclosure;

[0063] Figure 4 A flowchart illustrating another image processing method provided as an exemplary embodiment of this disclosure;

[0064] Figure 5A schematic diagram of a module of an image processing system provided for an exemplary embodiment of this disclosure;

[0065] Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0066] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.

[0067] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and should not be construed as an unnecessary limitation. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0068] Before describing the embodiments, the inventive intent of this disclosure will be explained first. See [link to relevant documentation]. Figure 1 It is known that existing image signal processing generally involves an image capture module, an image signal processing module, and an output module. The image capture module includes an image sensor, typically a CMOS sensor, which converts received light signals into electrical signals through a photoelectric conversion process. The image signal processing module specifically includes modules for black level correction, lens shading correction, automatic white balance, and de-mosaic. The output module includes a display and a memory. The processed image signal can be directly displayed to the user or stored in the memory for later use. Lens shading includes luma shading and color shading. Color shading represents the color difference between the center and the edges of the image, and the lens shading correction module corrects the color shading in the image. Furthermore, the output of the lens shading correction module serves as input data for the automatic white balance correction module. If N images of the same scene are captured with some angular deviations, the output of the lens shading correction module will differ to some extent, potentially leading to differences in the input statistical images for automatic white balance, resulting in inconsistent colors in the final output image. This disclosed technical solution aims to solve the above-mentioned technical problems.

[0069] Example 1

[0070] Figure 2A flowchart of an image processing method provided as an exemplary embodiment of this disclosure is shown in the figure. The image processing method includes:

[0071] Step 101: Obtain the first pixel information of the first image and the second pixel information of the second image respectively.

[0072] In this step, the first and second images can be two consecutive frames from an image sequence (e.g., a video), or two images spaced several frames apart. The purpose of acquiring these two images is to compare their differences, thereby determining the subsequent processing flow. The pixel information includes, but is not limited to, the brightness value of each pixel (in a grayscale image) or the RGB color channel values ​​(in a color image). This information is crucial for tasks such as calculating differences between images, performing image registration, or color correction.

[0073] Here's a concrete example: Suppose we're processing two frames from a video sequence, the first being the current frame and the second a reference frame. We read the first image, scan each pixel, and record its RGB values. For example, a pixel at the same location in the second image might be recorded as R=210, G=160, B=110. Similarly, we read the second image, scan each pixel, and record its RGB values. For example, a pixel in the image might be recorded as R=200, G=150, B=100. This data can be stored in a two-dimensional array, where each element is a structure or class containing RGB values.

[0074] Step 102: Determine the deviation information between the first image and the second image based on the first pixel information and the second pixel information.

[0075] Optionally, the first pixel information includes several first pixel values, and the second pixel information includes several second pixel values ​​corresponding to the first pixel values. The goal of step 102 is to determine the deviation information between the two images based on their pixel information. In image processing, this step directly affects the accuracy and efficiency of subsequent processing. Optionally, step 102 specifically includes at least one of the following methods:

[0076] Method 1-1: Obtain the first average value of several first pixel values ​​and the second average value of the corresponding second pixel values, and determine the pixel mean difference based on the difference between the first average value and the second average value. Specifically, it is necessary to obtain the first average value of the first pixel value of each pixel in the first image and the second average value of the corresponding second pixel value in the second image, and calculate the difference between the first average value and the second average value. This difference represents the average difference between the two images at the pixel level, i.e., the pixel mean difference, which is an important indicator for evaluating image similarity. If this mean difference is small, it indicates that the two images are relatively similar in grayscale or color.

[0077] Method 1-2: Determine a first feature vector based on the first pixel value, and a second feature vector based on the corresponding second pixel value. Then, determine the cosine similarity based on the first and second feature vectors. Specifically, generate a feature vector based on all pixel values ​​of the first image; similarly, generate a corresponding feature vector for the second image. Typically, this process involves arranging a row or column of pixel values ​​sequentially to form a long vector. After obtaining the two feature vectors, calculate their cosine similarity. Cosine similarity is determined by measuring the cosine of the interior angles of the two vectors; the closer the value is to 1, the more similar the two vectors are.

[0078] Regarding the pixel mean difference and cosine similarity mentioned above, here is a specific example: Suppose there are two images of the same scene but taken at different times, both 3×3 pixels in size. To simplify the demonstration, we will only consider grayscale images.

[0079] 1. Regarding the calculation of the pixel mean difference, assume that the pixel values ​​of the first image (from left to right, from top to bottom) are: [40, 50, 60, 71, 80, 92, 103, 110, 120], and the pixel values ​​of the second image (from left to right, from top to bottom) are: [42, 52, 62, 72, 82, 92, 102, 112, 122].

[0080] Then, the sum of the pixel values ​​of the first image is calculated as follows:

[0081] 40+50+60+71+80+92+103+110+120=726

[0082] The first average value of the first pixel is: 726 / 9≈80.67.

[0083] The sum of the pixel values ​​in the second image is calculated as follows:

[0084] 42+52+62+72+82+92+102+112+122=728

[0085] The second average value of the second pixel is: 728 / 9≈80.89.

[0086] Finally, calculate the difference between the two averages: 80.89 - 80.67 = 0.22.

[0087] It should be understood that if the final difference is negative, the absolute value of the difference can be taken. If the image has three RGB channels, then the same calculation needs to be performed separately for each of the three RGB channels.

[0088] 2. Regarding the calculation of cosine similarity, assume the feature vector of the first image is [40, 50, 60, 70, 0, 90, 100, 110, 120], and the feature vector of the second image is [42, 52, 62, 72, 82, 92, 102, 112, 122]. Calculate the cosine similarity between these two vectors. The dot product of the two feature vectors is: (40*42+50*52+60*62+70*72+80*82+90*92+100*102+110*112+120*122)=113360. Then, calculate the magnitudes of the two feature vectors as 394.35 and 420.37 respectively, and divide the dot product by the product of the magnitudes of the two vectors. Therefore, the cosine similarity is: 113360 / (394.35*420.37)≈0.669.

[0089] Step 103: In response to the deviation information meeting the preset conditions, the smoothing coefficients are determined based on the deviation information; the smoothing coefficients are used to update the first compensation table of the first image.

[0090] This step involves determining smoothing coefficients based on the deviation information between two images, and then updating the first compensation table of the first image. This step is crucial for achieving coherence and natural transitions in the image sequence. Optionally, step 103 specifically includes: in response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, determining smoothing coefficients based on the pixel mean difference and / or cosine similarity. Conversely, in response to the pixel mean difference being greater than a difference threshold and / or the cosine similarity being less than a similarity threshold, it indicates that the difference between the first pixel information and the second pixel information is large, and there is no need for smoothing; LSC processing can be performed based on the current first compensation table of the first image. The difference threshold and similarity threshold can be set according to actual needs. It should be understood that the pixel mean difference being less than or equal to the difference threshold and the cosine similarity being greater than or equal to the similarity threshold are conditions for triggering the determination of smoothing coefficients; at least one of these conditions needs to be met. Furthermore, once the above conditions are met, the smoothing coefficients can be calculated based on at least one of the pixel mean difference and cosine similarity.

[0091] Optionally, the smoothing coefficients are determined based on the difference in pixel mean and / or cosine similarity, specifically including the following three processing methods:

[0092] Method 2-1: Determine the smoothing coefficient based on the pixel mean difference and the difference threshold. Optionally, this can be achieved using the following formula:

[0093]

[0094] Where r is the smoothing coefficient, a m For the difference in pixel mean, a t This is the difference threshold.

[0095] Method 2-2: Determine the smoothing coefficients based on cosine similarity and a similarity threshold. Optionally, this can be achieved using the following formula:

[0096]

[0097] Where r is the smoothing coefficient, b m For cosine similarity, b t This is the similarity threshold.

[0098] Method 2-3: Determine the first smoothing coefficient based on the pixel mean difference and the difference threshold; determine the second smoothing coefficient based on the cosine similarity and the similarity threshold; determine the smoothing coefficient based on the first smoothing coefficient and the second smoothing coefficient.

[0099] Optionally, the first smoothing coefficient in methods 2-3, based on the difference in pixel mean and the difference threshold, is determined using the following formula:

[0100]

[0101] Where r1 is the first smoothing coefficient, a m For the difference in pixel mean, a t The difference threshold;

[0102] And / or,

[0103] The second smoothing coefficient, determined in methods 2-3 based on cosine similarity and a similarity threshold, is achieved using the following formula:

[0104]

[0105] Where r2 is the second smoothing coefficient, b m For cosine similarity, b t The similarity threshold;

[0106] And / or,

[0107] The smoothing coefficients determined in methods 2-3 based on the first and second smoothing coefficients are implemented using the following formula:

[0108] r = k1*r1 + k2*r2;

[0109] Where r1 is the first smoothing coefficient, r2 is the second smoothing coefficient, r is the smoothing coefficient, k1 is the first weighting coefficient, and k2 is the second weighting coefficient. The first and second weighting coefficients can be set according to actual needs.

[0110] Here's a concrete example: Suppose we're processing two frames (the first and second images) from a video sequence: the mean pixel difference between the two frames is 5, and the difference threshold is set to 10. The cosine similarity between the two frames is 0.95, and the similarity threshold is set to 0.8. Then, the three methods for calculating the smoothing coefficient are:

[0111] Method 2-1: The smoothing coefficient calculated based on the pixel mean difference and the difference threshold is: 1-(5 / 10)=0.5.

[0112] Method 2-2: The smoothing coefficient calculated based on cosine similarity and similarity threshold is: (0.95-0.8) / (1-0.8)=0.75.

[0113] Method 2-3: The first smoothing coefficient calculated based on the pixel mean difference and the difference threshold is: 1-(5 / 10)=0.5.

[0114] The second smoothing coefficient, calculated based on cosine similarity and similarity threshold, is: (0.95-0.8) / (1-0.8)=0.75.

[0115] Assuming the first and second weighting coefficients are 0.4 and 0.6 respectively, the final smoothing coefficient is: 0.4*0.5+0.6*0.75=0.65.

[0116] In summary, step 103, by evaluating the deviation information, reasonably determines the smoothing coefficients, thereby effectively adjusting the compensation table in the image sequence to achieve a more coherent and natural visual effect. The compensation table can effectively maintain color consistency across multiple adjacent frames.

[0117] Figure 3 This is a flowchart illustrating another image processing method provided as an exemplary embodiment of the present disclosure. Optionally, based on the above steps 101 to 103, the image processing method may further include at least one of steps 104 and 105, wherein step 105 may follow step 102, and step 104 may follow step 103.

[0118] Step 104: Update the first compensation table according to the smoothing coefficients, the first compensation table of the first image, and the second compensation table of the second image; perform lens shading correction on the first image based on the updated first compensation table. Optionally, step 104 can be implemented using the following formula:

[0119] L = L1*r + L2*(1-r), or L = L2*r + L1*(1-r);

[0120] Where L1 is the first compensation table, L2 is the second compensation table, L is the updated first compensation table, and r is the smoothing coefficient.

[0121] The purpose of this step is to update the first compensation table of the first image using smoothing coefficients, making it more adaptable to the current image environment, thereby improving the coherence and naturalness of the image sequence.

[0122] Step 105: In response to the deviation information not meeting the preset conditions, lens shadow correction is performed on the first image based on the first compensation table of the first image.

[0123] The purpose of this step is to correct lens shading when the deviation information does not meet the preset conditions, indicating a significant difference between the two images. Directly updating the compensation table may not yield ideal results. In this case, the system selects the original compensation table based on the first image for lens shading correction to ensure image quality. This step typically involves using the compensation table of the first image to correct lens shading of the current image, without relying on information related to the second image. This approach ensures effective lens shading correction even when image content changes significantly or ambient lighting conditions change substantially, avoiding poor correction results due to excessive differences between images. For example, if a video suddenly transitions from an indoor scene to an outdoor scene, the significant changes in lighting conditions and shadow characteristics may render the smoothing coefficients and compensation table based on the indoor scene inapplicable. In this situation, the system recognizes that the deviation information does not meet the preset conditions (e.g., a sudden increase in the difference in pixel mean), and then selects the compensation table of the current image (outdoor scene) for lens shading correction to ensure the accuracy of the correction.

[0124] Through these steps, the image processing method can flexibly adapt to different types of scene changes and ensure that the best image processing results can be obtained under different conditions.

[0125] Optionally, step 101, which involves obtaining the first pixel information of the first image and the second pixel information of the second image respectively, includes at least one of the following preprocessing steps:

[0126] Preprocessing step 1: Obtain several preset compensation tables and combine two different preset compensation tables into several compensation table combinations; calculate the error of the compensation table combination in lens shadow correction of the target image; determine the first compensation table based on the target compensation table combination corresponding to the minimum error and the corresponding interpolation coefficient.

[0127] The purpose of this step is to select the optimal combination of compensation tables from multiple preset compensation tables for subsequent lens shading correction, thereby ensuring optimal correction results. Specifically, the system acquires multiple preset compensation tables, typically generated in advance based on different types of scenes or lighting conditions. Then, the system combines these compensation tables in pairs to form multiple compensation table combinations. For each combination, the system calculates the error after lens shading correction of the target image using that combination. This typically involves applying the combination to a series of test images and evaluating the difference between the corrected result and the standard image. Based on the minimum error, the system selects the optimal compensation table combination. This combination and its corresponding interpolation coefficients are used to determine the final first compensation table. For example, suppose there are four preset compensation tables A, B, C, and D. The system will form combinations such as AB, AC, AD, BC, BD, and CD. Each combination will be used to perform lens shading correction on a series of test images, and the mean square error between the corrected image and the standard image will be calculated. Assuming that combination BD has the smallest error value, then BD will be selected as the optimal combination, and its corresponding interpolation coefficients will be used to determine the first compensation table.

[0128] Preprocessing step 2: In response to the activation of the voting mechanism, the number of times the first compensation table has been used, starting from the Nth frame, is counted according to the preset frame number, where N is a positive integer; the first compensation table whose number of uses meets the preset quantity condition is used as the first compensation table of the first image.

[0129] The purpose of this step, when the voting mechanism is activated, is to select the most suitable first compensation table for the current image sequence by analyzing historical data, ensuring consistency in lens shading correction. Specifically, the system counts the number of times the first compensation table is used starting from frame N, based on a preset frame number N. The system retrieves historical statistics of the used first compensation tables. Based on these statistics, the system selects the first compensation table that meets a preset quantity condition as the current first compensation table. For example, assuming the voting mechanism is activated at frame 10 (N=10), the system counts the number of times the first compensation table is used starting from frame 10. Historical statistics show that compensation table a was used 30 times, compensation table b was used 25 times, and compensation table c was used only 5 times. If the preset quantity condition is the compensation table with the most uses, then compensation table a will be selected as the first compensation table for the first image. This preset quantity condition can also be a threshold value; exceeding this threshold satisfies the preset quantity condition.

[0130] These two preprocessing steps provide a flexible compensation table selection mechanism for image processing methods, which can optimize the lens shadow correction effect according to different scenes and historical data, thereby obtaining a better visual experience when processing images.

[0131] Based on the above steps, the complete process of the image processing method is described by taking the camera function of a mobile phone as an example, for details, please refer to Figure 4 . In a dark room environment, adjust the parameters of the mobile phone to meet the shooting requirements, cover the mobile phone camera with frosted glass, align it with the center of the light box, and shoot the original images for calibration respectively under uniform low color temperature light source A, medium color temperature light source B, and high color temperature light source C. Calibrate the original images to obtain the compensation tables of the original images under each light source, which are recorded as table_A, table_B, and table_C respectively.

[0132] Step 201: Initialize a plurality of compensation table combinations. Select several fixed pairs of compensation tables, recorded as (table_A, table_B), (table_B, table_C) respectively, or combine them according to other principles to obtain N groups of compensation table combinations.

[0133] Step 202: Calculate the optimal interpolation coefficients and errors of the N groups of compensation table combinations respectively.

[0134] Step 203: Determine the target compensation table combination corresponding to the minimum error and the corresponding interpolation coefficient.

[0135] Step 204: Determine whether the voting mechanism is enabled. If it is enabled, go to step 205; if it is not enabled, go to step 206.

[0136] Step 205: Determine the target compensation table combination and the corresponding interpolation coefficient through the voting mechanism. The specific method is as follows:

[0137] (1) Find the minimum error value among the N groups of compensation table combinations, record it as min_error, and save the sequence number i of the corresponding compensation table into min_index. min_index[j]=i, when error_i<min_error, i∈0~(N-1), and j=cnt%(M+1), j∈0~M, cnt∈R. error_i is the i-th compensation table combination. N, M, and R are positive integers.

[0138] (2) For the first 0 to K frames, the compensation table of the current light source obtained by interpolation using the smoothing coefficient and compensation table corresponding to min_error. K is a positive integer.

[0139] (3) For the (K+1)th to (K+M)th frames, the table selection result of the camera just started may be unstable. In order to make the camera stabilize for several frames, the target compensation table combination determined in step 203 and the corresponding interpolation coefficient are used to determine the first compensation table.

[0140] (4) Starting from frame (M+1), calculate the number of times the current light source compensation table appears in the record, denoted as cnt_index. cnt_index[i]+=1, when i=min_index[j], where i∈0~N, j∈0~M.

[0141] (5) The voting threshold is set to vote_th. When cnt_index[i]>vote_th, the compensation table and smoothing coefficients corresponding to i are used to interpolate to obtain the compensation table of the current light source.

[0142] Step 206: Determine the first compensation table based on the target compensation table combination determined in step 203 or step 205 and the corresponding interpolation coefficients.

[0143] Step 207: Determine whether the smoothing function is enabled. If the smoothing function is enabled, proceed to step 208; otherwise, proceed to step 211.

[0144] Step 208: Calculate the deviation information between the first image and the second image. Based on the differences in statistical values ​​between adjacent frames, determine whether the first compensation table needs smoothing, and the degree of smoothing. The method for calculating the differences in statistical values ​​between adjacent frames includes the difference in the mean values ​​of the three statistical channels R, G, and B, and the cosine similarity. See method 1-2 in step 102 for details.

[0145] Step 209: Determine whether the deviation information meets the preset conditions. If the preset conditions are met, proceed to step 210; otherwise, proceed to step 211. See method 2-3 in step 103 for details.

[0146] Step 210: Smoothing of the compensation table. Smoothing involves updating the first compensation table based on the smoothing coefficients, as detailed in step 104.

[0147] Step 211: Output the first compensation table. Used for LSC processing of the first image.

[0148] Step 212, LSC processing. That is, LSC processing is performed on the first image using the first compensation table.

[0149] The image processing method in this embodiment achieves stability in LSC processing of images under various shooting conditions. This not only improves the accuracy of image processing but also optimizes the user's visual experience.

[0150] Example 2

[0151] Corresponding to the foregoing image processing method embodiments, this disclosure also provides embodiments of image processing systems.

[0152] Figure 5This is a schematic diagram of a module of an image processing system provided in an exemplary embodiment of the present disclosure. The image processing system includes:

[0153] The acquisition module 21 is used to acquire the first pixel information of the first image and the second pixel information of the second image, respectively;

[0154] The first determining module 22 is used to determine the deviation information between the first image and the second image based on the first pixel information and the second pixel information;

[0155] The second determining module 23 is used to determine the smoothing coefficient based on the deviation information in response to the deviation information meeting the preset conditions; the smoothing coefficient is used to update the first compensation table of the first image.

[0156] Optionally, the first pixel information includes several first pixel values, and the second pixel information includes several second pixel values ​​corresponding to the first pixel values; the first determining module 22 is specifically used for:

[0157] The first average value of several first pixel values ​​and the second average value of the corresponding second pixel values ​​are obtained respectively, and the pixel mean difference is determined based on the difference between the first average value and the second average value.

[0158] And / or,

[0159] A first feature vector is determined based on the first pixel value, and a second feature vector is determined based on the corresponding second pixel value. Cosine similarity is then determined based on the first feature vector and the second feature vector.

[0160] Optionally, the second determining module 22 is specifically used for:

[0161] In response to a pixel mean difference being less than or equal to a difference threshold and / or a cosine similarity being greater than or equal to a similarity threshold, a smoothing coefficient is determined based on the pixel mean difference and / or cosine similarity.

[0162] Optionally, the second determining module 22 is specifically used for:

[0163] The smoothing coefficient is determined based on the pixel mean difference and the difference threshold.

[0164] or,

[0165] The smoothing coefficients are determined based on cosine similarity and similarity threshold.

[0166] or,

[0167] The first smoothing coefficient is determined based on the pixel mean difference and the difference threshold; the second smoothing coefficient is determined based on the cosine similarity and the similarity threshold; and the smoothing coefficient is determined based on the first smoothing coefficient and the second smoothing coefficient.

[0168] Optionally, the second determining module 22 is used to implement the following formula:

[0169]

[0170] Where r is the smoothing coefficient, a m For the difference in pixel mean, a t The difference threshold;

[0171] or,

[0172] The second determining module 22 is used to implement the following formula:

[0173]

[0174] Where r is the smoothing coefficient, b m For cosine similarity, b t This is the similarity threshold.

[0175] Optionally, the second determining module 22 is used to implement the following formula:

[0176]

[0177] Where r1 is the first smoothing coefficient, a m For the difference in pixel mean, a t The difference threshold;

[0178] And / or,

[0179] The second determining module 22 is used to implement the following formula:

[0180]

[0181] Where r2 is the second smoothing coefficient, b m For cosine similarity, b t The similarity threshold;

[0182] And / or,

[0183] The second determining module 22 is used to implement the following formula:

[0184] r = k1*r1 + k2*r2;

[0185] Where r1 is the first smoothing coefficient, r2 is the second smoothing coefficient, r is the smoothing coefficient, k1 is the first weighting coefficient, and k2 is the second weighting coefficient.

[0186] Optionally, the image processing system also includes:

[0187] The update module is used to update the first compensation table according to the smoothing coefficient, the first compensation table of the first image, and the second compensation table of the second image; and to perform lens shading correction on the first image based on the updated first compensation table.

[0188] And / or,

[0189] The correction module is used to perform lens shadow correction on the first image based on the first compensation table of the first image in response to the deviation information not meeting the preset conditions.

[0190] Optionally, the update module is used to implement the following formula:

[0191] L = L1*r + L2*(1-r), or L = L2*r + L1*(1-r);

[0192] Where L1 is the first compensation table, L2 is the second compensation table, L is the updated first compensation table, and r is the smoothing coefficient.

[0193] Optionally, the image processing system also includes:

[0194] The compensation table generation module is used to obtain several preset compensation tables and combine two different preset compensation tables into several compensation table combinations; calculate the error of the compensation table combination in lens shadow correction of the target image; and determine the first compensation table based on the target compensation table combination corresponding to the minimum error and the corresponding interpolation coefficient.

[0195] And / or,

[0196] The compensation table generation module is also used to, in response to the activation of the voting mechanism, determine the number of times the first compensation table has been used since the Nth frame image based on the preset number of frames, where N is a positive integer; and use the first compensation table whose number of uses meets the preset quantity condition as the first compensation table of the first image.

[0197] The image processing system in this embodiment achieves stable LSC processing of images under various shooting conditions. This not only improves the accuracy of image processing but also optimizes the user's visual experience.

[0198] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.

[0199] Example 3

[0200] Figure 6 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the image processing method described in any of the above embodiments. Figure 6 The electronic device 90 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0201] like Figure 6 As shown, the electronic device 90 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including memory 92 and processor 91).

[0202] Bus 93 includes a data bus, an address bus, and a control bus.

[0203] The memory 92 may include volatile memory, such as random access memory (RAM) 921 and / or cache memory 922, and may further include read-only memory (ROM) 923.

[0204] The memory 92 may also include a program tool 925 (or utility) having a set (at least one) program module 924, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0205] The processor 91 executes various functional applications and data processing, such as the image processing method provided in any of the above embodiments, by running a computer program stored in the memory 92.

[0206] Electronic device 90 can also communicate with one or more external devices 94 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 95. Furthermore, electronic device 90 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 96. As shown, network adapter 96 communicates with other modules of electronic device 90 via bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0207] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0208] Example 4

[0209] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method provided in any of the above embodiments.

[0210] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0211] Example 5

[0212] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method described in any of the preceding embodiments.

[0213] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0214] Example 6

[0215] This disclosure also provides a chip that stores a computer program, which, when executed by the chip, implements the image processing method described in any of the above embodiments.

[0216] Example 7

[0217] This disclosure also provides a chip module for use in an electronic device, including a transceiver component and a chip, wherein the chip is used to implement the image processing method described in any of the above embodiments.

[0218] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. An image processing method, characterized in that, The image processing method includes: The first pixel information of the first image and the second pixel information of the second image are obtained respectively, wherein the first image and the second image are adjacent frames; The deviation information between the first image and the second image is determined based on the first pixel information and the second pixel information; In response to the deviation information satisfying a preset condition, namely, the difference between the first image and the second image is less than or equal to a difference threshold, or the similarity between the first image and the second image is greater than or equal to a similarity threshold, a smoothing coefficient is determined based on the deviation information; the smoothing coefficient is used to update the first compensation table of the first image. The image processing method further includes: The first compensation table is updated based on the smoothing coefficient, the first compensation table of the first image, and the second compensation table of the second image; lens shading correction is performed on the first image based on the updated first compensation table; And / or, In response to the deviation information not meeting the preset conditions, namely, the difference between the first image and the second image is greater than the difference threshold, or the similarity between the first image and the second image is less than the similarity threshold, lens shadow correction is performed on the first image based on the first compensation table of the first image.

2. The image processing method according to claim 1, characterized in that, The first pixel information includes a plurality of first pixel values, and the second pixel information includes a plurality of second pixel values ​​corresponding to the first pixel values; determining the deviation information between the first image and the second image based on the first pixel information and the second pixel information includes: The first average value of several first pixel values ​​and the second average value of corresponding second pixel values ​​are obtained respectively, and the pixel mean difference is determined based on the difference between the first average value and the second average value. And / or, A first feature vector is determined based on the first pixel value, and a second feature vector is determined based on the corresponding second pixel value. Cosine similarity is then determined based on the first feature vector and the second feature vector.

3. The image processing method according to claim 2, characterized in that, The step of determining a smoothing coefficient based on the deviation information in response to the deviation information satisfying a preset condition includes: In response to the pixel mean difference being less than or equal to a difference threshold and / or the cosine similarity being greater than or equal to a similarity threshold, the smoothing coefficient is determined based on the pixel mean difference and / or the cosine similarity.

4. The image processing method according to claim 3, characterized in that, Determining the smoothing coefficient based on the pixel mean difference and / or the cosine similarity includes: The smoothing coefficient is determined based on the pixel mean difference and the difference threshold. or, The smoothing coefficients are determined based on the cosine similarity and the similarity threshold. or, A first smoothing coefficient is determined based on the pixel mean difference and the difference threshold; a second smoothing coefficient is determined based on the cosine similarity and the similarity threshold; and the smoothing coefficient is determined based on the first smoothing coefficient and the second smoothing coefficient.

5. The image processing method according to claim 1, characterized in that, Before the step of acquiring the first pixel information of the first image and the second pixel information of the second image respectively, the following steps are included: Several preset compensation tables are obtained, and two different preset compensation tables are combined into several compensation table combinations; the error of the compensation table combination in lens shading correction of the target image is calculated respectively; the first compensation table is determined according to the target compensation table combination corresponding to the minimum value of the error and the corresponding interpolation coefficient; And / or, In response to the activation of the voting mechanism, the number of times the first compensation table has been used, starting from the Nth frame, is counted according to a preset number of frames, where N is a positive integer; the first compensation table whose number of uses meets a preset quantity condition is used as the first compensation table for the first image.

6. An image processing system, characterized in that, The image processing system includes: The acquisition module is used to acquire the first pixel information of the first image and the second pixel information of the second image, wherein the first image and the second image are adjacent frames; The first determining module is used to determine the deviation information between the first image and the second image based on the first pixel information and the second pixel information; The second determining module is used to determine a smoothing coefficient based on the deviation information in response to the deviation information satisfying a preset condition, namely, the difference between the first image and the second image is less than or equal to a difference threshold, or the similarity between the first image and the second image is greater than or equal to a similarity threshold; the smoothing coefficient is used to update the first compensation table of the first image. The image processing system also includes: The update module is configured to update the first compensation table according to the smoothing coefficient, the first compensation table of the first image, and the second compensation table of the second image; and perform lens shading correction on the first image based on the updated first compensation table. And / or, The correction module is used to perform lens shading correction on the first image based on the first compensation table of the first image in response to the deviation information not meeting the preset conditions, namely, the difference between the first image and the second image is greater than the difference threshold, or the similarity between the first image and the second image is less than the similarity threshold.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the image processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image processing method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1 to 5.

Images