Image Processing Method, Apparatus, and Storage Medium

Abstract

The present disclosure relates to an image processing method, apparatus, and storage medium. The method includes: determining a first transformation coefficient corresponding to the light source feature according to the light source feature of the environment where the image to be processed is currently located and the mapping relationship between the transformation coefficient and the set light source; correcting the color shadow of the image to be processed according to the first transformation coefficient to obtain a preliminary corrected image; determining pixels with residual color shadows from the preliminary corrected image and obtaining a residual color shadow contour; obtaining a second transformation coefficient according to the residual color shadow contour and the residual color shadow model; and correcting the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient. Compared with the correction methods in the related art, the present disclosure can obtain a higher correction accuracy within a shorter algorithm running time, has a strong adjustment ability for local correction, can meet the photographing requirements of complex light source scenarios, and has high industrial application value.

Description

Technical Field

[0001] The present disclosure relates to the field of image technologies, and in particular, to an image processing method, apparatus, and storage medium. Background Art

[0002] Currently, most terminal devices have a photographing function. For different terminal devices, the image sensors used have different characteristics. Due to the different refractive indices of the microlenses of the image sensors for lights of different wavelength bands (red, green, and blue lights), a prism splitting phenomenon will occur. Coupled with the process errors in the manufacturing and installation processes of different image sensors, the color distribution of different regions of the image formed by imaging will be uneven. This phenomenon is called color shading.

[0003] The specific manifestation of color shading is that the center of the image is biased towards red, while the periphery is biased towards blue. The color shading of the image has a devastating impact on accurately restoring the scene color. Therefore, a module for correcting image shading needs to be added to the processing link of the Image Signal Processor (ISP) of the terminal device to achieve accurate restoration of the scene color.

[0004] In related technologies, mainly a calibration method is used to correct the image, that is, a uniform white field image is collected under a standard light source as a standard image. Through the color shading model of the standard image sensor, two gain maps of the red channel and the blue channel are obtained. By multiplying the gain maps with the images of the corresponding channels respectively, the color-biased part in the standard image can be corrected into a white field image without color bias and with uniform color. These two gain maps are the correction gain maps of the image sensor under the light source of this type of color temperature scene. During the implementation process, the image in the actual scene can be corrected by using this correction gain map. However, the image effect after correction by this method is not good. Summary of the Invention

[0005] The present disclosure provides an image processing method, apparatus, and storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, including:

[0007] Determine a first transformation coefficient corresponding to the light source feature according to the light source feature of the environment where the image to be processed is currently located and the mapping relationship between the transformation coefficient and the set light source; wherein, the transformation coefficient is used to characterize the strength of the color shading;

[0008] Correct the color shading of the image to be processed according to the first transformation coefficient to obtain a preliminary corrected image;

[0009] Determine the pixels with residual color shading from the preliminary corrected image, and obtain a residual color shading contour;

[0010] Obtain a second transformation coefficient according to the residual color shadow contour and the residual color shadow model;

[0011] Correct the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient.

[0012] Optionally, determining the pixels with residual color shadow from the preliminarily corrected image includes:

[0013] Determine the pixels with gradient values less than the first gradient threshold in the preliminarily corrected image as the pixels with residual color shadow.

[0014] Optionally, the method further includes:

[0015] Determine a set number of control points from the preliminarily corrected image;

[0016] Construct the residual color shadow model according to the set number of control points and a set curve function.

[0017] Optionally, the obtaining the second transformation coefficient according to the residual color shadow contour and the residual color shadow model includes:

[0018] Adjust the model parameters of the residual color shadow model based on the residual color shadow contour until a set stop condition is reached;

[0019] Determine the model parameters when the set stop condition is reached as the second transformation coefficient.

[0020] Optionally, the adjusting the model parameters of the residual color shadow model based on the residual color shadow contour until a set stop condition is reached includes:

[0021] Convert both the residual color shadow contour and the residual color shadow model to the gradient domain;

[0022] Use the least squares method to calculate the error between the converted residual color shadow contour and the converted residual color shadow model;

[0023] When the error is greater than or equal to a set error threshold, update the model parameters of the residual color shadow model until the set stop condition is reached.

[0024] Optionally, the method further includes:

[0025] Collect original images with color shadows respectively under multiple set light sources;

[0026] Using a standard image without color shading as a calibration target to calibrate the original image, and obtaining transformation coefficients corresponding to the plurality of the set light sources;

[0027] Establishing the mapping relationship between the transformation coefficients and the set light sources.

[0028] Optionally, the calibrating the color shading of the image to be processed according to the first transformation coefficient and the second transformation coefficient includes:

[0029] Obtaining a target transformation coefficient based on the product of the first transformation coefficient and the second transformation coefficient;

[0030] Calibrating the color shading of the image to be processed based on the target transformation coefficient.

[0031] According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, including:

[0032] A first determination module configured to determine a first transformation coefficient corresponding to the light source feature according to the light source feature of the environment where the image to be processed is currently located and the mapping relationship between the transformation coefficient and the set light source; wherein, the transformation coefficient is used to characterize the intensity of the color shading;

[0033] A first acquisition module configured to calibrate the color shading of the image to be processed according to the first transformation coefficient to obtain a preliminary calibrated image;

[0034] A second determination module configured to determine pixels with residual color shading from the preliminary calibrated image and obtain a residual color shading contour;

[0035] A second acquisition module configured to obtain a second transformation coefficient according to the residual color shading contour and the residual color shading model;

[0036] A calibration module configured to calibrate the color shading of the image to be processed according to the first transformation coefficient and the second transformation coefficient.

[0037] Optionally, the second determination module is further configured to:

[0038] Determine pixels with residual color shading in the preliminary calibrated image as pixels whose gradient value is less than a first gradient threshold.

[0039] Optionally, the apparatus further includes:

[0040] A third determination module configured to determine a set number of control points from the preliminary calibrated image;

[0041] A first construction module, configured to construct the residual color shadow model according to the set number of control points and the set curve function.

[0042] Optionally, the second acquisition module is further configured to:

[0043] Adjust the model parameters of the residual color shadow model based on the residual color shadow profile until a set stop condition is reached;

[0044] Determine the model parameters when the set stop condition is reached as the second transformation coefficient.

[0045] Optionally, the second acquisition module is further configured to:

[0046] Convert both the residual color shadow profile and the residual color shadow model to the gradient domain;

[0047] Use the least squares method to calculate the error between the converted residual color shadow profile and the converted residual color shadow model;

[0048] When the error is greater than or equal to a set error threshold, update the model parameters of the residual color shadow model until the set stop condition is reached.

[0049] Optionally, the apparatus further includes:

[0050] An acquisition module, configured to respectively acquire original images with color shadows under multiple set light sources;

[0051] A third acquisition module, configured to use a standard image without color shadow as a correction target to correct the original image and obtain transformation coefficients corresponding to multiple set light sources;

[0052] A second construction module, configured to establish the mapping relationship between the transformation coefficient and the set light source.

[0053] Optionally, the correction module is further configured to:

[0054] Obtain a target transformation coefficient based on the product of the first transformation coefficient and the second transformation coefficient;

[0055] Correct the color shadow of the image to be processed based on the target transformation coefficient.

[0056] According to a third aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, including:

[0057] A processor;

[0058] A memory configured to store processor-executable instructions;

[0059] Wherein, the processor is configured to: when executed, implement the steps in any one of the above-mentioned first aspect of the image processing methods.

[0060] According to a fourth aspect of the embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by a processor of an image processing device, enables the device to execute any one of the above-mentioned first aspect of the image processing methods.

[0061] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0062] In the embodiments of the present disclosure, based on the light source characteristics of the current environment where the image to be processed is located, a first transformation coefficient corresponding to the light source characteristics can be determined, and the image to be processed can be preliminarily corrected based on the first transformation coefficient to obtain a preliminarily corrected image. After obtaining the preliminarily corrected image, pixels with residual color shadows are determined from the preliminarily corrected image, and then a second transformation coefficient is obtained, and the color shadows in the image to be processed are corrected based on the first transformation coefficient and the second transformation coefficient.

[0063] The present disclosure can correct the image to be processed based on the first transformation coefficient and the second transformation coefficient. Since the first transformation coefficient is related to the light source characteristics of the current environment where the image to be processed is located, and the second transformation coefficient is related to the residual color shadows in the preliminarily corrected image. Through the technical solutions in the present disclosure, not only can the color shadows brought by the image sensor of the terminal device to the image be corrected, but also the local color cast phenomenon in the image can be corrected. Moreover, compared with the correction methods in the related art, the present disclosure can obtain a higher correction accuracy within a shorter algorithm running time, has a strong local correction adjustment ability, can meet the photographing requirements of complex light source scenarios, and has a high industrial application value.

[0064] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0066] Figure 1 is a flowchart showing an image processing method according to an exemplary embodiment.

[0067] Figure 2 is a schematic diagram of an image before correction according to an exemplary embodiment.

[0068] Figure 3It is a schematic diagram of a corrected image shown according to an exemplary embodiment.

[0069] Figure 4 It is a block diagram of an image processing device shown according to an exemplary embodiment.

[0070] Figure 5 It is a block diagram of an image processing device 1200 shown according to an exemplary embodiment.

[0071] Figure 6 It is another block diagram of an image processing device 1300 shown according to an exemplary embodiment. Detailed implementation

[0072] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0073] An image processing method is provided in an embodiment of the present disclosure. Figure 1 It is a schematic flowchart of an image processing method shown according to an exemplary embodiment. As Figure 1 shown, the method mainly includes the following steps:

[0074] In step 101, according to the light source characteristics of the current environment where the image to be processed is located, and the mapping relationship between the transformation coefficient and the set light source, a first transformation coefficient corresponding to the light source characteristics is determined; wherein, the transformation coefficient is used to characterize the strength of the color shadow.

[0075] In step 102, according to the first transformation coefficient, the color shadow of the image to be processed is corrected to obtain a preliminarily corrected image.

[0076] In step 103, pixels with residual color shadows are determined from the preliminarily corrected image, and a residual color shadow contour is obtained.

[0077] In step 104, according to the residual color shadow contour and the residual color shadow model, a second transformation coefficient is obtained.

[0078] In step 105, according to the first transformation coefficient and the second transformation coefficient, the color shadow of the image to be processed is corrected.

[0079] The image processing method involved in the embodiments of the present disclosure can be applied to electronic devices, where the electronic devices include mobile terminals and fixed terminals. Among them, mobile terminals include: mobile phones, tablet computers, laptop computers, etc.; fixed terminals include: personal computers. In other alternative embodiments, the image processing method can also run on network-side devices, where network-side devices include: servers, processing centers, etc.

[0080] In the embodiments of the present disclosure, the light source characteristics of the environment where the image to be processed is currently located can be determined. After determining the light source characteristics, based on the mapping relationship between the transformation coefficient and the set light source, the first transformation coefficient corresponding to the current light source characteristics can be determined. Among them, the light source characteristics can include the brightness characteristics of the current environment, the color characteristics of the light source, etc.

[0081] Since the transformation coefficient is used to characterize the strength of color shadows, in the embodiments of the present disclosure, after determining the first transformation coefficient, the color shadows of the image to be processed can be corrected according to the first transformation coefficient to obtain a preliminary corrected image. Among them, the first transformation coefficient can be in the form of a matrix. In some embodiments, each pixel in the image to be processed can have a corresponding first transformation coefficient. When there are N pixels in the image to be processed, there can be N corresponding first transformation coefficients. These N first transformation coefficients can form a gain map, where N is a positive integer. In the implementation process, the pixel values of each pixel in the image to be processed can be multiplied by the corresponding first transformation coefficient in the gain map, and the preliminary corrected image can be obtained based on the product result.

[0082] That is, when performing color shadow correction processing on the image to be processed, the light source characteristics of the environment where the image to be processed is currently located can be judged, the first transformation coefficient corresponding to the light source characteristics can be selected, and the pixel values of each pixel of the image to be processed can be multiplied by the corresponding first transformation coefficient to obtain the preliminary compensation result image of the color shadows of the image to be processed, that is, the preliminary corrected image. From a global perspective, after correcting the image to be processed based on the first transformation coefficient, the color shadows of the image to be processed can be better corrected and compensated. However, for the local color shadows of the image, refined adjustment and compensation can be performed to achieve a satisfactory color shadow compensation effect.

[0083] In the embodiments of the present disclosure, after obtaining the preliminary corrected image, pixels with residual color shadows can be determined from the preliminary corrected image to obtain a residual color shadow contour, and the second transformation coefficient can be obtained according to the residual color shadow contour and the residual color shadow model.

[0084] For example, pixels in the preliminary corrected image with gradient values less than a set gradient threshold can be determined as pixels having residual color shadows, thereby forming a color shadow contour. Here, the color shadow contour is used to characterize information related to the color shadow, and can also be referred to as a color shadow component or color shadow information.

[0085] In some embodiments, the residual color shadow model can be a mathematical model for characterizing the mapping relationship between transformation coefficients and the residual color shadow contour. For example, during implementation, the transformation coefficients can be used as model parameters of the residual color shadow model, and the model parameters can be updated by adjusting the residual color shadow contour to obtain second transformation coefficients.

[0086] In the embodiments of the present disclosure, after obtaining the first transformation coefficient and the second transformation coefficient, the color shadow of the image to be processed can be corrected according to the first transformation coefficient and the second transformation coefficient. For example, the first transformation coefficient and the second transformation coefficient can be multiplied by the pixel values of each pixel in the image to be processed respectively to obtain a corrected image. Another example is that the first transformation coefficient and the second transformation coefficient can be weighted, and the weighted first transformation coefficient and the second transformation coefficient can be multiplied by the pixel values of each pixel in the image to be processed respectively to obtain a corrected image, etc.

[0087] In the embodiments of the present disclosure, the image to be processed can be corrected based on the first transformation coefficient and the second transformation coefficient. Since the first transformation coefficient is related to the light source characteristics of the environment where the image to be processed is currently located, and the second transformation coefficient is related to the residual color shadow in the preliminary corrected image. Through the technical solution in the present disclosure, not only can the color shadow brought by the image sensor of the terminal device to the image be corrected, but also the local color cast phenomenon in the image can be corrected. Moreover, compared with the correction methods in the related art, the present disclosure can obtain a higher correction accuracy within a shorter algorithm running time, has a strong local correction adjustment ability, can meet the photographing requirements of complex light source scenarios, and has high industrial application value.

[0088] In some embodiments, determining the pixels having residual color shadows from the preliminary corrected image includes:

[0089] Pixels in the preliminary corrected image with gradient values less than a first gradient threshold are determined as pixels having residual color shadows.

[0090] Since during the process of processing the image to be processed, the pixel values of each pixel in the image to be processed will all change to varying degrees, different gradient values will be generated. Some of these changes are normal changes caused by the conversion of the picture, and some are changes caused by the influence of color shadows.

[0091] According to the feature that the gradient values of pixels with color shadows in the image are close to 0 in the gradient domain, the preliminary corrected image can be transformed into the gradient domain. By setting a first gradient threshold, the pixels where the gradient is less than the first gradient threshold are determined as pixels with residual color shadows. In this way, the pixels with residual color shadows can be determined from the preliminary corrected image, and then the residual color shadow contour can be obtained.

[0092] Since the gradient values generated by the normal changes of pixels are relatively large, while the gradient values generated by the influence of color shadows are relatively small. In the embodiments of the present disclosure, the pixels in the preliminary corrected image with gradient values less than the first gradient threshold can be determined as pixels with residual color shadows. By screening out the pixels with relatively small gradient values, on the one hand, the data calculation amount in the image processing process can be reduced, improving the efficiency of image processing. On the other hand, the possibility of determining pixels without color shadows as pixels with color shadows can be reduced, thereby improving the accuracy of image correction.

[0093] In some embodiments, the method further includes:

[0094] Determining a set number of control points from the preliminary corrected image;

[0095] Constructing the residual color shadow model according to the set number of control points and a set curve function.

[0096] Although the shapes of color shadow contours are variable and complex, they all have a property that the color shadow contours are symmetric about the center of the image. Based on the property that the color shadow contours are symmetric about the center of the image, the present disclosure can select pixel points with equal position intervals as the control points of the model. In the implementation process, the control points can be pixel points located at set positions on the preliminary corrected image, and the distances between the control points are equal.

[0097] By selecting some pixel points with equal position intervals as the control points of the residual color shadow model, the present disclosure can control the basic shape of the residual color shadow model. For example, M*M pixel points can be determined as the control points. Among them, the set number can be set as needed, as long as it can represent the basic shape of the residual color shadow model, and no specific limitation is made here. For example, the number of selected control points can be 9*9, and these 9*9 control points basically determine the shape of the entire model. For another example, the number of selected control points can also be 5*5, etc.

[0098] Here, the setting curve function can be a cubic spline curve function. Based on the preliminary color shading compensation for the image to be processed, a color shading model of the image residue, i.e., the residual color shading model, can be established through a cubic spline curve and control points. When establishing the residual color shading model, cubic spline curves are used to connect between the control points. For example, by combining a cubic spline curve with 9*9 control points, the morphology of the residual color shading model is depicted. In this way, while ensuring the smoothness of the residual color shading model, the shape of the residual color shading model can be restored as accurately as possible.

[0099] In some embodiments, obtaining the second transformation coefficient according to the residual color shading contour and the residual color shading model includes:

[0100] Adjusting the model parameters of the residual color shading model based on the residual color shading contour until a set stop condition is reached;

[0101] Determining the model parameters when the set stop condition is reached as the second transformation coefficient.

[0102] In some embodiments, the preliminary corrected image can be transformed into the gradient domain to obtain partial morphological features of the residual color shading in the preliminary corrected image, i.e., the residual color shading contour, and then the morphological contours of the residual color shading model and the residual color shading contour are fitted by the least squares method.

[0103] In some embodiments, adjusting the model parameters of the residual color shading model based on the residual color shading contour until a set stop condition is reached includes:

[0104] Converting both the residual color shading contour and the residual color shading model to the gradient domain;

[0105] Calculating the error between the transformed residual color shading contour and the transformed residual color shading model using the least squares method;

[0106] When the error is greater than or equal to a set error threshold, updating the model parameters of the residual color shading model until the set stop condition is reached.

[0107] In the embodiments of the present disclosure, according to the characteristic that the pixels with color shading in the image have gradient values close to 0 in the gradient domain, the preliminary corrected image can be transformed into the gradient domain, and by setting a first gradient threshold, the pixels where the gradient is less than the first gradient threshold are determined as the pixels with residual color shading. In this way, the pixels with residual color shading can be determined from the preliminary corrected image, and then the residual color shading contour can be obtained.

[0108] The residual color shadow model formed by the control points can also be transformed into the gradient domain. In this way, the least squares method can be used to calculate the error between the residual color shadow contour and the residual color shadow model, and determine whether the error between the residual color shadow contour and the residual color shadow model is greater than or equal to a set error threshold. When the error between the residual color shadow contour and the residual color shadow model is greater than or equal to the set error threshold, the model parameters of the residual color shadow model are updated until the set stop condition is reached. Among them, the set stop condition may include: the error between the residual color shadow contour and the residual color shadow model is less than the set error threshold. In the embodiments of the present disclosure, after reaching the stop condition, the model parameters when the set stop condition is reached can be determined as the second transformation coefficient.

[0109] In the embodiments of the present disclosure, the residual color shadow model is adjusted by the obtained residual color shadow contour, so that the error between the residual color shadow model and the residual color shadow contour is minimized. At this time, the formed residual color shadow model is closest to the residual color shadow contour. In some embodiments, the derivative of the residual color shadow model can be taken to obtain the model parameters when the stop condition is reached, that is, the second transformation coefficient.

[0110] In the embodiments of the present disclosure, the least squares method can be used to fit the morphological contours of the residual color shadow model and the residual color shadow contour to obtain the second transformation coefficient. Since the morphology of the finally obtained residual color shadow model is closest to the morphological contour of the residual color shadow contour, the finally obtained second transformation coefficient is more accurate, thereby improving the accuracy of image correction.

[0111] In some embodiments, the method further includes:

[0112] Acquire original images with color shadows respectively under multiple said set light sources;

[0113] Use a standard image without color shadow as a correction target to correct the original image, and obtain transformation coefficients corresponding to multiple said set light sources;

[0114] Establish the mapping relationship between the transformation coefficients and the set light sources.

[0115] In the implementation process, an interrupt device (such as a mobile phone) can be used to acquire original images with color shadows in a uniform white field, and obtain the color shadow contour of the original image. Then, using a real standard image without color shadow as a correction target, the transformation coefficients corresponding to the pixels of the original image are obtained by back-calculating through a calibration algorithm to form a compensation gain map. For example, it can be carried out under multiple representative set light sources, so as to obtain the transformation coefficients corresponding to multiple set light sources, and establish the mapping relationship between the transformation coefficients and the set light sources.

[0116] In some embodiments, the process of establishing the mapping relationship is carried out in a uniform white field formed by setting the light source irradiation. In the implementation process, the original image with color shadows is multiplied by the corresponding transformation coefficients pixel by pixel, so as to correct the original image into a standard image without color shadows. In this way, the transformation coefficients corresponding to each pixel form a gain map with the same size as the original image.

[0117] In some embodiments, the correcting the color shadows of the image to be processed according to the first transformation coefficient and the second transformation coefficient includes:

[0118] Obtaining a target transformation coefficient based on the product of the first transformation coefficient and the second transformation coefficient;

[0119] Correcting the color shadows of the image to be processed based on the target transformation coefficient.

[0120] Here, the first transformation coefficient (preliminary correction compensation gain map) is combined with the second transformation coefficient (local adjustment compensation gain map), and finally applied to the image to be processed to complete the correction of the color shadows of the image to be processed. Finally, the target transformation coefficient (color shadow correction gain) of the image to be processed is obtained by multiplying the first transformation coefficient (preliminary correction compensation gain map) by the second transformation coefficient (local adjustment compensation gain map). Multiplying the pixel value of each pixel of the image to be processed by the target transformation coefficient (color shadow correction gain) completes the correction of the color shadows of the image to be processed, and then the target image without color shadows is obtained.

[0121] Figure 2 is a schematic diagram of an image before correction shown according to an exemplary embodiment, as Figure 2 shown, Figure 2 the image in Figure 3 is an image with color shadows, Figure 3 is a schematic diagram of an image after correction shown according to an exemplary embodiment, Figure 2 the image in Figure 3 shown, Figure 3 the image in

[0122] The present disclosure can correct the color shadows and local color cast phenomena brought by the image sensor of the terminal device to the image to be processed. Compared with the correction methods in the related art, the present disclosure can obtain a higher correction accuracy within a shorter algorithm running time, has a strong local correction adjustment ability, can meet the photographing requirements of complex light source scenarios, and has high industrial application value.

[0123] Figure 4A block diagram of an image processing device shown according to an exemplary embodiment, as Figure 4 shown, the image processing device 300 mainly includes:

[0124] A first determination module 301, configured to determine a first transformation coefficient corresponding to the light source feature according to the light source feature of the environment where the image to be processed is currently located and the mapping relationship between the transformation coefficient and the set light source; wherein, the transformation coefficient is used to characterize the strength of the color shadow;

[0125] A first acquisition module 302, configured to correct the color shadow of the image to be processed according to the first transformation coefficient to obtain a preliminary corrected image;

[0126] A second determination module 303, configured to determine pixels with residual color shadow from the preliminary corrected image and obtain a residual color shadow contour;

[0127] A second acquisition module 304, configured to obtain a second transformation coefficient according to the residual color shadow contour and the residual color shadow model;

[0128] A correction module 305, configured to correct the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient.

[0129] In some embodiments, the second determination module 303 is further configured to:

[0130] Determine the pixels in the preliminary corrected image with a gradient value less than the first gradient threshold as the pixels with residual color shadow.

[0131] In some embodiments, the device 300 further includes:

[0132] A third determination module, configured to determine a set number of control points from the preliminary corrected image;

[0133] A first construction module, configured to construct the residual color shadow model according to the set number of control points and the set curve function.

[0134] In some embodiments, the second acquisition module 304 is further configured to:

[0135] Adjust the model parameters of the residual color shadow model based on the residual color shadow contour until a set stop condition is reached;

[0136] Determine the model parameters when the set stop condition is reached as the second transformation coefficient.

[0137] In some embodiments, the second acquisition module 304 is further configured to:

[0138] Convert both the residual color shadow contour and the residual color shadow model to the gradient domain;

[0139] Calculate the error between the converted residual color shadow contour and the converted residual color shadow model using the least squares method;

[0140] When the error is greater than or equal to a set error threshold, update the model parameters of the residual color shadow model until the set stop condition is reached.

[0141] In some embodiments, the apparatus 300 further includes:

[0142] An acquisition module configured to respectively acquire original images with color shadows under a plurality of the set light sources;

[0143] A third acquisition module configured to correct the original images using a standard image without color shadows as a correction target and obtain transformation coefficients corresponding to the plurality of the set light sources;

[0144] A second construction module configured to establish the mapping relationship between the transformation coefficients and the set light sources.

[0145] In some embodiments, the correction module 305 is further configured to:

[0146] Obtain a target transformation coefficient based on the product of the first transformation coefficient and the second transformation coefficient;

[0147] Correct the color shadows of the image to be processed based on the target transformation coefficient.

[0148] Regarding the apparatus in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0149] Figure 5 is a block diagram of an image processing apparatus 1200 shown according to an exemplary embodiment. For example, the apparatus 1200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0150] Referring to Figure 5 , the apparatus 1200 may include one or more of the following components: a processing component 1202, a memory 1204, a power component 1206, a multimedia component 1208, an audio component 1210, an input / output (I / O) interface 1212, a sensor component 1214, and a communication component 1216.

[0151] The processing component 1202 generally controls the overall operation of the device 1200, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 1202 may include one or more processors 1220 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 1202 may include one or more modules to facilitate the interaction between the processing component 1202 and other components. For example, the processing component 1202 may include a multimedia module to facilitate the interaction between the multimedia component 1208 and the processing component 1202.

[0152] The memory 1204 is configured to store various types of data to support the operation of the device 1200. Examples of such data include instructions for any application or method operating on the device 1200, contact data, phone book data, messages, pictures, videos, etc. The memory 1204 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0153] The power component 1206 provides power to various components of the device 1200. The power component 1206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the device 1200.

[0154] The multimedia component 1208 includes a screen that provides an output interface between the device 1200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1208 includes a front camera and / or a rear camera. When the device 1200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each of the front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0155] The audio component 1210 is configured to output and / or input audio signals. For example, the audio component 1210 includes a microphone (MIC) that is configured to receive external audio signals when the device 1200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 1204 or transmitted via the communication component 1216. In some embodiments, the audio component 1210 further includes a speaker for outputting audio signals.

[0156] The I / O interface 1212 provides an interface between the processing component 1202 and a peripheral interface module, and the peripheral interface module may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.

[0157] The sensor component 1214 includes one or more sensors for providing status assessments of various aspects of the device 1200. For example, the sensor component 1214 can detect the on / off state of the device 1200, the relative positioning of components, such as the display and keypad of the device 1200, the sensor component 1214 can also detect a change in the position of the device 1200 or a component of the device 1200, the presence or absence of user contact with the device 1200, the orientation or acceleration / deceleration of the device 1200, and the temperature change of the device 1200. The sensor component 1214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 1214 may further include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1214 may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0158] The communication component 1216 is configured to facilitate communication between the device 1200 and other devices in a wired or wireless manner. The device 1200 can access a wireless network based on a communication standard, such as WiFi, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component 1216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1216 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0159] In an exemplary embodiment, the apparatus 1200 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0160] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 1204 including instructions, may be provided, and the above instructions may be executed by a processor 1220 of the apparatus 1200 to complete the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0161] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an image processing apparatus, enables the image processing apparatus to execute an image processing method, the method including:

[0162] Determining a first transformation coefficient corresponding to the light source feature according to the light source feature of the environment where the image to be processed is currently located and the mapping relationship between the transformation coefficient and the set light source; wherein the transformation coefficient is used to characterize the strength of the color shadow;

[0163] Correcting the color shadow of the image to be processed according to the first transformation coefficient to obtain a preliminary corrected image;

[0164] Determining pixels with residual color shadow from the preliminary corrected image and obtaining a residual color shadow contour;

[0165] Obtaining a second transformation coefficient according to the residual color shadow contour and the residual color shadow model;

[0166] Correcting the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient.

[0167] Figure 6 is another block diagram of an image processing apparatus 1300 shown according to an exemplary embodiment. For example, the apparatus 1300 may be provided as a server. Refer to Figure 6, the apparatus 1300 includes a processing component 1322, which further includes one or more processors, and memory resources represented by a memory 1332 for storing instructions executable by the processing component 1322, such as application programs. The application programs stored in the memory 1332 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1322 is configured to execute instructions to perform the above image processing method, the method including:

[0168] Determine a first transformation coefficient corresponding to the light source feature according to the light source feature of the environment where the image to be processed is currently located and the mapping relationship between the transformation coefficient and the set light source; wherein, the transformation coefficient is used to characterize the strength of the color shadow;

[0169] Correct the color shadow of the image to be processed according to the first transformation coefficient to obtain a preliminary corrected image;

[0170] Determine the pixels with residual color shadow from the preliminary corrected image, and obtain a residual color shadow contour;

[0171] Obtain a second transformation coefficient according to the residual color shadow contour and the residual color shadow model;

[0172] Correct the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient.

[0173] The apparatus 1300 may further include a power supply component 1326 configured to perform power management of the apparatus 1300, a wired or wireless network interface 1350 configured to connect the apparatus 1300 to a network, and an input / output (I / O) interface 1358. The apparatus 1300 may operate based on an operating system stored in the memory 1332, such as Windows ServerTM, MacOS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0174] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed herein. The specification and embodiments are to be considered as exemplary only, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0175] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. An image processing method, characterized in that, comprising: Determining a first transformation coefficient corresponding to the light source feature according to the light source feature of the current environment of the image to be processed and the mapping relationship between the transformation coefficient and the set light source; wherein, the first transformation coefficient is used to characterize the strength of the color shadow; Correcting the color shadow of the image to be processed according to the first transformation coefficient to obtain a preliminary corrected image; Determining the pixels with a gradient value less than the first gradient threshold in the preliminary corrected image as the pixels with residual color shadow, and determining the residual color shadow contour based on the pixels with residual color shadow; Determining a set number of control points from the preliminary corrected image; constructing a residual color shadow model according to the set number of control points and the cubic spline curve; wherein, the distances between the control points are equal; Obtaining a second transformation coefficient according to the error between the residual color shadow contour and the residual color shadow model; the error corresponding to the second transformation coefficient is less than the set error threshold; Correcting the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient.

2. The method according to claim 1, characterized in that, The obtaining the second transformation coefficient according to the error between the residual color shadow contour and the residual color shadow model includes: Adjusting the model parameters of the residual color shadow model based on the residual color shadow contour until a set stop condition is reached; Determining the model parameters when the set stop condition is reached as the second transformation coefficient.

3. The method according to claim 2, characterized in that, The adjusting the model parameters of the residual color shadow model based on the residual color shadow contour until a set stop condition is reached includes: Converting both the residual color shadow contour and the residual color shadow model to the gradient domain; Calculating the error between the converted residual color shadow contour and the converted residual color shadow model by using the least squares method; When the error is greater than or equal to the set error threshold, updating the model parameters of the residual color shadow model until the set stop condition is reached.

4. The method according to claim 1, characterized in that, The method further includes: Collecting original images with color shadows under multiple set light sources respectively; Using the standard image without color shadow as the correction target to correct the original image, and obtaining the transformation coefficients corresponding to the multiple set light sources; Establishing the mapping relationship between the transformation coefficient and the set light source.

5. The method according to any one of claims 1 to 4, characterized in that, The correcting the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient includes: Obtaining a target transformation coefficient based on the product of the first transformation coefficient and the second transformation coefficient; Correcting the color shadow of the image to be processed based on the target transformation coefficient.

6. An image processing apparatus, characterized in that, comprising: A first determination module, configured to determine a first transformation coefficient corresponding to the light source feature according to the light source feature of the environment where the image to be processed is currently located and the mapping relationship between the transformation coefficient and the set light source; wherein, the transformation coefficient is used to characterize the strength of the color shadow; A first acquisition module, configured to correct the color shadow of the image to be processed according to the first transformation coefficient to obtain a preliminary corrected image; A second determination module, configured to determine the pixels with a gradient value less than the first gradient threshold in the preliminary corrected image as the pixels with residual color shadow, and determine the residual color shadow contour based on the pixels with residual color shadow; A third determination module, configured to determine a set number of control points from the preliminary corrected image; A first construction module, configured to construct a residual color shadow model according to the set number of control points and a cubic spline curve; wherein, the distances between the control points are equal; A second acquisition module, configured to obtain a second transformation coefficient according to the error between the residual color shadow contour and the residual color shadow model; the error corresponding to the second transformation coefficient is less than the set error threshold; A correction module, configured to correct the color shadow of the image to be processed according to the first transformation coefficient and the second transformation coefficient.

7. The apparatus according to claim 6, wherein, the second acquisition module is further configured to: adjust the model parameters of the residual color shadow model based on the residual color shadow contour until a set stop condition is reached; determine the model parameters when the set stop condition is reached as the second transformation coefficient.

8. The apparatus according to claim 7, wherein, the second acquisition module is further configured to: convert both the residual color shadow contour and the residual color shadow model to the gradient domain; calculate the error between the converted residual color shadow contour and the converted residual color shadow model by using the least squares method; when the error is greater than or equal to the set error threshold, update the model parameters of the residual color shadow model until the set stop condition is reached.

9. The apparatus according to claim 6, wherein, the apparatus further includes: an acquisition module, configured to respectively acquire original images with color shadows under a plurality of the set light sources; a third acquisition module, configured to correct the original images by using a standard image without color shadow as a correction target and obtain the transformation coefficients corresponding to the plurality of the set light sources; a second construction module, configured to establish the mapping relationship between the transformation coefficient and the set light source.

10. The apparatus according to any one of claims 6 to 9, wherein, the correction module is further configured to: obtain a target transformation coefficient based on the product of the first transformation coefficient and the second transformation coefficient; correct the color shadow of the image to be processed based on the target transformation coefficient.

11. An image processing apparatus, wherein, it includes: a processor; a memory configured to store instructions executable by the processor; Wherein, the processor is configured to: when executed, implement the steps in any one of the above-mentioned image processing methods of claims 1 to 5.

12. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an image processing device, enabling the device to execute the steps in any one of the above-mentioned image processing methods of claims 1 to 5.

Images