Image processing method and device

Abstract

The present application discloses an image processing method and device, which belongs to the field of communication technology. The image processing method includes: when there is a shadow area in the first image, inputting the first image into a preset model; using the preset model to perform color correction on the shadow area to obtain a second image; wherein the training stop condition of the preset model is that the color loss value of the predicted shadow-free image of the training sample image and the labeled shadow-free image of the training sample image meets the preset condition. The image processing method provided by the present application can remove the shadow area in the image, improve the information integrity and clarity of the image, improve the image recognizability, and improve the user experience.

Description

Technical Field

[0001] The present application belongs to the field of communication technology, and specifically relates to an image processing method and device. Background Art

[0002] Images, as an information carrier that can contain various elements such as text, pictures, tables, etc., can be used to record and transmit information.

[0003] However, once a shadow area appears in an image, it may affect the information integrity and clarity of the image, resulting in lower image recognizability, which in turn leads to a poor user experience. Therefore, there is an urgent need for an image processing method that can remove the shadow area in an image to improve the information integrity and clarity of the image and improve the recognizability of the image. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image processing method and device that can remove shadow areas in an image, improve the information integrity and clarity of the image, and improve the image recognizability.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which may include:

[0007] In the case where there is a shadow area in the first image, the first image is input into a preset model; and the shadow area is color corrected using the preset model to obtain a second image;

[0008] The training stopping condition of the preset model is that the color loss value of the predicted shadow-free image of the training sample image and the labeled shadow-free image of the training sample image meets the preset condition.

[0009] In a second aspect, an embodiment of the present application provides an image processing device, which may include:

[0010] A first input module, used for inputting the first image into a preset model when there is a shadow area in the first image;

[0011] A correction module, used for performing color correction on the shadow area using the preset model to obtain a second image;

[0012] The training stopping condition of the preset model is that the color loss value of the predicted shadow-free image of the training sample image and the labeled shadow-free image of the training sample image meets the preset condition.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0016] In an embodiment of the present application, when there is a shadow area in the first image, the first image is input into a preset model, and the preset model is used to perform color correction on the shadow area. Among them, the training stop condition of the preset model is that the predicted shadow-free image of the training sample image and the color loss value of the labeled shadow-free image of the training sample image meet the preset condition. In this way, on the one hand, not only can the preset model be used to perform color correction on the shadow area in the first image, but also illumination compensation and brightness restoration of the shadow area in the first image can be achieved through color correction, and the shadow area in the first image can be removed. In this way, the color of the second image can be made more realistic, the information integrity and clarity of the second image can be improved, and the recognizability of the second image can be improved, thereby effectively improving the user experience. On the other hand, using the preset model to correct the color of the first image and removing the shadow area in the first image to obtain the second image can also make the second image more beautiful, thereby further improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0018] Figure 2 It is a flowchart of another image processing method provided in an embodiment of the present application;

[0019] Figure 3 is a structural schematic diagram of an image processing device provided in an embodiment of the present application;

[0020] Figure 4 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0023] Based on the background technology, it can be known that in the prior art, due to the shadow area in the image, the information integrity and clarity of the image are low, the image recognizability is low, and thus the user experience is poor.

[0024] Specifically, images can usually contain elements such as text, pictures, and tables, which are used to record and transmit information, and are common in learning, work, and life. Images usually need to ensure high information integrity and clarity. Once there is a shadow area in the image, the image color in the shadow area may be distorted, resulting in low integrity and clarity of the information in the shadow area of ​​the image, which will lead to poor image recognizability. In addition, the shadow area in the image will affect the beauty of the image, resulting in a poor user experience.

[0025] In the prior art, image processing technology is usually used to remove shadow areas in images. However, image processing technology can usually only restore the brightness of the image, but cannot correct the color of the shadow area. Brightness restoration can usually only play the role of diluting the shadow area, but cannot correct the color of the shadow area. In addition, the image after brightness restoration may also have color distortion, resulting in mottled image color. In this way, the image after brightness restoration usually still has technical problems such as low information integrity and clarity, low image recognizability, and unsightly image.

[0026] Based on the above findings, the embodiment of the present application provides an image processing method and device, which can input the first image into a preset model when there is a shadow area in the first image, and use the preset model to perform color correction on the shadow area. Among them, the training stop condition of the preset model is that the predicted shadow-free image of the training sample image and the color loss value of the labeled shadow-free image of the training sample image meet the preset condition. In this way, on the one hand, not only can the preset model be used to perform color correction on the shadow area in the first image, but also illumination compensation and brightness restoration of the shadow area in the first image can be achieved by color correction, and the shadow area in the first image can be removed. In this way, the color of the second image can be made more realistic, the information integrity and clarity of the second image can be improved, and the recognizability of the second image can be improved, thereby effectively improving the user experience. On the other hand, using the preset model to correct the color of the first image and removing the shadow area in the first image to obtain the second image can also make the second image more beautiful, thereby further improving the user experience.

[0027] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0028] Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present application, and the execution subject of the method may be an electronic device. Figure 1 As shown, the image processing method may include the following steps:

[0029] S110: When there is a shadow area in the first image, the electronic device inputs the first image into a preset model.

[0030] The first image may be any image containing a shadow area.

[0031] The preset model may be a pre-trained model for removing shadow areas in the first image.

[0032] When processing a first image with a shadow area, the electronic device may obtain a pre-trained preset model, and then input the first image into the preset model to remove the shadow area in the first image using the preset model.

[0033] S120: The electronic device performs color correction on the shadow area using a preset model to obtain a second image.

[0034] The second image may be an image output after a preset model performs color correction on the shadow area of ​​the first image.

[0035] The training stop condition of the preset model may be that the predicted shadow-free image of the training sample image and the color loss value of the labeled shadow-free image of the training sample image meet the preset condition, and the color loss value may be the distance between the predicted shadow-free image and the labeled shadow-free image in the HSV color model.

[0036] The predicted shadow-free image may be an image obtained by inputting the training sample image into the preset model, processing the preset model and outputting the image by the preset model; the labeled shadow-free image may be a real image without shadows corresponding to the training sample image. For example, taking the cover of a book as an example, the image with shadow areas obtained by photographing the cover is the training sample image, the image output after the training sample image is input into the preset model and color correction is performed by the preset model is the predicted shadow-free image, and the image of the shadow-free area obtained by photographing the cover is the labeled shadow-free image, i.e., the real image.

[0037] After the electronic device inputs the first image into the preset model, it can use the preset model to perform color correction on the first image according to the logic pre-trained by the preset model, and achieve illumination compensation and brightness restoration of the shadow area in the first image through color correction, thereby removing the shadow area in the first image and obtaining a second image.

[0038] It should be noted that the preset model can be a model that uses training samples containing paired images with shadow areas (i.e., the above-mentioned training sample images) and images without shadow areas (i.e., the above-mentioned labeled shadow-free images) to train a deep convolutional generative adversarial network, and the training sample images can be processed to obtain a model of a predicted shadow-free image whose color loss value satisfies the preset conditions with the labeled shadow-free image. Compared with the existing de-shadowing method based on image processing technology, the embodiment of the present application can use the guidance of paired data and the function fitting ability of the deep convolutional generative adversarial network to learn a preset model with scene universality, and the prediction model can be applicable to de-shadowing processing of various image types such as black and white images, color images, etc. containing shadow areas.

[0039] Moreover, in order to perform color correction on the shadow area, the embodiment of the present invention can also randomly change the color of the shadow area of ​​the input training sample image with shadow area during the preset model training stage to construct a training sample image with color shadow. When using the labeled shadow-free image to supervise the preset model training, the color loss value can be added and improved in addition to the conventional reconstruction loss value, perception loss value, and adversarial loss value, so that the predicted shadow-free image output by the preset model can be close to the labeled shadow-free image, thereby achieving color correction of the shadow area. Among them, the color loss value can be the predicted shadow-free image corresponding to the training sample image, and the distance between the labeled shadow-free image corresponding to the training sample image in the HSV color model, such as the cosine distance, and the calculation formula of the color loss value can be as shown in formula (1).

[0040] color_loss=1–cos((Ho,So,Vo),(Hgt,Sgt,Vgt)) (1)

[0041] Among them, color_loss represents the color loss value, Ho represents the predicted hue of the shadow-free image, So predicts the saturation of the shadow-free image, Vo predicts the brightness of the shadow-free image, Hgt represents the hue of the label shadow-free image, Sgt represents the saturation of the label shadow-free image, and Vgt represents the brightness of the label shadow-free image.

[0042] It can be understood that the training sample images and the labeled shadow-free images in the training samples can be collected real images or artificially synthesized images.

[0043] In an embodiment of the present application, when there is a shadow area in the first image, the first image is input into a preset model, and the preset model is used to perform color correction on the shadow area. Among them, the training stop condition of the preset model is that the predicted shadow-free image of the training sample image and the color loss value of the labeled shadow-free image of the training sample image meet the preset condition. In this way, on the one hand, not only can the preset model be used to perform color correction on the shadow area in the first image, but also illumination compensation and brightness restoration of the shadow area in the first image can be achieved through color correction, and the shadow area in the first image can be removed. In this way, the color of the second image can be made more realistic, the information integrity and clarity of the second image can be improved, and the recognizability of the second image can be improved, thereby effectively improving the user experience. On the other hand, using the preset model to correct the color of the first image and removing the shadow area in the first image to obtain the second image can also make the second image more beautiful, thereby further improving the user experience.

[0044] In some embodiments, before performing color correction on the first image, the electronic device may first determine whether there is a shadow area in the first image. Accordingly, before the above step S110, the following steps may also be performed:

[0045] The electronic device calculates the proportion of the shadow area in the first image in the first image;

[0046] When the proportion is greater than or equal to a preset proportion, the electronic device determines that a shadow area exists in the first image.

[0047] Considering that when the shadow area in the first image is small, it usually does not affect the information integrity and clarity of the first image, and does not affect the recognizability of the first image. Therefore, before the electronic device inputs the first image with the shadow area into the preset model, it can first calculate the total area of ​​the shadow area in the first image and the full image area of ​​the first image. Then calculate the proportion of the shadow area in the first image in the first image, that is, the proportion of the total area of ​​the shadow area in the first image in the full image area of ​​the first image. The specific calculation method can be referred to formula (2):

[0048] shadow_ratio=sum_area / whole_area (2)

[0049] Among them, shadow_ratio represents the ratio of the total area of ​​the shadow area to the whole area of ​​the first image, sum_area represents the total area of ​​the shadow area in the first image, and whole_area represents the whole area of ​​the first image.

[0050] After calculating the proportion of the shadow area in the first image in the first image, a preset proportion can be obtained, such as the preset proportion can be 0.01, 0.02, etc., and the specific data can be set according to actual needs. Then compare the proportion of the shadow area in the first image in the first image with the preset proportion to determine whether the proportion of the shadow area in the first image in the first image is greater than or equal to the preset proportion. In the case where the proportion of the shadow area in the first image in the first image is greater than or equal to the preset proportion, it can be considered that the shadow area in the first image may affect the information integrity and clarity of the first image and affect the recognizability of the first image. At this time, it can be determined that there is a shadow area in the first image. After determining that there is a shadow area in the first image, the above steps S110 and S120 can be performed.

[0051] In this way, image processing is performed on the first image only when the proportion of the shadow area in the first image is greater than the preset proportion, that is, the shadow area existing in the first image may affect the information integrity and clarity of the first image and the recognizability of the first image. This can avoid some unnecessary image processing processes, thereby effectively reducing power consumption and improving image processing efficiency.

[0052] In some embodiments, the electronic device may calculate the proportion of the shadow area in the first image in the first image based on the third image that can reflect the lighting information of the first image. Accordingly, the specific implementation method of calculating the proportion of the shadow area in the first image in the first image in the above step may be as follows:

[0053] The electronic device performs Gaussian filtering on the first image to obtain a third image;

[0054] The electronic device performs binarization processing on the third image to obtain a fourth image;

[0055] The electronic device performs connected component analysis on the fourth image to calculate the area of ​​the connected components in the fourth image;

[0056] The electronic device calculates the proportion of the area of ​​the connected component in the entire area of ​​the first image.

[0057] The third image may be obtained by performing Gaussian filtering on the first image, and may be used to reflect the illumination information of the first image.

[0058] Considering that using the lighting information of the first image makes it easier to calculate the proportion of the shadow area in the first image, when calculating the proportion of the shadow area in the first image, the electronic device can first perform Gaussian filtering on the first image. For example, the first image can be subjected to large-scale Gaussian filtering to weaken the local texture details in the first image and retain only the low-frequency components in the first image, that is, the lighting information of the first image.

[0059] After performing Gaussian filtering on the first image to obtain the third image, the electronic device may perform binarization on the third image so that the third image presents an obvious black and white effect, thereby obtaining a fourth image. For example, pixels whose grayscale values ​​of the third image are less than or equal to a preset pixel threshold (such as 50) may be classified as candidate shadow pixels, and the values ​​of the candidate shadow pixels are marked as 1, and pixels whose grayscale values ​​are greater than the preset pixel threshold may be classified as non-shadow pixels, and the values ​​of the non-shadow pixels are marked as 0, thereby obtaining an image after binarization, i.e., the fourth image.

[0060] After the electronic device performs binarization processing on the third image to obtain the fourth image, it can perform connected component analysis on the fourth image to obtain all connected components in the fourth image, that is, all shadow areas in the fourth image. Then, the area of ​​the connected components in the fourth image is calculated, that is, the total area of ​​all connected components in the fourth image, and the full image area of ​​the first image can be calculated. After calculating the area of ​​the connected components in the fourth image, the proportion of the area of ​​the connected components in the fourth image to the full image area of ​​the first image can be calculated.

[0061] In this way, by binarizing the third image reflecting the illumination information of the first image to obtain the fourth image, the amount of data in the fourth image can be greatly reduced. In this way, by calculating the proportion of the area of ​​the connected components in the fourth image to the total area of ​​the first image based on the fourth image with less data, the amount of data calculation can be effectively reduced, thereby further improving the efficiency of image processing.

[0062] In some embodiments, in the above steps, the Gaussian filter processing is performed on the first image to obtain the third image in a specific implementation manner as follows:

[0063] The electronic device performs scaling processing on the first image to obtain a fifth image;

[0064] The electronic device performs Gaussian filtering on the fifth image to obtain a third image.

[0065] The fifth image may be an image with an area less than or equal to the first preset area, such as less than or equal to 1000 pixels×1000 pixels, that is, the fifth image is an image with a side length less than or equal to 1000 pixels.

[0066] Considering that the first image is usually large in area, image processing of a large image may result in high power and power consumption. Therefore, before the electronic device performs Gaussian filtering on the first image to obtain the third image, it may first perform scaling processing on the first image to scale the first image to a fifth image that is less than or equal to the first preset area. Then the fifth image is subjected to Gaussian filtering to obtain the third image. In this way, scaling the first image to a fifth image that is less than or equal to the first preset area can reduce power and power consumption, thereby further improving image processing efficiency.

[0067] In some embodiments, in the above steps, the electronic device performs connected component analysis on the fourth image and calculates the area of ​​the connected components in the fourth image in the following specific implementation manner:

[0068] The electronic device performs connected component analysis on the fourth image to obtain M connected regions in the fourth image;

[0069] The electronic device removes a first connected area from the M connected areas;

[0070] The electronic device calculates a total area of ​​a second connected region among the M connected regions;

[0071] At this time, the above calculation of the proportion of the area of ​​the connected component in the whole area of ​​the first image may include:

[0072] The electronic device calculates the proportion of the total area of ​​the second connected region in the entire area of ​​the first image.

[0073] Wherein, M may be a positive integer. The first connected region may be a connected region whose area is smaller than the second preset area, that is, a shadow region whose area is smaller than the second preset area. The second preset area may be a preset minimum value of the area of ​​the connected region to be removed, such as 625 pixels. The connected region smaller than the second preset area may be considered as a region that will not affect the information integrity and clarity of the image, and will not affect the recognizability of the image.

[0074] The second connected region may be a connected region among the M connected regions except the first connected region, that is, a connected region having an area greater than the second preset area.

[0075] When the electronic device performs connected component analysis on the fourth image and calculates the area of ​​the connected components in the fourth image, the electronic device may perform connected component analysis on the fourth image to obtain M connected regions in the fourth image, that is, to obtain each connected region in the fourth image. The area of ​​each connected region in the M connected regions is calculated respectively, and a second preset area is obtained. The area of ​​each connected region in the M connected regions is then compared with the second preset area respectively, and a connected region in the M connected regions whose area is smaller than the second preset area is determined, that is, a first connected region. The first connected region may be one or more, or may not exist.

[0076] After the electronic device determines the first connected area whose area is smaller than the second preset area among the M connected areas, it can remove the aforementioned first connected area from the M connected areas of the fourth image to obtain the second connected area among the M connected areas. The second connected area is actually a connected area whose area is larger than the second preset area, that is, a shadow area that affects the information integrity and clarity of the image and the recognizability of the image. The area of ​​each second connected area among the aforementioned M connected areas is calculated respectively, the total area of ​​the second connected areas among the M connected areas is calculated, and then the proportion of the total area of ​​the second connected areas among the M connected areas in the total area of ​​the first image is calculated.

[0077] In this way, the first connected region whose area is smaller than the second preset area is removed from the fourth image obtained after the third image is binarized, that is, the shadow region in the fourth image that does not affect the information integrity and clarity of the image but affects the recognizability of the image is removed. In this way, unnecessary image processing of the first connected region can be reduced, thereby further reducing power consumption and improving image processing efficiency.

[0078] In some embodiments, the electronic device may further save the second image when the definition of the second image meets the requirement. Accordingly, after the above step S120, the following steps may be further performed:

[0079] The electronic device calculates a clarity value of the second image;

[0080] When the definition value of the second image is greater than or equal to a preset definition value, the electronic device saves the second image.

[0081] After the electronic device performs color correction on the first image using a preset model to obtain the second image, it can also calculate the clarity value of the second image, such as by using a clarity evaluation method of a spatial domain gradient field, a clarity evaluation method based on transform domain frequency analysis, etc. After calculating the clarity value of the second image, the preset clarity value can be obtained, and the clarity value of the second image can be compared with the preset clarity value to determine whether the clarity value of the second image is greater than or equal to the preset clarity value. If the clarity value of the second image is greater than or equal to the preset clarity value, the second image can be saved. On the contrary, if the clarity value of the second image is less than the preset clarity value, it is possible to continue to return to the above-mentioned image processing steps and re-process the first image.

[0082] In this way, on the one hand, since the second image has no shadow area, its information integrity and clarity value are higher than those of the first image, so the user can view and use an image with higher recognizability, thereby further improving the user experience. On the other hand, based on the fact that the second image has no shadow area, only the second image with a clarity value greater than or equal to the preset clarity value is saved for the user to view and use, so that the second image viewed and used by the user has a higher clarity value and higher recognizability, thereby further improving the user experience.

[0083] In some embodiments, the above image processing method may also be applied to a photographing process. In this case, the above first image may be a photographing preview image. Accordingly, before the above step S110, the following steps may also be performed:

[0084] When the shooting program is in the target shooting mode and there is a shadow area in the shooting preview image, the electronic device inputs the shooting preview image into the preset model.

[0085] At this time, after the above step S120, the following steps may also be performed:

[0086] The electronic device takes a photo of the second image and saves the second image.

[0087] When the above-mentioned image processing method is applied to the process of photographing, the electronic device can first open the shooting program and turn on the target shooting mode of the shooting program, such as the document shooting mode, the shadow shooting mode, etc. When the shooting program is in the target shooting mode, it can be determined whether there is a shadow area in the shooting preview image. When the shooting program is in the target shooting mode and there is a shadow area in the shooting preview image, the shooting preview image is input into the preset model, and the preset model is used to perform color correction on the shooting preview image to obtain a second image. After obtaining the second image, the second image can be photographed, such as triggering a shooting button to take a picture, and the second image can be saved, such as being saved in an album. The user can open the album application to directly view and use the image without a shadow area, with clear information and harmonious and beautiful colors.

[0088] It is understandable that after obtaining the second image, the clarity value of the second image can also be calculated; when the clarity value of the second image is greater than or equal to the preset clarity value, the second image is saved, and its implementation principle and technical effect are similar to those of the above method embodiment, and for the sake of brevity, they are not repeated here. Moreover, after obtaining the second image, the second image can be displayed on the photo preview interface, so that the user can intuitively see the second image on the photo preview interface.

[0089] It should be noted that, when the shooting program is in the target shooting mode, but there is no shadow area in the photo preview image, the clarity value of the photo preview image can be calculated, and when the clarity value of the photo preview image is greater than or equal to the preset clarity value, the photo preview image is photographed and saved. When the clarity value of the second image obtained by color correction of the photo preview image using the preset model is less than the preset clarity value, the next image can be obtained to determine whether there is a shadow area in the image, and the above-mentioned image processing process is performed.

[0090] In this way, during the photo-taking process, the color correction of the photo preview image with shadow areas can be achieved, and the shadow areas in the photo preview image can be removed. This allows users to directly obtain shadow-free images with high information integrity and high clarity during the photo-taking process, thereby improving the recognizability of the images obtained by taking photos, thereby further improving the user experience.

[0091] Figure 2Taking the application of the image processing method provided in the embodiment of the present application to the process of taking a photo as an example, another image processing method provided in the embodiment of the present application is shown, such as Figure 2 As shown, the method may include the following steps:

[0092] S210, the electronic device starts a target photographing mode of a photographing program.

[0093] S220: The electronic device determines whether there is a shadow area in the photo preview image.

[0094] When determining whether there is a shadow area in the photo preview image, the electronic device can perform Gaussian filtering on the photo preview image to obtain a third image, and the third image is used to reflect the lighting information of the photo preview image. The third image is then binarized to obtain a fourth image, and the fourth image is analyzed for connected components to calculate the area of ​​the connected components in the fourth image. Then, the proportion of the area of ​​the connected components in the full image area of ​​the photo preview image is calculated. When the proportion of the area of ​​the connected components in the full image area of ​​the photo preview image is greater than or equal to the preset proportion, it is determined that there is a shadow area in the photo preview image, and step S230 is executed. Otherwise, step S230 is not executed, and step S240 is directly executed.

[0095] S230: The electronic device inputs the camera preview image into a preset model, and uses the preset model to perform color correction on the shadow area to obtain a second image.

[0096] S240: The electronic device calculates a clarity value of the second image.

[0097] S250: The electronic device determines whether the definition value of the second image is greater than or equal to a preset definition value.

[0098] When the definition value of the second image is greater than or equal to the preset definition value, the electronic device executes step S260. Otherwise, the electronic device executes step S270.

[0099] S260: The electronic device takes a photo of the second image and saves the second image.

[0100] S270: The electronic device acquires the next image.

[0101] The implementation principle and technical effect of the image processing method provided in the embodiment of the present application are similar to those of the above-mentioned method embodiment and will not be described again for the sake of brevity.

[0102] It should be noted that the image processing method provided in the embodiment of the present application can be executed by an image processing device, or by a control module in the image processing device for executing the loaded image processing method. In the embodiment of the present application, the image processing method provided in the embodiment of the present application is described by taking the image processing device executing the loaded image processing method as an example.

[0103] Figure 3 FIG. 1 is a schematic diagram showing the structure of an image processing device provided by an embodiment of the present application. Figure 3 As shown, the image processing device 300 may include:

[0104] The first input module 310 may be used to input the first image into a preset model when there is a shadow area in the first image;

[0105] The correction module 320 may be used to perform color correction on the shadow area using a preset model to obtain a second image;

[0106] Among them, the training stopping condition of the preset model is that the color loss value of the predicted shadow-free image of the training sample image and the labeled shadow-free image of the training sample image meets the preset condition.

[0107] In an embodiment of the present application, when there is a shadow area in the first image, the first image is input into a preset model, and the preset model is used to perform color correction on the shadow area. Among them, the training stop condition of the preset model is that the predicted shadow-free image of the training sample image and the color loss value of the labeled shadow-free image of the training sample image meet the preset condition. In this way, on the one hand, not only can the preset model be used to perform color correction on the shadow area in the first image, but also illumination compensation and brightness restoration of the shadow area in the first image can be achieved through color correction, and the shadow area in the first image can be removed. In this way, the color of the second image can be made more realistic, the information integrity and clarity of the second image can be improved, and the recognizability of the second image can be improved, thereby effectively improving the user experience. On the other hand, using the preset model to correct the color of the first image and removing the shadow area in the first image to obtain the second image can also make the second image more beautiful, thereby further improving the user experience.

[0108] In some embodiments, the image processing apparatus 300 may further include:

[0109] A first calculation module may be used to calculate a proportion of a shadow area in a first image in the first image;

[0110] The determination module may be configured to determine that a shadow area exists in the first image when the proportion is greater than or equal to a preset proportion.

[0111] In this way, image processing is performed on the first image only when the proportion of the shadow area in the first image is greater than the preset proportion, that is, the shadow area existing in the first image may affect the information integrity and clarity of the first image and the recognizability of the first image. This can avoid some unnecessary image processing processes, thereby effectively reducing power consumption and improving image processing efficiency.

[0112] In some embodiments, the first computing module may include:

[0113] The filtering unit may be used to perform Gaussian filtering on the first image to obtain a third image; the third image is used to reflect the illumination information of the first image;

[0114] A binarization unit may be used to perform binarization processing on the third image to obtain a fourth image;

[0115] The first calculation unit may be used to perform connected component analysis on the fourth image and calculate the area of ​​the connected components in the fourth image;

[0116] The second calculation unit may be used to calculate the proportion of the area of ​​the connected component in the entire area of ​​the first image.

[0117] In this way, by binarizing the third image reflecting the illumination information of the first image to obtain the fourth image, the amount of data in the fourth image can be greatly reduced. In this way, by calculating the proportion of the area of ​​the connected components in the fourth image to the total area of ​​the first image based on the fourth image with less data, the amount of data calculation can be effectively reduced, thereby further improving the efficiency of image processing.

[0118] In some embodiments, the filtering unit may include:

[0119] The scaling subunit may be used to scale the first image to obtain a fifth image; the area of ​​the fifth image is less than or equal to the first preset area;

[0120] The filtering subunit may be used to perform Gaussian filtering on the fifth image to obtain the third image.

[0121] In this way, scaling the first image to a fifth image having an area smaller than or equal to the first preset area can reduce power and consumption, thereby further improving image processing efficiency.

[0122] In some embodiments, the first computing unit includes:

[0123] The analysis component can be used to perform connected component analysis on the fourth image to obtain M connected regions in the fourth image; M is a positive integer;

[0124] A removal component may be used to remove a first connected area from the M connected areas; the first connected area is a connected area having an area smaller than a second preset area;

[0125] A calculation component may be used to calculate a total area of ​​a second connected region among the M connected regions; the second connected region is a connected region among the M connected regions except the first connected region;

[0126] The second computing unit may be specifically used for:

[0127] The ratio of the total area of ​​the second connected region to the total area of ​​the first image is calculated.

[0128] In this way, the first connected region whose area is smaller than the second preset area is removed from the fourth image obtained after the third image is binarized, that is, the shadow region in the fourth image that does not affect the information integrity and clarity of the image but affects the recognizability of the image is removed. In this way, unnecessary image processing of the first connected region can be reduced, thereby further reducing power consumption and improving image processing efficiency.

[0129] In some embodiments, the image processing apparatus 300 may further include:

[0130] A second calculation module may be used to calculate a clarity value of a second image;

[0131] The saving module may be used to save the second image when the clarity value of the second image is greater than or equal to a preset clarity value.

[0132] In this way, on the one hand, since the second image has no shadow area, its information integrity and clarity value are higher than those of the first image, so the user can view and use an image with higher recognizability, thereby further improving the user experience. On the other hand, based on the fact that the second image has no shadow area, only the second image with a clarity value greater than or equal to the preset clarity value is saved for the user to view and use, so that the second image viewed and used by the user has higher clarity and higher recognizability, thereby further improving the user experience.

[0133] In some embodiments, the first image is a photo preview image;

[0134] The image processing device may further include:

[0135] The second input module can be used to input the photo preview image into a preset model when the shooting program is in a target photo mode and there is a shadow area in the photo preview image;

[0136] The camera module can be used for:

[0137] The second image is photographed and saved.

[0138] In this way, during the photo-taking process, the color correction of the photo preview image with shadow areas can be achieved, and the shadow areas in the photo preview image can be removed. This allows users to directly obtain shadow-free images with high information integrity and high clarity during the photo-taking process, thereby improving the recognizability of the images obtained by taking photos, thereby further improving the user experience.

[0139] The image processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0140] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0141] The image processing device provided in the embodiment of the present application can achieve Figure 1 to Figure 2 To avoid repetition, the various processes implemented by the image processing device in the method embodiment will not be described here.

[0142] Optionally, an embodiment of the present application also provides an electronic device, including a processor 410, a memory 409, and a program or instruction stored in the memory 409 and executable on the processor 410. When the program or instruction is executed by the processor 410, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0143] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0144] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0145] The electronic device 400 includes but is not limited to components such as a radio frequency unit 401 , a network module 402 , an audio output unit 403 , an input unit 404 , a sensor 405 , a display unit 406 , a user input unit 407 , an interface unit 408 , a memory 409 , and a processor 410 .

[0146] Those skilled in the art will appreciate that the electronic device 400 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 410 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0147] The processor 410 is used to input the first image into a preset model when there is a shadow area in the first image; and perform color correction on the shadow area using the preset model to obtain a second image.

[0148] In an embodiment of the present application, when there is a shadow area in the first image, the first image is input into a preset model, and the preset model is used to perform color correction on the shadow area. Among them, the training stop condition of the preset model is that the predicted shadow-free image of the training sample image and the color loss value of the labeled shadow-free image of the training sample image meet the preset condition. In this way, on the one hand, not only can the preset model be used to perform color correction on the shadow area in the first image, but also illumination compensation and brightness restoration of the shadow area in the first image can be achieved through color correction, and the shadow area in the first image can be removed. In this way, the color of the second image can be made more realistic, the information integrity and clarity of the second image can be improved, and the recognizability of the second image can be improved, thereby effectively improving the user experience. On the other hand, using the preset model to correct the color of the first image and removing the shadow area in the first image to obtain the second image can also make the second image more beautiful, thereby further improving the user experience.

[0149] Optionally, the processor 410 is further configured to:

[0150] Calculate the proportion of the shadow area in the first image in the first image;

[0151] When the proportion is greater than or equal to the preset proportion, it is determined that there is a shadow area in the first image.

[0152] In this way, image processing is performed on the first image only when the proportion of the shadow area in the first image is greater than the preset proportion, that is, the shadow area existing in the first image may affect the information integrity and clarity of the first image and the recognizability of the first image. This can avoid some unnecessary image processing processes, thereby effectively reducing power consumption and improving image processing efficiency.

[0153] Optionally, the processor 410 is further configured to:

[0154] Performing Gaussian filtering on the first image to obtain a third image; the third image is used to reflect the illumination information of the first image;

[0155] Binarizing the third image to obtain a fourth image;

[0156] Performing connected component analysis on the fourth image to calculate the areas of the connected components in the fourth image;

[0157] Calculate the ratio of the area of ​​the connected components to the total area of ​​the first image.

[0158] In this way, by binarizing the third image reflecting the illumination information of the first image to obtain the fourth image, the amount of data in the fourth image can be greatly reduced. In this way, by calculating the proportion of the area of ​​the connected components in the fourth image to the total area of ​​the first image based on the fourth image with less data, the amount of data calculation can be effectively reduced, thereby further improving the efficiency of image processing.

[0159] Optionally, the processor 410 is further configured to:

[0160] Performing scaling processing on the first image to obtain a fifth image;

[0161] Gaussian filtering is performed on the fifth image to obtain a third image.

[0162] In this way, scaling the first image to a fifth image having an area smaller than or equal to the first preset area can reduce power and consumption, thereby further improving image processing efficiency.

[0163] Optionally, the processor 410 is further configured to:

[0164] Performing connected component analysis on the fourth image to obtain M connected regions in the fourth image;

[0165] Remove the first connected area among the M connected areas;

[0166] Calculate the total area of ​​the second connected region among the M connected regions;

[0167] The ratio of the total area of ​​the second connected region to the total area of ​​the first image is calculated.

[0168] In this way, the first connected region whose area is smaller than the second preset area is removed from the fourth image obtained after the third image is binarized, that is, the shadow region in the fourth image that does not affect the information integrity and clarity of the image but affects the recognizability of the image is removed. In this way, unnecessary image processing of the first connected region can be reduced, thereby further reducing power consumption and improving image processing efficiency.

[0169] Optionally, the processor 410 is further configured to:

[0170] Calculating a clarity value of the second image;

[0171] When the definition value of the second image is greater than or equal to the preset definition value, the second image is saved.

[0172] In this way, on the one hand, since the second image has no shadow area, its information integrity and clarity value are higher than those of the first image, so the user can view and use an image with higher recognizability, thereby further improving the user experience. On the other hand, based on the fact that the second image has no shadow area, only the second image with a clarity value greater than or equal to the preset clarity value is saved for the user to view and use, so that the second image viewed and used by the user has higher clarity and higher recognizability, thereby further improving the user experience.

[0173] Optionally, the processor 410 is further configured to:

[0174] When the shooting program is in the target shooting mode and there is a shadow area in the shooting preview image, inputting the shooting preview image into the preset model;

[0175] The second image is photographed and saved.

[0176] In this way, during the photo-taking process, the color correction of the photo preview image with shadow areas can be achieved, and the shadow areas in the photo preview image can be removed. This allows users to directly obtain shadow-free images with high information integrity and high clarity during the photo-taking process, thereby improving the recognizability of the images obtained by taking photos, thereby further improving the user experience.

[0177] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0178] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0179] The present application also provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the above xxx The various processes of the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0180] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0181] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0182] In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0183] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0184] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: include: Performing Gaussian filtering on the first image to obtain a third image; The third image is used to reflect the lighting information of the first image; Binarizing the third image to obtain a fourth image; Performing connected component analysis on the fourth image to calculate the areas of connected components in the fourth image; Calculating the proportion of the area of ​​the connected component in the entire area of ​​the first image; When the proportion is greater than or equal to a preset proportion, determining that a shadow area exists in the first image; In the case where there is a shadow area in the first image, inputting the first image into a preset model; Performing color correction on the shadow area using the preset model to obtain a second image; The training stopping condition of the preset model is that the color loss value of the predicted shadow-free image of the training sample image and the labeled shadow-free image of the training sample image meets the preset condition, and the predicted shadow-free image is the image input into the preset model, processed by the preset model and output by the preset model. The labeled shadow-free image is a real image without shadow corresponding to the training sample image.

2. The method according to claim 1, characterized in that: The performing Gaussian filtering on the first image to obtain a third image includes: Scaling the first image to obtain a fifth image; the area of ​​the fifth image is less than or equal to the first preset area; Perform Gaussian filtering on the fifth image to obtain a third image.

3. The method according to claim 1, characterized in that The performing connected component analysis on the fourth image to calculate the area of ​​the connected components in the fourth image includes: Performing connected component analysis on the fourth image to obtain M connected regions in the fourth image, where M is a positive integer; Remove the first connected area among the M connected areas; the first connected area is a connected area whose area is smaller than the second preset area; Calculate the total area of ​​the second connected region among the M connected regions; the second connected region is the connected region among the M connected regions except the first connected region; The calculating the proportion of the area of ​​the connected component in the whole area of ​​the first image includes: The ratio of the total area of ​​the second connected region to the entire area of ​​the first image is calculated.

4. The method according to any one of claims 1 to 3, characterized in that: After the color correction of the shadow area is performed using the preset model to obtain the second image, the method further includes: Calculating a clarity value of the second image; When the definition value of the second image is greater than or equal to a preset definition value, the second image is saved.

5. An image processing device, characterized in that: include: A filtering unit, configured to perform Gaussian filtering on the first image to obtain a third image; The third image is used to reflect the lighting information of the first image; A binarization unit, used for performing binarization processing on the third image to obtain a fourth image; a first calculation unit, configured to perform connected component analysis on the fourth image, and calculate areas of connected components in the fourth image; A second calculation unit, used for calculating the proportion of the area of ​​the connected component in the whole area of ​​the first image; A determination module, configured to determine that a shadow area exists in the first image when the proportion is greater than or equal to a preset proportion; A first input module, used for inputting the first image into a preset model when there is a shadow area in the first image; A correction module, used for performing color correction on the shadow area using the preset model to obtain a second image; The training stopping condition of the preset model is that the color loss value of the predicted shadow-free image of the training sample image and the labeled shadow-free image of the training sample image meets the preset condition, and the predicted shadow-free image is the image input into the preset model, processed by the preset model and output by the preset model. The labeled shadow-free image is a real image without shadow corresponding to the training sample image.

6. The device according to claim 5, characterized in that The filtering unit comprises: A scaling subunit, configured to scale the first image to obtain a fifth image; the area of ​​the fifth image is less than or equal to a first preset area; The filtering subunit is used to perform Gaussian filtering on the fifth image to obtain a third image.

7. The device according to claim 5, characterized in that The first computing unit comprises: An analysis component, used for performing connected component analysis on the fourth image to obtain M connected regions in the fourth image, where M is a positive integer; A removal component, used to remove a first connected area from the M connected areas; the first connected area is a connected area whose area is smaller than a second preset area; A calculation component, used for calculating the total area of ​​a second connected region among the M connected regions; the second connected region is a connected region among the M connected regions except the first connected region; The second computing unit is specifically configured to: The ratio of the total area of ​​the second connected region to the entire area of ​​the first image is calculated.

8. The device according to any one of claims 5 to 7, characterized in that: The image processing device further includes: A second calculation module, used for calculating the clarity value of the second image; A saving module is used to save the second image when the clarity value of the second image is greater than or equal to a preset clarity value.

Images