Model training method and device

Abstract

The invention discloses a model training method and device, and belongs to the technical field of image processing. The model training method comprises the following steps: acquiring a dazzle light weakening reference image corresponding to a dazzle light reference image, and acquiring a first residual image corresponding to a first floodlight image; based on the first residual image, performing image processing on the first floodlight image to obtain a second residual image; obtaining a dazzle light training data pair based on the dazzle light weakening reference image and the non-dazzle light reference image, and obtaining a floodlight training data pair based on the second residual image and the first floodlight image; and training the initial model based on the dazzle light training data pair and the floodlight training data pair to obtain a first model.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a model training method and a device thereof. Background Art

[0002] Currently, when an electronic device captures an image with a light source, if the brightness of the light source is high, the brightness difference between the brightness of the captured background and the brightness of the light source will be large, so glare and blooming may occur around the light source in the image captured by the electronic device.

[0003] In the prior art, electronic devices can remove glare and blooming in images through relative smoothness technology. Specifically, the electronic device can identify the high-brightness image area in the image through the brightness distribution of the image; then, filter the brightness value of the pixel points in the high-brightness image area to adjust the brightness value of each pixel point in the high-brightness image area to the brightness value of the non-high-brightness area in the image; thereby reducing the glare and blooming in the image.

[0004] However, in the above method, since the glare phenomenon usually includes high-frequency image details and the blooming phenomenon usually includes low-frequency image details, the electronic device filters the high-brightness image area, usually by adjusting the brightness of the low-frequency image details, resulting in the above method being basically ineffective against the glare phenomenon; thus, the image quality of the image captured by the electronic device is poor. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a model training method and device thereof, which can reduce blooming and glare phenomena in captured images, thereby improving the image quality of images captured by electronic devices.

[0006] In a first aspect, an embodiment of the present application provides a model training method, which includes: obtaining a glare attenuation reference image corresponding to a glare reference image, and obtaining a first residual map corresponding to a first floodlight image, wherein the image brightness of the glare attenuation reference image is less than the image brightness of the glare reference image, and the first residual map is used to characterize the brightness difference between the brightness of each two adjacent pixels in the first floodlight image; based on the first residual map, performing image processing on the first floodlight image to obtain a second residual map, wherein the brightness residual value contained in the second residual map is greater than the brightness residual value contained in the first residual map; based on the glare attenuation reference image and the glare-free reference image, obtaining a glare training data pair, and based on the second residual map and the first floodlight image, obtaining a floodlight training data pair; based on the glare training data pair and the floodlight training data pair, training an initial model to obtain a first model.

[0007] In a second aspect, an embodiment of the present application provides a model training device, which includes: an acquisition module, a processing module, and a training module. The acquisition module is used to acquire a glare reduction reference image corresponding to a glare reference image, and to acquire a first residual image corresponding to a first floodlight image, wherein the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of each two adjacent pixels in the first floodlight image. The processing module is used to perform image processing on the first floodlight image based on the first residual image acquired by the acquisition module to obtain a second residual image, wherein the brightness residual value contained in the second residual image is greater than the brightness residual value contained in the first residual image; and to obtain a glare training data pair based on the glare reduction reference image and the glare-free reference image, and to obtain a floodlight training data pair based on the second residual image and the first floodlight image. The training module is used to train the initial model based on the glare training data pair and the floodlight training data pair obtained by the processing module to obtain a first model.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, a glare reduction reference image corresponding to a glare reference image and a first residual image corresponding to a first floodlight image can be obtained, wherein the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of each two adjacent pixels in the first floodlight image; then, based on the first residual image, image processing is performed on the first floodlight image to obtain a second residual image, wherein the brightness residual value contained in the second residual image is greater than the brightness residual value contained in the first residual image; and based on the glare reduction reference image and the glare-free reference image, a glare training data pair is obtained, and based on the second residual image and the first floodlight image, a floodlight training data pair is obtained; finally, based on the glare training data pair and the floodlight training data pair, an initial model is trained to obtain a first model. In this solution, since the first model is trained based on the glare training data pair and the bloom training data pair, the first model can have the ability to simultaneously reduce glare and bloom in the image; and, since the brightness residual value contained in the second residual map is greater than the brightness residual value contained in the first residual map, the first model can restore the original brightness of the image while removing the glare and bloom in the image, and can avoid the reduction in the brightness of the image after removing the glare and bloom in the image; in this way, the electronic device can reduce glare and bloom in the image while ensuring the original brightness of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is one of the flow charts of a model training method provided in an embodiment of the present application;

[0014] Figure 2 This is the second flowchart of a model training method provided in an embodiment of the present application;

[0015] Figure 3 This is the third flowchart of a model training method provided in an embodiment of the present application;

[0016] Figure 4 This is the fourth flowchart of a model training method provided in an embodiment of the present application;

[0017] Figure 5 This is the fifth flowchart of a model training method provided in an embodiment of the present application;

[0018] Figure 6 This is the sixth flowchart of a model training method provided in an embodiment of the present application;

[0019] Figure 7 This is the seventh flowchart of a model training method provided in an embodiment of the present application;

[0020] Figure 8 This is the eighth flowchart of a model training method provided in an embodiment of the present application;

[0021] Fig. 9 This is the ninth flowchart of a model training method provided in an embodiment of the present application;

[0022] Fig.10 This is the tenth flowchart of a model training method provided in an embodiment of the present application;

[0023] Fig.11 This is one of the example schematic diagrams of a shooting interface provided in an embodiment of the present application;

[0024] Fig.12 This is a second example schematic diagram of a shooting interface provided in an embodiment of the present application;

[0025] Fig.13 It is a structural schematic diagram of a model training device provided in an embodiment of the present application;

[0026] Fig.14 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0027] Fig.15 This is the second hardware structure diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The terms "at least one (item)", "at least one of" and the like in the specification and claims of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0031] The model training method and device provided in the embodiments of the present application are described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0032] The model training method and device provided in the embodiments of the present application can be applied to shooting scenes for removing bloom and glare, such as photo shooting scenes and video shooting scenes.

[0033] In scenario 1, the user can actually take pictures to obtain a glare reference image, a bloom reference image and a glare-free reference image, and then input the glare reference image, the bloom reference image and the glare-free reference image into the electronic device respectively, so that the electronic device can perform image processing on the glare reference image to obtain a glare reduction reference image, and process the bloom reference image through a relative smoothness technique to obtain a first residual image; then, the electronic device can perform brightness restoration processing on the bloom reference image according to the first residual image, and obtain a second residual image; then, the electronic device can obtain a glare training data pair according to the glare reduction reference image and the glare-free reference image, and obtain a bloom training data pair according to the second residual image and the bloom reference image; then, the electronic device can input the glare training data pair and the glare training data pair into the initial model, and train the initial model through the glare training data pair and the glare training data pair to obtain a first model; finally, the electronic device can input the taken photos into the first model to reduce bloom and glare in the taken photos.

[0034] In scenario 2, the user can actually take pictures to obtain a glare reference image, a bloom reference image, and a glare-free reference image, and then input the glare reference image, the bloom reference image, and the glare-free reference image into the electronic device respectively, so that the electronic device can perform image processing on the glare reference image to obtain a glare reduction reference image, and process the bloom reference image through a relative smoothness technique to obtain a first residual image; then, the electronic device can perform brightness restoration processing on the bloom reference image according to the first residual image, and obtain a second residual image; then, the electronic device can obtain a glare training data pair according to the glare reduction reference image and the glare-free reference image, and obtain a bloom training data pair according to the second residual image and the bloom reference image; then, the electronic device can input the glare training data pair and the glare training data pair into the initial model, and train the initial model through the glare training data pair and the glare training data pair to obtain a first model; finally, the electronic device can input the captured video into the first model, and the first model can perform image processing on each frame of the video to reduce bloom and glare in the captured video.

[0035] The execution subject of the model training method provided in the embodiment of the present application may be a model training device, and the model training device may be an electronic device, or a functional module in an electronic device. The technical solution provided in the embodiment of the present application is described below using an electronic device as an example.

[0036] The present application embodiment provides a model training method. Figure 1 A flow chart of a model training method provided in an embodiment of the present application is shown. Figure 1 As shown, the model training method provided in the embodiment of the present application may include the following steps 201 to 204.

[0037] Step 201: The electronic device obtains a glare reduction reference image corresponding to a glare reference image, and obtains a first residual image corresponding to a first floodlight image.

[0038] In the embodiment of the present application, the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of every two adjacent pixels in the first floodlight image.

[0039] In the embodiment of the present application, the glare reference image may be a reference image containing a glare phenomenon.

[0040] Optionally, in an embodiment of the present application, the above-mentioned glare reference image may be one or more.

[0041] Optionally, in an embodiment of the present application, the electronic device may capture the glare reference image; or the electronic device may obtain the glare reference image from a photo album application; or the electronic device may download the glare reference image through a browser.

[0042] Preferably, the electronic device can obtain the above-mentioned glare reference image by photographing.

[0043] For example, taking the electronic device as a mobile phone, the user can prepare a mobile phone and dirty the camera of the mobile phone. In a dark room in a laboratory environment, the user can use the mobile phone to actually shoot white light, requiring the light to directly shine on the camera of the electronic device, changing the light position and light type, and obtaining multiple sets of real-shot glare reference images containing only light glare.

[0044] It can be understood that in the case where the glare reference image is a plurality of glare reference images, the glare reduction reference image is also a plurality of glare reduction reference images, and the plurality of glare reference images and the plurality of glare reduction reference images are Figure 1 One to one correspondence.

[0045] In the embodiment of the present application, the first floodlight image is an image including a floodlight phenomenon.

[0046] Optionally, in the embodiment of the present application, the above-mentioned floodlight image may be one or more.

[0047] Optionally, in an embodiment of the present application, the electronic device may capture the above-mentioned floodlight image; or the electronic device may obtain the above-mentioned floodlight image from a photo album application; or the electronic device may download the floodlight image through a browser.

[0048] Preferably, the electronic device can obtain the above-mentioned floodlight image by photographing.

[0049] For example, a user may prepare a mobile phone and dirty the camera of the mobile phone, and take images at night in an environment where light halos are easily produced, such as advertising light billboards, to obtain multiple flood images with blurred light.

[0050] Optionally, in the embodiment of the present application, the camera may be any of the following: a wide-angle camera, a macro camera, a panoramic camera, or an ultra-wide-angle camera, etc. The specific camera may be determined according to actual use requirements, and the embodiment of the present application does not limit the camera.

[0051] Optionally, in the embodiment of the present application, the electronic device may input the first floodlight image into a relative smoothing algorithm to obtain the above-mentioned first residual image through the algorithm.

[0052] Exemplarily, the electronic device may obtain the first residual image through an image decomposition algorithm in a relative smoothing algorithm, which may be specifically implemented through the following formula (1).

[0053] d init ,J0=H(I f ) (1)

[0054] Among them, d init is the first residual image, J0 is the clean image, H is the image decomposition algorithm, I f Flood image.

[0055] It can be understood that, in the case where the first floodlight image is a plurality of first floodlight images, the first residual map also has a plurality of first residual maps, and the plurality of first floodlight images and the plurality of first residual maps are Figure 1 One to one correspondence.

[0056] Step 202: The electronic device performs image processing on the first floodlight image based on the first residual image to obtain a second residual image.

[0057] In the embodiment of the present application, the brightness residual value included in the second residual image is greater than the brightness residual value included in the first residual image.

[0058] Optionally, in an embodiment of the present application, the second residual map may be one or more.

[0059] Exemplarily, when there are multiple first residual maps and multiple first floodlight images, the electronic device may perform image processing on the first floodlight image corresponding to each residual map based on each of the multiple first residual maps to obtain multiple second residual maps.

[0060] It can be understood that the multiple first floodlight images and the multiple second residuals Figure 1 One to one correspondence.

[0061] In an embodiment of the present application, the electronic device can perform color correction and brightness restoration processing on the first floodlight image based on the first residual map to obtain a second residual map. The specific implementation process can be found in the following embodiments, and will not be described here to avoid repetition.

[0062] Step 203: The electronic device obtains a glare training data pair based on the glare reduction reference image and the glare-free reference image, and obtains a floodlight training data pair based on the second residual image and the first floodlight image.

[0063] Optionally, in an embodiment of the present application, the glare-free reference image may be one or more.

[0064] Optionally, in an embodiment of the present application, the electronic device may capture the glare-free reference image; or the electronic device may obtain the glare-free reference image from an album application; or the electronic device may download the glare-free reference image through a browser.

[0065] Preferably, the electronic device can obtain the above-mentioned glare-free reference image by photographing.

[0066] For example, the user can prepare a mobile phone and ensure that the camera lens of the mobile phone is clean without scratches or dirt, and use the mobile phone to collect real-life images. In an environment without obvious light sources, collect several sets of non-laboratory scene glare-free reference images.

[0067] Optionally, in an embodiment of the present application, the electronic device may combine the glare reduction reference image and the glare-free reference image to obtain a glare training data pair; and combine the second residual image and the first floodlight image to obtain a floodlight training data pair.

[0068] It can be understood that when there are multiple glare reduction reference images, there may be multiple glare training data pairs; when there are multiple second residual images and multiple first flood images, there may be multiple flood training data pairs.

[0069] Step 204: The electronic device trains the initial model based on the glare training data pair and the floodlight training data pair to obtain a first model.

[0070] Optionally, in an embodiment of the present application, the above-mentioned initial module can be any one of an AI model or a neural network model.

[0071] Exemplarily, the neural network model can be any of the following: a full convolutional network model, a Gaussian mixture model, or a convolutional network for biomedical image segmentation (U-Net) model, etc. The specific model can be determined according to actual use requirements, and the embodiments of the present application are not limited thereto.

[0072] In an embodiment of the present application, the electronic device can input the glare training data pair and the floodlight training data pair into the initial model, obtain a loss function through the glare training data pair and the floodlight training data pair, and then adjust the model parameters of the initial model through the loss function to obtain the trained initial model, that is, the first model mentioned above.

[0073] In the model training method provided in the embodiment of the present application, the electronic device can obtain a glare reduction reference image corresponding to the glare reference image, and obtain a first residual image corresponding to the first floodlight image, the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of each two adjacent pixels in the first floodlight image; then based on the first residual image, the first floodlight image is subjected to image processing to obtain a second residual image, and the brightness residual value contained in the second residual image is greater than the brightness residual value contained in the first residual image; and based on the glare reduction reference image and the glare-free reference image, a glare training data pair is obtained, and based on the second residual image and the first floodlight image, a floodlight training data pair is obtained; finally, based on the glare training data pair and the floodlight training data pair, the initial model is trained to obtain the first model. In this solution, since the first model is trained based on the glare training data pair and the bloom training data pair, the first model can have the ability to simultaneously reduce glare and bloom in the image; and, since the brightness residual value contained in the second residual map is greater than the brightness residual value contained in the first residual map, the first model can restore the original brightness of the image while removing the glare and bloom in the image, and can avoid the reduction in the brightness of the image after removing the glare and bloom in the image; in this way, the electronic device can reduce glare and bloom in the image while ensuring the original brightness of the image.

[0074] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 2 As shown, the "electronic device obtains a glare reduction map corresponding to the glare image" in the above step 201 can be specifically implemented through the following steps 201a to 201c.

[0075] Step 201a: The electronic device determines a first image area from a glare reference image based on a first preset brightness threshold.

[0076] In the embodiment of the present application, the brightness value of each pixel in the above-mentioned first image area is greater than or equal to the first preset brightness threshold.

[0077] Optionally, in an embodiment of the present application, the first preset brightness threshold may be preset by the electronic device; or customized by the user.

[0078] Optionally, in an embodiment of the present application, the electronic device may directly determine, based on the above-mentioned first preset brightness threshold, an area consisting of pixels in the glare reference image whose brightness values ​​are greater than or equal to the first preset brightness threshold as the first image area.

[0079] Optionally, in an embodiment of the present application, the electronic device may binarize the glare reference image to obtain a binarized glare reference image, and then compare the pixel value of each pixel in the binarized glare reference image with a first threshold value, and determine the area composed of pixels in the glare image that are greater than the first threshold value as the first image area.

[0080] Exemplarily, the user can determine a threshold value based on the laboratory shooting environment, i.e., the above-mentioned first preset brightness threshold value, and then, the electronic device can binarize the glare reference image according to the threshold value to obtain a binary overexposed mask image; then, the area composed of pixel points whose pixel values ​​in the overexposed mask image are greater than 0.5, i.e., the above-mentioned first threshold value, is determined as the first image area.

[0081] It can be understood that the first image area mentioned above is a high brightness area in the glare reference image.

[0082] Step 201b: The electronic device calculates a glare attenuation reference image based on coordinate information of at least one pixel point of the glare reference image and coordinate information of at least one first pixel point in the first image area.

[0083] In an embodiment of the present application, the at least one first pixel point is a pixel point among the edge pixels of the first image area that intersects with at least one first line, and each of the at least one first line is a line between the central pixel point and a pixel point in the glare reference image.

[0084] In the embodiment of the present application, the electronic device can perform a mean operation on the coordinate information of each pixel point in the first image area to obtain the coordinate information of the above-mentioned central pixel point.

[0085] Optionally, in an embodiment of the present application, the above-mentioned edge pixel point may be one or more.

[0086] Optionally, in an embodiment of the present application, the electronic device may multiply the brightness value of each pixel in the glare reference image by the distance difference between the coordinate information of each pixel in the glare reference image and the coordinate information of at least one first pixel to obtain a glare attenuation reference image, which may be specifically implemented by the following formula (2).

[0087]

[0088] Among them, I d is the glare attenuation diagram, sigma is a constant of 10, distance (P x ,P b ) is the absolute distance between each pixel in the glare reference image and at least one first pixel.

[0089] Optionally, in an embodiment of the present application, the electronic device can obtain a first glare attenuation reference map through the above formula (2), and then filter the first glare attenuation reference map based on the glare reference image to obtain a glare attenuation reference map, which can be specifically achieved through the following formula (3).

[0090] I d '=g(I d ,I g ) (3)

[0091] Among them, I d ' is the glare attenuation reference diagram, I d is the first glare attenuation reference graph, I g is the glare reference image and g is the guided filter.

[0092] It should be noted that the above-mentioned glare attenuation reference map may be a weight map.

[0093] Step 201c: The electronic device performs a multiplication operation on the brightness between the glare attenuation reference map and the glare reference image to obtain a glare reduction reference map.

[0094] In an embodiment of the present application, the electronic device may multiply the brightness value of each pixel in the glare attenuation reference map with the brightness value of each pixel in the glare reference image to obtain a glare attenuation reference map.

[0095] It can be understood that each pixel point in the above glare attenuation reference image corresponds one-to-one to each pixel point in the glare reference image.

[0096] For example, Figure 3 As shown, the above embodiment is specifically explained below through specific examples, which can be implemented through the following steps 20 to 25.

[0097] Step 20: Take a real shot of the electronic device to obtain a glare reference image.

[0098] Step 21: The electronic device binarizes the glare reference image and detects the overexposed area.

[0099] In the embodiment of the present application, the overexposed area is the first image area.

[0100] Step 22: The electronic device obtains edge pixel points of the overexposed area, and calculates the center pixel point of the overexposed area through the coordinate information of each pixel point in the overexposed area.

[0101] Step 23: The electronic device connects the central pixel point with each pixel point in the glare reference image, and determines the edge pixel point intersecting with the connecting line as at least one first pixel point.

[0102] Step 24: The electronic device obtains a weight map by multiplying the brightness value of each pixel in the glare reference image by the distance difference between the coordinate information of each pixel in the glare reference image and the coordinate information of at least one first pixel.

[0103] Step 25: The electronic device multiplies the weight map by the brightness value of the glare reference image to obtain a glare reduction map.

[0104] In the embodiments of the present application, the electronic device can simulate the real physical process of glare reduction through the above embodiments, retain the overexposed area in the center of the light, and there is no need to superimpose the overexposed light on the clean glare image later, thereby improving the flexibility of the electronic device in obtaining the glare reduction reference image.

[0105] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 4 As shown, the above step 202 can be specifically implemented through the following steps 202a to 202c.

[0106] Step 202a: The electronic device performs color correction on the non-flood area in the first flood image based on the first residual image to obtain a second flood image.

[0107] Optionally, in an embodiment of the present application, the electronic device may determine an image area consisting of pixels whose brightness value of each pixel in the first floodlight image is less than a second preset threshold as a non-floodlight area, and then perform color correction on the non-floodlight area through the first residual map to obtain a second floodlight image.

[0108] Optionally, in the embodiment of the present application, the electronic device may adaptively determine a non-flood area from the first flood image through the first residual map, and then perform color correction on the non-flood area to obtain a second flood image.

[0109] It should be noted that the above specific process can be found in the following embodiments, and will not be described again here to avoid repetition.

[0110] Optionally, in the embodiment of the present application, combined with Figure 4 ,like Figure 5 As shown, the above step 202a can be specifically implemented through the following steps 301 to 304.

[0111] Step 301: The electronic device performs grayscale conversion on the first residual image to obtain a converted first residual image, and normalizes the converted first residual image to obtain a weight image.

[0112] In the embodiment of the present application, the electronic device may perform grayscale conversion on the first residual image using a first algorithm, and normalize the converted first residual image to obtain a weight image.

[0113] Optionally, in an embodiment of the present application, the first algorithm may be any one of the following: using an average algorithm, using a weight algorithm, using a maximum algorithm, or using a minimum algorithm.

[0114] Step 302: The electronic device performs a multiplication operation on the pixel values ​​between the weight map and the first residual map to obtain a third residual map.

[0115] In an embodiment of the present application, the electronic device may multiply each weight value in the weight map by the brightness residual value corresponding to each weight value in the weight map in the first residual map to obtain a third residual map, which may be specifically implemented by the following formula (4).

[0116] d flare =d init_map ·d init (4)

[0117] Among them, d flare is the third residual graph, d init_map is the weight graph, d init is the first residual graph.

[0118] Step 303: The electronic device filters the first floodlight image based on the third residual image to obtain a filtered first floodlight image.

[0119] In the embodiment of the present application, the electronic device performs guided filtering on the first floodlight image through the third residual image to obtain the filtered first floodlight image, which can be specifically achieved by the following formula (5).

[0120] d blue =g(d flare ,I f ) (5)

[0121] Among them, d blue is the first floodlight image after filtering, g is the guided filter, d flare is the third residual graph, I f is the first flood image.

[0122] Step 304: The electronic device performs a subtraction operation on the brightness between the first floodlight image and the filtered first floodlight image to obtain a second floodlight image.

[0123] In the embodiment of the present application, the electronic device can perform a subtraction operation on the brightness value of each pixel in the first floodlight image and the brightness value of each pixel in the filtered first floodlight image to obtain a second floodlight image, which can be specifically achieved by the following formula (6).

[0124] J1=I f -d blue (6)

[0125] Where, J1 is the second floodlight image, I f is the first flood image, d blue is the first flood image after filtering.

[0126] It should be noted that the second floodlight image is a clean image with floodlight removed.

[0127] In the embodiment of the present application, the electronic device removes the bloom in the bloom image through the above method compared with the prior art using relative smoothness technology. The second bloom image only removes the bloom, and the brightness of the non-flood area changes little compared with the original image.

[0128] Step 202b: The electronic device performs color correction on the floodlight area in the first floodlight image based on the second floodlight image to obtain a third floodlight image.

[0129] Optionally, in an embodiment of the present application, the electronic device may determine an image area consisting of pixels whose brightness value of each pixel in the first floodlight image is greater than or equal to a second preset threshold as a floodlight area, and then perform color correction on the floodlight area through the second floodlight image to obtain a third floodlight image.

[0130] Optionally, in the embodiment of the present application, the electronic device may adaptively determine a floodlight area from the first floodlight image through the second floodlight image, and then perform color correction on the floodlight area to obtain a third floodlight image.

[0131] It should be noted that the above specific process can be found in the following embodiments, and will not be described again here to avoid repetition.

[0132] Optionally, in the embodiment of the present application, combined with Figure 4 ,like Figure 6 As shown, the above step 202b can be specifically implemented through the following steps 401 to 403.

[0133] Step 401: The electronic device performs grayscale conversion on the second floodlight image to obtain a converted second floodlight image, and performs grayscale conversion on the first floodlight image to obtain a converted first floodlight image.

[0134] In the embodiment of the present application, the electronic device can perform grayscale conversion on the second floodlight image using the first algorithm to obtain a converted second floodlight image, and perform grayscale conversion on the first floodlight image using the second algorithm to obtain a converted first floodlight image.

[0135] Step 402: The electronic device performs a division operation on the brightness of the converted second floodlight image and the converted first floodlight image to obtain a fourth residual image.

[0136] In the embodiment of the present application, the electronic device may divide the brightness value of each pixel in the converted second floodlight image by the brightness value of each pixel in the converted first floodlight image to obtain a fourth residual image.

[0137] Step 403: The electronic device performs a multiplication operation on the pixel values ​​between the fourth residual image and the first floodlight image to obtain a third floodlight image.

[0138] It should be noted that the third floodlight image is a clean image with floodlight and color cast removed.

[0139] In an embodiment of the present application, the electronic device can perform pixel channel expansion on the fourth residual image to obtain an expanded fourth residual image, and then multiply each brightness residual value in the expanded fourth residual image by the brightness value of each pixel point in the first floodlight image to obtain a third floodlight image.

[0140] Exemplarily, the pixel channel expansion of the fourth residual image may be performed by duplicating each brightness residual value in the fourth residual image twice and adding the copies to the fourth residual image to obtain the expanded fourth residual image.

[0141] In the embodiment of the present application, the electronic device can perform color correction on the floodlight area in the first floodlight image by multiplying the pixel values ​​between the fourth residual image and the first floodlight image, thereby obtaining a color-corrected, de-flooded clean image. In this way, there will be no color cast in the third floodlight image obtained by the electronic device.

[0142] Step 202c: The electronic device determines a second image region from the first floodlight image based on a second preset brightness threshold, and calculates a second residual image based on the second image region and the third floodlight image.

[0143] In the embodiment of the present application, the brightness value of each pixel in the above-mentioned second image area is greater than or equal to the second preset brightness threshold.

[0144] Optionally, in the embodiment of the present application, the second preset brightness threshold may be preset by the electronic device; or customized by the user.

[0145] In the embodiment of the present application, the electronic device may determine the second image area from the first floodlight image by using the following formula (7), and the formula (7) may specifically be:

[0146] M over =where(I f =255) (7)

[0147] Among them, M over is the second image area, I f=255 is an area where the brightness value of the pixel points in the first floodlight image is equal to 255.

[0148] Optionally, in the embodiment of the present application, combined with Figure 4 ,like Figure 7 As shown, the above step 202c can be specifically implemented through the following steps 501 and 502.

[0149] Step 501: The electronic device calculates a fourth floodlight image based on the brightness value of the second image area, the brightness value of the third floodlight image, and the brightness value of the first floodlight image.

[0150] In the embodiment of the present application, the electronic device can fuse the brightness values ​​of each pixel of the first floodlight image, the second image area and the third floodlight image through a pyramid fusion algorithm to obtain a clean image with normal highlights restored, that is, the fourth floodlight image mentioned above, which can be specifically implemented through the following formula (8).

[0151]

[0152] Among them, J M is the fourth flood image, is the pyramid fusion algorithm, I f is the first floodlight image, M over is the second image area, and J is the third floodlight image.

[0153] Step 502: The electronic device performs a subtraction operation on the brightness values ​​of the fourth floodlight image and the first floodlight image to obtain a second residual image.

[0154] In the embodiment of the present application, the electronic device may perform a subtraction operation on the brightness value of each pixel in the fourth floodlight image and the brightness value of each pixel in the first floodlight image to obtain a second residual image.

[0155] Optionally, in the embodiment of the present application, the electronic device may perform a subtraction operation on the brightness value of each pixel in the fourth floodlight image from the brightness value of each pixel in the first floodlight image, and then perform guided filtering to obtain a second residual image.

[0156] In the embodiment of the present application, the electronic device can obtain a clean image with normal highlight restoration through a pyramid fusion algorithm, thereby avoiding the reduction in brightness of the image after removing glare and bloom in the image.

[0157] For example, Figure 8 As shown, the above embodiment is specifically explained below through a specific example, which can be implemented through the following steps 30 to 40.

[0158] Step 30: The electronic device actually captures a first floodlight image.

[0159] Step 31: The electronic device performs image processing on the first floodlight image by using an image decomposition algorithm in the relative smoothing technology to obtain a first residual image.

[0160] Step 32: The electronic device performs grayscale conversion on the first residual image and performs normalization processing to obtain a weight image.

[0161] Step 33: The electronic device multiplies the weight of the weight map by the brightness residual value of the first residual map to obtain a third residual map.

[0162] Step 34: The electronic device performs guided filtering on the first floodlight image using the third residual image to obtain a filtered first floodlight image.

[0163] Step 35: The electronic device subtracts the brightness value of the filtered first floodlight image from the brightness value of the first floodlight image to obtain a clean first floodlight image.

[0164] Step 36: The electronic device performs grayscale conversion on the clean first floodlight image to obtain a converted clean first floodlight image.

[0165] Step 37: The electronic device performs grayscale conversion on the first floodlight image to obtain a converted first floodlight image.

[0166] Step 38: The electronic device divides the brightness value of the converted clean first floodlight image by the brightness value of the converted first floodlight image to obtain a processed first floodlight image.

[0167] Step 39: The electronic device performs channel expansion on the processed first floodlight image, and multiplies the brightness value of the first floodlight image after the channel expansion by the brightness value of the first floodlight image to obtain a third floodlight image.

[0168] Step 40: The electronic device calculates the high brightness area in the first floodlight image, and fuses the brightness value of the high brightness area, the brightness value of the first floodlight image, and the brightness value of the third floodlight image to obtain a fourth floodlight image.

[0169] Step 41: The electronic device may perform a subtraction operation on the brightness value of the fourth floodlight image and the brightness value of the first floodlight image to obtain a fifth residual image, and filter the first floodlight image based on the fifth residual image to obtain a second residual image.

[0170] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Fig. 9 As shown, in the above step 203, "the electronic device obtains a glare training data pair based on the glare reduction reference image and the glare-free reference image" can be specifically implemented through the following steps 203a to 203d.

[0171] Step 203a: The electronic device performs color augmentation on the glare reference image to obtain an augmented glare reference image.

[0172] In the embodiment of the present application, the electronic device may perform color augmentation on the glare reference image by changing the ratio of the three RGB channels of the glare image to obtain an augmented glare reference image.

[0173] Exemplarily, the electronic device may change the ratio of two channels RG of the glare reference image, thereby obtaining a red / yellow / blue glare reference image.

[0174] It can be understood that when there are multiple glare reference images, there are also multiple augmented glare reference images, and the multiple glare reference images correspond one-to-one to the multiple augmented glare reference images.

[0175] Step 203b: The electronic device performs an addition operation on the brightness between the augmented glare reference image and the non-glare reference image to obtain a first image.

[0176] In an embodiment of the present application, the electronic device may add the brightness value of each pixel in the augmented glare reference image and the brightness value of each pixel in the non-glare reference image to obtain a degraded glare reference image, namely the first image mentioned above.

[0177] Optionally, in an embodiment of the present application, when there are multiple augmented glare reference images and one non-glare reference image, the electronic device may perform addition operations on the brightness between the multiple augmented glare reference images and one non-glare reference image respectively to obtain multiple first images, and the multiple augmented glare reference images correspond one-to-one to the multiple first images.

[0178] Optionally, in an embodiment of the present application, when there are multiple augmented glare reference images and multiple non-glare reference images, the multiple augmented glare reference images correspond one-to-one to the multiple non-glare reference images, and the electronic device can add the brightness between each augmented glare reference image in the multiple augmented glare reference images and the corresponding non-glare reference image to obtain multiple first images, and the multiple augmented glare reference images correspond one-to-one to the multiple first images.

[0179] Step 203c: The electronic device performs an addition operation on the brightness between the first image and the glare reduction reference image to obtain a second image.

[0180] In an embodiment of the present application, the electronic device may add the brightness value of each pixel in the first image and the brightness value of each pixel in the glare reduction reference image to obtain a degraded image with glare removed, namely the second image mentioned above.

[0181] Optionally, in an embodiment of the present application, when there are multiple first images and glare reduction reference images, the electronic device can add the brightness between each first image in the multiple first images and the corresponding glare reduction reference image to obtain multiple second images, and the multiple first images correspond one-to-one to the multiple second images.

[0182] Step 203d: The electronic device combines the first image and the second image to obtain a glare training data pair.

[0183] Optionally, in an embodiment of the present application, the glare training data pair may be one or more.

[0184] Exemplarily, when there is one first image and one second image, there may be one glare training data pair; when there are multiple first images and multiple second images, there may be multiple glare training data pairs.

[0185] In an embodiment of the present application, the electronic device can train an initial model using glare training data, so that the trained initial model, that is, the first model, has the ability to remove glare from an image.

[0186] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Fig.10 As shown, the "obtaining a floodlight training data pair based on the second residual image and the first floodlight image" in the above step 203 can be specifically implemented through the following steps 203e to 203g.

[0187] Step 203e: The electronic device performs a subtraction operation on the pixel values ​​between the second residual image and the first floodlight image to obtain a fifth floodlight image.

[0188] In the embodiment of the present application, the electronic device may perform a subtraction operation on each brightness residual value in the second residual image and the brightness value of each pixel point in the first floodlight image to obtain a fifth floodlight image.

[0189] Step 203f: The electronic device performs an addition operation on the brightness between the first floodlight image and the glare reduction reference image to obtain a third image.

[0190] In the embodiment of the present application, the electronic device can add the brightness value of each pixel in the first floodlight image and the brightness value of each pixel in the glare reduction reference image to obtain an image with both glare and floodlight, that is, the third image mentioned above.

[0191] Step 203g: The electronic device combines the fifth floodlight image and the third image to obtain a floodlight training data pair.

[0192] Optionally, in the embodiment of the present application, the above-mentioned floodlight training data pair may be one or more.

[0193] In the embodiment of the present application, the electronic device can train the initial model through the bloom training data, so that the trained initial model, that is, the first model, can have the ability to remove bloom in the image.

[0194] Exemplarily, the model training method provided in the embodiment of the present application is specifically explained below through specific examples, which can be specifically implemented through the following steps 50 to 54.

[0195] Step 50: The electronic device collects data.

[0196] In the embodiment of the present application, the above-mentioned data collection includes shooting a glare reference image in a laboratory, shooting a floodlight reference image in a laboratory, and shooting a glare-free clean reference image.

[0197] Step 51: The electronic device performs data processing on the collected glare reference image.

[0198] In an embodiment of the present application, the above-mentioned data processing includes: obtaining overexposed area information, obtaining a glare attenuation map, and obtaining a glare reduction map.

[0199] Optionally, in an embodiment of the present application, the above-mentioned overexposed area information includes: an overexposed area, a center coordinate point of the overexposed area, and a boundary coordinate set of the overexposed area.

[0200] Step 52: The electronic device performs data processing on the collected floodlight reference image.

[0201] In the embodiment of the present application, the above data processing includes: floodlight decomposition, non-floodlight area color correction, floodlight area color correction and highlight area restoration.

[0202] Step 53: The electronic device synthesizes and augments the training data.

[0203] Step 54: The electronic device performs model training using the training data.

[0204] Exemplarily, the effect achieved by the model training method provided in the embodiment of the present application is specifically explained below through specific examples.

[0205] Example 1, such as Fig.11 As shown, the electronic device can photograph the street light to obtain an image containing the street light, and then input the image into the first model. The image is processed by the first model. The first model can simulate the real physical process of glare reduction, retain the overexposed area in the center of the light, maintain the color consistency before and after glare reduction, and at the same time remove the floodlight and glare phenomena in the night scene.

[0206] Example 2, such as Fig.12 As shown, the electronic device can shoot the billboard to obtain an image containing the billboard, and then input the image into the first model. The image is processed by the first model. The first model can simulate the real physical process of glare reduction, retain the overexposed area in the center of the light, maintain the color consistency before and after glare reduction, and at the same time remove the floodlight and glare phenomena in the night scene.

[0207] The above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0208] It should be noted that the model training method provided in the embodiment of the present application can be executed by a model training device. In the embodiment of the present application, the model training device executing the model training method is taken as an example to illustrate the model training device provided in the embodiment of the present application.

[0209] Fig.13 A possible structural diagram of the model training device involved in the embodiment of the present application is shown. Fig.13 As shown, the model training device 70 may include: an acquisition module 71, a processing module 72 and a training module 73.

[0210] Among them, the acquisition module 71 is used to acquire a glare reduction reference image corresponding to the glare reference image, and to acquire a first residual image corresponding to the first floodlight image, the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of each two adjacent pixels in the first floodlight image. The processing module 72 is used to perform image processing on the first floodlight image based on the first residual image acquired by the acquisition module 71 to obtain a second residual image, the brightness residual value contained in the second residual image is greater than the brightness residual value contained in the first residual image; and to obtain a glare training data pair based on the glare reduction reference image and the glare-free reference image, and to obtain a floodlight training data pair based on the second residual image and the first floodlight image. The training module 73 is used to train the initial model based on the glare training data pair and the floodlight training data pair obtained by the processing module 72 to obtain a first model.

[0211] In one possible implementation, the acquisition module 71 is specifically used to determine a first image area from a glare reference image based on a first preset brightness threshold, wherein the brightness value of each pixel in the first image area is greater than or equal to the first preset brightness threshold; and based on the coordinate information of at least one pixel of the glare reference image and the coordinate information of at least one first pixel in the first image area, calculate a glare attenuation reference map, wherein at least one first pixel is a pixel among edge pixels of the first image area that intersects with at least one first line, and the first line is a line between a central pixel and a pixel in the glare image; and perform a multiplication operation on the brightness between the glare attenuation reference map and the glare reference image to obtain a glare attenuation reference map.

[0212] In a possible implementation, the acquisition module 71 is specifically configured to perform color correction on a non-flood area in a first flood image based on the first residual image to obtain a second flood image; and perform color correction on a flood area in the first flood image based on the second flood image to obtain a third flood image; and determine a second image area from the first flood image based on a second preset brightness threshold, and calculate a second residual map based on the second image area and the third flood image; wherein the brightness value of each pixel in the second image area is greater than or equal to the second preset brightness threshold.

[0213] In a possible implementation, the acquisition module 71 is specifically used to perform grayscale conversion on the first residual image to obtain a converted first residual image, and normalize the first residual image to obtain a weight image; and multiply the pixel values ​​between the weight image and the first residual image to obtain a third residual image; and based on the third residual image, filter the first floodlight image to obtain a filtered first floodlight image; and subtract the brightness between the first floodlight image and the filtered first floodlight image to obtain a second floodlight image.

[0214] In a possible implementation, the acquisition module 71 is specifically configured to perform grayscale conversion on the second floodlight image to obtain a converted second floodlight image, and perform grayscale conversion on the converted first floodlight image to obtain a converted first floodlight image; and perform a division operation on the brightness between the converted second floodlight image and the converted first floodlight image to obtain a fourth residual image; and perform a multiplication operation on the pixel values ​​between the fourth residual image and the first floodlight image to obtain a third floodlight image.

[0215] In a possible implementation, the acquisition module 71 is specifically configured to calculate a fourth floodlight image based on the brightness value of the second image area, the brightness value of the third floodlight image, and the brightness value of the first floodlight image; and perform a subtraction operation on the brightness value between the fourth floodlight image and the first floodlight image to obtain a second residual image.

[0216] In one possible implementation, the processing module 72 is specifically used to perform color augmentation on the glare reference image to obtain an augmented glare reference image; and perform an addition operation on the brightness between the augmented glare reference image and the glare-free reference image to obtain a first image; and perform an addition operation on the brightness between the first image and the glare reduction reference image to obtain a second image; and combine the first image and the second image to obtain a glare training data pair.

[0217] In one possible implementation, the processing module 72 is specifically used to perform a subtraction operation on the pixel values ​​between the second residual image and the first floodlight image to obtain a fifth floodlight image; perform an addition operation on the brightness between the first floodlight image and the glare attenuation reference image to obtain a third image; and combine the fifth floodlight image and the third image to obtain a floodlight training data pair.

[0218] An embodiment of the present application provides a model training device. Since the first model is trained based on a glare training data pair and a bloom training data pair, the first model can have the ability to simultaneously reduce glare and bloom in an image. Moreover, since the brightness residual value included in the second residual map is greater than the brightness residual value included in the first residual map, the first model can restore the original brightness of the image while removing the glare and bloom in the image, and can avoid a decrease in the brightness of the image after removing the glare and bloom in the image. In this way, the model training device can reduce glare and bloom in the image while ensuring the original brightness of the image.

[0219] The model training device in the embodiment of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplary, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a palmtop computer, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobilepersonal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0220] The model training device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0221] The model training device provided in the embodiment of the present application can implement each process implemented in the above embodiment. To avoid repetition, it will not be repeated here.

[0222] Alternatively, if Fig.14 As shown, an embodiment of the present application also provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores programs or instructions that can be executed on the processor 91, and when the program or instructions are executed by the processor 91, the various steps of the above-mentioned model training method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0223] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0224] Fig.15 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0225] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0226] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.15 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0227] Among them, the processor 110 is used to obtain a glare reduction reference image corresponding to the glare reference image, and to obtain a first residual image corresponding to the first floodlight image, the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of each two adjacent pixels in the first floodlight image, and based on the first residual image, perform image processing on the first floodlight image to obtain a second residual image, and the brightness residual value contained in the second residual image is greater than the brightness residual value contained in the first residual image; and based on the glare reduction reference image and the glare-free reference image, obtain a glare training data pair, and based on the second residual map and the first floodlight image, obtain a floodlight training data pair; and based on the glare training data pair and the floodlight training data pair, train the initial model to obtain the first model.

[0228] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to determine a first image area from a glare reference image based on a first preset brightness threshold, and the brightness value of each pixel in the first image area is greater than or equal to the first preset brightness threshold; and based on the coordinate information of at least one pixel of the glare reference image and the coordinate information of at least one first pixel in the first image area, calculate a glare attenuation reference map, at least one first pixel is a pixel among the edge pixels of the first image area that intersects with at least one first line, and the first line is a line between a central pixel and a pixel in the glare reference image; and perform a multiplication operation on the brightness between the glare attenuation reference map and the glare reference image to obtain a glare attenuation reference map.

[0229] Optionally, in an embodiment of the present application, the processor 110 is specifically configured to perform color correction on a non-flood area in a first flood image based on the first residual map to obtain a second flood image; perform color correction on a flood area in the first flood image based on the second flood image to obtain a third flood image; determine a second image area from the first flood image based on a second preset brightness threshold, and calculate a second residual map based on the second image area and the third flood image; wherein the brightness value of each pixel in the second image area is greater than or equal to the second preset brightness threshold.

[0230] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to perform grayscale conversion on the first residual image to obtain a converted first residual image, and normalize the first residual image to obtain a weight map; multiply the pixel values ​​between the weight map and the first residual image to obtain a third residual map; filter the first floodlight image based on the third residual map to obtain a filtered first floodlight image; subtract the brightness between the first floodlight image and the filtered first floodlight image to obtain a second floodlight image.

[0231] Optionally, in the embodiment of the present application, the processor 110 is specifically configured to perform grayscale conversion on the second floodlight image to obtain a converted second floodlight image, and perform grayscale conversion on the converted first floodlight image to obtain a converted first floodlight image; perform a division operation on the brightness between the converted second floodlight image and the converted first floodlight image to obtain a fourth residual image; and perform a multiplication operation on the pixel values ​​between the fourth residual image and the first floodlight image to obtain a third floodlight image.

[0232] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to calculate a fourth floodlight image based on the brightness value of the second image area, the brightness value of the third floodlight image and the brightness value of the first floodlight image; and perform a subtraction operation on the brightness value between the fourth floodlight image and the first floodlight image to obtain a second residual image.

[0233] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to perform color augmentation on the glare reference image to obtain an augmented glare reference image; perform addition operation on the brightness between the augmented glare reference image and the glare-free reference image to obtain a first image; perform addition operation on the brightness between the first image and the glare attenuation reference image to obtain a second image; and combine the first image and the second image to obtain a glare training data pair.

[0234] Optionally, in an embodiment of the present application, the above-mentioned processor 110 is specifically used to perform a subtraction operation on the pixel values ​​between the second residual image and the first floodlight image to obtain a fifth floodlight image; perform an addition operation on the brightness between the first floodlight image and the glare attenuation reference image to obtain a third image; and combine the fifth floodlight image and the third image to obtain a floodlight training data pair.

[0235] An embodiment of the present application provides an electronic device. Since the first model is trained based on a glare training data pair and a bloom training data pair, the first model can have the ability to simultaneously reduce glare and bloom in an image. Moreover, since the brightness residual value included in the second residual map is greater than the brightness residual value included in the first residual map, the first model can restore the original brightness of the image while removing the glare and bloom in the image, and can avoid a decrease in the brightness of the image after removing the glare and bloom in the image. In this way, the electronic device can reduce glare and bloom in the image while ensuring the original brightness of the image.

[0236] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0237] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.

[0238] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0239] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0240] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0241] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0242] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0243] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0244] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0245] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned model training method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0246] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0247] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0248] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Acquire a glare reduction reference image corresponding to the glare reference image, and acquire a first residual image corresponding to the first floodlight image, wherein the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of every two adjacent pixels in the first floodlight image; Based on the first residual map, performing image processing on the first floodlight image to obtain a second residual map, wherein a brightness residual value included in the second residual map is greater than a brightness residual value included in the first residual map; Based on the glare reduction reference image and the glare-free reference image, a glare training data pair is obtained, and based on the second residual image and the first floodlight image, a floodlight training data pair is obtained; Based on the glare training data pair and the floodlight training data pair, an initial model is trained to obtain a first model.

2. The method according to claim 1, characterized in that The step of obtaining a glare reduction reference image corresponding to the glare reference image includes: Based on a first preset brightness threshold, determining a first image area from the glare reference image, wherein the brightness value of each pixel in the first image area is greater than or equal to the first preset brightness threshold; Calculating a glare attenuation reference map based on coordinate information of at least one pixel of the glare reference image and coordinate information of at least one first pixel in the first image area, wherein the at least one first pixel is a pixel among edge pixels of the first image area that intersects with at least one first connecting line, and the first connecting line is a connecting line between the central pixel and a pixel in the glare reference image; A multiplication operation is performed on the brightness between the glare attenuation reference map and the glare reference image to obtain the glare reduction reference map.

3. The method according to claim 1, characterized in that The performing image processing on the first floodlight image based on the first residual image to obtain a second residual image includes: Based on the first residual image, color correction is performed on a non-flood area in the first flood image to obtain a second flood image; Based on the second floodlight image, color correction is performed on the floodlight area in the first floodlight image to obtain a third floodlight image; Based on a second preset brightness threshold, determining a second image area from the first flood image, and calculating the second residual map based on the second image area and the third flood image; Wherein, the brightness value of each pixel in the second image area is greater than or equal to the second preset brightness threshold.

4. The method according to claim 3, characterized in that The step of performing color correction on a non-flood area in the first flood image based on the first residual image to obtain a second flood image includes: Performing grayscale conversion on the first residual map to obtain the converted first residual map, and normalizing the first residual map to obtain a weight map; Performing a multiplication operation on pixel values ​​between the weight map and the first residual map to obtain a third residual map; filtering the first floodlight image based on the third residual image to obtain a filtered first floodlight image; A subtraction operation is performed on the brightness of the first floodlight image and the first floodlight image after filtering to obtain the second floodlight image.

5. The method according to claim 1, characterized in that The step of obtaining a glare training data pair based on the glare reduction reference image and the glare-free reference image comprises: Performing color augmentation on the glare reference image to obtain the augmented glare reference image; Performing an addition operation on the brightness of the augmented glare reference image and the non-glare reference image to obtain a first image; performing an addition operation on the brightness of the first image and the glare reduction reference image to obtain a second image; The first image and the second image are combined to obtain the glare training data pair.

6. A model training device, characterized in that: The model training device comprises: an acquisition module, a processing module and a training module; The acquisition module is used to acquire a glare reduction reference image corresponding to the glare reference image, and to acquire a first residual image corresponding to the first floodlight image, wherein the image brightness of the glare reduction reference image is less than the image brightness of the glare reference image, and the first residual image is used to characterize the brightness difference between the brightness of every two adjacent pixels in the first floodlight image; The processing module is configured to perform image processing on the first floodlight image based on the first residual map acquired by the acquisition module to obtain a second residual map, wherein the brightness residual value included in the second residual map is greater than the brightness residual value included in the first residual map; and obtain a glare training data pair based on the glare reduction reference map and the glare-free reference image, and obtain a floodlight training data pair based on the second residual map and the first floodlight image; The training module is used to train the initial model based on the glare training data pair and the floodlight training data pair obtained by the processing module to obtain a first model.

7. The device according to claim 6, characterized in that The acquisition module is specifically configured to determine a first image area from the glare reference image based on a first preset brightness threshold, wherein the brightness value of each pixel in the first image area is greater than or equal to the first preset brightness threshold; And based on the coordinate information of at least one pixel point of the glare reference image and the coordinate information of at least one first pixel point in the first image area, a glare attenuation reference map is calculated, the at least one first pixel point is a pixel point among the edge pixels of the first image area that intersects with at least one first line, and the first line is a line between the center pixel point and a pixel point in the glare reference image; and a brightness multiplication operation is performed between the glare attenuation reference map and the glare reference image to obtain the glare attenuation reference map.

8. The device according to claim 6, characterized in that The acquisition module is specifically configured to perform color correction on a non-flood area in the first flood image based on the first residual image to obtain a second flood image; and based on the second floodlight image, performing color correction on the floodlight area in the first floodlight image to obtain a third floodlight image; And based on a second preset brightness threshold, determining a second image area from the first floodlight image, and calculating the second residual map based on the second image area and the third floodlight image; wherein the brightness value of each pixel in the second image area is greater than or equal to the second preset brightness threshold.

9. The device according to claim 8, characterized in that The acquisition module is specifically configured to perform grayscale conversion on the first residual image to obtain the converted first residual image, and normalize the first residual image to obtain a weight map; and perform multiplication operation on pixel values ​​between the weight map and the first residual image to obtain a third residual map; and based on the third residual map, filter the first floodlight image to obtain the filtered first floodlight image; And performing a subtraction operation on the brightness between the first floodlight image and the first floodlight image after filtering to obtain the second floodlight image.

10. The device according to claim 6, characterized in that The processing module is specifically configured to perform color augmentation on the glare reference image to obtain the augmented glare reference image; perform an addition operation on the brightness between the augmented glare reference image and the non-glare reference image to obtain a first image; and perform an addition operation on the brightness between the first image and the glare reduction reference image to obtain a second image; And combining the first image and the second image to obtain the glare training data pair.

Images