Image defogging method and device, electronic equipment and readable storage medium

Abstract

The invention relates to an image defogging method and device, electronic equipment and a readable storage medium. The method comprises the steps of identifying an environment category corresponding to a vehicle under the condition that a to-be-processed image collected by a vehicle electronic rearview mirror is obtained, determining a defogging parameter based on the to-be-processed image under the condition that the environment category corresponding to the vehicle is determined to be a rain and fog environment, and obtaining a defogged target image according to the to-be-processed image and the defogging parameter; and under the condition of determining that the environment category corresponding to the vehicle is a particle environment, performing normalization processing on the to-be-processed image, determining a defogging parameter based on the normalized image, and obtaining a defogged target image according to the normalized image and the defogging parameter. On the basis of the environment category corresponding to the vehicle, targeted defogging processing is carried out by adopting the mode corresponding to the environment category, so that the image quality of the defogged target image in different scenes can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image defogging method, device, electronic device and readable storage medium. Background Art

[0002] With the development of image processing technology, electronic rearview mirror cameras are key components in vehicle safety assistance systems, and their image quality directly affects the driver's field of vision and driving safety. Especially in severe weather conditions such as fog, rain, and snow, the images collected by the camera are easily affected by the weather, resulting in blurred images, which seriously affects driving safety.

[0003] In traditional technology, images collected by electronic rearview mirror cameras are generally defogged by relying on complex image preprocessing and postprocessing techniques to improve image clarity. Commonly used image preprocessing and postprocessing techniques generally include histogram equalization, dark channel prior (DCP), convolutional neural network, etc. Although these methods can improve the visual effect of images to a certain extent, they have problems such as large amount of calculation and slow processing speed, and the defogging effect is unstable in different scenes. Summary of the invention

[0004] Based on this, it is necessary to provide an image defogging method, device, electronic device, computer-readable storage medium and computer program product that can adapt to different scenes and improve the defogging effect in response to the above technical problems.

[0005] In a first aspect, the present application provides an image defogging method, the method comprising:

[0006] When an image to be processed collected by an electronic rearview mirror of a vehicle is acquired, identifying an environment category corresponding to the vehicle, the environment category including a rainy fog environment or a particle environment;

[0007] In the case where it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, a defogging parameter is determined based on the image to be processed, and a defogging target image is obtained according to the image to be processed and the defogging parameter;

[0008] When it is determined that the environment category corresponding to the vehicle is a particle environment, the image to be processed is normalized, a defogging parameter is determined based on the normalized image, and a defogging target image is obtained according to the normalized image and the defogging parameter.

[0009] In one of the embodiments, the defogging parameters are determined based on the image to be processed, and the target image after defogging is obtained according to the image to be processed and the defogging parameters, including: performing convolution processing on the image to be processed to obtain image features of the image to be processed; performing adaptive learning based on the image features to determine corresponding defogging parameters, wherein the defogging parameters are used to characterize a mapping relationship for defogging the image to be processed; and determining the target image after defogging according to the defogging parameters and the image features.

[0010] In one of the embodiments, the normalizing the image to be processed, determining the defogging parameters based on the normalized image, and obtaining the defogging target image according to the normalized image and the defogging parameters include: performing convolution processing on the image to be processed to obtain image features of the image to be processed; normalizing the image features to obtain normalized image features after normalization; performing adaptive learning based on the normalized image features to determine corresponding defogging parameters, the defogging parameters being used to characterize the mapping relationship for defogging the image to be processed; and determining the defogging target image according to the defogging parameters and the normalized image features.

[0011] In one of the embodiments, before determining the defogging parameters, the method also includes: obtaining a training data set, the training data set including sample images and corresponding clear images of multiple target environments; the target environment includes a rainy fog environment or a particle environment; using an initial convolutional neural network to defog the sample image to obtain a defogged image; obtaining the mean absolute error loss and the structural similarity loss between the defogged image and the corresponding clear image; determining a target loss based on the mean absolute error loss and the structural similarity loss; adjusting the network parameters of the initial convolutional neural network based on the target loss until the target loss converges to obtain a target convolutional neural network; determining the defogging parameters includes: using the target convolutional neural network to determine the defogging parameters.

[0012] In one embodiment, determining the target loss according to the mean absolute error loss and the structural similarity loss includes: Among them, L Mix is the target loss, is the mean absolute error loss, L SSIM is the structural similarity loss, α is the preset weight, and Gσ is the Gaussian distribution parameter.

[0013] In one embodiment, the identifying the environment category corresponding to the vehicle includes: obtaining environmental information of the vehicle's location, the environmental information including at least one of meteorological element information and particulate matter concentration information; when the particulate matter concentration information at the vehicle's location reaches a set concentration threshold, determining that the vehicle's environment category is a particle environment; or, when it is determined based on the meteorological element information that there is rainfall at the vehicle's location, or when it is determined based on the meteorological element information that the humidity at the vehicle's location reaches a set humidity threshold, determining that the vehicle's environment category is a rainy and foggy environment.

[0014] In one embodiment, the identifying the environment category corresponding to the vehicle includes: obtaining image features of the image to be processed, the image features including at least one of texture features, saturation and contrast of the image to be processed; obtaining the similarity between the texture features of the image to be processed and the first texture features of a preset particle environment and the second texture features of a preset rain and fog environment, respectively, and determining the environment category with the highest similarity as the environment category of the vehicle; or, when at least one of the saturation and contrast of the image to be processed is less than a corresponding threshold, determining the environment category of the vehicle as a rain and fog environment.

[0015] In a second aspect, the present application provides an image defogging device, the device comprising:

[0016] An environment recognition module is used to identify the environment category corresponding to the vehicle when acquiring the image to be processed collected by the vehicle electronic rearview mirror, wherein the environment category includes a rainy fog environment or a particle environment;

[0017] A defogging processing module, configured to determine defogging parameters based on the image to be processed when it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, and to obtain a defogging target image according to the image to be processed and the defogging parameters;

[0018] And when it is determined that the environment category corresponding to the vehicle is a particle environment, the image to be processed is normalized, a defogging parameter is determined based on the normalized image, and a defogging target image is obtained according to the normalized image and the defogging parameter.

[0019] In a third aspect, the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0021] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0022] The above-mentioned image defogging method, device, electronic device, computer-readable storage medium and computer program product, when the electronic rearview mirror system obtains the image to be processed collected by the electronic rearview mirror of the vehicle, identifies the environment category corresponding to the vehicle, and when it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, determines the defogging parameters based on the image to be processed, and obtains the defogged target image according to the image to be processed and the defogging parameters; and when it is determined that the environment category corresponding to the vehicle is a particle environment, normalizes the image to be processed, determines the defogging parameters based on the normalized image, and obtains the defogged target image according to the normalized image and the defogging parameters. It performs targeted defogging based on the environment category corresponding to the vehicle and in a manner corresponding to the environment category, thereby improving the image quality of the defogged target image in different scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 is a schematic diagram of a flow chart of an image defogging method in one embodiment;

[0025] Figure 2 A schematic diagram of a flow chart of the optimization steps of the AOD-Net model in one embodiment;

[0026] Figure 3 A schematic diagram of the network structure of an AOD-Net model in one embodiment;

[0027] Figure 4 A schematic diagram of a PONO algorithm in one embodiment;

[0028] Figure 5 A schematic diagram of a PONO-MS algorithm in one embodiment;

[0029] Figure 6 A schematic diagram of a PONO-DMS algorithm in one embodiment;

[0030] Figure 7 is a schematic flow chart of an image defogging method in another embodiment;

[0031] Figure 8 is a structural block diagram of an image defogging device in one embodiment;

[0032] Fig. 9 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] Traditional algorithms for defogging images collected by electronic rearview mirror cameras mainly include:

[0035] Dehazing algorithm based on image enhancement: It performs histogram equalization, homomorphic filtering and other operations on the image to improve the image contrast, thereby achieving the dehazing effect. However, this type of algorithm is prone to over-enhancement of the image, resulting in color distortion and other problems.

[0036] Dehazing algorithms based on physical models: such as dark channel prior algorithms, restore clear images by estimating parameters in the atmospheric scattering model. However, such algorithms are computationally complex and difficult to meet real-time requirements.

[0037] Dehazing algorithms based on deep learning: such as Convolutional Neural Networks (CNN), etc. Although deep learning algorithms have improved the dehazing effect, they often rely on complex pre-processing and post-processing steps, and have poor adaptability to different scenes, resulting in unstable dehazing effects.

[0038] With the rise of deep learning technology, image dehazing algorithms based on neural networks have gradually become a research hotspot. Among them, AOD-Net (All-in-One Dehazing Network), as an end-to-end dehazing network, has attracted widespread attention for its simple architecture and efficient performance. AOD-Net is designed based on a new atmospheric scattering model. Unlike traditional models that estimate the transmission matrix and atmospheric light separately, it directly generates clean images through a lightweight CNN.

[0039] Although AOD-Net achieves fast processing of image defogging tasks by directly learning the decomposition of atmospheric light and transmission maps, its performance is still limited by the network structure and feature processing method. However, in severe weather conditions such as fog, rain, snow, and different lighting environments, the characteristics of images vary greatly. Traditional AOD-Net defogging algorithms often have difficulty adapting to various complex scenes, resulting in poor defogging effects.

[0040] Based on this, the embodiment of the present application provides an image defogging method, which identifies the environment category corresponding to the vehicle when the image to be processed is acquired by the electronic rearview mirror of the vehicle, determines the defogging parameters based on the image to be processed when the environment category corresponding to the vehicle is determined to be a rainy and foggy environment, and obtains the defogging target image according to the image to be processed and the defogging parameters; when the environment category corresponding to the vehicle is determined to be a particle environment, normalizes the image to be processed, determines the defogging parameters based on the normalized image, and obtains the defogging target image according to the normalized image and the defogging parameters. Since the defogging process is based on the environment category corresponding to the vehicle and the defogging process is performed in a manner corresponding to the environment category, the image quality of the defogging target image can be improved.

[0041] In one embodiment, Figure 1 As shown, an image defogging method is provided, which is applied to an electronic rearview mirror system and includes the following steps:

[0042] Step 102 : when the image to be processed collected by the electronic rearview mirror of the vehicle is obtained, the environment category corresponding to the vehicle is identified.

[0043] Among them, the image to be processed refers to the image that needs to be defogged and is collected by the electronic rearview mirror of the vehicle. Specifically, it can be determined whether the corresponding image is an image to be processed based on at least one of the image quality and image features of the image collected by the electronic rearview mirror of the vehicle. Image quality can be measured based on quality indicators such as resolution and PSNR (Peak signal-to-noise ratio), and image features include but are not limited to image saturation, contrast, etc. For example, when the resolution of the collected image is less than a set resolution threshold, or when the PSNR of the collected image is less than a preset PSNR threshold, or when the saturation of the collected image is less than a set saturation threshold, or when the contrast of the collected image is less than a set contrast threshold, it can be determined that the collected image is an image to be processed. It can be understood that the above-mentioned quality indicators and image features can also be combined to determine whether the collected image is an image to be processed. Specifically, the judgment criteria (such as various thresholds) can be flexibly set based on the accuracy requirements of the actual scene, and this embodiment does not limit this.

[0044] The environment category can be determined based on the specific environment information of the current location of the vehicle, such as a rain fog environment or a particle environment. Among them, rain fog is composed of tiny water droplets or ice crystals suspended in the air near the ground. It is the product of condensation (or sublimation) of water vapor in the air near the ground layer. The formation of rain fog generally requires certain water vapor and cooling conditions. For example, mountain fog environment, high humidity area, hot and cold alternating environment, rainy day and other environments are all rain fog environments. In this environment, the visibility at the ground level decreases. For the electronic rearview mirror, the image quality collected in this environment is reduced.

[0045] The particle environment usually refers to the particles in the atmosphere, such as PM2.5, which are formed by solid or liquid particles suspended in the air, mainly from combustion processes, industrial emissions, dust, etc. The formation of particles does not require water vapor and cooling conditions, mainly when the (dry) particles in the air reach a certain concentration. The appearance of these particles makes the atmosphere turbid and the field of vision blurred, which also affects visibility. For electronic rearview mirrors, due to the presence of suspended particles, the light entering the electronic rearview mirror will be absorbed and scattered in the propagation diagram, which seriously affects the image quality.

[0046] Due to the differences between rainy and foggy environments and particle environments, the scattering characteristics in different environments are also significantly different. For example, in rainy and foggy environments, the scattering is relatively more uniform, and the difference in scattering intensity at different wavelengths is small. In particle environments, the scattering is usually more directional, and the difference in scattering intensity at different wavelengths is large. Based on this, the characteristics of images collected by electronic rearview mirrors in different environments are also quite different. If the same defogging process is performed on images collected in different environments, the defogging effect will be seriously affected.

[0047] Therefore, in this embodiment, when the electronic rearview mirror system obtains the image to be processed captured by the vehicle's electronic rearview mirror, it identifies the environment category corresponding to the vehicle, and then based on subsequent steps, performs targeted defogging on the image to be processed according to different environment categories, thereby improving the defogging effect.

[0048] Step 104 , when it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, defogging processing is performed based on the first processing method to obtain a defogged target image.

[0049] Among them, the first processing method includes determining the defogging parameters based on the image to be processed, and performing defogging processing according to the image to be processed and the defogging parameters. Specifically, the defogging parameters are used to characterize the mapping relationship of defogging the image to be processed. The defogging parameters can be noise level information extracted from the original image, that is, the image to be processed. For example, it can be a parameter combined with the atmospheric light value and the atmospheric transmittance. The target image is a clear image obtained after the image to be processed is defogged.

[0050] In this embodiment, when the electronic rearview mirror system determines that the environment category corresponding to the vehicle is a rainy and foggy environment based on the above steps, it can extract noise level information from the image to be processed to determine the defogging parameters, and perform defogging on the image to be processed according to the defogging parameters to obtain a defogged target image.

[0051] Step 106 , when it is determined that the environment category corresponding to the vehicle is a particle environment, defogging is performed based on the second processing method to obtain a defogged target image.

[0052] Among them, the second processing method includes normalizing the image to be processed, determining the defogging parameters based on the normalized image, and performing the defogging process according to the normalized image and the defogging parameters. Specifically, the defogging parameters are used to characterize the mapping relationship of defogging the image to be processed. The defogging parameters can be the noise level information extracted from the image after the original image, i.e., the image to be processed, is normalized. For example, it can be a parameter that combines the atmospheric light value and the atmospheric transmittance. The target image is a clear image obtained after the image to be processed is defogged.

[0053] Normalization is a processing method for optimizing features in neural networks in deep learning. It helps to improve the problem of uneven data distribution and enables the network to better process images of different scales and different noise levels. Specifically, it can be implemented using Positional Normalization (PONO).

[0054] In this embodiment, when the electronic rearview mirror system determines that the environment category corresponding to the vehicle is a particle environment based on the above steps, it can normalize the image to be processed to obtain a normalized image, extract noise level information from the normalized image to determine defogging parameters, and then defog the normalized image according to the defogging parameters to obtain a defogged target image.

[0055] Since the features of images collected by the electronic rearview mirror in different environments vary greatly, in the above-mentioned image defogging method, when the electronic rearview mirror system obtains the image to be processed collected by the electronic rearview mirror of the vehicle, it identifies the environment category corresponding to the vehicle, and when it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, it determines the defogging parameters based on the image to be processed, and obtains the defogging target image according to the image to be processed and the defogging parameters; and when it is determined that the environment category corresponding to the vehicle is a particle environment, it normalizes the image to be processed, determines the defogging parameters based on the normalized image, and obtains the defogging target image according to the normalized image and the defogging parameters. It is based on the environment category corresponding to the vehicle, and adopts a method corresponding to the environment category to perform targeted defogging, so as to improve the image quality of the defogging target image in different scenes.

[0056] In an exemplary embodiment, in step 102, identifying the environment category corresponding to the vehicle may specifically include: acquiring environmental information of the location of the vehicle, and determining the environment category corresponding to the vehicle according to the environmental information.

[0057] The environmental information may include at least one of meteorological element information and particulate matter concentration information. Specifically, the meteorological element information includes but is not limited to temperature, humidity, precipitation (such as rain, snow, hail, etc.) and visibility, etc. The particulate matter concentration information refers to the concentration of particulate matter in the atmosphere.

[0058] In this embodiment, when the image to be processed collected by the electronic rearview mirror of the vehicle is obtained, the environmental information of the vehicle's location can be further obtained, and the environmental category corresponding to the vehicle can be determined based on the environmental information. For example, when it is determined that there is rainfall at the current location based on the environmental information of the vehicle's location, such as meteorological factors, or when the humidity at the vehicle's location reaches a set humidity threshold, it can be determined that the environmental category corresponding to the vehicle is a rainy and foggy environment. Alternatively, when it is determined that the particulate matter concentration information in the atmosphere at the current location reaches a set concentration threshold based on the environmental information of the vehicle's location, it can be determined that the environmental category corresponding to the vehicle is a particle environment. Thereby, the environmental category corresponding to the vehicle is determined based on the environmental information of the vehicle's location.

[0059] In one scenario, image features of the image to be processed may also be obtained, and the environment category corresponding to the vehicle may be determined based on the image features. The image features may include at least one of texture features, saturation, and contrast of the image to be processed.

[0060] Since the scattering characteristics of rainy fog environment and particle environment are obviously different, for example, in rainy fog environment, the difference in scattering intensity of images at different wavelengths is small, that is, the scattering characteristics are relatively more uniform. In particle environment, the scattering has stronger directionality, and the difference in scattering intensity at different wavelengths is large. The texture features of the image, such as Local Binary Patterns (LBP) and Gray-level co-occurrence matrix (GLCM), can capture the microstructure and pattern in the image. These features reflect the roughness, shape and arrangement of the surface of the object. These properties affect the response of the ground object target to the radar beam, because different surface states will lead to different scattering characteristics. For example, when the surface roughness is high, the radar beam will produce stronger diffuse reflection or scattering. Therefore, by analyzing the texture features of the image, some characteristics of the scattering intensity can be indirectly inferred. Based on this, the environment category corresponding to the vehicle can be determined by the scattering characteristics reflected by the texture features of the image.

[0061] In the HSV color space (a color space created based on the intuitive color characteristics of Hue, Saturation, and Value), images in smoke scenes usually have lower saturation and contrast. Therefore, the color features such as saturation and contrast can also be obtained by calculating the saturation histogram of the image, and then the environment category corresponding to the vehicle can be determined based on the saturation or contrast, and the size of the saturation and contrast.

[0062] Specifically, the color image to be processed can be converted into a grayscale image to reduce the computational complexity, and Gaussian blur can be applied to reduce noise and improve the robustness of subsequent processing. Then the local binary pattern (LBP) or gray level co-occurrence matrix (GLCM) is used to extract the texture features of the image. And the saturation histogram is calculated. The Sobel operator (an edge detection operator used in the field of image processing) is used to calculate the gradients in the x and y directions to determine the gradient amplitude, and then the contrast features such as the mean and standard deviation of the gradient amplitude are calculated. In this way, the image features of the image to be processed are extracted, and the environment category corresponding to the vehicle's location is classified according to the image features.

[0063] In one scenario, the first texture feature of the image in the particle environment and the second texture feature of the image in the rain and fog environment can be pre-configured. Among them, the first texture feature can be determined based on the commonality of the texture features of the sample images in the particle environment, and the second texture feature can be determined based on the commonality of the texture features of the sample images in the rain and fog environment. After the texture features of the image to be processed are extracted, the similarity between the texture features of the image to be processed and the first texture features of the particle environment, and the similarity between the texture features of the image to be processed and the second texture features of the rain and fog environment can be calculated respectively, and the two similarities are compared, and the environment category with the highest similarity is determined as the environment category of the vehicle. For example, if after comparing the two similarities, it is determined that the similarity between the texture features of the image to be processed and the first texture features of the particle environment is the highest, then the environment category of the vehicle can be determined as the particle environment.

[0064] In another scenario, since images in smoke scenes usually have lower saturation and contrast, the maximum saturation threshold and contrast threshold of images in smoke scenes can also be pre-configured based on the characteristics of image saturation and contrast in smoke scenes. After the saturation and contrast of the image to be processed are extracted, they can be compared with the pre-configured saturation threshold and contrast threshold. For example, if the saturation of the image to be processed is less than the corresponding saturation threshold, the vehicle's environment category can be determined to be a rainy and foggy environment; or, if the contrast of the image to be processed is less than the corresponding contrast threshold, the vehicle's environment category can be determined to be a rainy and foggy environment.

[0065] It is understandable that when determining the environment category corresponding to the vehicle, the environment information of the vehicle's location and the image features of the image to be processed may also be comprehensively considered. For example, different weights may be set for the environment information of the vehicle's location and the image features of the image to be processed, and then the final environment category is determined by combining the classification results and weights of the environment categories, so that the identification of the environment category is more accurate.

[0066] In an exemplary embodiment, the first processing method in the above step 104 may specifically include: performing convolution processing on the image to be processed to obtain image features of the image to be processed; performing adaptive learning based on the image features to determine corresponding defogging parameters, and determining the target image after defogging according to the defogging parameters and image features.

[0067] Among them, the defogging parameters are used to characterize the mapping relationship of defogging the image to be processed. Convolution processing can be implemented by a convolutional neural network for extracting image features. By performing a convolution operation on the image to be processed, image features such as edges and textures of the image can be extracted. Then, a noise estimation network can be used to predict the noise level information in the image to be processed, and the noise estimation network can be implemented based on an architecture similar to U-Net. Among them, U-Net is an image segmentation network with a symmetrical encoder-decoder structure (Encoder-Decoder) based on a fully convolutional network (Fully Convolutional Networks, referred to as FCN). U-Net includes two parts, an encoder (downsampling path) and a decoder (upsampling path), and its shape is U-shaped, hence the name. The encoder part captures the context information of the image by gradually reducing the size of the feature map and increasing the number of channels. The decoder part performs upsampling and feature fusion operations on the feature map of the encoder to gradually restore the resolution and generate a segmentation result of the same size as the input image. Thereby, the noise level information is extracted and segmented from the image features of the original image to determine the corresponding defogging parameters. Then, according to the segmented noise level information and the image features of the image to be processed, a noise removal network can be used to perform defogging to obtain a clear target image after defogging. Specifically, the noise removal network can use the structure of a residual network. The residual network introduces residual connections, which makes it easier for the network to learn the mapping relationship between input and output, so that it can adapt to the predicted noise level and achieve efficient denoising effects under different noise levels.

[0068] In one scenario, the first processing method in the above step 104 can also be implemented by using an optimized target convolutional neural network. Among them, the optimized target convolutional neural network can be obtained by optimizing the traditional convolutional neural network based on the environment category such as rain and fog environment, so as to improve the processing effect of the model. Specifically, in this embodiment, the optimized target convolutional neural network is an AOD-Net neural network as an example. Then, when it is determined that the environment category corresponding to the vehicle is a rain and fog environment, the image to be processed can be input into the AOD-Net neural network, and a clear target image can be directly generated by the AOD-Net neural network. Because the core principle of AOD-Net is to use a convolutional neural network to directly generate a clear image from a foggy image without pre-estimating the transmission matrix and atmospheric light. In this embodiment, the design of the AOD-Net neural network is based on a reconfigured atmospheric scattering model, which can merge the atmospheric light value and the atmospheric transmittance into a parameter variable. Through a lightweight CNN, the AOD-Net neural network can directly generate a clear image. This end-to-end design can not only improve the defogging effect, but also improve the processing speed.

[0069] In an exemplary embodiment, Figure 2 As shown, before the above step of determining the defogging parameters, a network optimization process may also be included, which may specifically include the following steps:

[0070] Step 202: Obtain a training data set.

[0071] The training data set may include sample images of multiple target environments and corresponding clear images. The target environment includes a rainy foggy environment or a particle environment. It is understandable that if the network to be optimized is used in a rainy foggy scene, the training data set may include sample images of multiple rainy foggy environments and corresponding clear images; if the network to be optimized is used in a particle scene, the training data set may include sample images of multiple particle environments and corresponding clear images.

[0072] Specifically, the sample image can be a foggy image collected in a corresponding target environment (such as a rainy foggy environment or a particle environment), and the clear image can be a standard image collected in a normal environment, that is, a non-foggy image, which can be used as a sample label for training. In this embodiment, when performing network optimization, the corresponding training data set needs to be obtained first.

[0073] Step 204: Dehazing the sample image using an initial convolutional neural network to obtain a dehazed image.

[0074] The initial convolutional neural network can be a traditional convolutional neural network. In this embodiment, the initial convolutional neural network is a traditional AOD-Net neural network as an example for explanation, and its network structure diagram is as follows: Figure 3 As shown in the figure, the input is a foggy image and the output is a clear image. The middle part is the core part of the image dehazing algorithm. The K value is obtained through the estimation module, that is, K(x) is output, and a fog-free clear image is restored and output according to the physical model.

[0075] Due to the presence of haze, a large number of suspended particles appear in the atmosphere. The presence of these particles reduces the quality of the image obtained by the electronic rearview mirror. Due to the presence of suspended particles, the light entering the electronic rearview mirror will be absorbed and scattered in the propagation diagram. The model when sunlight is scattered by suspended particles in the air can be expressed as:

[0076] I(x)=J(x)t(x)+A(1-t(x))

[0077] Among them, I(x) is the image obtained by the electronic rearview mirror. Due to the existence of haze and other environments, there are often a large number of suspended particles in the air, so the image obtained by the electronic rearview mirror is a foggy image. J(x) is the corresponding fog-free and clear image obtained by the electronic rearview mirror. A is the global atmospheric light intensity, t(x) is the transmission matrix, also known as image transmittance. According to the atmospheric light model, t(x) can be rewritten as:

[0078] t(x)=e -βd(x) ,0 <t(x)≤1

[0079] Among them, β represents the global atmospheric light scattering coefficient, d represents the depth of field, and the value of d(x) represents the distance between the target position and the electronic rearview mirror.

[0080] The core idea of ​​the AOD-Net neural network is to treat the above two parameters t(x) and A as a whole, recorded as K(x), and directly reconstruct the minimum pixel threshold error, then:

[0081] J(x)=K(x)I(x)-K(x)+b

[0082] Combining the above three formulas, K(x) can be expressed as:

[0083]

[0084] Specifically, AOD-Net is a lightweight dehazing algorithm based on convolutional neural networks. It models the atmospheric scattering process, uses CNN to learn the features of the input image, and predicts the transmission map K(x). The transmission map represents the transmittance value of each pixel in the image, reflecting the degree of brightness attenuation and color shift before and after the haze. Then, the transmission map K(x) and the input image I(x), i.e. the sample image, are used as input, and a simplified atmospheric scattering model is used to obtain the clear image J(x), i.e. the dehazed image, after dehazing.

[0085] Step 206 , obtaining the mean absolute error loss and the structural similarity loss between the dehazed image and the corresponding clear image.

[0086] Among them, the loss is the quantification of the difference in probability distribution between the model output value and the true value (i.e., the label result). The mean absolute error loss is also the L1 loss, which takes the absolute value of the difference between the model output value and the true value. In this embodiment, the mean absolute error loss is mainly calculated by gradually comparing the differences between pixels and then finding the absolute value. It can be specifically expressed as:

[0087]

[0088] Among them, x represents the model output value, i.e., the defogging image, y represents the true value, i.e., the corresponding clear image, and p represents the corresponding pixel point in the image. The mean absolute error loss between the dehazed image and the corresponding clear image.

[0089] Structural Similarity (SSIM) loss is an indicator to measure the similarity between two images. In this embodiment, SSIM is used to measure the difference between the defogging image output by the model and the real fog-free image, that is, the corresponding clear image, so as to guide the optimization and learning of the model during the training process. The similarity of the two images is evaluated by comparing their brightness, contrast and structural information. The value range of SSIM is [-1,1]. The closer the value is to 1, the more similar the two images are, and the better the defogging effect is. During the model training process, it is necessary to continuously reduce the loss to make the network converge.

[0090] Typically, SSIM consists of three parts: illumination, contrast, and structure, which are expressed as:

[0091]

[0092] The specific SSIM loss can be expressed as:

[0093] Among them, μ x , σ x Respectively represent the mean and variance corresponding to the image x, μ y , σ y Respectively represent the mean and variance corresponding to the image y, σ xy represents the covariance of image x and image y. 1 、c 2 、c 3 is a constant, c 1 =(k 1 L) 2 , c 2 =(k 2 L) 2 , c 3 Usually defined as c 2 half of k 1 , k 2 Generally, it is 0.01 or 0.03, and L is generally 255.

[0094] Step 208, determining the target loss according to the mean absolute error loss and the structural similarity loss.

[0095] Since the SSIM function takes into account the brightness, contrast, and structural indicators of the image, that is, it comprehensively considers human visual perception, the results obtained by SSIM have better details and more prominent edge information. However, if only SSIM is used, although the high-frequency information of the image can be retained, it is very easy to cause the deviation of the image brightness and color. If the loss function L1 is used alone, although the image brightness and color can be better maintained without changing, it is easy to fall into the local optimal solution.

[0096] Based on this, in order to obtain an image that is more in line with human visual perception, this embodiment linearly combines the mean absolute error loss and the structural similarity loss based on preset weights to obtain the target loss.

[0097] Specifically, the target loss can be expressed as:

[0098]

[0099] Among them, L Mix is the target loss, is the mean absolute error loss, L SSIM is the structural similarity loss, α is a preset weight, which is generally obtained through experiments. In this embodiment, α can be set to 0.84, and Gσ is a Gaussian distribution parameter used to optimize the mean absolute error loss.

[0100] Step 210, adjusting the network parameters of the initial convolutional neural network based on the target loss until the target loss converges to obtain an optimized target convolutional neural network.

[0101] In this embodiment, the model parameter of the initial AOD-Net model, i.e., the K value, is adjusted based on the target loss determined in the above steps to train the initial AOD-Net model until the target loss converges to obtain an optimized AOD-Net model. Since the main function of the loss function is to calculate the gap between the predicted value and the target value, the smaller the loss function, the closer the predicted result is to the target. Therefore, when the target loss converges, it means that the model converges, that is, the training is completed.

[0102] In the above embodiment, by obtaining a training data set including a sample image and a corresponding clear image in a target environment, an initial convolutional neural network is used to perform defogging on the sample image to obtain a defogged image, and the mean absolute error loss and structural similarity loss between the defogged image and the corresponding clear image are obtained, the target loss is determined according to the mean absolute error loss and the structural similarity loss, and the network parameters of the initial convolutional neural network are adjusted based on the target loss until the target loss converges to obtain an optimized target convolutional neural network. Since the optimized target convolutional neural network is obtained after training the initial convolutional neural network based on the training data set in the target environment and the target loss determined by linearly combining the mean absolute error loss and the structural similarity loss, the adaptability and robustness of the network can be effectively improved, and the optimized target convolutional neural network can have a better defogging effect in the corresponding environment.

[0103] In specific applications, the initial convolutional neural network can be trained in the above manner based on the training data set in a rainy and foggy environment to obtain a target convolutional neural network. Then, when it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, the corresponding image to be processed can be input into the target convolutional neural network, thereby directly generating a clear target image through the target convolutional neural network.

[0104] The weak penetration of vision in a particle environment poses a greater challenge to the defogging algorithm. The pose normalization PONO aims to optimize the feature processing in the front and back layers of the neural network. This method brings a new perspective to deep learning by processing the features of each position independently. It normalizes across channels independently at the spatial position of each feature, so the semantics (mean) and standard deviation (std) extracted by this method can capture the coarse structural information of the input image.

[0105] Based on this, in order to solve the intelligent defogging problem of particle scenes, this embodiment combines the PONO algorithm to enhance the model's processing ability for position-related information and improve the image restoration quality, so that the processed images have significant improvements in indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity), further improving the defogging performance and ensuring that the driver can obtain clear and accurate image information.

[0106] In an exemplary embodiment, the second processing method in the above step 106 may specifically include: performing convolution processing on the image to be processed to obtain image features of the image to be processed; normalizing the image features to obtain normalized image features after normalization; performing adaptive learning based on the normalized image features to determine corresponding defogging parameters, and determining the target image after defogging according to the defogging parameters and the normalized image features.

[0107] Among them, the convolution processing can be implemented by a convolutional neural network for extracting image features. By performing a convolution operation on the image to be processed, image features such as the edge and texture of the image can be extracted. Normalization processing is a processing method for optimizing features in a neural network in deep learning. Normalization processing can be specifically implemented by positional normalization (Positional Normalization, referred to as PONO). In this embodiment, PONO can be used to normalize the image features of the image to be processed, and the normalized image features such as the mean and standard deviation of the feature map can be extracted to independently normalize the activation function of each position on the channel. This normalization operation helps to improve the problem of data distribution being not concentrated, so that the network can better process images of different scales and different noise levels. Then a noise estimation network can be used to predict the noise level information in the image to be processed. Similarly, the noise estimation network can be implemented based on an architecture similar to U-Net to extract and segment the noise level information from the normalized image features to determine the corresponding defogging parameters. Then, according to the segmented noise level information and the image features of the image to be processed itself, a noise removal network can be used for defogging to obtain a clear target image after defogging. Specifically, the noise removal network can also use the structure of the residual network.

[0108] In one scenario, the second processing method in the above step 106 can also be implemented by using a target convolutional neural network combined with pose normalization, such as using an AOD-Net neural network combined with PONO (i.e., combining the PONO mechanism with an end-to-end AOD-Net neural network). Among them, AOD-Net can also be obtained by optimizing a traditional convolutional neural network based on an environmental category such as a particle environment, thereby improving the processing effect of the model. The AOD-Net neural network combined with PONO is a network obtained by combining the PONO mechanism with an end-to-end AOD-Net neural network.

[0109] Specifically, in this embodiment, when it is determined that the environment category corresponding to the vehicle is a particle environment, the image to be processed can be input into the AOD-Net neural network combined with PONO, and the image to be processed can be normalized by PONO to obtain the normalized features of the image to be processed, and then the end-to-end AOD-Net neural network is used to perform defogging according to the normalized features, thereby generating a clear target image. Among them, the PONO normalization technology normalizes the feature map so that the AOD-Net neural network can achieve secondary enhancement of the image defogging performance, thereby helping to improve the generalization ability and training stability of the network, and can effectively solve the defogging process in the particle environment.

[0110] It can be understood that the end-to-end AOD-Net neural network used in this embodiment is also an optimized AOD-Net neural network. The specific optimization process can refer to the above Figure 2 Since the AOD-Net neural network combined with the pose normalization module in this embodiment is applied to a particle environment, in the process of optimizing the end-to-end AOD-Net neural network, its training data set can select sample images and corresponding clear images in a particle environment. In addition, the other optimization training processes are the same as Figure 2 The embodiment shown is the same, and this embodiment will not be repeated. In this embodiment, the sample image can be a foggy image collected in a particle environment, and the clear image can be a standard image collected in a normal environment, that is, a non-foggy image, which can be used as a sample label for training. Thereby, the optimized AOD-Net neural network can have a better defogging effect in a particle environment, so as to improve the adaptability and robustness of the defogging process.

[0111] In addition, the core idea of ​​PONO is to use PONO to regularize the convolutional features in the encoder, and use MS (Moment Shortcut) in the decoder. Furthermore, DMS (Dynamic Moment Shortcut) can use the mean (semantics) and standard deviation extracted by PONO to generate β and γ to adapt to the feature transformation in the decoding process. This design improves the robustness of the model to input changes and can improve the performance of various tasks.

[0112] Specifically, the PONO algorithm in this application can be as follows Figure 4As shown in the figure, B is the batch axis (i.e., batch dimension). In a neural network, a batch usually contains multiple data samples for parallel computing to improve training efficiency. C is the channel axis, which refers to the channel dimension considered when performing feature normalization. (H, W) is the spatial axis. The "spatial axis" in the algorithm refers to the spatial dimension of the feature map, that is, the height (H) and width (W).

[0113] Specifically, the PONO algorithm traverses each position in each batch when calculating the normalized statistics, and calculates the mean μ and standard deviation σ of that position over all channels. These statistics are then used to normalize the activation values ​​at the same position, rather than calculating statistics over the batch, height, and width dimensions like other normalization methods (such as Batch Normalization). This approach allows the network to process features independently at each position, thereby capturing richer spatial information, which is particularly useful for generating models and other tasks that require spatial perception.

[0114] In neural networks, a channel corresponds to a feature map in a feature map, for example, in a color image, a channel can be a red, green, or blue channel. In the PONO algorithm, for an activation value X∈R in a given layer B×C×H×W (where B is the batch size, C is the number of channels, H and W are the height and width respectively), the algorithm processes the features of each spatial position independently. This means that for each fixed spatial position, the algorithm calculates the mean μ and standard deviation σ of that position over all channels, and then uses these statistics to normalize the activation value of that position. This processing allows the network to process features independently at each position, thereby capturing richer spatial information.

[0115] like Figure 4 As shown, the colored parts (green and blue in the figure) are normalized using the same mean and std, including batch normalization, instance normalization, grop normalization, layer normalization, and position normalization. PONO can process each position independently and calculate two statistics, namely the mean μ and standard deviation σ, across channels, which is more efficient and accurate than other algorithms.

[0116] The mean and standard deviation algorithm of PONO in this embodiment can be expressed as follows:

[0117]

[0118] Among them, μ b,h,w is the mean, i.e., semantics, σ b,h,w is the standard deviation, X b,c,h,w is the activation value of the input image at channel b, channel c, position (h, w), ∈ is a stable constant, for example, ∈ can be 10 -5 B is the batch size, C is the number of channels, H is the height, and W is the width.

[0119] PONO is suitable for generative models, which use instantaneous cropping of MS to achieve image generation tasks with structured information. It extracts μ and σ with positional moment information in the encoder and then feeds them to the corresponding layer of the decoder. Figure 5 As shown, PONO-MS directly uses μ and σ as β and γ. Then the calculation of MS is as follows:

[0120] MS(x)=γF(x)+β

[0121] Among them, F(x) is modeled by the middle layer, and γ, β and μ, σ are obtained from the input x respectively. MS makes the activation information generated by the layers in the decoder similar to the statistical information generated by the corresponding layers in the encoder. MS can be used alone or in combination with PONO. The characteristic of PONO-MS is that it can extract the necessary structural information from the input image, so it can be used to implement the migration of the structure of an image from its encoder to the decoder of another image.

[0122] The schematic diagram of dynamic instantaneous cutting DMS is as follows Figure 6 As shown in the figure, PONO-DMS uses a shallow network (such as a convolutional neural network Conv Net) to predict β and γ based on μ and σ, so it is a dynamic instantaneous cropping.

[0123] This embodiment flexibly applies MS and DMS, and together with PONO, improves the intelligent defog effect in particle scenes, so that indicators such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity) are significantly improved, thereby further improving the defogging performance to ensure that the driver can obtain clear and accurate image information.

[0124] In an exemplary embodiment, Figure 7As shown, the above-mentioned image defogging method is further described below. The method can be applied to electronic devices, and an image defogging model is constructed in the electronic devices. Among them, the image defogging model includes an optimized AOD-Net model and an AOD-Net model combined with a pose normalization module. Specifically, the optimization process of the optimized AOD-Net model can refer to the above Figure 2 In the embodiment shown, the AOD-Net model combined with the posture normalization module is a model obtained by combining the PONO mechanism with the optimized end-to-end AOD-Net neural network. The specific implementation of the model can refer to the above embodiment, and this embodiment will not be repeated. Specifically, the method may include the following steps:

[0125] Step 702 : when the image to be processed collected by the electronic rearview mirror of the vehicle is obtained, the environment category corresponding to the vehicle is identified.

[0126] The environment category includes a rainy fog environment or a particle environment. Specifically, the environment category corresponding to the vehicle can be determined based on the environment information of the vehicle's location, or based on the image features of the image to be processed, or in combination with the environment information of the vehicle's location and the image features of the image to be processed.

[0127] In step 704 , if it is determined that the environment type is a rainy or foggy environment, then step 706 is executed.

[0128] Step 706: Use the optimized AOD-Net model to perform defogging on the image to be processed to obtain a defogged target image.

[0129] Specifically, when the environment category is determined to be a rainy and foggy environment, the optimized AOD-Net model is used to perform defogging on the image to be processed to obtain a defogged target image. Since the optimized AOD-Net model refers to a model obtained by optimizing the traditional end-to-end AOD-Net neural network based on a rainy and foggy environment. It can directly defog the input image without relying on complex preprocessing and post-processing steps. This design not only simplifies the process of the traditional multi-step defogging algorithm, but also improves processing efficiency and real-time performance. And because it is based on the optimization of the AOD-Net neural network in a rainy and foggy environment, it can also improve the defogging performance in this environment.

[0130] In step 708 , if it is determined that the environment category is a particle environment, then step 710 is executed.

[0131] Step 710, using the AOD-Net model combined with the pose normalization module to perform defogging on the image to be processed, to obtain a defogged target image.

[0132] Specifically, when the environment category is determined to be a particle environment, the AOD-Net model combined with the pose normalization module is used to perform defogging on the image to be processed to obtain the defogged target image. Since the AOD-Net model combined with the pose normalization module is based on the optimized AOD-Net neural network and combined with the efficient feature processing algorithm PONO, the model's processing ability for location-related information is enhanced, and the model's adaptability to different scenes is improved, ensuring that good defogging effects can be obtained under different conditions, and fast processing can be achieved, meeting the real-time requirements of the on-board electronic rearview mirror camera.

[0133] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0134] Based on the same inventive concept, the embodiment of the present application also provides an image defogging device for implementing the above-mentioned image defogging method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image defogging device embodiments provided below can refer to the limitations of the image defogging method above, and will not be repeated here.

[0135] In an exemplary embodiment, Figure 8 As shown, an image defogging device is provided, comprising: an environment recognition module 802 and a defogging processing module 804, wherein:

[0136] The environment recognition module 802 is used to identify the environment category corresponding to the vehicle when the image to be processed collected by the electronic rearview mirror of the vehicle is obtained, and the environment category includes a rainy fog environment or a particle environment;

[0137] A defogging processing module 804 is used to determine defogging parameters based on the image to be processed when it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, and to obtain a defogging target image according to the image to be processed and the defogging parameters;

[0138] And when it is determined that the environment category corresponding to the vehicle is a particle environment, the image to be processed is normalized, a defogging parameter is determined based on the normalized image, and a defogging target image is obtained according to the normalized image and the defogging parameter.

[0139] In an exemplary embodiment, the defogging processing module is specifically used to: perform convolution processing on the image to be processed to obtain image features of the image to be processed; perform adaptive learning based on the image features to determine corresponding defogging parameters, and the defogging parameters are used to characterize the mapping relationship for defogging the image to be processed; determine the target image after defogging based on the defogging parameters and the image features.

[0140] In an exemplary embodiment, the defogging processing module is specifically used to: perform convolution processing on the image to be processed to obtain image features of the image to be processed; perform normalization processing on the image features to obtain normalized image features after normalization processing; perform adaptive learning based on the normalized image features to determine corresponding defogging parameters, and the defogging parameters are used to characterize the mapping relationship for defogging the image to be processed; determine the target image after defogging based on the defogging parameters and the normalized image features.

[0141] In an exemplary embodiment, the device also includes a network optimization module, which is used to: obtain a training data set, the training data set including sample images and corresponding clear images of multiple target environments; the target environment includes a rainy fog environment or a particle environment; use an initial convolutional neural network to defog the sample image to obtain a defogged image; obtain the mean absolute error loss and the structural similarity loss between the defogged image and the corresponding clear image; determine a target loss based on the mean absolute error loss and the structural similarity loss; adjust the network parameters of the initial convolutional neural network based on the target loss until the target loss converges to obtain a target convolutional neural network; the determination of defogging parameters includes: using the target convolutional neural network to determine the defogging parameters.

[0142] In an exemplary embodiment, the environment identification module is also used to: obtain environmental information of the location of the vehicle, the environmental information including at least one of meteorological element information and particulate matter concentration information; when the particulate matter concentration information at the location of the vehicle reaches a set concentration threshold, determine that the environment category of the vehicle is a particle environment; or, when it is determined according to the meteorological element information that there is rainfall at the location of the vehicle, or when it is determined according to the meteorological element information that the humidity at the location of the vehicle reaches a set humidity threshold, determine that the environment category of the vehicle is a rainy and foggy environment.

[0143] In an exemplary embodiment, the environment recognition module is also used to: obtain image features of the image to be processed, the image features including at least one of texture features, saturation and contrast of the image to be processed; obtain the similarity between the texture features of the image to be processed and the first texture features of a preset particle environment and the second texture features of a preset rain and fog environment, and determine the environment category with the highest similarity as the environment category of the vehicle; or, when at least one of the saturation and contrast of the image to be processed is less than the corresponding threshold, determine the environment category of the vehicle as a rain and fog environment.

[0144] Each module in the above-mentioned image defogging device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0145] In an exemplary embodiment, an electronic device is provided. The electronic device may be a vehicle-mounted terminal, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, an image defogging method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0146] Those skilled in the art will understand that Fig. 9The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0147] In an exemplary embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0148] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0149] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0150] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0151] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0152] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0153] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image defogging method, characterized in that: The method comprises: When an image to be processed collected by an electronic rearview mirror of a vehicle is acquired, identifying an environment category corresponding to the vehicle, the environment category including a rainy fog environment or a particle environment; In the case where it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, a defogging parameter is determined based on the image to be processed, and a defogging target image is obtained according to the image to be processed and the defogging parameter; When it is determined that the environment category corresponding to the vehicle is a particle environment, the image to be processed is normalized, a defogging parameter is determined based on the normalized image, and a defogging target image is obtained according to the normalized image and the defogging parameter.

2. The method according to claim 1, characterized in that The step of determining the defogging parameters based on the image to be processed, and obtaining the defogging target image according to the image to be processed and the defogging parameters includes: Performing convolution processing on the image to be processed to obtain image features of the image to be processed; Performing adaptive learning based on the image features to determine corresponding defogging parameters, wherein the defogging parameters are used to characterize a mapping relationship for defogging the image to be processed; A target image after defogging is determined according to the defogging parameters and the image features.

3. The method according to claim 1, characterized in that The step of normalizing the image to be processed, determining a defogging parameter based on the normalized image, and obtaining a defogged target image according to the normalized image and the defogging parameter comprises: Performing convolution processing on the image to be processed to obtain image features of the image to be processed; Normalizing the image features to obtain normalized image features after normalization; Adaptively learning based on the normalized image features to determine corresponding defogging parameters, wherein the defogging parameters are used to characterize a mapping relationship for defogging the image to be processed; A target image after defogging is determined according to the defogging parameters and the normalized image features.

4. The method according to any one of claims 1 to 3, characterized in that: Before determining the defogging parameters, the method further includes: Acquire a training data set, wherein the training data set includes sample images of multiple target environments and corresponding clear images; the target environment includes a rainy fog environment or a particle environment; Using an initial convolutional neural network to perform defogging on the sample image to obtain a defogged image; Obtaining a mean absolute error loss and a structural similarity loss between the dehazed image and the corresponding clear image; Determining a target loss according to the mean absolute error loss and the structural similarity loss; Adjusting the network parameters of the initial convolutional neural network based on the target loss until the target loss converges to obtain a target convolutional neural network; The determining of the defogging parameters includes: using the target convolutional neural network to determine the defogging parameters.

5. The method according to claim 4, characterized in that The determining the target loss according to the mean absolute error loss and the structural similarity loss includes: Among them, L Mix is the target loss, is the mean absolute error loss, L SSIM is the structural similarity loss, α is the preset weight, and Gσ is the Gaussian distribution parameter.

6. The method according to any one of claims 1 to 5, characterized in that: The identifying the environment category corresponding to the vehicle includes: Acquiring environmental information of the location of the vehicle, the environmental information comprising at least one of meteorological element information and particulate matter concentration information; When the particle concentration information at the location of the vehicle reaches a set concentration threshold, determining that the environment category of the vehicle is a particle environment; or, When it is determined according to the meteorological element information that there is rainfall at the location of the vehicle, or when it is determined according to the meteorological element information that the humidity at the location of the vehicle reaches a set humidity threshold, the environment category of the vehicle is determined to be a rainy and foggy environment.

7. The method according to any one of claims 1 to 5, characterized in that: The identifying the environment category corresponding to the vehicle includes: Acquire image features of the image to be processed, wherein the image features include at least one of texture features, saturation and contrast of the image to be processed; Obtaining the similarities between the texture features of the image to be processed and the first texture features of the preset particle environment and the second texture features of the preset rain and fog environment, and determining the environment category with the highest similarity as the environment category of the vehicle; or, When at least one of the saturation and the contrast of the image to be processed is less than a corresponding threshold, it is determined that the environment category of the vehicle is a rainy and foggy environment.

8. An image defogging device, characterized in that: The device comprises: An environment recognition module is used to identify the environment category corresponding to the vehicle when acquiring the image to be processed collected by the vehicle electronic rearview mirror, wherein the environment category includes a rainy fog environment or a particle environment; A defogging processing module, configured to determine defogging parameters based on the image to be processed when it is determined that the environment category corresponding to the vehicle is a rainy and foggy environment, and to obtain a defogging target image according to the image to be processed and the defogging parameters; And when it is determined that the environment category corresponding to the vehicle is a particle environment, the image to be processed is normalized, a defogging parameter is determined based on the normalized image, and a defogging target image is obtained according to the normalized image and the defogging parameter.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images