Image defogging model and training method, defogging method, equipment and medium

Abstract

The present invention relates to an image defogging model and training method, a defogging method, a device and a medium, wherein the image defogging model comprises an encoder and a decoder; the encoder comprises a plurality of cascaded first processing groups, the first processing group comprises a connected image segmentation module and a first defogging unit; the decoder comprises a plurality of cascaded second processing groups, the second processing group comprises a connected upsampling module and a second defogging unit; the first defogging unit of the last first processing group is connected to the upsampling module of the first second processing group, and the second defogging unit of the last second processing group is connected to the upsampling module; each first defogging unit except the last first processing group is jump-connected to the corresponding second defogging unit. The present application combines the encoder and decoder structures, adopts an end-to-end training method, and trains the image defogging model from the input image to the final defogging output at one time, reduces the transmission error problem in multiple stages, and improves the image defogging quality and efficiency.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to an image defogging model and training method, a defogging method, a device and a medium. Background Art

[0002] Facial expression recognition is an important branch of computer vision and artificial intelligence. It infers a person's emotional state by analyzing the facial expression information in facial images or videos, providing important basis for many fields such as human-computer interaction, emotion analysis, security monitoring, medical health, education and training.

[0003] Face images collected in some scenarios may be obscured by smoke, such as smoking scenes and environmental smoke scenes. Environmental smoke includes automobile exhaust, industrial emissions, kitchen cooking smoke, food smoke, indoor exhaled smoke, and smoke produced by combustion. Therefore, face images collected in these scenes need to be dehazed.

[0004] At present, the existing dehazing algorithms have the following problems:

[0005] (1) When processing cigarette evaluation images, the effect is often not ideal and smoke cannot be completely removed, resulting in inaccurate evaluation results;

[0006] (2) Many dehazing algorithms have high computational complexity and are time-consuming, and cannot process a large number of images in real time, which affects the evaluation efficiency;

[0007] (3) Existing algorithms have poor adaptability to different types of smoke and shooting environments and cannot handle smoke removal problems in a variety of scenarios well;

[0008] (IV) Due to the existence of the above problems, existing dehazing algorithms may cause poor user experience during use, such as reduced image quality and long processing time. Summary of the invention

[0009] In order to solve one of the above problems, the present application provides an image defogging model and training method, a defogging method, a device and a medium.

[0010] In the first aspect, the present application provides an image defogging model, which adopts the following technical solutions:

[0011] An image dehazing model, comprising an encoder and a decoder;

[0012] The encoder comprises a plurality of first processing groups, wherein the first processing groups comprise an image segmentation module and a first defogging unit, wherein an output end of the image segmentation module is connected to an input end of the first defogging unit, and the plurality of first processing groups are cascaded;

[0013] The decoder comprises a plurality of second processing groups, the second processing groups comprising an upsampling module and a second defogging unit, the output end of the upsampling module is connected to the input end of the second defogging unit, and the plurality of second processing groups are cascaded;

[0014] The output end of the first defogging unit of the last one of the first processing groups is connected to the input end of the upsampling module of the first one of the second processing groups, and the output end of the second defogging unit of the last one of the second processing groups is connected to the upsampling module;

[0015] The output end of each of the first demisting units except the last first demisting unit of the first processing group is jump-connected to the input end of the corresponding second demisting unit;

[0016] The first defogging unit and the second defogging unit both include an attention module and a multi-feature extraction module, and the attention module and the multi-feature extraction module are connected.

[0017] By adopting the above technical solution and combining the structure of the encoder and decoder, an end-to-end training method can be used for training, so that the image defogging model can be trained from the input image to the final defogging output at one time, thereby reducing the problem of transmission errors in multiple stages and improving the image defogging quality and efficiency.

[0018] Optionally, the attention module includes a spatial self-attention module and a channel self-attention module, and the spatial self-attention module and the channel self-attention module are set in parallel.

[0019] Optionally, the multi-feature extraction module includes a first channel, a second channel, a third channel, a fourth channel and a fifth channel, wherein the first channel includes a 1*1 convolution, the second channel includes three 3×3 deconvolution (DeConv) layers arranged in parallel and a 1×1 depthwise separable convolution; the third channel includes a 1*1 convolution, the fourth channel includes an upsampling module, and the fifth channel includes a LayerScale module;

[0020] The input end of the first channel is connected to the output end of the attention module;

[0021] The output end of the first channel is connected to the input end and the output end of the second channel respectively, wherein the output end of the first channel is residually connected to the output end of the second channel, and the image processed by the first channel is residually connected and then superimposed with the image processed by the second channel to obtain a superimposed image;

[0022] The output end of the second channel is connected to the input end of the third channel, and the output end of the third channel is connected to the input end of the fourth channel.

[0023] The output end of the fourth channel is respectively connected to the input end and the output end of the fifth channel, and the output end of the fourth channel is residually connected to the output end of the fifth channel; the superimposed image passes through the third channel, the fourth channel and the fifth channel in sequence, and the image processed by the fourth channel is residually connected to obtain a processed image, and the processed image is superimposed with the image passed through the fifth channel to obtain a multi-scale feature image.

[0024] Optionally, the attention module is connected to the multi-feature extraction module, including:

[0025] The input end of the attention meter module is connected to the input end of the multi-feature extraction module, wherein the initial image that has not entered the attention module is superimposed with the image after passing through the attention module to obtain a first processed image, and the first processed image enters the input end of the multi-feature extraction module;

[0026] The output end of the attention module is connected to the multi-feature extraction module;

[0027] The input end connected to the multi-feature extraction module is connected to the output end connected to the multi-feature extraction module, wherein the first processed image and the first processed image after the multi-feature extraction module are superimposed to obtain the second processed image.

[0028] In a second aspect, the present application provides a method for training an image defogging model, which adopts the following technical solution:

[0029] A training method for an image defogging model, comprising:

[0030] Obtain foggy images and fog-free images in a public data set, wherein the foggy images and fog-free images correspond one to one, form the foggy images and the corresponding fog-free images into a training image pair, and divide the plurality of training image pairs into a training set, a validation set, and a test set;

[0031] Inputting the training set into an image defogging model as claimed in any one of claims 1 to 4;

[0032] Generate a defogging image corresponding to the smoke image in the training set by using the image defogging model;

[0033] Calculating the error value between the defogging image and the non-fog image in the training image pair using a loss function;

[0034] Updating the weight of the image dehazing model based on the error value;

[0035] Inputting the training set into the image defogging model after updating the weights, and repeatedly using the image defogging model to generate defogging images corresponding to the smoke images in the training set one by one, until the error value no longer changes or is within a preset range, thereby obtaining an initial image defogging model;

[0036] Acquire a target data set, the target data set comprising target image pairs under multiple target scenes, the target image pairs comprising a fog-free face image and a foggy face image corresponding to the fog-free face image;

[0037] The initial image dehazing model is used to perform transfer learning on the target data set to obtain a trained image dehazing model.

[0038] By adopting the above technical solution and acquiring a large number of smoke images and foggy images from public datasets, we can effectively utilize existing resources, reduce the cost and time of data collection, use the target dataset for transfer learning, and enhance the generalization ability of the image dehazing model so that it can process foggy images of the target scene.

[0039] Optionally, obtaining the target data set includes:

[0040] Get a fog-free face image;

[0041] Determine the center point of the face in the fog-free face image, and randomly shift the center point;

[0042] Generate a foggy face image corresponding to the fog-free face image based on an atmospheric scattering model and a center point fog synthesis method;

[0043] The fog-free face image and the foggy face image corresponding to the fog-free face image form a target image pair;

[0044] A plurality of the target image pairs are combined into the target data set.

[0045] By adopting the above technical solution, a corresponding foggy image is generated from a fog-free image of a target scene, thereby obtaining a required target image pair, making the image defogging model at the training site more suitable for the target scene.

[0046] In a third aspect, the present application provides an image defogging method, which adopts the following technical solution:

[0047] An image defogging method, characterized by comprising:

[0048] Obtain the image to be dehazed;

[0049] The image to be defogged is input into an image defogging model trained by the image defogging model training method according to any one of claims 3 to 6, and a defogged image is output.

[0050] By adopting the above technical solution,

[0051] Optionally, after outputting the defogging image, the method further includes:

[0052] The output dehazed image is processed using the prior knowledge to obtain a processed image;

[0053] The processed image is subjected to optical compensation and contrast adjustment to obtain the final dehazed image.

[0054] By adopting the above technical solution, the quality of the defogging image is improved by using the existing prior knowledge, and the optical compensation and contrast adjustment are performed by superimposing the defogging image to improve the image quality and ensure the stability and consistency of the defogging effect.

[0055] In a fourth aspect, the present application provides an electronic device, which adopts the following technical solution:

[0056] An electronic device comprises a processor and a memory, wherein the processor is coupled to the memory;

[0057] The processor is used to execute the computer program stored in the memory so that the electronic device performs the method as described in any one of the second aspect or the third aspect.

[0058] In a fifth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0059] A computer-readable storage medium includes a computer program or an instruction. When the computer program or the instruction is executed on a computer, the computer is caused to execute the method as described in any one of the second aspect or the third aspect.

[0060] In the technical solution of the present application, the image dehazing model is based on the Unet encoder-decoder network. The encoder is used to extract the features of the input image, and the decoder is used to map these features back to a clear image. The model is trained in an end-to-end manner so that the entire image dehazing model can be trained from the input image to the final dehazed output at one time, which helps to reduce the problem of transmission errors in multiple stages, so that the image dehazing model can better optimize the entire dehazing process, thereby improving the quality and efficiency of dehazing. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a structural block diagram of an image defogging model in an embodiment of the present application.

[0062] Figure 2 It is a structural block diagram of the first demisting unit and the second demisting unit in the embodiment of the present application.

[0063] Figure 3 It is a structural block diagram reflecting the structure of the attention module in the embodiment of the present application.

[0064] Figure 4 It is a structural block diagram of a multi-feature extraction module in an embodiment of the present application.

[0065] Figure 5 It is a flowchart of a training method for an image defogging model in an embodiment of the present application.

[0066] Figure 6 It is a flow chart of an image defogging method in an embodiment of the present application.

[0067] Figure 7 It is a structural block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The present application is further described in detail below in conjunction with the accompanying drawings.

[0069] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0070] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0071] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0072] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0073] This embodiment provides an image defogging model. Figure 1 and Figure 2The image defogging model includes an encoder and a decoder; the encoder includes multiple first processing groups, the first processing group includes an image segmentation module and a first defogging unit, the output end of the image segmentation module is connected to the input end of the first defogging unit, and the multiple first processing groups are cascaded; the decoder includes multiple second processing groups, the second processing group includes an upsampling module and a second defogging unit, the output end of the upsampling module is connected to the input end of the second defogging unit, and the multiple second processing groups are cascaded; the output end of the first defogging unit of the last first processing group is connected to the input end of the upsampling module of the first second processing group, and the output end of the second defogging unit of the last second processing group is connected with the upsampling module; the output end of each first defogging unit except the first defogging unit of the last first processing group is jump-connected to the input end of the corresponding second defogging unit; the first defogging unit and the second defogging unit both include an attention module and a multi-feature extraction module, and the attention module and the multi-feature extraction module are connected.

[0074] In this embodiment, the encoder includes four first processing groups, and the decoder includes three second processing groups. For the encoder, the processing order of the first processing groups is from top to bottom, and the first processing group in the lower left corner is the last first processing group. For the decoder, the processing order is from bottom to top, among which the second processing group in the lower right corner is the first second processing group.

[0075] After the defogging image is processed by each first processing group, the processed image is input to the input end of the second defogging unit of the corresponding second processing group, wherein the first first processing group corresponds to the last second processing group, the second first processing group corresponds to the second to last second processing group, and so on.

[0076] After the image to be defogged is output at the output end of the first defogging unit of the last first processing group, it enters the input end of the upsampling module of the first second processing group, and then passes through an upsampling module after the output end of the second defogging unit of the last second processing group, and finally outputs a defogged image corresponding to the image to be defogged.

[0077] It should be noted that the structures of the first demisting unit and the second demisting unit are completely consistent.

[0078] The attention module and the multi-feature extraction module are explained below respectively.

[0079] 1. Attention Module

[0080] The attention module includes a spatial self-attention module and a channel self-attention module, and the spatial self-attention module and the channel self-attention module are set in parallel.

[0081] like Figure 3As shown in the figure, when the dehazed image enters the attention module, four Token vectors are generated, namely Q, K, Vs, and Vc. Then, the spatial self-attention and channel self-attention are calculated in parallel to obtain the spatial self-attention weight matrix and the channel self-attention matrix respectively. The spatial self-attention weight matrix and the channel self-attention matrix are superimposed to obtain the image features. The calculation of channel self-attention can make up for the fact that the conventional multi-head self-attention method only extracts features in the spatial dimension and ignores the information differences between different channels.

[0082] (II) Multi-feature extraction module

[0083] like Figure 4 As shown, the multi-feature extraction module includes a first channel, a second channel, a third channel, a fourth channel and a fifth channel, wherein the first channel includes a 1*1 convolution, the second channel includes three 3×3 deconvolution (DeConv) layers arranged in parallel and a 1×1 depthwise separable convolution; the third channel includes a 1*1 convolution, the fourth channel includes an upsampling module, and the fifth channel includes a LayerScale module; the input end of the first channel is connected to the output end of the attention module; the output end of the first channel is connected to the input end and the output end of the second channel respectively, wherein the output end of the first channel is connected to the residual of the output end of the second channel The image processed by the first channel is residually connected and then superimposed with the image processed by the second channel to obtain a superimposed image; the output end of the second channel is connected to the input end of the third channel, and the output end of the third channel is connected to the input end of the fourth channel; the output end of the fourth channel is respectively connected to the input end and the output end of the fifth channel, and the output end of the fourth channel is residually connected to the output end of the fifth channel; the superimposed image passes through the third channel, the fourth channel and the fifth channel in sequence, and the image processed by the fourth channel is residually connected to obtain a processed image, and the processed image is superimposed with the image passed through the fifth channel to obtain a multi-scale feature image.

[0084] In this embodiment, the image features obtained after processing by the attention module are input to the input of the first channel, and the image features are processed by a 1×1 convolution, and then enter three parallel 3×3 deconvolution (DeConv) layers with different dilation rates (dilation) - 1, 2 and 3 to capture features of different scales. In addition, there is a 1×1 depth separable convolution (DWConv) for further refined feature extraction. After these parallel operations, the features are residually connected and integrated through 1×1 convolution, and then passed through an upsampling module (UpConv) to restore the spatial resolution. Finally, the LayerScale module is used to adjust the feature amplitude and perform residual connection output to obtain multi-scale image features, thereby improving the dehazing effect.

[0085] The input end of the attention module is connected to the input end of the multi-feature extraction module, wherein the initial image that has not entered the attention module is superimposed with the image after passing through the attention module to obtain a first processed image, and the first processed image enters the input end of the multi-feature extraction module; the output end of the attention module is connected to the multi-feature extraction module; the input end of the multi-feature extraction module is connected to the output end of the multi-feature extraction module, wherein the first processed image and the first processed image after the multi-feature extraction module are superimposed to obtain a second processed image, that is, Figure 4 Multi-scale image features in .

[0086] The attention modules in the first defogging unit and the second defogging unit calculate the attention weights to guide the image defogging model to pay more attention to important areas when processing images, so that the image defogging model can reduce the influence of unnecessary irrelevant information when processing images, thereby improving the performance of the method; the features extracted by the multi-feature extraction module can form a more compact and effective feature representation after being screened and fused by the attention module, which not only improves the defogging effect, but also reduces the computing resources required for subsequent processing.

[0087] In this application, the image dehazing model is based on the Unet encoder-decoder network, in which the encoder is used to extract the features of the input image, and the decoder is used to map these features back to a clear image. It is trained in an end-to-end manner so that the entire image dehazing model can be trained from the input image to the final dehazed output at one time, which helps to reduce the problem of transmission errors in multiple stages, so that the image dehazing model can better optimize the entire dehazing process, thereby improving the quality and efficiency of dehazing.

[0088] The embodiment of the present application also provides a training method for an image defogging model, which can be executed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a desktop computer, etc., but is not limited thereto.

[0089] like Figure 5 As shown, a training method for an image defogging model is described as follows (steps S101 to S108):

[0090] Step S101, obtaining foggy images and fog-free images in a public data set, wherein the foggy images and the fog-free images correspond one to one, forming a training image pair with the foggy images and the corresponding fog-free images, and dividing the multiple training image pairs into a training set, a validation set, and a test set;

[0091] In this embodiment, after obtaining the fog-free image and the foggy image corresponding to the fog-free image, cropping processing is performed so that the fog-free image and the foggy image corresponding to the fog-free image have the same size and ratio, and the resolution of the fog-free image and the foggy image after cropping can also be adjusted so that the fog-free image and the foggy image corresponding to the fog-free image have the same resolution. The cropping processing method can adopt a neural network model for image processing, and can also adopt other methods, which are not specifically limited.

[0092] Step S102, inputting the training set into the image defogging model;

[0093] Step S103, using the image defogging model to generate a defogging image that corresponds one-to-one to the smoke image in the training set;

[0094] Step S104, using the loss function to calculate the error value between the defogging image and the non-fogging image in the training image pair;

[0095] In this embodiment, the loss function L1 is used to calculate the error value.

[0096] Step S105, updating the weight of the image defogging model based on the error value;

[0097] Step S106, inputting the training set into the image defogging model after updating the weights, and repeatedly using the image defogging model to generate a defogging image that corresponds one-to-one to the smoke image in the training set until the error value no longer changes or is within a preset range, thereby obtaining an initial image defogging model;

[0098] Step S107, obtaining a target data set, the target data set including target image pairs under multiple target scenes, the target image pairs including a fog-free face image and a foggy face image corresponding to the fog-free face image;

[0099] Specifically, a fog-free face image is obtained; the center point of the face in the fog-free face image is determined, and the center point is randomly offset; a foggy face image corresponding to the fog-free face image is generated based on an atmospheric scattering model and a center point fog synthesis method; the fog-free face image and the foggy face image corresponding to the fog-free face image are combined into a target image pair; and multiple target image pairs are combined into a target data set.

[0100] In this embodiment, a fog-free face image in an actual business scenario, i.e., a target scenario, is collected, and a face detection model is used to obtain the center point in the fog-free face image. The fog concentration is controlled by adjusting the scattering coefficient β, and a local area is defined with the offset face center point as the center. A higher fog concentration is applied in this area (i.e., the transmittance of the area is reduced). According to the atmospheric scattering model, the original clear image and the locally reduced transmittance are combined to generate an image with a local fog effect, i.e., a foggy face image corresponding to the fog-free face image.

[0101] Among them, the target scenes include smoking scenes and environmental smoke scenes. Environmental smoke includes automobile exhaust, industrial emissions, kitchen cooking smoke, food smoke, indoor exhaled smoke, smoke produced by combustion, etc.

[0102] Step S108, using the initial image dehazing model to perform transfer learning on the target data set to obtain a trained image dehazing model.

[0103] In this embodiment, the initial image dehazing model trained by the training set is used for transfer learning in the target data set, thereby obtaining a trained image dehazing model to adapt to the business scenario.

[0104] It should be noted that the image dehazing model of the present application can not only be used for image dehazing processing in the process of cigarette evaluation, but also can be used for dehazing processing of images collected under automobile exhaust, industrial emissions, kitchen cooking smoke, food smoke, indoor exhaled smoke, and smoke produced by combustion. The target data set is collected according to the corresponding target scene.

[0105] The embodiment of the present application also provides an image defogging method, which can be executed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a desktop computer, etc., but is not limited thereto.

[0106] like Figure 6 As shown, an image defogging method, the main process of which is described as follows (steps S201-S202):

[0107] Step S201, obtaining an image to be defogged;

[0108] Step S202: input the image to be defogged into the trained image defogging model, and output the defogged image.

[0109] In this embodiment, the image to be defogged is input into a trained image defogging model to obtain a defogged image corresponding to the defogged image.

[0110] Among them, the images to be dehazed include face images collected when smoke obscures the face, and the smoke includes smoke in smoking scenes, car exhaust, industrial emissions, kitchen cooking smoke, food smoke, indoor exhaled smoke, smoke produced by combustion, etc.

[0111] In order to improve the defogging effect of the image to be defogged, after outputting the defogged image, the following is also included:

[0112] Specifically, the output defogging image is processed using the prior knowledge to obtain a processed image; the processed image is optically compensated and contrast adjusted to obtain a final defogging image.

[0113] In this embodiment, the prior knowledge includes physical prior, and the physical prior includes bright channel prior and dark channel prior. The output dehazed image is enhanced by color histogram redistribution and other technologies to further improve the quality of the dehazed image and obtain a processed image. Optical compensation and contrast adjustment are performed on the processed image to further improve the image quality and ensure the stability and consistency of the dehazed effect.

[0114] Figure 7 FIG. 3 is a structural block diagram of an electronic device 300 provided in an embodiment of the present application. Figure 7 As shown, the electronic device 300 includes a memory 301, a processor 302 and a communication bus 303; the memory 301 and the processor 302 are connected via the communication bus 303. The memory 301 stores a training method for an image defogging model and an image defogging method provided in the above embodiment, which can be loaded and executed by the processor 302.

[0115] The memory 301 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 301 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the training method of the image defogging model and the image defogging method provided in the above embodiment, etc.; the data storage area may store the training method of the image defogging model and the data involved in the image defogging method provided in the above embodiment, etc.

[0116] The processor 302 may include one or more processing cores. The processor 302 calls the data stored in the memory 301 by running or executing the instructions, programs, code sets or instruction sets stored in the memory 301, and performs various functions and processes data of the present application. The processor 302 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller and a microprocessor. It is understandable that for different devices, the electronic device used to implement the above-mentioned processor 302 function can also be other, and the embodiment of the present application is not specifically limited.

[0117] The communication bus 303 may include a path to transmit information between the above components. The communication bus 303 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 303 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one double arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0118] An embodiment of the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executes a training method for an image defogging model and an image defogging method as provided in the above embodiments.

[0119] In this embodiment, the computer-readable storage medium may be a tangible device that holds and stores instructions used by the instruction execution device. The computer-readable storage medium may be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. Specifically, the computer-readable storage medium may be a portable computer disk, a hard disk, a USB flash drive, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a podium random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, an optical disk, a magnetic disk, a mechanical encoding device, and any combination thereof.

[0120] The terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus.

[0121] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned application concept. For example, the above features are replaced with (but not limited to) technical features with similar functions applied in the present application.

Claims

1. An image dehazing model, characterized in that: Includes encoder and decoder; The encoder comprises a plurality of first processing groups, wherein the first processing groups comprise an image segmentation module and a first defogging unit, wherein an output end of the image segmentation module is connected to an input end of the first defogging unit, and the plurality of first processing groups are cascaded; The decoder comprises a plurality of second processing groups, the second processing groups comprising an upsampling module and a second defogging unit, the output end of the upsampling module is connected to the input end of the second defogging unit, and the plurality of second processing groups are cascaded; The output end of the first defogging unit of the last one of the first processing groups is connected to the input end of the upsampling module of the first one of the second processing groups, and the output end of the second defogging unit of the last one of the second processing groups is connected to the upsampling module; The output end of each of the first demisting units except the last first demisting unit of the first processing group is jump-connected to the input end of the corresponding second demisting unit; The first defogging unit and the second defogging unit both include an attention module and a multi-feature extraction module, and the attention module and the multi-feature extraction module are connected.

2. The image defogging model according to claim 1, characterized in that: The attention module includes a spatial self-attention module and a channel self-attention module, and the spatial self-attention module and the channel self-attention module are arranged in parallel.

3. An image defogging model according to claim 1 or 2, characterized in that: The multi-feature extraction module includes a first channel, a second channel, a third channel, a fourth channel and a fifth channel, wherein the first channel includes a 1*1 convolution, the second channel includes three 3×3 deconvolution (DeConv) layers arranged in parallel and a 1×1 depthwise separable convolution; the third channel includes a 1*1 convolution, the fourth channel includes an upsampling module, and the fifth channel includes a LayerScale module; The input end of the first channel is connected to the output end of the attention module; The output end of the first channel is connected to the input end and the output end of the second channel respectively, wherein the output end of the first channel is residually connected to the output end of the second channel, and the image processed by the first channel is residually connected and then superimposed with the image processed by the second channel to obtain a superimposed image; The output end of the second channel is connected to the input end of the third channel; the output end of the third channel is connected to the input end of the fourth channel; The output end of the fourth channel is respectively connected to the input end and the output end of the fifth channel, and the output end of the fourth channel is residually connected to the output end of the fifth channel; the superimposed image passes through the third channel, the fourth channel and the fifth channel in sequence, and the image processed by the fourth channel is residually connected to obtain a processed image, and the processed image is superimposed with the image passed through the fifth channel to obtain a multi-scale feature image.

4. The image defogging model according to claim 3, characterized in that: The attention module is connected to the multi-feature extraction module and includes: The input end of the attention meter module is connected to the input end of the multi-feature extraction module, wherein the initial image that has not entered the attention module is superimposed with the image after passing through the attention module to obtain a first processed image, and the first processed image enters the input end of the multi-feature extraction module; The output end of the attention module is connected to the multi-feature extraction module; The input end connected to the multi-feature extraction module is connected to the output end of the multi-feature extraction module, wherein the first processed image and the first processed image after the multi-feature extraction module are superimposed to obtain the second processed image.

5. A training method for an image defogging model, characterized in that: include: Obtain foggy images and fog-free images from a public data set, wherein the foggy images and fog-free images correspond one to one, form the foggy images and the corresponding fog-free images into a training image pair, and divide the plurality of training image pairs into a training set, a validation set, and a test set; Inputting the training set into an image defogging model as claimed in any one of claims 1 to 4; Generate a defogging image corresponding to the smoke image in the training set by using the image defogging model; Calculating the error value between the defogging image and the non-fog image in the training image pair using a loss function; Updating the weight of the image dehazing model based on the error value; Inputting the training set into the image defogging model after updating the weights, and repeatedly using the image defogging model to generate defogging images corresponding to the smoke images in the training set one by one, until the error value no longer changes or is within a preset range, thereby obtaining an initial image defogging model; Acquire a target data set, the target data set comprising target image pairs under multiple target scenes, the target image pairs comprising a fog-free face image and a foggy face image corresponding to the fog-free face image; The initial image dehazing model is used to perform transfer learning on the target data set to obtain a trained image dehazing model.

6. The method for training an image defogging model according to claim 5, characterized in that: The step of obtaining a target data set includes: Get a fog-free face image; Determine the center point of the face in the fog-free face image, and randomly shift the center point; Generate a foggy face image corresponding to the fog-free face image based on an atmospheric scattering model and a center point fog synthesis method; The fog-free face image and the foggy face image corresponding to the fog-free face image form a target image pair; A plurality of the target image pairs are combined into the target data set.

7. An image defogging method, characterized in that: include: Obtain the image to be dehazed; The image to be defogged is input into the image defogging model trained by the image defogging model training method according to claim 5 or 6, and a defogged image is output.

8. The image defogging method according to claim 7, characterized in that: After outputting the defogging image, the method further includes: The output dehazed image is processed using the prior knowledge to obtain a processed image; The processed image is subjected to optical compensation and contrast adjustment to obtain the final dehazed image.

9. An electronic device, characterized in that: comprising a processor and a memory, wherein the processor is coupled to the memory; The processor is configured to execute the computer program stored in the memory, so that the electronic device executes the method according to any one of claims 5 to 8.

10. A computer-readable storage medium, characterized in that: The method comprises a computer program or an instruction, which, when executed on a computer, causes the computer to execute the method according to any one of claims 5 to 8.

Images