Image defogging method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of data processing, and provides an image defogging method and device, equipment and a storage medium, and the method comprises the steps: obtaining a first sample image and a second sample image; the first sample image is obtained by carrying out data classification on a third sample image, and the third sample image is an unlabeled foggy image; the second sample image is a foggy image with a label; according to the second sample image, training the first model to obtain a trained first model; training a second model according to the first sample image and the trained first model to obtain a target defogging model; and carrying out defogging processing on the to-be-defogged image by using the target defogging model to obtain a target defogged image. According to the invention, the image defogging effect of the to-be-defogged image is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an image defogging method, device, equipment and storage medium. Background Art

[0002] Image defogging is to defog the photos and videos taken by image sensors (such as cameras) on foggy days, so that the defogging images are close to the images on fog-free days. Image defogging technology is one of the hot research areas in the field of computer vision and has been widely used in key areas such as autonomous driving, security monitoring, and target detection in highly complex environments.

[0003] In the prior art, most deep learning methods are used to defog images. The deep learning method is to train the deep learning model for defogging by collecting data with and without fog to achieve the effect of image defogging. However, since training the deep learning model requires a large amount of labeled data, that is, data collection is difficult, and the same scene needs to be collected in foggy and clear weather respectively, if the model is trained based on only a small amount of labeled data to perform image defogging, the effect is poor. Summary of the invention

[0004] The present invention provides an image defogging method, device, equipment and storage medium, which are used to solve the defect of poor image defogging effect in the prior art and improve the defogging effect.

[0005] In a first aspect, the present invention provides an image defogging method, the method comprising:

[0006] Acquire a first sample image and a second sample image; the first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; the second sample image is a labeled foggy image;

[0007] Training the first model according to the second sample image to obtain a trained first model;

[0008] Training the second model according to the first sample image and the trained first model to obtain a target defogging model;

[0009] The target defogging model is used to defog the image to be defogged to obtain a target defogged image.

[0010] Optionally, the training the second model according to the first sample image and the trained first model to obtain a target defogging model includes:

[0011] For each round, determining the first model parameters corresponding to the trained first model as the model parameters of the second model to obtain an updated second model;

[0012] Using the trained first model to perform defogging on the first sample image, to obtain a first defogging image processed by the trained first model;

[0013] Performing unsupervised training on the updated second model according to the first sample image and the first dehazed image to obtain a trained second model;

[0014] Inputting the second sample image and the first sample image into the trained second model to obtain second model parameters corresponding to the trained second model;

[0015] Determine the second model parameters as the model parameters of the trained first model to obtain an updated first model;

[0016] The trained second model corresponding to the last round is determined as the target defogging model.

[0017] Optionally, before performing unsupervised training on the updated second model according to the first sample image and the first defogging image to obtain the trained second model, the method further includes:

[0018] Screening the first defogged image according to a preset first confidence threshold to obtain a second defogged image;

[0019] Accordingly, performing unsupervised training on the updated second model according to the first sample image and the first defogging image to obtain the trained second model includes:

[0020] The updated second model is subjected to unsupervised training according to the first sample image and the second defogging image to obtain the trained second model.

[0021] Optionally, the first sample image may be obtained by the following steps, including:

[0022] For any of the third sample images, input the third sample image into the second model to obtain a third defogging image processed by the second model;

[0023] Calculating a second confidence level corresponding to the third dehazed image;

[0024] According to the second confidence levels corresponding to the third defogging images, data grading is performed on the third sample images corresponding to the second confidence levels to obtain graded third sample images;

[0025] The classified third sample image is determined as the first sample image.

[0026] Optionally, the training the first model according to the second sample image to obtain the trained first model includes:

[0027] Using the first model to perform defogging on the second sample image, to obtain a fourth defogging image processed by the first model;

[0028] The first model is iteratively trained according to the labels correspondingly contained in the fourth defogging image and the second sample image to obtain the trained first model.

[0029] Optionally, the first model and the second model are constructed based on an attention mechanism.

[0030] In a second aspect, the present invention provides an image defogging device, the device comprising:

[0031] an acquisition module, configured to acquire a first sample image and a second sample image; the first sample image is obtained after data classification of a third sample image, and the third sample image is an unlabeled foggy image; and the second sample image is a labeled foggy image;

[0032] A model training module, used for training the first model according to the second sample image to obtain a trained first model;

[0033] Training the second model according to the first sample image and the trained first model to obtain a target defogging model;

[0034] The processing module is used to perform defogging processing on the image to be defogged using the target defogging model to obtain a target defogged image.

[0035] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an image defogging method as described above is implemented.

[0036] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described image defogging methods.

[0037] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described image defogging methods.

[0038] The present invention provides an image defogging method, device, equipment and storage medium, which obtains a first sample image and a second sample image, wherein the first sample image is obtained after data classification of the third sample image, the third sample image is an unlabeled foggy image, and the second sample image is a labeled foggy image; then, according to the second sample image, the first model is trained to obtain the trained first model, and then, the second model is trained according to the first sample image and the trained first model to obtain a target defogging model; finally, the target defogging model is used to perform defogging on the image to be defogged to obtain the target defogged image. In the present invention, since the first sample image is obtained based on the third sample image, i.e., the unlabeled foggy image, the data collection difficulty is relatively small, and the second model is trained using a large-scale first sample image and the trained first model to obtain the target defogging model, thereby improving the feature extraction capability of the defogging model. Finally, the target defogging model is used to perform defogging on the image to be defogged, thereby improving the defogging effect of the image to be defogged. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 This is one of the flow charts of the image defogging method provided by the present invention;

[0041] Figure 2 This is the second flow chart of the image defogging method provided by the present invention;

[0042] Figure 3 This is the third flow chart of the image defogging method provided by the present invention;

[0043] Figure 4 It is a structural schematic diagram of the image defogging device provided by the present invention;

[0044] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The terms "first", "second", etc. in the specification and claims of the present disclosure are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable under appropriate circumstances, so that the embodiments of the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0047] In order to more clearly understand the various embodiments provided by the present invention, the relevant technical contents involved in the present invention are first introduced as follows.

[0048] Image defogging technology is to defog the photos and videos taken by image sensors (such as cameras) on foggy days, so that the defogging images are close to the images on fog-free days. Image defogging technology is one of the hot research areas in the field of computer vision and has been widely used in key areas such as autonomous driving, security monitoring, and target detection in highly complex environments.

[0049] In the prior art, most deep learning methods are currently used to defog images, which is to collect data with fog and no fog - labels to train the deep learning model for defogging to achieve the effect of image defogging. However, since training deep learning models requires a large amount of labeled data, that is, data collection is difficult, it is necessary to collect data for the same scene in foggy and clear weather.

[0050] In order to solve the problem of data collection, the conventional method of data synthesis is usually used to perform fogging on fog-free images. However, there is a large difference between the image effect of data synthesis and the image actually collected. The model is trained based on the data samples of data synthesis, and the image defogging effect of the deep learning network obtained by training is poor.

[0051] Based on the above shortcomings, the present invention provides an unsupervised pre-trained model image defogging method to improve the effect of image defogging.

[0052] Combine the following Figure 1-Figure 5The present invention describes an image defogging method.

[0053] Figure 1 This is one of the flow charts of the image defogging method provided by the present invention. Figure 1 As shown, the present invention provides an image defogging method, the method comprising:

[0054] Step 101, obtaining a first sample image and a second sample image; the first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; the second sample image is a labeled foggy image;

[0055] Specifically, the sample data obtained in this embodiment includes two parts. The first part is the third sample image, that is, the originally obtained unlabeled foggy image, and the second part is the second sample image, that is, the labeled foggy image.

[0056] It can be understood that the third sample image, that is, the unlabeled foggy image, is easier to obtain. For example, by searching for foggy images through keywords from various image websites, various browsers, etc. through a web crawler, and then saving the searched foggy images to a memory; or, by using cameras, image sensors and other image acquisition devices to actually collect data in foggy or rainy and foggy days. The collection process is relatively simple and easy to operate, and only requires simple recording.

[0057] Optionally, after acquiring the third sample image, data classification may be performed on the third sample image. For example, a deep learning model may be used to first defog the third sample image to obtain a defogged third sample image, and all defogged sample images may be sorted according to corresponding confidence levels to obtain a sorted third sample image, thereby implementing hierarchical management of the third sample images and determining the third sample image after data classification as the first sample image; or other methods of performing data classification on the third sample image may be used to obtain the first sample image.

[0058] The second sample image, that is, the labeled foggy image, can usually be downloaded from a public dataset related to labeled foggy images, or obtained by performing fogging processing on a fog-free image through an image synthesis method.

[0059] Step 102: training the first model according to the second sample image to obtain a trained first model;

[0060] Specifically, the model structure of the defogging model used in the embodiment of the present invention adopts, for example, a neural network model based on an attention mechanism based on a transformer structure to enhance the weight of detail feature information and achieve a better defogging effect; further, the defogging model used in the embodiment of the present invention includes two identical models, namely a first model and a second model, wherein the first model can be supervised trained based on a small-scale labeled foggy image to obtain a trained first model, and the trained second model can be obtained based on a large-scale unlabeled foggy image and the trained first model.

[0061] First, the first model is trained according to the second sample image, i.e., a large-scale unlabeled foggy image, to obtain the trained first model, wherein the model training parameters can be set as needed, for example, the initial learning rate is 0.001, the learning rate is reduced to one-tenth after every 50 iterations, the number of samples in each batch is set to 16, and the initial resolution of the image is set to (512, 512, 3). According to the above second sample image and model training parameters, after a preset number of iterations, such as 200 times, the trained first model can be obtained.

[0062] Step 103: training the second model according to the first sample image and the trained first model to obtain a target defogging model;

[0063] Specifically, for example, the unsupervised training of the second model can be completed by using a course learning method, wherein the course learning can be understood as first training sample data with low data difficulty and then training sample data with high data difficulty during the model training process.

[0064] Further, the second model is trained according to the first sample image and the trained first model to obtain the target defogging model. Optionally, the first model parameters corresponding to the trained first model are first determined as the parameters of the second model; then, the first sample image is input into the trained first model to obtain the first defogging image processed by the first model; further, the second model is unsupervisedly trained based on the first defogging image and the first sample image to obtain the trained second model; further, the second model parameters corresponding to the trained first model are copied to the trained first model to obtain the updated first model. The above process is iterated for a preset number of times, and the second model of the last round is determined as the target defogging model.

[0065] Step 104: Defogging the image to be defogged using the target defogging model to obtain a target defogged image.

[0066] Specifically, after obtaining the target defogging model, if it is necessary to defog the image to be defogged, the image to be defogged is input into the target defogging model, and the feature extraction module of the target defogging model is used to extract the feature information of the image, such as detail information and global information. Then, based on the feature information, an end-to-end defogging effect is achieved for the defogged image.

[0067] In the method provided in this embodiment, a first sample image and a second sample image are obtained, wherein the first sample image is obtained after data classification of the third sample image, the third sample image is an unlabeled foggy image, and the second sample image is a labeled foggy image; then, according to the second sample image, the first model is trained to obtain the trained first model, and then, the second model is trained according to the first sample image and the trained first model to obtain the target defogging model; finally, the target defogging model is used to perform defogging on the image to be defogged to obtain the target defogged image. In the present invention, since the first sample image is obtained based on the third sample image, i.e., the unlabeled foggy image, the data collection difficulty is relatively small, and the second model is trained using a large-scale first sample image and the trained first model to obtain the target defogging model, thereby improving the feature extraction capability of the defogging model. Finally, the target defogging model is used to perform defogging on the image to be defogged, thereby improving the defogging effect of the image to be defogged.

[0068] Optionally, training the second model according to the first sample image and the trained first model to obtain a target defogging model includes:

[0069] For each round, the first model parameters corresponding to the trained first model are determined as the model parameters of the second model to obtain an updated second model;

[0070] Defogging the first sample image using the trained first model to obtain a first defogging image processed by the trained first model;

[0071] According to the first sample image and the first dehazed image, unsupervised training is performed on the updated second model to obtain a trained second model;

[0072] Inputting the second sample image and the first sample image into the trained second model to obtain second model parameters corresponding to the trained second model;

[0073] Determine the second model parameters as the model parameters of the trained first model to obtain an updated first model;

[0074] The second model after training corresponding to the last round is determined as the target defogging model.

[0075] Specifically, step 103 can be implemented by the following steps:

[0076] It is understandable that using the trained first model to guide the training of the second model may include multiple rounds, and the rounds may be preset, for example, the training rounds are preset to 100 rounds, and after every 10 rounds of training, the parameters of the second model are copied to the first model, and the cycle is repeated until the training is completed after 100 rounds. The following takes each round as an example to introduce the process of unsupervised training of the model:

[0077] Figure 2 FIG. 2 is a flow chart of the image defogging method provided by the present invention, which is a schematic diagram showing the principle of using the trained first model to guide the training of the second model. Figure 2 As shown, the method includes:

[0078] For each round, the first model parameters corresponding to the trained first model are determined as the model parameters of the second model to obtain an updated second model, that is, the first model parameters corresponding to the trained first model are copied to the second model to obtain an updated second model.

[0079] Further, the first sample image, that is, the third sample image after data classification (the unlabeled foggy image), is input into the trained first model to obtain the first defogging image R1 processed by the trained first model;

[0080] Further, according to the first sample image and the first defogged image R1, the updated second model is unsupervisedly trained to obtain the trained second model. Specifically, the first sample image is input into the updated second model to obtain the defogged image fifth defogged image processed by the second model, and the first defogged image corresponding to each first sample image is used as a quasi-label to perform supervised training on the fifth defogged image processed by the second model. Optionally, the training parameters can be set as needed.

[0081] Optionally, the training order of the first sample images input into the updated second model can be set as needed, for example, by adopting a course learning method to first train the first sample images with low defogging difficulty and then train the first sample images with high defogging difficulty. The defogging difficulty is, for example, represented by the confidence of the defogged image obtained by corresponding processing using the second model. The higher the confidence of the data, the closer the result of the data processing by the model is to the real effect, indicating that the data difficulty of the first sample image is lower and the defogging difficulty of the first sample image is lower. On the contrary, the lower the confidence of the data, the worse the result of the data processing by the model is, indicating that the data difficulty of the first sample image is greater and the defogging difficulty of the first sample image is higher. Finally, the confidence of the defogged image obtained by corresponding processing by the second model can be used to achieve hierarchical management of the first sample images from easy to difficult.

[0082] Furthermore, the second sample image and the first sample image can be input into the trained second model to obtain second model parameters corresponding to the trained second model, and the second sample image and the first sample image can be mixed and input into the second model for mixed training. When a labeled foggy image, i.e., the second sample image, is input, the second model is supervised by the true value label, but when the first sample image without a true value label is input, the pseudo label, i.e., the first foggy image R1 after the first sample image is processed by the first model, is used to supervise the second model for training, thereby obtaining the trained second model and the second model parameters corresponding to the trained second model.

[0083] Furthermore, the second model parameters can be determined as the model parameters of the trained first model to obtain an updated first model, that is, the second model parameters are copied to the trained first model to complete one round of training.

[0084] Finally, the second model trained in the last round can be determined as the target defogging model. For example, the preset round is 100 rounds, and the second model trained in the 100th round is determined as the target defogging model.

[0085] In the method provided in this embodiment, the second model is trained based on the first sample image and the trained first model, that is, the second model can be trained using large-scale unlabeled foggy images, thereby enhancing the feature extraction capability of the target defogging model and improving the application effect in different scenarios, and the model defogging effect is better.

[0086] Optionally, before performing unsupervised training on the updated second model according to the first sample image and the first defogging image to obtain the trained second model, the method further includes:

[0087] The first defogging image is screened according to a preset first confidence threshold to obtain a second defogging image;

[0088] Accordingly, unsupervised training is performed on the updated second model according to the first sample image and the first defogging image to obtain the trained second model, including:

[0089] According to the first sample image and the second defogging image, unsupervised training is performed on the updated second model to obtain a trained second model.

[0090] Specifically, before unsupervised training is performed on the updated second model according to the first sample image and the first defogged image to obtain the trained second model, the first defogged image obtained by processing the first sample image with the trained first model may be optimized. An example of the process of optimizing the first defogged image is as follows:

[0091] In some embodiments, the first defogged image may be screened according to a preset first confidence threshold to obtain a second defogged image. Specifically, after obtaining the first defogged images corresponding to the first sample images, the pixels or images that do not meet the first confidence threshold in the first defogged image may be screened according to the preset first confidence threshold to obtain the first defogged image that meets the first confidence, that is, the second defogged image.

[0092] Accordingly, performing unsupervised training on the updated second model according to the first sample image and the first defogged image to obtain the trained second model may include: performing unsupervised training on the updated second model according to the first sample image and the second defogged image (obtained after optimizing the first defogged image) to obtain the trained second model. Specifically, the first sample image is input into the updated second model to obtain the defogged image processed by the second model, and the second defogged image P1 corresponding to each first sample image is used as a quasi-label to perform supervised training on the defogged image processed by the second model. Optionally, the training parameters may be set as needed, and then the trained second model may be obtained.

[0093] In the method provided in this embodiment, the first dehazed image is optimized according to a preset first confidence threshold, for example, the first dehazed image that meets the first confidence threshold is retained to obtain a second dehazed image, and then the second model is trained based on the second dehazed image, and the dehazing effect of the trained first model is better.

[0094] Optionally, the first sample image may be obtained by the following steps, including:

[0095] For any third sample image, input the third sample image into the second model to obtain a third defogging image processed by the second model;

[0096] Calculating a second confidence level corresponding to the third dehazed image;

[0097] According to the second confidence levels corresponding to the third defogging images, data grading is performed on the third sample images corresponding to the second confidence levels to obtain graded third sample images;

[0098] The classified third sample image is determined as the first sample image.

[0099] Specifically, it can be understood that the first sample image is obtained after data classification of the third sample image, the third sample image is the originally acquired unlabeled foggy image, and the process of obtaining the first sample image through the third sample image is as follows:

[0100] For any third sample image, first, the third sample image is input into the second model to obtain a third dehazed image processed by the second model. Optionally, the second model may be a trained second model, and the third sample image (that is, a large-scale unlabeled foggy image) is dehazed using the second model to obtain the third sample image after dehazing, that is, the third dehazed image.

[0101] Then, the second confidence level corresponding to the third dehazed image is calculated, wherein a higher second confidence level corresponding to the third dehazed image indicates that the result of processing the third sample image by the second model is closer to the real effect, indicating that the data difficulty of the third sample image is lower; a lower second confidence level corresponding to the third dehazed image indicates that the result of processing the third sample image by the second model is worse, indicating that the data difficulty of the third sample image is greater, and ultimately, data hierarchical management of the third sample image data from easy to difficult can be achieved.

[0102] Furthermore, data classification can be performed on the third sample images corresponding to the second confidence levels corresponding to the third defogging images to obtain the classified third sample images. For example, the second confidence levels of all the third defogging images are sorted from high to low to obtain the confidence ranking results of the third defogging images, and then data classification is performed on the third sample images corresponding to the second confidence levels according to the confidence ranking results of the third defogging images and the second confidence levels corresponding to the third defogging images.

[0103] Furthermore, the classified third sample image can be determined as the first sample image.

[0104] In the method provided in this embodiment, the first sample image is obtained by using the second model to perform data classification on the third sample image, and then the first sample image is input into the second model in order from easy to difficult data to perform unsupervised training on the second model, which can speed up the training process of the model and reduce the oscillation phenomenon during model training.

[0105] Optionally, training the first model according to the second sample image to obtain the trained first model includes:

[0106] Defogging the second sample image using the first model to obtain a fourth defogging image processed by the first model;

[0107] The first model is iteratively trained according to the labels correspondingly included in the fourth defogging image and the second sample image to obtain a trained first model.

[0108] Specifically, step 102 can be implemented by the following steps:

[0109] First, the second sample image, that is, the small-scale labeled foggy image, is input into the first model, and the second sample image is defogged to obtain a fourth defogged image processed by the first model.

[0110] Then, according to the labels corresponding to the fourth defogging image and the second sample image, such as the fog information of the image, the first model is iteratively trained to obtain the trained first model. The number of iterations and training parameters can be preset, for example, the training parameters are set to an initial learning rate of 0.001, and the learning rate is reduced to one tenth after every 50 iterations; the number of samples in each batch is set to 16, and the initial resolution of the input second sample image is set to (512, 512, 3), where 512 represents the width and height of the image, respectively, and 3 represents the number of channels of the image, and the number of iterations is set to 200. After 200 iterations, the trained first model can be obtained.

[0111] In the method provided in this embodiment, the first model is trained according to the second sample image, wherein the second sample image is a labeled foggy image, and then the trained first model is obtained, that is, the supervised training of the first model is completed based on the second sample image, and then the trained first model and the first sample image can be used to guide the unsupervised training of the second model to obtain the target defogging model.

[0112] Optionally, the first model and the second model are constructed based on an attention mechanism.

[0113] Specifically, in some embodiments, the first model and the second model can be constructed based on an attention mechanism, such as a neural network model constructed based on an attention mechanism of a transformer structure, and the model structure of the first model and the second model is based on the stacking of multiple transformer structures, thereby improving the feature extraction capability of the target dehazing model.

[0114] Optionally, in some embodiments, the first model and the second model may be other network models, which is not limited in this embodiment.

[0115] In the method provided in this embodiment, the first model and the second model are constructed based on the attention mechanism. The target defogging model obtained by training the first model and the second model has a strong feature extraction capability, and the defogging effect of the image to be defogged based on the target defogging model is better.

[0116] Figure 3 FIG. 3 is a flow chart of the image defogging method provided by the present invention, which shows the specific process of model training. Figure 3 As shown, the method includes:

[0117] Step 301: Sample data preparation;

[0118] Specifically, firstly, sample data required for model training is prepared, and the sample data includes two parts, one part is the third sample image, i.e., the unlabeled foggy image, and the other part is the second sample image, i.e., the labeled foggy image.

[0119] Step 302: Model design;

[0120] Furthermore, an image defogging model is designed. The model in this embodiment includes two identical models, a first model and a second model.

[0121] Step 303: Perform supervised training on the first model;

[0122] Furthermore, the first model is supervisedly trained using the second sample image to obtain a trained first model.

[0123] Step 304: Perform preliminary unsupervised training on the second model;

[0124] Furthermore, the second model can be preliminarily trained unsupervised, specifically:

[0125] Step (1): copying the first model parameters corresponding to the trained first model to the second model;

[0126] Step (2): Train the second model according to the third sample image and the trained first model to obtain the trained second model:

[0127] First, the third sample image is input into the trained first model to obtain the defogging result R1;

[0128] Then, images and pixels meeting the standards are selected from the defogging results according to a preset confidence threshold, and the defogging results processed by the trained first model are obtained;

[0129] Secondly, the third sample image is input into the second model, and the dehazing result processed by the trained first model is used as the quasi-label to perform pseudo-label supervised training on the output of the second model. At the beginning of the training, the confidence value of the input image is set to a small value. As the training process lengthens, the confidence value of the input image increases continuously, and finally increases to 0.85.

[0130] Step (3): The second sample image and the third sample image are mixed and input into the second model for mixed training. When the second sample image is input, supervision is performed through the label corresponding to the second sample image, but when the third sample image is input, supervised training is performed using the pseudo label, i.e., the result of defogging processed by the trained first model. After every 10 rounds of training, the parameters of the second model are copied to the first model, and the cycle is repeated until 100 rounds of training are completed.

[0131] Furthermore, the second model corresponding to the last round is determined as the trained second model.

[0132] Step 305: classify the sample data using the second model;

[0133] Specifically, the third sample image is dehazed using the (trained) second model to obtain a dehazed image processed by the (trained) second model, and all third sample images are sorted according to the confidence of the dehazed image processed by the (trained) second model to obtain a graded third sample image, that is, the first sample image.

[0134] Among them, the higher the confidence of the data, the closer the result of the data processing by the second model (after training) is to the real effect, indicating that the data difficulty is lower; the lower the confidence of the data, the worse the result of the data processing by the second model (after training) is, indicating that the data difficulty is greater; ultimately, the hierarchical management of the first sample image from easy to difficult is achieved.

[0135] Step 306: adopt a course learning approach to train the second model based on the sample data and the first model.

[0136] Specifically, the third sample image after data classification, that is, the first sample image, is sequentially input into the (trained) second model, the trained first model guides the (trained) second model, and the training is performed according to the first sample image to obtain the target defogging model.

[0137] The defogging method of unsupervised model training provided by the embodiment of the present invention can be trained using large-scale unlabeled foggy images, which can increase the feature extraction capability of the defogging model and improve the application effect in different scenes, and the defogging effect is good. In addition, in this embodiment, the second model is used to implement data hierarchical management of the third sample image, thereby realizing the learning of data from easy to difficult, which can speed up the model training process and reduce the oscillation phenomenon during learning.

[0138] The image defogging device provided by the present invention is described below. The image defogging device described below and the image defogging method described above can be referred to each other.

[0139] Figure 4: is a schematic diagram of the structure of the image defogging device provided by the present invention, such as Figure 4 As shown, the device comprises:

[0140] The acquisition module 410 is used to acquire a first sample image and a second sample image; the first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; the second sample image is a labeled foggy image;

[0141] A model training module 420, configured to train the first model according to the second sample image to obtain a trained first model;

[0142] Training the second model according to the first sample image and the trained first model to obtain a target defogging model;

[0143] The processing module 430 is used to perform defogging processing on the image to be defogged using the target defogging model to obtain a target defogged image.

[0144] In the device provided in this embodiment, the first sample image and the second sample image are acquired by the acquisition module 410, wherein the first sample image is obtained after data classification of the third sample image, the third sample image is an unlabeled foggy image, and the second sample image is a labeled foggy image; then, the model training module 420 trains the first model according to the second sample image to obtain the trained first model, and then, the second model is trained according to the first sample image and the trained first model to obtain the target defogging model; finally, the processing module 430 uses the target defogging model to perform defogging on the image to be defogged to obtain the target defogging image. In the present invention, since the first sample image is obtained based on the third sample image, i.e., the unlabeled foggy image, the data collection difficulty is relatively small, and the second model is trained using the large-scale first sample image and the trained first model to obtain the target defogging model, thereby improving the feature extraction capability of the defogging model. Finally, the target defogging model is used to perform defogging on the image to be defogged, thereby improving the defogging effect of the image to be defogged.

[0145] Optionally, the model training module 420 is specifically used to:

[0146] For each round, determining the first model parameters corresponding to the trained first model as the model parameters of the second model to obtain an updated second model;

[0147] Using the trained first model to perform defogging on the first sample image, to obtain a first defogging image processed by the trained first model;

[0148] Performing unsupervised training on the updated second model according to the first sample image and the first dehazed image to obtain a trained second model;

[0149] Inputting the second sample image and the first sample image into the trained second model to obtain second model parameters corresponding to the trained second model;

[0150] Determine the second model parameters as the model parameters of the trained first model to obtain an updated first model;

[0151] The trained second model corresponding to the last round is determined as the target defogging model.

[0152] Optionally, the image defogging device further includes a screening module, which is used to:

[0153] Screening the first defogged image according to a preset first confidence threshold to obtain a second defogged image;

[0154] Accordingly, the model training module 420 is specifically used for:

[0155] The updated second model is subjected to unsupervised training according to the first sample image and the second defogging image to obtain the trained second model.

[0156] Optionally, the first sample image may be obtained by the following steps, including:

[0157] For any of the third sample images, input the third sample image into the second model to obtain a third defogging image processed by the second model;

[0158] Calculating a second confidence level corresponding to the third dehazed image;

[0159] According to the second confidence levels corresponding to the third defogging images, data grading is performed on the third sample images corresponding to the second confidence levels to obtain graded third sample images;

[0160] The classified third sample image is determined as the first sample image.

[0161] Optionally, the model training module 420 is further used to:

[0162] Using the first model to perform defogging on the second sample image, to obtain a fourth defogging image processed by the first model;

[0163] The first model is iteratively trained according to the labels correspondingly contained in the fourth defogging image and the second sample image to obtain the trained first model.

[0164] Optionally, the first model and the second model are constructed based on an attention mechanism.

[0165] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the image defogging method, which includes:

[0166] Acquire a first sample image and a second sample image; the first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; the second sample image is a labeled foggy image;

[0167] Training the first model according to the second sample image to obtain a trained first model;

[0168] Training the second model according to the first sample image and the trained first model to obtain a target defogging model;

[0169] The target defogging model is used to defog the image to be defogged to obtain a target defogged image.

[0170] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0171] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the image defogging method provided by the above methods, the method includes:

[0172] Acquire a first sample image and a second sample image; the first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; the second sample image is a labeled foggy image;

[0173] Training the first model according to the second sample image to obtain a trained first model;

[0174] Training the second model according to the first sample image and the trained first model to obtain a target defogging model;

[0175] The target defogging model is used to defog the image to be defogged to obtain a target defogged image.

[0176] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the image defogging method provided by the above methods, the method comprising:

[0177] Acquire a first sample image and a second sample image; the first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; the second sample image is a labeled foggy image;

[0178] Training the first model according to the second sample image to obtain a trained first model;

[0179] Training the second model according to the first sample image and the trained first model to obtain a target defogging model;

[0180] The target defogging model is used to defog the image to be defogged to obtain a target defogged image.

[0181] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0182] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image defogging method, characterized in that: include: Acquire a first sample image and a second sample image; The first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; The second sample image is a labeled foggy image; Training the first model according to the second sample image to obtain a trained first model; Training the second model according to the first sample image and the trained first model to obtain a target defogging model; The target defogging model is used to defog the image to be defogged to obtain a target defogged image.

2. The image defogging method according to claim 1, characterized in that: The step of training the second model according to the first sample image and the trained first model to obtain a target defogging model includes: For each round, determining the first model parameters corresponding to the trained first model as the model parameters of the second model to obtain an updated second model; Using the trained first model to perform defogging on the first sample image, to obtain a first defogging image processed by the trained first model; Performing unsupervised training on the updated second model according to the first sample image and the first dehazed image to obtain a trained second model; Inputting the second sample image and the first sample image into the trained second model to obtain second model parameters corresponding to the trained second model; Determine the second model parameters as the model parameters of the trained first model to obtain an updated first model; The trained second model corresponding to the last round is determined as the target defogging model.

3. The image defogging method according to claim 2, characterized in that: Before performing unsupervised training on the updated second model according to the first sample image and the first defogging image to obtain the trained second model, the method further includes: Screening the first defogged image according to a preset first confidence threshold to obtain a second defogged image; Accordingly, performing unsupervised training on the updated second model according to the first sample image and the first defogging image to obtain the trained second model includes: The updated second model is subjected to unsupervised training according to the first sample image and the second defogging image to obtain the trained second model.

4. The image defogging method according to any one of claims 1 to 3, characterized in that: The first sample image can be obtained by the following steps, including: For any of the third sample images, input the third sample image into the second model to obtain a third defogging image processed by the second model; Calculating a second confidence level corresponding to the third dehazed image; According to the second confidence levels corresponding to the third defogging images, data grading is performed on the third sample images corresponding to the second confidence levels to obtain graded third sample images; The classified third sample image is determined as the first sample image.

5. The image defogging method according to any one of claims 1 to 3, characterized in that: The step of training the first model according to the second sample image to obtain the trained first model includes: Using the first model to perform defogging on the second sample image, to obtain a fourth defogging image processed by the first model; The first model is iteratively trained according to the labels correspondingly contained in the fourth defogging image and the second sample image to obtain the trained first model.

6. The image defogging method according to any one of claims 1 to 3, characterized in that: The first model and the second model are constructed based on the attention mechanism.

7. An image defogging device, characterized in that: include: An acquisition module, used for acquiring a first sample image and a second sample image; The first sample image is obtained after data classification of the third sample image, and the third sample image is an unlabeled foggy image; The second sample image is a labeled foggy image; A model training module, used for training the first model according to the second sample image to obtain a trained first model; Training the second model according to the first sample image and the trained first model to obtain a target defogging model; The processing module is used to perform defogging processing on the image to be defogged using the target defogging model to obtain a target defogged image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the image defogging method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image defogging method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image defogging method according to any one of claims 1 to 6 is implemented.

Images