Image defogging method and device based on self-supervised learning, equipment and medium

Abstract

The application is suitable for the technical field of image processing, and provides an image defogging method and device based on self-supervised learning, equipment and medium, wherein the method comprises: acquiring multiple clear and non-fog images; for each clear and non-fog image, processing the clear and non-fog image according to multiple preset image processing angles by using an atmospheric scattering model to obtain multiple foggy images of the clear and non-fog image; for each foggy image, taking the foggy image and the corresponding clear and non-fog image as an image pair to obtain multiple image pairs; training an image defogging model by using the multiple image pairs to obtain a trained image defogging model; and performing defogging processing on a to-be-defogged image by using the trained image defogging model. The application can improve the efficiency of image defogging.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image defogging method and device based on self-supervised learning, equipment and a medium. BACKGROUND

[0002] With the wide popularity and application of computer vision technology, a large amount of image data needs to be analyzed and processed, and the collection environment of image data in reality is easily disturbed by factors such as weather. For example, in foggy weather, images captured in outdoor scenes are often affected by poor visibility, reduced contrast, surface blooming and color shift. The existence of fog and haze caused by aerosols such as dust, fog and smoke adds complex, nonlinear and data-dependent noise to images, making fog and haze removal a challenging image restoration and enhancement problem. In many application scenarios, such as autonomous driving, the presence of fog and haze can obscure vehicle cameras, causing the collected data quality to degrade seriously and affecting the further decision-making of intelligent driving systems. Therefore, effectively defogging images and restoring high-quality haze-free images have become an important process for many computer systems and application scenarios, and various image defogging models have emerged accordingly.

[0003] However, these image defogging models need to be trained using a large amount of image data (i.e., image pairs composed of haze-free images and foggy images) before being put into actual use, and the large amount of image data is usually collected and manually annotated by humans, thereby causing low defogging efficiency. SUMMARY

[0004] The embodiments of the present application provide an image defogging method and device based on self-supervised learning, which can solve the problem of low image defogging efficiency.

[0005] In a first aspect, the embodiments of the present application provide an image defogging method based on self-supervised learning, comprising:

[0006] obtaining a plurality of clear haze-free images;

[0007] respectively for each clear haze-free image, processing the clear haze-free image according to a plurality of preset image processing angles by using an atmospheric scattering model to obtain a plurality of foggy images of the clear haze-free image; wherein the plurality of image processing angles correspond one-to-one to the plurality of foggy images;

[0008] respectively for each foggy image, taking the foggy image and the clear haze-free image corresponding to the foggy image as an image pair to obtain a plurality of image pairs;

[0009] training an image defogging model using the plurality of image pairs to obtain a trained image defogging model;

[0010] The trained image defogging model is used to defog the to-be-defogged image.

[0011] Optionally, the transmission map of the atmospheric scattering model is:

[0012]

[0013] wherein t(x) represents the transmission map of the atmospheric scattering model, x represents the position of a pixel in the clear and fog-free image, k represents the dissipation coefficient, the value range of k is 2 to 4, t0(x) represents the initialized transmission map, p represents the adjustment coefficient, the value range of p is 1 to 1.6, and d represents the bias coefficient, the value range of d is 0 to 0.15.

[0014] Optionally, before the clear and fog-free image is processed according to the preset image processing angle by using the atmospheric scattering model, the image defogging method further comprises:

[0015] a value in the value range of the dissipation coefficient is selected as the value of the dissipation coefficient by using a random function;

[0016] a value in the value range of the adjustment coefficient is selected as the value of the adjustment coefficient by using a random function;

[0017] a value in the value range of the bias coefficient is selected as the value of the bias coefficient by using a random function.

[0018] Optionally, the value range of the atmospheric light in the atmospheric scattering model is 0.6 to 1.

[0019] Optionally, before the clear and fog-free image is processed according to the preset image processing angle by using the atmospheric scattering model, the image defogging method further comprises:

[0020] a value in the value range of the atmospheric light is selected as the value of the atmospheric light by using a random function.

[0021] Optionally, the plurality of image processing angles include: from the bottom to the top of the clear and fog-free image, from the top to the bottom of the clear and fog-free image, from the left side to the right side of the clear and fog-free image, from the right side to the left side of the clear and fog-free image, from the lower left corner to the upper right corner of the clear and fog-free image, from the upper right corner to the lower left corner of the clear and fog-free image, from the upper left corner to the lower right corner of the clear and fog-free image, and from the lower right corner to the upper left corner of the clear and fog-free image.

[0022] In a second aspect, an image defogging device based on self-supervised learning is provided, comprising:

[0023] an acquisition module configured to acquire a plurality of clear and fog-free images;

[0024] The processing module is configured to process, for each clear image, the clear image according to a plurality of preset image processing angles based on the atmospheric scattering model, to obtain a plurality of foggy images corresponding to the clear image.

[0025] The data set module is configured to obtain a plurality of image pairs by taking each foggy image and the clear image corresponding to the foggy image as an image pair.

[0026] The training module is configured to train the image dehazing model based on the plurality of image pairs, to obtain a trained image dehazing model.

[0027] The dehazing module is configured to perform dehazing on the image to be dehazed based on the trained image dehazing model.

[0028] In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the image dehazing method described above when executing the computer program.

[0029] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the image dehazing method described above.

[0030] The above-mentioned scheme of the present application has the following advantages:

[0031] In the embodiments of the present application, a plurality of clear images are processed based on the atmospheric scattering model to obtain a foggy image corresponding to each clear image, and the clear image and the corresponding foggy image are taken as an image pair, then the image dehazing model is trained based on the image pair, and finally the trained image dehazing model is used to perform dehazing on the image to be dehazed. Since the paired image with artificial label information is not required to be input during the training of the image dehazing model, the training of the image dehazing model can be completed only by using the clear image itself, thereby greatly shortening the training period of the image dehazing model, and further greatly improving the efficiency of image dehazing.

[0032] In addition, when the clear image is processed based on the atmospheric scattering model, the clear image is processed from multiple image processing angles, so that the plurality of foggy images corresponding to the clear image can simulate any fog distribution, thereby making the distribution of fog in the data set for training the image dehazing model more balanced and diversified, which can increase the robustness of the image dehazing model and improve the effect of image dehazing.

[0033] Other advantages of the present application will be described in detail in the following specific embodiments. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart of an image dehazing method based on self-supervised learning provided in an embodiment of this application;

[0036] Figure 2 An optimized curve of an atmospheric scattering model provided in an embodiment of this application;

[0037] Figure 3 This is a schematic diagram of a foggy image obtained by an atmospheric scattering model in an example of this application;

[0038] Figure 4 This is a comparison chart of the results of different image dehazing methods in an example of this application;

[0039] Figure 5 This is a schematic diagram of the structure of an image dehazing device provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0041] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0042] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0043] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0044] As used in the specification and the appended claims, the term “if’ can be interpreted as meaning “when” or “upon” or “in response to determining” or “in response to detecting” depending on the context. Similarly, the phrase “if it is determined” or “if [the described condition or event] is detected” can be interpreted as meaning “upon determining” or “in response to determining” or “upon detecting [the described condition or event]” or “in response to detecting [the described condition or event]” depending on the context.

[0045] In addition, in the description of the present application and the appended claims, the terms “first”, “second”, “third”, etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0046] Reference in the specification to “one embodiment” or “some embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase “in one embodiment” or “in some embodiments” in various places in the specification are not necessarily all referring to the same embodiment, although it can. The terms “comprising”, “including”, “having” and their variants mean “including but not limited to”, unless otherwise expressly specified.

[0047] At present, before the image defogging model is put into practical use, it needs to be trained by using a large amount of image data, and the large amount of image data is usually manually labeled, thereby causing low defogging efficiency.

[0048] In view of the above problems, the embodiment of the present application provides an image defogging method based on self-supervised learning. The image defogging method processes a plurality of clear and fog-free images by using an atmospheric scattering model to obtain a foggy image of each clear and fog-free image, and takes the clear and fog-free image and the corresponding foggy image as an image pair, and then trains the image defogging model by using the image pair. Finally, the trained image defogging model is used to defog a to-be-defogged image. Since the paired image with artificial label information is not required to be input during training of the image defogging model, the training of the image defogging model can be completed only by using the clear and fog-free image itself, thereby greatly shortening the training period of the image defogging model, and further greatly improving the efficiency of image defogging.

[0049] In addition, since the clear and non-foggy image is processed from multiple image processing angles when the atmospheric scattering model is used to process the clear and non-foggy image, the clear and non-foggy image can simulate any fog distribution for multiple foggy images corresponding to the clear and non-foggy image, so that the distribution of the fog in the data set for training the image defogging model is more balanced and diversified, the robustness of the image defogging model can be increased, and the image defogging effect can be improved.

[0050] The image defogging method based on self-supervised learning provided by the present application will be exemplarily described below in combination with specific embodiments.

[0051] The image defogging method based on self-supervised learning provided by the present application can be executed by a terminal device or an apparatus (such as a chip) applied to the terminal device. The following embodiments take the terminal device as an example. As an example, the terminal device can be a tablet, a server, a notebook computer, or the like, and the embodiments of the present application do not limit this.

[0052] As shown in Figure 1 The image defogging method based on self-supervised learning provided by the present application includes the following steps:

[0053] Step 11, obtaining multiple clear and non-foggy images.

[0054] In some embodiments of the present application, the background of the clear and non-foggy image can be a gray background, that is, the weather corresponding to the clear and non-foggy image is overcast, so that the fog added to the clear and non-foggy image subsequently is more coordinated with the background and not conspicuous.

[0055] Specifically, in some embodiments of the present application, the clear and non-foggy image can be obtained from the Internet through a crawler technology.

[0056] As an optional example, the number of clear and non-foggy images can be 5000.

[0057] Step 12, for each clear and non-foggy image, an atmospheric scattering model is used to process the clear and non-foggy image according to multiple preset image processing angles, to obtain multiple foggy images of the clear and non-foggy image.

[0058] The multiple image processing angles correspond to the multiple foggy images one by one, that is, each image processing angle corresponds to one foggy image.

[0059] In some embodiments of the present application, for each clear and non-hazy image in step 11, the clear and non-hazy image needs to be processed multiple times by using the atmospheric scattering model to obtain multiple hazy images of the clear and non-hazy image. Specifically, before the atmospheric scattering model processes the clear and non-hazy image each time, the image processing angle for processing the clear and non-hazy image needs to be determined to ensure that the atmospheric scattering model can process the clear and non-hazy image from multiple image processing angles to obtain multiple hazy images with multiple angles and multiple distributions.

[0060] It should be noted that the above multiple image processing angles can be set in advance according to actual conditions. And from the above process of obtaining hazy images, it can be seen that in some embodiments of the present application, without the help of a depth scanning device or an existing depth estimation network, close-to-real hazy images can be efficiently and simply obtained.

[0061] Step 13: For each hazy image, the hazy image and the clear and non-hazy image corresponding to the hazy image are taken as an image pair to obtain multiple image pairs.

[0062] In some embodiments of the present application, the hazy image and the corresponding clear and non-hazy image are taken as an image pair to facilitate subsequent training of the image dehazing model.

[0063] Step 14: The image dehazing model is trained by using the multiple image pairs to obtain a trained image dehazing model.

[0064] In some embodiments of the present application, the above image dehazing model can be a commonly used depth dehazing network. Specifically, in some embodiments of the present application, during the training process, the clear and non-hazy image of the input image pair can be taken as a reference image, and the dehazed image obtained based on the depth dehazing network can be taken as a restored image. The average absolute error between the restored image and the reference image is calculated, and the error between the restored image and the reference image is optimized to make the restored image obtained by the depth dehazing network closer to the reference image. When the difference between the restored image and the reference image is stable within a small range, the training of the image dehazing model stops, and a trained image dehazing model is obtained.

[0065] It should be noted that since the process of training the image dehazing model by using multiple image pairs as a training data set is the same as the training process of the commonly used depth dehazing network, the specific steps of the training process will not be described in detail here.

[0066] It should be noted that in some embodiments of the present application, only a clear non-fog image needs to be input to generate a corresponding fog image, so the generation process of the fog image is a self-generation process. At the same time, since the training process of the image defogging model only needs to input a clear non-fog image without the help of any additional information, the training process is carried out in a self-supervised manner.

[0067] Step 15, using the trained image defogging model to perform defogging processing on the image to be defogged.

[0068] It is worth mentioning that in some embodiments of the present application, since when training the image defogging model, no paired images with artificial label information need to be input, and the training of the image defogging model can be completed only by the clear non-fog image itself, the training period of the image defogging model is greatly shortened, and the efficiency of image defogging is greatly improved.

[0069] In addition, since when processing the clear non-fog image by using the atmospheric scattering model, the clear non-fog image is processed from multiple image processing angles, so that the multiple fog images corresponding to the clear non-fog image can simulate any fog distribution, and the distribution of the fog in the data set for training the image defogging model is more balanced and diversified, which can increase the robustness of the image defogging model, improve the effect of image defogging, and obtain high-quality defogging images.

[0070] The atmospheric scattering model will be exemplarily described in combination with specific embodiments.

[0071] In some embodiments of the present application, the above-mentioned atmospheric scattering model is: I(x) = J(x)t(x) + A(1-t(x), I(x) represents the atmospheric scattering model, J(x) represents the target reflected light, t(x) represents the transmission map of the atmospheric scattering model, and A represents the atmospheric light value.

[0072] In some embodiments of the present application, the above-mentioned atmospheric scattering model is mainly used to generate a fog image. It should be noted that since the above-mentioned atmospheric scattering model is a commonly used atmospheric scattering model, in the embodiments of the present application, the principle of the atmospheric scattering model is not described in detail.

[0073] It should be noted that in some embodiments of the present application, the value range of the atmospheric light value A in the above-mentioned atmospheric scattering model is 0.6 to 1. In addition, in a natural environment, different wind speeds can be sufficient to change the distribution of fog in the air, so the distribution of fog density in the natural environment is changing. Based on this, the present application optimizes the above-mentioned t(x), and the optimized t(x) is: t(x) = exp(-0.6x) + 0.4. wherein, x represents the position of a pixel in the clear haze-free image, k represents the dissipation coefficient, the value range of k is 2 to 4, t0(x) represents the initialized transmission map (in some embodiments of the present application, the pixel value of the initialized transmission map can be 1), p represents the adjustment coefficient, the value range of p is 1 to 1.6, d represents the bias coefficient, and the value range of d is 0 to 0.15.

[0074] It should be noted that different values of k, p and d affect the color and density distribution of the fog in the finally generated fog image. That is, by adjusting the values of k, p and d, the color and density distribution of the fog in the fog image can be adjusted. As shown in Figure 2 , when k, p and d take different values, the density distribution of the fog in the fog image is different. Among them, Figure 2 The vertical axis represents the concentration of the fog in the fog image, the horizontal axis represents different positions in the fog image (for example, the horizontal coordinate 0 represents the bottom end of the fog image, and the horizontal coordinate 1 represents the top end of the fog image), the curve A is the optimization curve of the transmission map t(x) when k=4, p=1 and d=0, the curve B is the optimization curve of the transmission map t(x) when k=2, p=1.2 and d=0.15, the curve C is the optimization curve of the transmission map t(x) when k=3, p=1.3 and d=0.15, the curve D is the optimization curve of the transmission map t(x) when k=4, p=1.4 and d=0.15, and the curve E is the optimization curve of the transmission map t(x) when k=4, p=1.5 and d=0.05.

[0075] In some embodiments of the present application, regarding the values of A, k, p and d, after determining the image processing angle for processing the clear haze-free image, corresponding values can be generated from a specified value range by using a random function, so that the density distribution of the fog in the finally generated fog image is continuous and has no void, and is closer to the real fog image, thereby helping to improve the training accuracy of the image defogging model and improve the defogging performance of the image defogging model.

[0076] Specifically, in some embodiments of the present application, a value can be selected from the value range of the dissipation coefficient as the value of the dissipation coefficient by using a random function; a value can be selected from the value range of the adjustment coefficient as the value of the adjustment coefficient by using a random function; a value can be selected from the value range of the bias coefficient as the value of the bias coefficient by using a random function; and a value can be selected from the value range of the atmospheric light value as the value of the atmospheric light value by using a random function.

[0077] The image processing angle will be exemplarily described in combination with specific embodiments.

[0078] Generally, the concentration of fog perceived by a person is related to the depth of the scene. Specifically, the farther the distance of the depth of the scene, the greater the concentration of fog perceived by a person, and thus the concentration of fog can bring the perception of the depth of the scene to a person. When a camera is used to shoot a foggy image, the lower half of the image is generally closer to the lens, and the upper half of the scene is farther away from the lens, and the lower half of most foggy images is light in fog concentration, and the upper half is deep in fog concentration, so the concentration of fog in the image is consistent with the visual perception of a person, that is, the concentration of fog in the image is closely related to the depth of the scene.

[0079] However, for a clear and non-foggy image, the distribution of the scene in the image is unknown, that is, which position of the image is a near scene and which position of the image is a far scene. Therefore, in the embodiments of the present application, a multi-angle (i.e. multiple image processing angles) method is used to simulate any fog distribution, so that for any clear and non-foggy image of a scene, a multi-angle distributed foggy image can be synthesized, and at the same time, for the training of the image defogging model, not only is the amount of training data increased, but also the distribution of fog in the entire data set is more balanced and diversified, thereby increasing the robustness of the image defogging model.

[0080] In some embodiments of the present application, the above-mentioned image processing angle can be understood as the processing order of the pixels in the clear and non-foggy image, such as processing the pixels at the bottom of the clear and non-foggy image first and then processing the pixels at the top of the clear and non-foggy image, or processing the pixels on the left side of the clear and non-foggy image first and then processing the pixels on the right side of the clear and non-foggy image.

[0081] Specifically, the above-mentioned multiple image processing angles include: from the bottom of the clear and non-foggy image to the top of the clear and non-foggy image, from the top of the clear and non-foggy image to the bottom of the clear and non-foggy image, from the left side of the clear and non-foggy image to the right side of the clear and non-foggy image, from the right side of the clear and non-foggy image to the left side of the clear and non-foggy image, from the lower left corner of the clear and non-foggy image to the upper right corner of the clear and non-foggy image, from the upper right corner of the clear and non-foggy image to the lower left corner of the clear and non-foggy image, from the upper left corner of the clear and non-foggy image to the lower right corner of the clear and non-foggy image, from the lower right corner of the clear and non-foggy image to the upper left corner of the clear and non-foggy image, and from the center of the clear and non-foggy image to the edge of the clear and non-foggy image, etc.

[0082] It is worth mentioning that by processing the clear and non-foggy image from multiple image processing angles, multiple foggy images corresponding to different angles are obtained, thereby making the training data set of the image defogging model more abundant, which helps to improve the training accuracy of the image defogging model and improve the defogging performance of the image defogging model.

[0083] It can be understood that in order to obtain foggy images with uniform fog distribution and further enrich the training data set of the foggy images, the above-mentioned multiple image processing angles can also include randomly processing each pixel in the clear and non-foggy image, so that the fog in the finally generated foggy image is uniformly distributed.

[0084] In this case, the foggy image obtained by different image processing angles is exemplarily illustrated by a specific example. In this example, the clear and non-foggy image is as shown by a in Figure 3 , the foggy image obtained by processing from the bottom of the clear and non-foggy image to the top of the clear and non-foggy image is as shown by b in Figure 3 , the foggy image obtained by processing from the top of the clear and non-foggy image to the bottom of the clear and non-foggy image is as shown by c in Figure 3 , the foggy image obtained by processing from the right side of the clear and non-foggy image to the left side of the clear and non-foggy image is as shown by d in Figure 3 , the foggy image obtained by processing from the left side of the clear and non-foggy image to the right side of the clear and non-foggy image is as shown by e in Figure 3 , the foggy image obtained by processing from the lower right corner of the clear and non-foggy image to the upper left corner of the clear and non-foggy image is as shown by f in Figure 3 , the foggy image obtained by processing from the upper left corner of the clear and non-foggy image to the lower right corner of the clear and non-foggy image is as shown by g in Figure 3 , the foggy image obtained by processing from the upper right corner of the clear and non-foggy image to the lower left corner of the clear and non-foggy image is as shown by h in Figure 3 , the foggy image obtained by processing from the lower left corner of the clear and non-foggy image to the upper right corner of the clear and non-foggy image is as shown by i in Figure 3 , the foggy image obtained by randomly processing each pixel in the clear and non-foggy image is as shown by j in Figure 4 .

[0085] The above image defogging method is exemplarily illustrated by a specific example as follows.

[0086] In this example, the image defogging model is two existing deep defogging networks, which are: a multi-scale boosted defogging network with dense feature fusion (MSBDN, Multi-Scale Boosted Dehazing Network with DenseFeature Fusion) and GridNet. In this case, the foggy image as shown by a in Figure 4 is defogged by MSBDN, the image defogging method of the present application (the image defogging model in this method is MSBDN), GridNet and the image defogging method of the present application (the image defogging model in this method is GridNet), respectively. Among them, the image processed by MSBDN is as shown by b in Figure 4 , the image processed by the image defogging method of the present application (the image defogging model in this method is MSBDN) is as shown by c in Figure 4 , the image processed by GridNet is as shown by d in Figure 4As shown in FIG. d, the image processed by the image defogging method of the present application (the image defogging model in the method is GridNet) is as shown in FIG. d. Dehazing method As shown in FIG. e.

[0087] In this example, the effect of the image defogging method of the embodiments of the present application is also illustrated using the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM), the PSNR is the most common and most widely used objective quality evaluation method for evaluating image quality, the larger the value, the better the quality of the output image, the SSIM is an index for measuring the similarity between two images, which can better represent the visual perception, the larger the value, the closer the output image to the real image, and the better the image quality effect. The PSNR and SSIM values of MSBDN, the image defogging method of the present application (the image defogging model in the method is MSBDN), GridNet, and the image defogging method of the present application (the image defogging model in the method is GridNet) are shown in Table 1. It should be noted that MSBDN+ self-supervised in Table 1 represents the image defogging method of the present application (the image defogging model in the method is MSBDN), and GridNet+ self-supervised represents the image defogging method of the present application (the image defogging model in the method is GridNet).

[0088] PSNR SSIM MSBDN MSBDN + self-supervision 11.91 0.4786 GridNet 13.41 0.5824 GridNet + self-supervision 13.13 0.6020 Figure 4 16.29 0.6688

[0089] Table 1

[0090] From Figure 5 and Table 1, it can be seen that the defogging effect of the image defogging method provided by the embodiments of the present application is obviously better than that of the MSBDN and GridNet methods.

[0091] The image defogging device based on self-supervised learning provided by the present application will be described below in conjunction with specific embodiments.

[0092] As Figure 6 shown, the embodiments of the present application provide an image defogging device based on self-supervised learning, which comprises:

[0093] The acquisition module 501 is configured to acquire a plurality of clear and fog-free images.

[0094] The processing module 502 is configured to process each clear and fog-free image according to a preset plurality of image processing angles using an atmospheric scattering model to obtain a plurality of foggy images of the clear and fog-free image, wherein the plurality of image processing angles correspond to the plurality of foggy images one by one.

[0095] Dataset module 503 is used to take each foggy image and the corresponding clear, fog-free image as an image pair to obtain multiple image pairs.

[0096] Training module 504 is used to train the image dehazing model using multiple image pairs to obtain the trained image dehazing model;

[0097] The dehazing module 505 is used to perform dehazing processing on the image to be dehazed using the trained image dehazing model.

[0098] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0100] like Figure 6 As shown, embodiments of this application provide a terminal device, such as... Figure 6 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( ​ The diagram shows only one processor, a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 executes the computer program D102 to implement the steps in any of the above method embodiments.

[0101] Specifically, the processor D100 executes the computer program D102, processes multiple clear and non-fog images by using an atmospheric scattering model to obtain a fog image corresponding to each clear and non-fog image, takes the clear and non-fog image and the corresponding fog image as an image pair, trains an image defogging model by using the image pair, and finally performs defogging processing on a to-be-defogged image by using the trained image defogging model. Since the paired image with artificial label information does not need to be input during training of the image defogging model, the training of the image defogging model can be completed by using the clear and non-fog image itself, so that the training period of the image defogging model is greatly shortened, and the efficiency of image defogging is greatly improved.

[0102] The processor D100 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0103] The memory D101 can be an internal storage unit of the terminal device D10 in some embodiments, for example, a hard disk or a memory of the terminal device D10. The memory D101 can also be an external storage device of the terminal device D10 in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. Further, the memory D101 can include both the internal storage unit and the external storage device of the terminal device D10. The memory D101 is used to store an operating system, application programs, a boot loader, data, and other programs, for example, program codes of the computer program, and the like. The memory D101 can also be used to temporarily store data that has been output or is to be output.

[0104] It should be noted that the information interaction and execution process between the above apparatuses / units are based on the same concept as the method embodiments, and the specific functions and technical effects thereof can be referred to the method embodiments, which will not be described here.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0106] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in each method embodiment described above.

[0107] The embodiment of the present application provides a computer program product, when the computer program product runs on a terminal device, so that the terminal device executes to realize the steps in each method embodiment described above.

[0108] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the embodiment of the present application realizes all or part of the processes in the above method, which can be completed by a computer program to instruct related hardware, and the computer program can be stored in a computer readable storage medium. The computer program is executed by the processor to realize the steps in each method embodiment described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the image defogging device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0109] In the above embodiments, the description of each embodiment focuses on different aspects, and the parts not described or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0110] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0111] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0112] The units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0113] The above described embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An image defogging method based on self-supervised learning, characterized in that, The method comprises the following steps: obtaining a plurality of clear and non-foggy images; for each of the clear and non-foggy images, processing the clear and non-foggy image according to a plurality of preset image processing angles by using an atmospheric scattering model to obtain a plurality of foggy images corresponding to the clear and non-foggy image; the plurality of image processing angles correspond to the plurality of foggy images one by one; the plurality of foggy images are foggy images at different angles, and the plurality of foggy images corresponding to the clear and non-foggy image can simulate any fog distribution; the plurality of image processing angles include: from the bottom to the top of the clear and non-foggy image, from the top to the bottom of the clear and non-foggy image, from the left to the right of the clear and non-foggy image, from the right to the left of the clear and non-foggy image, from the lower left corner to the upper right corner of the clear and non-foggy image, from the upper right corner to the lower left corner of the clear and non-foggy image, from the upper left corner to the lower right corner of the clear and non-foggy image, and from the lower right corner to the upper left corner of the clear and non-foggy image; for each of the foggy images, taking the foggy image and the clear and non-foggy image corresponding to the foggy image as an image pair to obtain a plurality of image pairs; training an image defogging model by using the plurality of image pairs to obtain a trained image defogging model; performing defogging processing on a to-be-defogged image by using the trained image defogging model; the transmission map of the atmospheric scattering model is: wherein t(x) represents the transmission map of the atmospheric scattering model, x represents the position of a pixel in the clear and non-foggy image, k represents a dissipation coefficient, the value range of k is 2 to 4, t0(x) represents an initialized transmission map, the pixel value of the initialized transmission map is 1, p represents an adjustment coefficient, the value range of p is 1 to 1.6, and d represents a bias coefficient, the value range of d is 0 to 0.

15.

2. The image defogging method of claim 1, wherein, Before the clear and non-foggy image is processed according to the preset image processing angle by using the atmospheric scattering model, the image defogging method further comprises the following steps: selecting a value as the value of the dissipation coefficient from the value range of the dissipation coefficient by using a random function; selecting a value as the value of the adjustment coefficient from the value range of the adjustment coefficient by using a random function; selecting a value as the value of the bias coefficient from the value range of the bias coefficient by using a random function.

3. The image defogging method of claim 1, wherein, The value range of the atmospheric light value in the atmospheric scattering model is 0.6 to 1.

4. The method of claim 3, wherein, Before the clear and non-foggy image is processed according to the preset image processing angle by using the atmospheric scattering model, the image defogging method further comprises the following step: selecting a value as the value of the atmospheric light value from the value range of the atmospheric light value by using a random function.

5. An image defogging device based on self-supervised learning, characterized in that, The method comprises the following steps: an obtaining module is configured to obtain a plurality of clear and non-foggy images; The processing module is configured to process the clear haze-free image according to a preset image processing angle by using an atmospheric scattering model, to obtain a haze image corresponding to the clear haze-free image, for each of the clear haze-free images. The data set module is configured to obtain a plurality of image pairs by taking the haze image and the clear haze-free image corresponding to the haze image as an image pair, for each of the haze images. The training module is configured to train an image defogging model by using the plurality of image pairs, to obtain a trained image defogging model. The defogging module is configured to perform defogging processing on a to-be-defogged image by using the trained image defogging model. The transmission map of the atmospheric scattering model is as follows: wherein t(x) represents the transmission map of the atmospheric scattering model, x represents the position of a pixel in the clear haze-free image, k represents a dissipation coefficient, the value range of k is 2 to 4, t0(x) represents an initialized transmission map, the pixel value of the initialized transmission map is 1, p represents an adjustment coefficient, the value range of p is 1 to 1.6, and d represents a bias coefficient, the value range of d is 0 to 0.

15.

6. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the image defogging method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to implement the image defogging method according to any one of claims 1 to 4.

Images