Image defogging processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image defogging processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-defogged image which is a video frame image obtained by intercepting an underground monitoring video collected in a mine; a pre-trained defogging processing model is called, the to-be-defogged image is input into the defogging processing model, a fogless image matched with the to-be-defogged image and output by the defogging processing model is obtained, the defogging processing model is obtained through training of a first training sample data set, and the defogging processing model is obtained through training of a second training sample data set; the first training sample data set comprises a plurality of pieces of first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and generated historical underground fog video frame images matched with the historical underground fog-free video frame images. According to the invention, the sharpness processing of the foggy video frame image is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of fully mechanized mining working faces, and in particular to an image defogging method, device, electronic equipment and storage medium. Background Art

[0002] At present, in response to complex working face scenarios in mines, an intelligent video monitoring system for working faces based on computer vision perception technology has been built. This system has realized automatic detection and identification of some key parameters related to equipment, surrounding rock, and environmental conditions in the mining area, as well as intelligent monitoring of abnormal conditions. To a certain extent, it has innovated the technical means of mining area video monitoring and improved the level of visual intelligent management and control of working faces.

[0003] However, due to the normal influence of coal dust, fog, irregular light sources, and visual obstruction on the mining site, the monitoring video images are blurred and the remote visual monitoring effect of key equipment is still not ideal.

[0004] Therefore, finding a method that can effectively defog video frame images obtained in mines has become a current research hotspot. Summary of the Invention

[0005] The present invention provides an image defogging method, device, electronic device and storage medium, which can effectively defog video frame images acquired in mines, thereby obtaining a fog-free image that matches the image to be defogged, and realizing the clarity processing of foggy video frame images.

[0006] The present invention provides an image defogging method, which includes: obtaining an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; calling a pre-trained defogging processing model, and inputting the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, and the first training sample data set includes multiple first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and generated historical underground foggy video frame images that match the historical underground fog-free video frame images.

[0007] According to an image defogging processing method provided by the present invention, the first training sample data set is established in the following manner: a plurality of historical underground fog-free video frame images are obtained based on historical underground monitoring video intercepts collected from the mine; image depth values ​​corresponding to each of the historical underground fog-free video frame images are obtained respectively; an atmospheric scattering coefficient and an atmospheric light value are obtained, and based on the atmospheric scattering coefficient, the atmospheric light value, the image depth value, and the historical underground fog-free video frame image, a historical underground foggy video frame image that matches the historical underground fog-free video frame image is generated; based on each of the historical underground fog-free video frame images and the generated historical underground foggy video frame image that matches each of the historical underground fog-free video frame images, the first training sample data set is constructed.

[0008] According to an image defogging method provided by the present invention, after obtaining the image to be defogged, the method further includes: obtaining a pre-trained optimized defogging processing model, and using the optimized defogging processing model as the defogging processing model, wherein the optimized defogging processing model is obtained by training a second training sample data set; calling the pre-trained defogging processing model and inputting the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, specifically including: calling the pre-trained optimized defogging processing model, and inputting the image to be defogged and a video frame image that is an adjacent video frame to the image to be defogged into the optimized defogging processing model in parallel, to obtain a fog-free image output by the optimized defogging processing model that matches the image to be defogged.

[0009] According to an image defogging method provided by the present invention, the optimized defogging processing model includes an encoder, an inter-frame information compensation module, and a decoder; the optimized defogging processing model adopts the following method to output a fog-free image that matches the image to be defogged: based on the encoder, encoding feature processing is performed on the image to be defogged and the video frame image that is an adjacent video frame to the image to be defogged, respectively, to obtain a first encoding feature of the image to be defogged and a second encoding feature of the video frame image that is an adjacent video frame to the image to be defogged; based on the inter-frame information compensation module, the first encoding feature and the second encoding feature are superimposed and fused to obtain a fused encoding feature; based on the decoder, the fused encoding feature is decoded to obtain a fog-free image that matches the image to be defogged.

[0010] According to an image defogging processing method provided by the present invention, the second training sample data set is obtained in the following manner: obtaining a single foggy video frame image training sample data, foggy video frame image training sample data with adjacent video frames to the foggy video frame image training sample data, and a single fog-free video frame image training sample data that matches the single foggy video frame image training sample data; constructing the second training sample data based on the single foggy video frame image training sample data, the foggy video frame image training sample data with adjacent video frames to the foggy video frame image training sample data, and the single fog-free video frame image training sample data that matches the single foggy video frame image training sample data; and obtaining the second training sample data set based on multiple groups of the second training sample data.

[0011] According to an image defogging processing method provided by the present invention, the foggy video frame image training sample data is obtained in the following manner: obtaining a single image depth value corresponding to the fog-free video frame image training sample data, and randomly obtaining different atmospheric scattering coefficients and atmospheric light values, wherein the fog-free video frame image training sample data is obtained by intercepting historical underground monitoring videos collected in a mine; based on a single image depth value corresponding to the fog-free video frame image training sample data, and randomly obtaining multiple atmospheric scattering coefficients and atmospheric light values, a plurality of foggy video frame image training sample data that match the single fog-free video frame image training sample data are obtained.

[0012] The present invention also provides an image defogging processing device, which includes: an acquisition module for acquiring an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; a processing module for calling a pre-trained defogging processing model, and inputting the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, and the first training sample data set includes multiple first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and generated historical underground foggy video frame images that match the historical underground fog-free video frame images.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image defogging method described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described image defogging methods.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned image defogging methods.

[0016] The image defogging method, device, electronic device, and storage medium provided by the present invention obtain an image to be defogged, wherein the image to be defogged is a video frame image captured from underground surveillance video collected in a mine; call a pre-trained defogging processing model, and input the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained using a first training sample data set, the first training sample data set including a plurality of first training sample data, the first training sample data being composed of historical underground fog-free video frame images and generated historical underground foggy video frame images that match the historical underground fog-free video frame images. Since the first training sample data is determined based on the historical underground fog-free video frame images and the historical underground foggy video frame images that match the historical underground fog-free video frame images, and the defogging processing model is then trained, the accuracy of the trained defogging processing model can be ensured, and the video frame images obtained in the mine can be effectively defogged, thereby obtaining a fog-free image that matches the image to be defogged, thereby achieving clarity processing of the foggy video frame images. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is one of the flow charts of the image defogging method provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the process of obtaining first training sample data provided by the present invention.

[0020] Figure 3 This is the second flow chart of the image defogging method provided by the present invention.

[0021] Figure 4 It is a flow chart of outputting a fog-free image that matches the image to be defogged based on the optimized defogging processing model provided by the present invention.

[0022] Figure 5 It is a schematic diagram of the process of obtaining the second training sample data provided by the present invention.

[0023] Figure 6 It is a structural schematic diagram of the image defogging processing device provided by the present invention.

[0024] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0026] The image defogging method provided by the present invention addresses the problem of blurred video footage caused by dust and fog in working face scenes, and can achieve clearer processing of surveillance video images. It should be noted that the image defogging method provided by the present invention can be applied to underground coal mine working faces. The image defogging method can be used to perform corresponding processing on videos of underground coal mine working faces. Figure 1 This is one of the flow charts of the image defogging method provided by the present invention.

[0027] The following will be combined Figure 1 The process of the image defogging method provided by the present invention is described.

[0028] In an exemplary embodiment of the present invention, Figure 1 It can be seen that the image defogging method may include step 110 and step 120, and each step will be introduced below.

[0029] In step 110 , an image to be defogged is obtained, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine.

[0030] In step 120, a pre-trained defogging processing model is called, and the image to be defogged is input into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, and the first training sample data set includes multiple first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and historical underground foggy video frame images generated to match the historical underground fog-free video frame images.

[0031] In one embodiment, an image to be defogged may be obtained, wherein the image to be defogged may be a video frame image captured from an underground monitoring video collected in a mine.

[0032] Furthermore, a pre-trained defogging processing model can be called, and the image to be defogged is input into the defogging processing model, so that a fog-free image output by the defogging processing model that matches the image to be defogged can be obtained.

[0033] It should be noted that in order to improve the accuracy of the fog-free image output by the defogging processing model that matches the image to be defogged, it is necessary to ensure the model accuracy of the defogging processing model trained based on the first training sample data set. Since the accuracy of the model is related to the rationality of the first training sample data set, in this embodiment, the defogging processing model is trained based on the first training sample data set, wherein the first training sample data set includes a plurality of first training sample data. Since the first training sample data is composed of historical downhole fog-free video frame images and generated historical downhole foggy video frame images that match the historical downhole fog-free video frame images, the practical applicability of the first training sample data set is ensured, thereby ensuring the model accuracy and practical applicability of the defogging processing model trained based on the first training sample data set.

[0034] The image defogging method provided by the present invention obtains an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; calls a pre-trained defogging processing model, and inputs the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained using a first training sample data set, the first training sample data set including a plurality of first training sample data, the first training sample data being composed of historical underground fog-free video frame images and generated historical underground foggy video frame images that match the historical underground fog-free video frame images. Since the first training sample data is determined based on the historical underground fog-free video frame images and the historical underground foggy video frame images that match the historical underground fog-free video frame images, and then the defogging processing model is trained, the accuracy of the trained defogging processing model can be ensured, and the video frame images obtained in the mine can be effectively defogged, thereby obtaining a fog-free image that matches the image to be defogged, thereby achieving clarity processing of the foggy video frame images.

[0035] Figure 2 It is a schematic diagram of the process of obtaining the first training sample data set provided by the present invention.

[0036] The following will be combined Figure 2 The process of obtaining the first training sample data set provided by the present invention is described.

[0037] In an exemplary embodiment of the present invention, Figure 2It can be seen that obtaining the first training sample data set may include steps 210 to 240, and each step will be introduced below.

[0038] In step 210, a plurality of historical underground fog-free video frame images are obtained based on historical underground monitoring video captured from the mine; In step 220, image depth values ​​corresponding to each historical downhole fog-free video frame image are respectively obtained; In step 230, an atmospheric scattering coefficient and an atmospheric light value are obtained, and based on the atmospheric scattering coefficient, the atmospheric light value, the image depth value, and the historical downhole fog-free video frame image, a historical downhole foggy video frame image is generated that matches the historical downhole fog-free video frame image; In step 240 , a first training sample data set is constructed based on each historical underground fog-free video frame image and the generated historical underground foggy video frame image that matches each historical underground fog-free video frame image.

[0039] Since the deep learning method based on matching image pairs strictly requires that the backgrounds of foggy images and fog-free images are completely consistent, this is almost impossible to meet in actual scenarios. In order to solve this problem and thereby ensure the accuracy of the defogging processing model trained based on the first training sample data set, in one example, a plurality of historical underground fog-free video frame images are obtained based on the historical underground monitoring video captured from the mine. In other words, a plurality of historical underground fog-free video frame images in real application scenarios can be obtained. Furthermore, the image depth values ​​corresponding to each historical underground fog-free video frame image are obtained respectively. In one example, the DinoV2 visual large model can be used to obtain a depth map, that is, the image depth value d(x) value corresponding to each historical underground fog-free video frame image is obtained.

[0040] In another embodiment, the atmospheric scattering coefficient β and the atmospheric light value A can be obtained, and based on the atmospheric scattering coefficient β, the atmospheric light value A, the image depth value d(x), and the historical downhole fog-free video frame image J(x), the historical downhole foggy video frame image I(x) that matches the historical downhole fog-free video frame image can be obtained. In another example, the historical downhole foggy video frame image I(x) that matches the historical downhole fog-free video frame image can be obtained according to the following formulas (1) and (2): (1) (2) where t(x) is the transmittance of the medium.

[0041] In another embodiment, a first training sample data set can be obtained based on each historical underground fog-free video frame image and the generated historical underground foggy video frame image that matches each historical underground fog-free video frame image. In this embodiment, each historical underground fog-free video frame image is data from a real application scenario and has practical applicability, and the historical underground foggy video frame image is obtained based on the matching calculation of the historical underground fog-free video frame image, and also has practical applicability and matching, thereby ensuring that the first training sample data set has practical applicability and matching. Then, based on the first training sample data set, a defogging processing model is trained to ensure the accuracy of the trained defogging processing model, and then the video frame images obtained in the mine can be effectively defogged, so that a fog-free image that matches the image to be defogged can be obtained, and the foggy video frame image is cleared.

[0042] Figure 3 This is the second flow chart of the image defogging method provided by the present invention.

[0043] In order to further introduce the image defogging method provided by the present invention, the following Figure 3 Provide explanation.

[0044] In an exemplary embodiment of the present invention, Figure 3 It can be seen that after obtaining the image to be defogged, the image defogging processing method may further include the following steps 310 and 320: In step 310, a pre-trained optimized defogging processing model is obtained and used as the defogging processing model, wherein the optimized defogging processing model is obtained by training the second training sample data set.

[0045] In step 320, a pre-trained optimized defogging processing model is called, and the image to be defogged and the video frame image that is adjacent to the image to be defogged are input into the optimized defogging processing model in parallel to obtain a fog-free image that matches the image to be defogged and is output by the optimized defogging processing model.

[0046] In order to solve the problem of abrupt changes such as obvious light and dark flickering between defogging video image frames generated by a single-frame image defogging method when processing video defogging tasks, in this embodiment, an image defogging processing method that integrates the temporal and spatial features of the video is proposed.

[0047] In one embodiment, a pre-trained optimized defogging processing model can be obtained and used as the defogging processing model. Further, the pre-trained optimized defogging processing model is called, and the image to be defogged and video frames that are adjacent to the image to be defogged are input into the optimized defogging processing model in parallel, thereby obtaining a haze-free image that matches the image to be defogged and is output by the optimized defogging processing model.

[0048] In this embodiment, the input of the optimized defogging processing model is several consecutive frames, such as 3 frames of foggy video frame images. When defogging one of the foggy video frame images, the characteristics of the foggy video frame images of other frames are also considered, so that a fog-free image that matches the image to be defogged can be obtained without abrupt changes such as light and dark flickering.

[0049] In another example, the optimized dehazing processing model is obtained by training with a second training sample data set, wherein the second training sample data set will be described below and will not be repeated here.

[0050] Figure 4 It is a flow chart of outputting a fog-free image that matches the image to be defogged based on the optimized defogging processing model provided by the present invention.

[0051] The following will be combined Figure 4 The process of outputting a haze-free image that matches the image to be dehazed based on an optimized dehazing processing model is described.

[0052] In an exemplary embodiment of the present invention, the optimized defogging processing model may include an encoder, an inter-frame information compensation module, and a decoder; Figure 4 It can be seen that outputting a fog-free image that matches the image to be defogged based on the optimized defogging processing model may include steps 410 to 430, and each step will be described below.

[0053] In step 410, based on the encoder, encoding feature processing is performed on the image to be dehazed and the video frame image that is an adjacent video frame to the image to be dehazed, respectively, to obtain a first encoding feature of the image to be dehazed and a second encoding feature of the video frame image that is an adjacent video frame to the image to be dehazed.

[0054] In step 420, based on the inter-frame information compensation module, the first coding feature and the second coding feature are superimposed and fused to obtain a fused coding feature.

[0055] In step 430 , the fused encoded features are decoded based on the decoder to obtain a haze-free image that matches the image to be dehazed.

[0056] In one embodiment, three consecutive foggy images can be obtained, that is, the image to be defogged and the video frame image that is adjacent to the image to be defogged are obtained, wherein the image to be defogged and the video frame image that is adjacent to the image to be defogged can be expressed as F t 、F t-1 、F t-2 Furthermore, based on the encoder, the defogging image and the video frame image that is adjacent to the image to be defogging are respectively processed, for example, F t、F t-1 、F t-2 By performing coding feature processing, coding features of three frames of foggy images can be obtained, that is, a first coding feature of the image to be defogged and a second coding feature of a video frame image that is an adjacent video frame to the image to be defogged.

[0057] Furthermore, based on the inter-frame information compensation module, the first coding feature, that is, F t The coding feature and the second coding feature, that is, F t-1 、F t-2 The coded features are superimposed and fused to obtain fused coded features. The fused coded features are then decoded using a decoder, thereby predicting a haze-free image at time t that matches the image to be defogged. This embodiment effectively addresses the issue of abrupt changes such as noticeable light and dark flickering between frames in defogged video images generated by single-frame image defogging methods when processing video defogging tasks. This ensures that the final output haze-free image that matches the image to be defogged does not exhibit such abrupt changes.

[0058] Figure 5 It is a schematic diagram of the process of obtaining the second training sample data provided by the present invention.

[0059] The following will be combined Figure 5 The process of obtaining the second training sample data is described.

[0060] In an exemplary embodiment of the present invention, Figure 5 It can be seen that obtaining the second training sample data set may include steps 510 to 530, and each step will be described below.

[0061] In step 510, single foggy video frame image training sample data, foggy video frame image training sample data having adjacent video frames to the foggy video frame image training sample data, and single fog-free video frame image training sample data matching the single foggy video frame image training sample data are obtained.

[0062] In step 520, second training sample data is constructed based on the single foggy video frame image training sample data, the foggy video frame image training sample data whose video frames are adjacent to the foggy video frame image training sample data, and the single fog-free video frame image training sample data that matches the single foggy video frame image training sample data.

[0063] In step 530 , a second training sample data set is obtained based on multiple sets of second training sample data.

[0064] In one embodiment, a single foggy video frame image training sample data, foggy video frame image training sample data with adjacent video frames to the foggy video frame image training sample data, and a single fog-free video frame image training sample data that matches the single foggy video frame image training sample data can be obtained. Furthermore, second training sample data is constructed based on the single foggy video frame image training sample data, the foggy video frame image training sample data with adjacent video frames to the foggy video frame image training sample data, and the single fog-free video frame image training sample data that matches the single foggy video frame image training sample data. A second training sample dataset is then obtained based on multiple sets of second training sample data.

[0065] In this embodiment, the second training sample data includes multiple frames of continuous foggy video frame image training sample data and a single fog-free video frame image training sample data corresponding to one of the frames, so that the optimized defogging processing model can be trained based on the structure of the second training sample data to ensure that the trained optimized defogging processing model can solve the problem of abrupt changes such as obvious light and dark flickering between the defogged video image frames generated when processing the video defogging task based on the single-frame image defogging method.

[0066] In another exemplary embodiment of the present invention, continuing with the above Figure 5 Taking the above embodiment as an example, foggy video frame image training sample data can be obtained in the following way: Obtain a single image depth value corresponding to a fog-free video frame image training sample data, and randomly obtain different atmospheric scattering coefficients and atmospheric light values, wherein the fog-free video frame image training sample data is obtained by intercepting historical underground monitoring videos collected in the mine; Based on a single image depth value corresponding to fog-free video frame image training sample data and a plurality of randomly obtained atmospheric scattering coefficients and atmospheric light values, a plurality of foggy video frame image training sample data matching the single fog-free video frame image training sample data are obtained.

[0067] In one embodiment, in order to improve the enhancement processing effect of the training sample data, a plurality of foggy video frame image training sample data that matches the single fog-free video frame image training sample data can be generated by formulas (1) and (2). Thus, a plurality of foggy video frame image training sample data can be obtained on the basis of a small amount of fog-free video frame image training sample data, and further a plurality of second training sample data can be obtained, thereby achieving the enhancement effect of the training sample data.

[0068] In another example, by randomly transforming different atmospheric scattering coefficients and atmospheric light values ​​and combining formulas (1) and (2), multiple foggy video frame image training sample data that match a single fog-free video frame image training sample data can be generated.

[0069] According to the foregoing description, the image defogging method provided by the present invention obtains an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; calls a pre-trained defogging processing model, and inputs the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, the first training sample data set including a plurality of first training sample data, the first training sample data being composed of historical underground fog-free video frame images and generated historical underground foggy video frame images that match the historical underground fog-free video frame images. Since the first training sample data is determined based on the historical underground fog-free video frame images and the historical underground foggy video frame images that match the historical underground fog-free video frame images, and then the defogging processing model is trained, the accuracy of the trained defogging processing model can be ensured, and the video frame images obtained in the mine can be effectively defogged, thereby obtaining a fog-free image that matches the image to be defogged, and achieving clarity processing of the foggy video frame images.

[0070] The image defogging apparatus provided by the present invention is described below. The image defogging apparatus described below and the image defogging method described above can be referred to in correspondence with each other.

[0071] Figure 6 It is a structural schematic diagram of the image defogging processing device provided by the present invention.

[0072] The following will be combined Figure 6 The structure of the image defogging device is described.

[0073] In an exemplary embodiment of the present invention, Figure 6 It can be seen that the image defogging processing device may include an acquisition module 610 and a processing module 620. Each module will be introduced below.

[0074] The acquisition module 610 may be configured to acquire an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; The processing module 620 can be configured to call a pre-trained defogging processing model and input the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, and the first training sample data set includes multiple first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and historical underground foggy video frame images generated to match the historical underground fog-free video frame images.

[0075] In an exemplary embodiment of the present invention, the processing module 620 may obtain the first training sample data set in the following manner: A plurality of historical underground fog-free video frame images are obtained based on historical underground video intercepts collected from the mine; respectively obtaining image depth values ​​corresponding to the respective historical downhole fog-free video frame images; Obtaining an atmospheric scattering coefficient and an atmospheric light value, and generating a historical underground foggy video frame image that matches the historical underground fog-free video frame image based on the atmospheric scattering coefficient, the atmospheric light value, the image depth value, and the historical underground fog-free video frame image; The first training sample data set is obtained based on each of the historical underground fog-free video frame images and the generated historical underground foggy video frame images that match each of the historical underground fog-free video frame images.

[0076] In an exemplary embodiment of the present invention, the processing module 620 may also be configured to: Obtaining a pre-trained optimized defogging processing model, and using the optimized defogging processing model as the defogging processing model, wherein the optimized defogging processing model is obtained by training the second training sample data set; The processing module 620 may also use the following method to call a pre-trained defogging processing model, and input the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged: The pre-trained optimized defogging processing model is called, and the image to be defogged and the video frame image that is an adjacent video frame to the image to be defogged are input into the optimized defogging processing model in parallel to obtain a fog-free image output by the optimized defogging processing model that matches the image to be defogged.

[0077] In an exemplary embodiment of the present invention, the optimized defogging processing model includes an encoder, an inter-frame information compensation module, and a decoder; The processing module 620 may output a fog-free image that matches the image to be defogged based on the optimized defogging processing model in the following manner: Based on the encoder, encoding feature processing is performed on the image to be dehazed and a video frame image that is an adjacent video frame to the image to be dehazed, respectively, to obtain a first encoding feature of the image to be dehazed and a second encoding feature of the video frame image that is an adjacent video frame to the image to be dehazed; Based on the inter-frame information compensation module, the first coding feature and the second coding feature are superimposed and fused to obtain a fused coding feature; Based on the decoder, the fused encoded features are decoded to obtain a haze-free image that matches the image to be dehazed.

[0078] In an exemplary embodiment of the present invention, the processing module 620 may obtain the second training sample data set in the following manner: Acquire single foggy video frame image training sample data, foggy video frame image training sample data whose video frames are adjacent to the foggy video frame image training sample data, and single fog-free video frame image training sample data that matches the single foggy video frame image training sample data; Constructing second training sample data based on the single foggy video frame image training sample data, foggy video frame image training sample data having adjacent video frames to the foggy video frame image training sample data, and single fog-free video frame image training sample data matching the single foggy video frame image training sample data; Based on multiple sets of the second training sample data, the second training sample data set is obtained.

[0079] In an exemplary embodiment of the present invention, the processing module 620 may obtain foggy video frame image training sample data in the following manner: Obtaining a single image depth value corresponding to the fog-free video frame image training sample data, and randomly obtaining different atmospheric scattering coefficients and atmospheric light values, wherein the fog-free video frame image training sample data is obtained by intercepting historical underground monitoring videos collected in a mine; Based on a single image depth value corresponding to the fog-free video frame image training sample data and a plurality of randomly acquired atmospheric scattering coefficients and atmospheric light values, a plurality of foggy video frame image training sample data matching the single fog-free video frame image training sample data are obtained.

[0080] Figure 7 An example of a physical structure diagram of an electronic device is shown below. Figure 7As shown, the electronic device may include: a processor 70, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call logic instructions in the memory 730 to execute an image defogging method, which includes: obtaining an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; calling a pre-trained defogging processing model and inputting the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained using a first training sample data set, wherein the first training sample data set includes multiple first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and historical underground foggy video frame images generated to match the historical underground fog-free video frame images.

[0081] Furthermore, the logic instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0082] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image defogging processing method provided by the above methods, the method including: obtaining an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; calling a pre-trained defogging processing model, and inputting the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, and the first training sample data set includes multiple first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and generated historical underground foggy video frame images that match the historical underground fog-free video frame images.

[0083] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the image defogging processing method provided by the above-mentioned methods, the method comprising: obtaining an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; calling a pre-trained defogging processing model, and inputting the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, the first training sample data set including multiple first training sample data, the first training sample data being generated based on historical underground fog-free video frame images, and historical underground foggy video frame images generated to match the historical underground fog-free video frame images.

[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0085] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An image defogging method, characterized in that: The method comprises: Acquire an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; A pre-trained defogging processing model is called, and the image to be defogged is input into the defogging processing model to obtain a fog-free image output by the defogging processing model and matching the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, and the first training sample data set includes a plurality of first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and historical underground foggy video frame images generated to match the historical underground fog-free video frame images.

2. The image defogging method according to claim 1, characterized in that: The first training sample data set is established in the following manner: A plurality of historical underground fog-free video frame images are obtained based on the historical underground monitoring video captured from the mine; Respectively obtain image depth values ​​corresponding to each of the historical underground fog-free video frame images; Acquire an atmospheric scattering coefficient and an atmospheric light value, and generate a historical underground foggy video frame image that matches the historical underground foggy video frame image based on the atmospheric scattering coefficient, the atmospheric light value, the image depth value, and the historical underground fog-free video frame image; The first training sample data set is formed based on each of the historical underground fog-free video frame images and the generated historical underground foggy video frame images that match each of the historical underground fog-free video frame images.

3. The image defogging method according to claim 1, characterized in that: After obtaining the image to be defogged, the method further includes: Obtaining a pre-trained optimized defogging processing model, and using the optimized defogging processing model as the defogging processing model, wherein the optimized defogging processing model is obtained by training with a second training sample data set; The calling of a pre-trained defogging processing model, and inputting the image to be defogged into the defogging processing model, to obtain a non-fogging image output by the defogging processing model and matching the image to be defogged, specifically includes: The pre-trained optimized defogging processing model is called, and the image to be defogged and the video frame image which is an adjacent video frame to the image to be defogged are input into the optimized defogging processing model in parallel, so as to obtain a fog-free image which is output by the optimized defogging processing model and matches the image to be defogged.

4. The image defogging method according to claim 3, characterized in that: The optimized defogging processing model includes an encoder, an inter-frame information compensation module, and a decoder; The optimized defogging processing model outputs a defogging image that matches the image to be defogged in the following manner: Based on the encoder, respectively perform encoding feature processing on the image to be defogged and the video frame image that is an adjacent video frame to the image to be defogged, to obtain a first encoding feature of the image to be defogged and a second encoding feature of the video frame image that is an adjacent video frame to the image to be defogged; Based on the inter-frame information compensation module, the first coding feature and the second coding feature are superimposed and fused to obtain a fused coding feature; Based on the decoder, the fused encoded features are decoded to obtain a haze-free image that matches the image to be dehazed.

5. The image defogging method according to claim 3 or 4, characterized in that: The second training sample data set is obtained in the following manner: Acquire a single foggy video frame image training sample data, foggy video frame image training sample data whose video frames are adjacent to the foggy video frame image training sample data, and a single fog-free video frame image training sample data matching the single foggy video frame image training sample data; Constructing second training sample data based on the single foggy video frame image training sample data, foggy video frame image training sample data that are adjacent video frames to the foggy video frame image training sample data, and a single fog-free video frame image training sample data that matches the single foggy video frame image training sample data; Based on multiple groups of the second training sample data, the second training sample data set is obtained.

6. The image defogging method according to claim 5, characterized in that: The foggy video frame image training sample data is obtained in the following manner: Acquire a single image depth value corresponding to the fog-free video frame image training sample data, and randomly acquire different atmospheric scattering coefficients and atmospheric light values, wherein the fog-free video frame image training sample data is obtained by intercepting historical underground monitoring videos collected in a mine; Based on a single image depth value corresponding to the fog-free video frame image training sample data and a plurality of randomly acquired atmospheric scattering coefficients and atmospheric light values, a plurality of foggy video frame image training sample data matching the single fog-free video frame image training sample data are obtained.

7. An image defogging device, characterized in that: The device comprises: An acquisition module, used for acquiring an image to be defogged, wherein the image to be defogged is a video frame image captured from an underground monitoring video collected in a mine; A processing module is used to call a pre-trained defogging processing model, and input the image to be defogged into the defogging processing model to obtain a fog-free image output by the defogging processing model that matches the image to be defogged, wherein the defogging processing model is trained by a first training sample data set, and the first training sample data set includes multiple first training sample data, and the first training sample data is composed of historical underground fog-free video frame images and historical underground foggy video frame images generated to match the historical underground fog-free video frame images.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the image defogging method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image defogging method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the image defogging method according to any one of claims 1 to 6 is implemented.

Images