Image defogging method and system based on attention mechanism network enhanced by optical flow dynamic characteristics

Abstract

This application discloses an image dehazing method and system based on an optical flow dynamic feature enhanced attention mechanism network, belonging to the field of computer vision technology. The method includes: performing a first preprocessing on multiple surveillance video segments to obtain multiple preprocessed surveillance video segments; constructing a training set and a test set, and performing a second preprocessing on the training set to obtain a preprocessed training set; performing fogging processing on the multiple preprocessed surveillance video segments and constructing a fogged image stream set; constructing an optical flow dynamic feature enhanced attention mechanism network and a composite loss function, training and testing the optical flow dynamic feature enhanced attention mechanism network, inputting the image to be dehazed into the optical flow dynamic feature enhanced attention mechanism network, and outputting a dehazed image; processing the dehazed image according to a progressive optimization algorithm to obtain the target dehazed image. This method can efficiently dehaze surveillance videos in real-time or near real-time, helping to improve the monitoring security of surveillance scenes.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, specifically relating to an image dehazing method and system based on an optical flow dynamic feature-enhanced attention mechanism network. Background Technology

[0002] Outdoor surveillance scenarios often involve complex environmental conditions, such as atmospheric fog, haze, rain, and snow, which can significantly impact the clarity and accuracy of surveillance videos. To improve the efficiency and security of surveillance systems, dehazing technology has become an important research area. Dehazing technology not only enhances video quality but also effectively extracts key scene features, assisting in automated monitoring and judgment.

[0003] Currently, existing surveillance videos are generally dehazed using simple dehazing algorithms. However, these simple algorithms can damage the video and produce poor results. Therefore, how to achieve efficient dehazing of surveillance videos in surveillance scenarios is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] The purpose of this application is to provide an image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network, which can solve the technical problem that simple dehazing algorithms in the prior art result in poor dehazing effect for surveillance videos.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide an image dehazing method based on an optical flow dynamic feature-enhanced attention mechanism network, the method comprising:

[0007] Acquire multiple surveillance videos of the monitored scene, perform a first preprocessing on the multiple surveillance videos, and obtain corresponding multiple preprocessed surveillance videos;

[0008] Training and test sets are obtained from the multiple preprocessed surveillance videos. The training set is then subjected to a second preprocessing to obtain a preprocessed training set.

[0009] The multiple pre-processed surveillance videos are subjected to fogging processing to construct a fogged image stream set based on the fogged surveillance videos.

[0010] An optical flow dynamic feature enhanced attention mechanism network is constructed, which includes a background feature extraction module, a color feature extraction module, and a sliding window attention mechanism module.

[0011] A composite loss function is constructed, and the optical flow dynamic feature enhancement attention mechanism network is trained based on the preprocessed training set, the fogged image stream set, and the composite loss function.

[0012] The trained optical flow dynamic feature enhancement attention mechanism network is tested according to the test set. The image to be dehazed is input into the optical flow dynamic feature enhancement attention mechanism network that has passed the test, and the dehazed image is output.

[0013] The dehazed image is processed using a progressive optimization algorithm to obtain the final target dehazed image.

[0014] As an optional implementation of the first aspect of this application, the step of acquiring multiple surveillance video segments of a surveillance scene and performing a first preprocessing on the multiple surveillance video segments to obtain corresponding multiple preprocessed surveillance video segments is as follows:

[0015] Acquire multiple surveillance videos of the monitored scene, and set the same resolution and frame rate for each of the multiple surveillance videos;

[0016] The multiple surveillance video segments are standardized to ensure that each surveillance video segment has the same duration and format, resulting in the multiple pre-processed surveillance video segments.

[0017] As an optional implementation of the first aspect of this application, the step of obtaining a training set and a test set based on the multiple preprocessed surveillance videos, and performing a second preprocessing on the training set to obtain a preprocessed training set, specifically includes:

[0018] Image extraction is performed on each preprocessed surveillance video segment in the multiple preprocessed surveillance videos to obtain each set of real images corresponding to each preprocessed surveillance video.

[0019] For each real image in each real image set, random center fogging is performed to obtain each fogged image set corresponding to each real image set.

[0020] Each fogged image set is divided according to a preset ratio to obtain each training subset and each test subset corresponding to each fogged image set;

[0021] Extract any real image from each real image set to construct a real reference image set; construct a training set based on each training subset and the real reference image set; construct a test set based on each test subset and the real reference image set.

[0022] The training set is image-scaled to make the images in the test set the same size. The scaled training set is then randomly cropped and horizontally flipped to obtain the preprocessed training set.

[0023] As an optional implementation of the first aspect of this application, the step of performing fogging processing on the multiple pre-processed surveillance video segments to construct a fogged image stream set based on the fogged surveillance video segments is as follows:

[0024] The multiple preprocessed surveillance videos are processed using a fog synthesis algorithm based on depth image information to obtain multiple fogged videos.

[0025] N segments of fogging video are randomly selected from the multiple fogging video segments. Continuous image extraction is performed on each of the N fogging video segments to obtain each fogging image stream corresponding to each fogging video.

[0026] The set of fogged image streams is constructed based on each fogged image stream corresponding to each fogged video.

[0027] As an optional implementation of the first aspect of this application, the construction of the composite loss function, and the training of the optical flow dynamic feature enhancement attention mechanism network based on the preprocessed training set, the fogged image stream set, and the composite loss function, specifically involves:

[0028] A composite loss function is constructed, and the preprocessed training set and the fogged image stream set are input into the optical flow dynamic feature enhancement attention mechanism network for training.

[0029] The difference in loss between the training results of the optical flow dynamic feature-enhanced attention mechanism network on the preprocessed training set and the corresponding real images is obtained.

[0030] The composite loss function optimizes the parameters of the optical flow dynamic feature enhancement attention mechanism network based on the loss difference until the optical flow dynamic feature enhancement attention mechanism network converges, thus completing the training of the optical flow dynamic feature enhancement attention mechanism network by the composite loss function.

[0031] As an optional implementation of the first aspect of this application, the step of inputting the preprocessed training set and the fogged image stream set into the optical flow dynamic feature enhancement attention mechanism network for training is specifically as follows:

[0032] The fogged images in the preprocessed training set are input into a 3×3 convolutional layer, and after being processed by a 3×3 convolutional layer and a sliding window attention mechanism module, the first attention feature is obtained.

[0033] The same fogged image is input into the color feature extraction module, and the color features are obtained after processing by the color feature extraction module; the real reference image from the same preprocessed monitoring video as the fogged image in the preprocessed training set is input into the background feature extraction module, and the background features are obtained after processing by the background feature extraction module.

[0034] The first attention feature and the background feature are input into a downsampling layer for feature fusion processing to obtain the first fused feature. The first fused feature is then input into the sliding window attention mechanism module for processing to obtain the second attention feature.

[0035] The second attention feature and the color feature are input into a downsampling layer for feature fusion processing to obtain the second fused feature; the second fused feature is then input into a sliding window attention mechanism module and an upsampling layer for processing to obtain the third attention feature.

[0036] The third attention feature and the second attention feature are input into an SK channel attention fusion module for feature fusion processing to obtain the third fused feature; the third fused feature is then input into the sliding window attention mechanism module for processing to obtain the fourth attention feature.

[0037] The set of fogged image streams is input into an optical flow network for feature extraction to obtain optical flow features. The optical flow features and the fourth attention features are then input into an upsampling layer for feature fusion to obtain the fourth fused features.

[0038] The fourth fusion feature and the first attention feature are input into an SK channel attention fusion module for feature fusion to obtain a fifth fusion feature. The fifth fusion feature is then processed sequentially through a sliding window attention mechanism module and a 3×3 convolutional layer to obtain a dehazed image.

[0039] As an optional implementation of the first aspect of this application, the composite loss function is expressed by the following formula:

[0040]

[0041] L = L1 + SSIM(x,y),

[0042] Where L represents the composite loss function, x(p) represents the pixel value at pixel p in the dehazed image, y(p) represents the pixel value at pixel p in the corresponding ground truth image, and N represents the total number of pixels in the image. p∈P This represents summing over all pixels p in the image, μ x μ represents the pixel mean of the dehazed image. yσ represents the pixel mean of the real image corresponding to the dehazed image. x σ represents the standard deviation of pixel values ​​in a dehazed image. y σ represents the standard deviation of pixel values ​​in the real image corresponding to the dehazed image. xy The table shows the pixel value covariance between the dehazed image and the corresponding real image, where C1 and C2 represent constants.

[0043] The data processing procedure of the sliding window attention mechanism module is represented by the following formula:

[0044]

[0045] Where B represents the relative position deviation term, Q represents the query matrix, K represents the key matrix, V represents the value matrix, ||Q|| represents the L2 norm of Q, ||K|| represents the L2 norm of K, and β represents the learnable scaling factor.

[0046] Where B represents the relative position deviation term, Q represents the query matrix, K represents the key matrix, V represents the value matrix, ||Q|| represents the L2 norm of Q, ||K|| represents the L2 norm of K, and λ represents the learnable scaling factor.

[0047] Secondly, embodiments of this application provide an image dehazing system based on an optical flow dynamic feature-enhanced attention mechanism network, the system comprising:

[0048] First acquisition module: acquires multiple surveillance videos of the monitored scene, performs first preprocessing on the multiple surveillance videos, and obtains corresponding preprocessed surveillance videos;

[0049] The second acquisition module: acquires a training set and a test set based on the multiple preprocessed surveillance videos, and performs a second preprocessing on the training set to obtain a preprocessed training set;

[0050] First processing module: performs fogging processing on the multiple pre-processed surveillance video segments to construct a fogged image stream set based on the fogged surveillance video segments;

[0051] Construction Module: Construct an optical flow dynamic feature enhanced attention mechanism network, which includes a background feature extraction module, a color feature extraction module, and a sliding window attention mechanism module;

[0052] Training module: Constructs a composite loss function and trains the optical flow dynamic feature enhancement attention mechanism network based on the preprocessed training set, the fogged image stream set, and the composite loss function;

[0053] Test module: Test the trained optical flow dynamic feature enhancement attention mechanism network according to the test set, input the image to be dehazed into the optical flow dynamic feature enhancement attention mechanism network that has passed the test, and output the dehazed image;

[0054] The second processing module processes the dehazed image according to a progressive optimization algorithm to obtain the final target dehazed image.

[0055] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0056] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0057] In the embodiments of this application, compared with the prior art, the following beneficial effects are achieved:

[0058] (1) By constructing an optical flow dynamic feature enhanced attention mechanism network, key information in scene surveillance videos, especially optical flow features and color features, can be extracted and utilized more effectively, thereby enhancing the adaptability to foggy environments. The introduction of the composite loss function makes network training more accurate, which can minimize the difference between the defogging result and the real image and improve the defogging effect.

[0059] (2) Perform a first preprocessing on the surveillance video and a second preprocessing on the training set. The first preprocessing includes setting the same resolution and frame rate and standardization. The second preprocessing includes image scaling, random cropping and horizontal flipping, which helps to improve the network's adaptability to different scenarios and monitoring conditions. By performing random center fogging and constructing a fogged image stream set, the diversity and complexity of the training data are increased, further enhancing the network's generalization performance.

[0060] (3) The sliding window attention mechanism module in the optical flow dynamic feature enhancement network can pay more attention to the feature regions with obvious fogging in the input features, thereby achieving a more refined defogging effect; while the synergistic effect of the background feature extraction module and the color feature extraction module enables the network to more accurately identify and process feature elements in the monitoring scene.

[0061] (4) The application of progressive optimization algorithm further improves the quality of dehazed images, making the dehazed images clearer and more natural; while the introduction of the SK channel attention fusion module enhances the network's ability to allocate attention to different feature channels, which helps to improve the detail performance and overall visual effect of dehazed images.

[0062] (5) This method can perform defogging on the monitoring video of the monitoring scene in real time or near real time, which helps to improve the efficiency and accuracy of scene monitoring security; the defogging monitoring video can provide a clearer view, which helps monitoring personnel to discover and deal with potential security risks in a timely manner. Attached Figure Description

[0063] Figure 1 This is a flowchart of an image dehazing method based on an optical flow dynamic feature-enhanced attention mechanism network provided by some embodiments of this application;

[0064] Figure 2 This is a diagram of the optical flow dynamic feature enhanced attention mechanism network structure, which is provided by some embodiments of this application for an image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network.

[0065] Figure 3 This is a structural diagram of the background feature extraction module of an image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network provided by some embodiments of this application;

[0066] Figure 4 This is a structural diagram of the color feature extraction module of an image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network provided by some embodiments of this application;

[0067] Figure 5 These are dehazing effect diagrams of an image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network provided by some embodiments of this application;

[0068] Figure 6 This is a schematic diagram of a monitoring device for an image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network, provided by some embodiments of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0071] The following description, in conjunction with the accompanying drawings, details an image dehazing method and system based on an optical flow dynamic feature-enhanced attention mechanism network provided in this application, through specific embodiments and application scenarios.

[0072] See Figure 1 As shown, an image dehazing method based on an optical flow dynamic feature-enhanced attention mechanism network includes the following steps:

[0073] S100: Acquire multiple surveillance videos of the monitored scene, perform first preprocessing on the multiple surveillance videos, and obtain corresponding preprocessed surveillance videos;

[0074] It should be noted that scene monitoring includes railway operation scenes, airport operation scenes, and construction site work scenes, etc. The monitoring scene in this embodiment is described using the railway operation scene as an example.

[0075] It should be noted that S100 specifically refers to:

[0076] S110: Based on railway monitoring, acquire multiple monitoring videos of railway operation scenarios and set the same resolution and frame rate for each monitoring video in the multiple monitoring videos;

[0077] S120: Standardize multiple surveillance video segments to ensure that each surveillance video segment has the same duration and format, resulting in multiple pre-processed surveillance video segments.

[0078] Furthermore, in S110, multiple surveillance video segments are acquired and processed to become a uniform 1080*720 resolution with 30 frames per second; then in S120, the duration of each surveillance video segment is adjusted to 40 minutes and the format is set to mp4 to obtain multiple pre-processed surveillance video segments, which are then named video(1), video(2), ..., video(n) in sequence.

[0079] S200: Obtain training and test sets from multiple pre-processed surveillance videos, and perform a second preprocessing on the training set to obtain a pre-processed training set;

[0080] It should be noted that the S200 specifically refers to:

[0081] S210: Extract images from each preprocessed surveillance video in multiple preprocessed surveillance videos to obtain a set of real images corresponding to each preprocessed surveillance video.

[0082] S220: Perform random center fogging processing on each real image in each real image set to obtain each fogged image set corresponding to each real image set;

[0083] S230: Divide each fogged image set according to a preset ratio to obtain each training subset and each test subset corresponding to each fogged image set;

[0084] S240: Extract any real image from each real image set to construct a real reference image set, construct a training set based on each training subset and the real reference image set, and construct a test set based on each test subset and the real reference image set.

[0085] S250: Perform image scaling on the training set to ensure that the images in the test set are the same size. Then, perform random cropping and horizontal flipping on the scaled training set to obtain a preprocessed training set.

[0086] Furthermore, in S210, image extraction processing is performed on the preprocessed surveillance videos Video(1), Video(2), ..., Video(n). One real image is extracted every 2 seconds, and 1200 real images are extracted from each preprocessed surveillance video. Specifically, the real images extracted for preprocessed surveillance video Video(1) are Video(1-1), Video(1-2), ..., Video(1-1200), and the real images extracted for preprocessed surveillance video Video(2) are Video(1-1), Video(2-2), ..., Video(n). o(2-1), Video(2-2), ..., Video(2-1200), similarly, the real images extracted from the preprocessed surveillance video Video(n) are Video(n-1), Video(n-2), ..., Video(n-1200); for each preprocessed surveillance video, the 1200 images extracted from the preprocessed surveillance video constitute the real image set GT(1), GT(2), ..., GT(n) corresponding to the preprocessed surveillance video; specifically, the real image set GT(1) consists of Video( The real image set GT(2) consists of Video(2-1), Video(2-2), ..., Video(2-1200), and similarly, the real image set GT(n) consists of Video(n-1), Video(n-2), ..., Video(n-1200). In S220, each real image in GT(1), GT(2), ..., GT(n) is subjected to random center fogging processing to obtain the image set GT(n). The fogged image sets corresponding to GT(1), GT(2), ..., GT(n) are Hazy(1), Hazy(2), ..., Hazy(n); where Hazy(1) includes Hazy(1-1), Hazy(1-2), ..., Hazy(1-1200), Hazy(2) includes Hazy(2-1), Hazy(2-2), ..., Hazy(2-1200), and similarly, Hazy(n) includes Hazy(n-1), Hazy(n-2), ..., Hazy(n-1200);The Hazy(1-1), Hazy(1-2), ..., Hazy(1-1200) in Hazy(1) correspond one-to-one with the Video(1-1), Video(1-2), ..., Video(1-1200) in GT(1). Similarly, the Hazy(2-1), Hazy(2-2), ..., Hazy(2-1200) in Hazy(2) correspond one-to-one with the Video(2-1), Video(2-2), ..., Video(2-1200) in GT(2). Likewise, the Hazy(n-1), Hazy(n-2), ..., Hazy(n-1200) in Hazy(n) correspond one-to-one with the Video(n) in GT(n). (n-1), Video(n-2), ..., Video(n-1200) correspond one-to-one; in S230, according to the preset ratio, in this embodiment, the hazy image set Hazy(1), Hazy(2), ..., Hazy(n) are divided sequentially according to the ratio of 8:2 to obtain each training subset and each test subset corresponding to each hazy image set. Specifically, the hazy image set Hazy(1) corresponds to the training subset Train(1) and the test subset Test(1), the hazy image set Hazy(2) corresponds to the training subset Train(2) and the test subset Test(2), and similarly, the hazy image set Hazy(n) corresponds to the training subset Train(n). ) and test subset Test(n), where the ratio of fogged images in Train(1) and Test(1) is 8:2, the ratio of fogged images in Train(2) and Test(2) is 8:2, and similarly, the ratio of fogged images in Train(2) and Test(2) is 8:2; In S240, one image is randomly extracted from each of the real image sets GT(1), GT(2), ..., GT(n) as the real reference image Re(1), Re(2), ..., Re(n) corresponding to the real image set, and the real reference image set Re is constructed by all the extracted real reference images Re(1), Re(2), ..., Re(n); by training subsets Train(1), T The training set Train is composed of the training subsets Train(2), ..., Train(n) and the real reference image set Re. The test set Test is composed of the test subsets Test(1), Test(2), ..., Test(n) and the real reference image set Re. In S250, all images in the training set Train are scaled to keep the size of all images the same. Then, all images are randomly cropped and horizontally flipped to prevent overfitting, resulting in a preprocessed training set. The preprocessed training set consists of the training subsets Train(1), Train(2), ..., Train(n) after the second preprocessing (scaling, random cropping, and horizontal flipping) and the reference image set Re.

[0087] S300: Perform fogging processing on multiple segments of pre - processed surveillance videos to construct a fogged image stream set based on the multiple segments of fogged surveillance videos;

[0088] It should be noted that S300 specifically is:

[0089] S310: Perform fogging processing on multiple segments of pre - processed surveillance videos according to the fog synthesis algorithm based on depth image information to obtain multiple segments of fogged videos;

[0090] S320: Randomly sample N segments of fogged videos from the multiple segments of fogged videos, and perform continuous image extraction on each fogged video in the N segments of fogged videos to obtain each fogged image stream corresponding to each fogged video;

[0091] S330: Construct a fogged image stream set according to each fogged image stream corresponding to each fogged video.

[0092] Furthermore, in S310, perform fogging processing on multiple segments of pre - processed surveillance videos Video(1), Video(2), …, Video(n) according to the fog synthesis algorithm based on depth image information to obtain corresponding multiple segments of fogged videos WH(1), WH(2), …, WH(n); in S320, randomly sample N segments of fogged videos from these n segments of fogged videos WH(1), WH(2), …, WH(n), where N < n; perform continuous image extraction on each fogged video in the N segments of fogged videos to obtain each fogged image stream corresponding to each fogged video in the N segments of fogged videos; in S330, construct a fogged image stream set according to the N fogged image streams corresponding to the N segments of fogged videos obtained in S320.

[0093] S400: Construct an optical flow dynamic feature enhanced attention mechanism network, and the optical flow dynamic feature enhanced attention mechanism network includes a background feature extraction module, a color feature extraction module, and a sliding window attention mechanism module;

[0094] It should be noted that, referring to Figure 2 、 Figure 3 and Figure 4 as shown, the background feature extraction module is used to extract background features in the railway scene, and these features are crucial for distinguishing foreground (such as trains, personnel) and background (such as tracks, sky); the color feature extraction module is used to extract color information in the image, and fog usually causes loss or blurring of color information, so the extraction of color features helps to restore image details affected by fog; the sliding window attention mechanism module uses the optical flow dynamic feature enhanced attention mechanism to apply the attention mechanism to the image in a sliding window manner, enabling the network to focus on important local regions in the image, thereby improving the defogging effect.

[0095] S500: Construct a composite loss function and train the optical flow dynamic feature enhancement attention mechanism network based on the preprocessed training set, the fogged image stream set, and the composite loss function;

[0096] It should be noted that the S500 specifically refers to:

[0097] S510: Construct a composite loss function and input the preprocessed training set and the fogged image stream set into the optical flow dynamic feature enhancement attention mechanism network for training;

[0098] S520: Obtaining dynamic optical flow features to enhance the attention mechanism network, the loss difference between the training results of the preprocessed training set and the corresponding real images;

[0099] S530: The composite loss function optimizes the parameters of the optical flow dynamic feature enhancement attention mechanism network based on the loss difference until the optical flow dynamic feature enhancement attention mechanism network converges, thus completing the training of the optical flow dynamic feature enhancement attention mechanism network by the composite loss function.

[0100] It should be noted that in S510, the preprocessed training set and the fogged image stream set are input into the optical flow dynamic feature enhancement attention mechanism network for training, specifically as follows:

[0101] S511: Input the fogged image from the preprocessed training set into a 3×3 convolutional layer, and after passing through a 3×3 convolutional layer and a sliding window attention mechanism module, the first attention feature is obtained.

[0102] S512: Input the same fogged image into the color feature extraction module, and obtain the color features after processing by the color feature extraction module; input the real reference image from the same preprocessed monitoring video as the fogged image into the background feature extraction module, and obtain the background features after processing by the background feature extraction module;

[0103] S513: Input the first attention feature and background feature into a downsampling layer for feature fusion processing to obtain the first fused feature. Input the first fused feature into a sliding window attention mechanism module for processing to obtain the second attention feature.

[0104] S514: Input the second attention feature and color feature into a downsampling layer for feature fusion processing to obtain the second fused feature; input the second fused feature into a sliding window attention mechanism module and an upsampling layer in sequence to obtain the third attention feature;

[0105] S515: Input the third attention feature and the second attention feature into an SK channel attention fusion module for feature fusion processing to obtain the third fused feature; input the third fused feature into a sliding window attention mechanism module for processing to obtain the fourth attention feature;

[0106] S516: Input the set of fogged image streams into the optical flow network for feature extraction to obtain optical flow features. Input the optical flow features and the fourth attention features into an upsampling layer for feature fusion to obtain the fourth fused features.

[0107] S517: Input the fourth fusion feature and the first attention feature into an SK channel attention fusion module to perform feature fusion, and obtain the fifth fusion feature. After the fifth fusion feature is processed by a sliding window attention mechanism module and a 3×3 convolutional layer, the dehazed image is obtained.

[0108] Further, assuming the hazy image Hazy(1-2) belongs to the training subset Train(1); in S511, the hazy image Hazy(1-2) after the second preprocessing in the preprocessing training set is input into a 3×3 convolutional layer, and after being processed by a 3×3 convolutional layer and a sliding window attention mechanism module, the first attention feature is obtained. The depth of the first attention feature is 24, the length is 720, and the width is 1080; in S512, the hazy image Hazy(1-2) after the second preprocessing is input into the color feature extraction module, and after being processed by the color feature extraction module, the color feature is obtained. The network depth of the color feature is 12, the length is 180, and the width is 270; the preprocessing training... The concentrated and fogged images Hazy (1-2) are from the same preprocessed surveillance video's real reference image input to the background feature extraction module. Since the fogged image Hazy (1-2) comes from the preprocessed surveillance video Video (1), the real reference image corresponding to the preprocessed surveillance video Video (1) is Re (1). Therefore, the real reference image Re (1) is input to the background feature extraction module. After processing by the background feature extraction module, the background features are obtained. The depth of the background features is 48, the length is 360, and the width is 540. In S513, the first attention feature and the background feature are input into a downsampling layer for feature fusion processing to obtain the first fused feature. The depth of the first fused feature is 48, and the length is 360. The first fused feature is processed by a sliding window attention mechanism module to obtain a second attention feature with a depth of 48, a length of 360, and a width of 540. In S514, the second attention feature and the color feature are input into a downsampling layer for feature fusion processing to obtain a second fused feature with a depth of 96, a length of 180, and a width of 270. The second fused feature is then processed by a sliding window attention mechanism module and an upsampling layer to obtain a third attention feature with a depth of 48, a length of 360, and a width of 540. In S515, the third attention feature and the second attention feature are input into an SK channel. The attention fusion module performs feature fusion processing to obtain a third fused feature with a depth of 48, a length of 360, and a width of 540. This third fused feature is then processed by a sliding window attention mechanism module to obtain a fourth attention feature with a depth of 48, a length of 360, and a width of 540. In S516, the fogged image stream set is input into an optical flow network for feature extraction processing to obtain an optical flow feature with a depth of 48, a length of 360, and a width of 540. The optical flow feature and the fourth attention feature are then input into an upsampling layer for feature fusion processing to obtain a fourth fused feature with a depth of 24, a length of 720, and a width of 1080.In S517, the fourth fusion feature and the first attention feature are input into an SK-channel attention fusion module for feature fusion to obtain the fifth fusion feature. The fifth fusion feature has a depth of 24, a length of 720, and a width of 1080. The fifth fusion feature is then processed sequentially through a sliding window attention mechanism module and a 3×3 convolutional layer to obtain the dehazed image.

[0109] It should be noted that before the set of fogged image streams is input into the optical flow network for feature extraction in S516, the optical flow network has already been trained. That is, when constructing the optical flow dynamic feature enhancement attention mechanism network, the optical flow network used is already trained and has the best weights.

[0110] It should be noted that there is a difference in loss between the training results of the optical flow dynamic feature enhancement attention mechanism network in S520 on the preprocessed training set and the corresponding real images; specifically:

[0111] Suppose that the optical flow dynamic feature enhancement attention mechanism network in S510 is trained with the hazy image Hazy(1-2) and the output is the dehazed image QHazy(1-2). The difference between the dehazed image QHazy(1-2) and the corresponding real image value is obtained. Since Hazy(1-2) comes from the random center haze of the real image Video(1-2), the real image corresponding to the dehazed image QHazy(1-2) is Video(1-2). Therefore, the loss difference between the dehazed image QHazy(1-2) and the real image Video(1-2) is obtained.

[0112] It should be noted that in S530, the composite loss function optimizes the parameters of the optical flow dynamic feature enhancement attention mechanism network based on the loss difference until the network converges, thus completing the training of the optical flow dynamic feature enhancement attention mechanism network using the composite loss function; specifically:

[0113] Assuming that in S510, the optical flow dynamic feature enhancement attention mechanism network is trained using the hazy image Hazy(1-2) to output a dehazed image, in S530, the composite loss function optimizes the parameters of the optical flow dynamic feature enhancement attention mechanism network based on the loss difference between the dehazed image QHazy(1-2) and the real image Video(1-2) obtained in S520. Through iterative optimization during the training process, the optical flow dynamic feature enhancement attention mechanism network converges, thus completing the training of the optical flow dynamic feature enhancement attention mechanism network using the composite loss function.

[0114] It should be noted that the composite loss function is expressed by the following formula:

[0115]

[0116] L = L1 + SSIM(x,y),

[0117] Where L represents the composite loss function, x(p) represents the pixel value at pixel p in the dehazed image, y(p) represents the pixel value at pixel p in the corresponding ground truth image, and N represents the total number of pixels in the image. p∈P This represents summing over all pixels p in the image, μ x μ represents the pixel mean of the dehazed image. y σ represents the pixel mean of the real image corresponding to the dehazed image. x σ represents the standard deviation of pixel values ​​in a dehazed image. y σ represents the standard deviation of pixel values ​​in the real image corresponding to the dehazed image. xy The table shows the pixel value covariance between the dehazed image and the corresponding real image, where C1 and C2 represent constants.

[0118] The data processing procedure of the sliding window attention mechanism module can be represented by the following formula:

[0119]

[0120] Where B represents the relative position deviation term, Q represents the query matrix, K represents the key matrix, V represents the value matrix, ||Q|| represents the L2 norm of Q, ||K|| represents the L2 norm of K, and β represents the learnable scaling factor.

[0121] S600: Test the trained optical flow dynamic feature enhancement attention mechanism network according to the test set, input the image to be dehazed into the optical flow dynamic feature enhancement attention mechanism network that has passed the test, and output the dehazed image;

[0122] It should be noted that the trained network was tested using a test set to verify its dehazing effect. After passing the test, dehazing images were extracted from the railway operation scene monitoring video that needed dehazing at 15 frames per second, resulting in a set of continuous images to be dehazed. This set of continuous images was then sequentially input into the tested optical flow dynamic feature enhancement attention mechanism network to output a set of continuous dehazed images. (See reference...) Figure 5 As shown, this is the dehazed image obtained after processing one of the images to be dehazed.

[0123] S700: The dehazed image is processed according to the progressive optimization algorithm to obtain the final target dehazed image;

[0124] It should be noted that in the S700, see [reference]. Figure 6As shown, considering the characteristics of surveillance cameras, most cameras are tilted slightly downwards. Real-time video images show increasing y-axis pixel values ​​from top to bottom and increasing x-axis pixel values ​​from left to right. Smaller y-axis pixels indicate a longer real-time detection distance; the greater the distance, the thicker the fog, and the greater the impact of the fog.

[0125] Therefore, a progressive optimization algorithm is designed to optimize the obtained dehazed image to obtain a more accurate target dehazed image. The progressive optimization algorithm is expressed by the following formula:

[0126]

[0127] T(x)=t(x)+δ·f(t(x)),

[0128] Where, δ min Let δ represent the minimum position change factor, H represent the height of the dehazed image, y represent the current pixel position, T(x) represent the target dehazed image, t(x) represent the dehazed image, and f(t(x)) represent the image enhancement processing performed on the dehazed image.

[0129] This embodiment of the image dehazing method based on an optical flow dynamic feature-enhanced attention mechanism network (OCN) can more effectively extract and utilize key information in railway scene monitoring videos, especially optical flow and color features, thereby enhancing its adaptability to foggy environments. The introduction of a composite loss function makes network training more accurate, minimizing the difference between the dehazing result and the real image, thus improving the dehazing effect. The monitoring video undergoes a first preprocessing step, and the training set undergoes a second preprocessing step. The first preprocessing includes setting the same resolution and frame rate, and standardization. The second preprocessing includes image scaling, random cropping, and horizontal flipping, which helps improve the network's adaptability to different railway scenes and monitoring conditions. By using random centering for fogging and constructing a fogged image stream set, the diversity and complexity of the training data are increased, further enhancing the network's generalization ability. Performance: The sliding window attention mechanism module in the optical flow dynamic feature-enhanced attention mechanism network can pay more attention to the feature regions with obvious fogging in the input features, thereby achieving a more refined defogging effect. The synergistic effect of the background feature extraction module and the color feature extraction module enables the network to more accurately identify and process feature elements in railway scenes. The application of the progressive optimization algorithm further improves the quality of the defogging image, making the defogging image clearer and more natural. The introduction of the SK channel attention fusion module enhances the network's ability to allocate attention to different feature channels, which helps to improve the detail performance and overall visual effect of the defogging image. This method can perform defogging processing on railway scene monitoring videos in real time or near real time, which helps to improve the efficiency and accuracy of railway safety monitoring. The defogging monitoring video can provide a clearer field of view, which helps monitoring personnel to discover and deal with potential safety hazards in a timely manner.

[0130] It should be noted that the image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network provided in this application embodiment can be executed by an image dehazing system based on an optical flow dynamic feature enhanced attention mechanism network, or by a control module within that system for executing the image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network. This application embodiment uses the execution of the image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network by an image dehazing system as an example to illustrate the image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network provided in this application embodiment.

[0131] An image dehazing system based on an optical flow dynamic feature-enhanced attention mechanism network includes:

[0132] First acquisition module: acquires multiple surveillance videos of the monitored scene, performs first preprocessing on the multiple surveillance videos, and obtains corresponding preprocessed surveillance videos;

[0133] The second acquisition module: acquires training and test sets based on multiple pre-processed surveillance videos, and performs a second pre-processing on the training set to obtain a pre-processed training set;

[0134] The first processing module performs fogging processing on multiple pre-processed surveillance videos to construct a fogged image stream set based on the fogged surveillance video segments.

[0135] Construction Modules: Construct an optical flow dynamic feature enhanced attention mechanism network, which includes a background feature extraction module, a color feature extraction module, and a sliding window attention mechanism module;

[0136] Training module: Construct a composite loss function and train the optical flow dynamic feature enhancement attention mechanism network based on the preprocessed training set, the fogged image stream set, and the composite loss function;

[0137] Test module: Test the trained optical flow dynamic feature enhancement attention mechanism network according to the test set. Input the image to be dehazed into the optical flow dynamic feature enhancement attention mechanism network that has passed the test, and output the dehazed image.

[0138] The second processing module processes the dehazed image using a progressive optimization algorithm to obtain the final target dehazed image.

[0139] The image dehazing system based on an optical flow dynamic feature-enhanced attention mechanism network in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), etc. This application embodiment does not specifically limit the specific implementation.

[0140] The image dehazing system based on an optical flow dynamic feature-enhanced attention mechanism network in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0141] This application provides an image dehazing system based on an optical flow dynamic feature-enhanced attention mechanism network, which can achieve... Figures 1 to 6 The various processes implemented in the method embodiment of the image dehazing method based on the optical flow dynamic feature enhanced attention mechanism network are not described in detail here to avoid repetition.

[0142] According to this embodiment, an image dehazing system based on an optical flow dynamic feature-enhanced attention mechanism network preprocesses multiple surveillance videos of a railway scene through a first acquisition module. This improves the quality and consistency of the video data, laying a solid foundation for subsequent processing steps. This helps reduce noise, enhance contrast, or perform other necessary image enhancement operations, thereby improving the dehazing effect. A second acquisition module acquires training and testing sets from multiple preprocessed surveillance videos, ensuring that model training and testing are performed on independent datasets. This helps improve the model's generalization ability and avoid overfitting. The first processing module performs fogging processing on the preprocessed surveillance videos, constructing a fogged image stream set. This method, simulating a real foggy environment, helps the model learn dehazing strategies under different fog levels, improving the model's adaptability in practical applications. The optical flow dynamic feature-enhanced attention mechanism network in the construction module, through a background feature extraction module, a color feature extraction module, and a sliding window attention mechanism module, can more accurately capture key information in the railway scene. This mechanism helps the model focus more on important visual cues when processing complex scenes, thereby improving the accuracy and efficiency of dehazing. The training module trains the optical flow dynamic feature-enhanced attention mechanism network by constructing a composite loss function. This multi-objective optimization method helps improve the overall performance of the model, enabling it to achieve the best dehazing effect while meeting various requirements. The testing module tests the trained model to ensure its stability and reliability in practical applications. By inputting the image to be dehazed into the tested network, a high-quality dehazed image can be output to meet practical application needs. The second processing module uses a progressive optimization algorithm to further process the dehazed image to obtain the final target dehazed image. This algorithm, through gradual iteration and optimization, can further improve the image quality, making the dehazing effect more natural and clear.

[0143] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described image dehazing method embodiment based on optical flow dynamic feature enhanced attention mechanism network, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0144] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image dehazing method embodiment based on an optical flow dynamic feature enhanced attention mechanism network, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0145] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0146] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0148] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image dehazing method based on an optical flow dynamic feature-enhanced attention mechanism network, characterized in that, The method includes: Acquire multiple surveillance videos of the monitored scene, perform a first preprocessing on the multiple surveillance videos, and obtain corresponding multiple preprocessed surveillance videos; Training and test sets are obtained from the multiple preprocessed surveillance videos. The training set is then subjected to a second preprocessing to obtain a preprocessed training set. The multiple pre-processed surveillance videos are subjected to fogging processing to construct a fogged image stream set based on the fogged surveillance videos. An optical flow dynamic feature enhanced attention mechanism network is constructed, which includes a background feature extraction module, a color feature extraction module, and a sliding window attention mechanism module. A composite loss function is constructed, and the optical flow dynamic feature enhancement attention mechanism network is trained based on the preprocessed training set, the fogged image stream set, and the composite loss function. The construction of the composite loss function involves training the optical flow dynamic feature enhancement attention mechanism network based on the preprocessed training set, the fogged image stream set, and the composite loss function; specifically: A composite loss function is constructed, and the preprocessed training set and the fogged image stream set are input into the optical flow dynamic feature enhancement attention mechanism network for training. Specifically, the step of inputting the preprocessed training set and the fogged image stream set into the optical flow dynamic feature enhancement attention mechanism network for training is as follows: The fogged images in the preprocessed training set are input into a 3×3 convolutional layer, and after being processed by a 3×3 convolutional layer and a sliding window attention mechanism module, the first attention feature is obtained. The same fogged image is input into the color feature extraction module, and the color features are obtained after processing by the color feature extraction module; the real reference image from the same preprocessed monitoring video as the fogged image in the preprocessed training set is input into the background feature extraction module, and the background features are obtained after processing by the background feature extraction module. The first attention feature and the background feature are input into a downsampling layer for feature fusion processing to obtain the first fused feature. The first fused feature is then input into the sliding window attention mechanism module for processing to obtain the second attention feature. The second attention feature and the color feature are input into a downsampling layer for feature fusion processing to obtain the second fused feature; the second fused feature is then input into a sliding window attention mechanism module and an upsampling layer for processing to obtain the third attention feature. The third attention feature and the second attention feature are input into an SK channel attention fusion module for feature fusion processing to obtain the third fused feature; the third fused feature is then input into the sliding window attention mechanism module for processing to obtain the fourth attention feature. The set of fogged image streams is input into an optical flow network for feature extraction to obtain optical flow features. The optical flow features and the fourth attention features are then input into an upsampling layer for feature fusion to obtain the fourth fused features. The fourth fusion feature and the first attention feature are input into an SK channel attention fusion module for feature fusion to obtain a fifth fusion feature. The fifth fusion feature is then processed through a sliding window attention mechanism module and a 3×3 convolutional layer to obtain a dehazed image. The loss difference between the training results of the optical flow dynamic feature enhanced attention mechanism network on the preprocessed training set and the corresponding real image is obtained; The composite loss function optimizes the parameters of the optical flow dynamic feature enhancement attention mechanism network based on the loss difference until the optical flow dynamic feature enhancement attention mechanism network converges, thus completing the training of the optical flow dynamic feature enhancement attention mechanism network by the composite loss function. The trained optical flow dynamic feature enhancement attention mechanism network is tested according to the test set. The image to be dehazed is input into the optical flow dynamic feature enhancement attention mechanism network that has passed the test, and the dehazed image is output. The dehazed image is processed using a progressive optimization algorithm to obtain the final target dehazed image.

2. The image dehazing method based on optical flow dynamic feature-enhanced attention mechanism network according to claim 1, characterized in that, The process involves acquiring multiple segments of surveillance video from the monitored scene, performing a first preprocessing step on the multiple segments of surveillance video, and obtaining corresponding multiple preprocessed surveillance videos. Specifically: Acquire multiple surveillance videos of the monitored scene, and set the same resolution and frame rate for each of the multiple surveillance videos; The multiple surveillance video segments are standardized to ensure that each surveillance video segment has the same duration and format, resulting in the multiple pre-processed surveillance video segments.

3. The image dehazing method based on optical flow dynamic feature-enhanced attention mechanism network according to claim 2, characterized in that, The training set and test set are obtained based on the multiple preprocessed surveillance videos, and the training set is subjected to a second preprocessing to obtain a preprocessed training set. Specifically: Image extraction is performed on each preprocessed surveillance video segment in the multiple preprocessed surveillance videos to obtain each set of real images corresponding to each preprocessed surveillance video. For each real image in each real image set, random center fogging is performed to obtain each fogged image set corresponding to each real image set. Each fogged image set is divided according to a preset ratio to obtain each training subset and each test subset corresponding to each fogged image set; Extract any real image from each real image set to construct a real reference image set; construct a training set based on each training subset and the real reference image set; construct a test set based on each test subset and the real reference image set. The training set is image-scaled to make the images in the test set the same size. The scaled training set is then randomly cropped and horizontally flipped to obtain the preprocessed training set.

4. The image dehazing method based on optical flow dynamic feature-enhanced attention mechanism network according to claim 3, characterized in that, The process involves performing a fogging process on the multiple pre-processed surveillance video segments to construct a fogged image stream set based on the fogged surveillance video segments; specifically: The multiple preprocessed surveillance videos are processed using a fog synthesis algorithm based on depth image information to obtain multiple fogged videos. N segments of fogging video are randomly selected from the multiple fogging video segments. Continuous image extraction is performed on each of the N fogging video segments to obtain each fogging image stream corresponding to each fogging video. The set of fogged image streams is constructed based on each fogged image stream corresponding to each fogged video.

5. The image dehazing method based on an optical flow dynamic feature-enhanced attention mechanism network according to claim 1, characterized in that, The composite loss function is expressed by the following formula: ， in, Represents the composite loss function. Indicates the dehazed image at the pixel level Pixel value at that location, This represents the true image at the pixel level corresponding to the dehazed image. Pixel value at that location, This represents the total number of pixels in the image. This represents all pixels in the image. Summation, This represents the pixel mean of the dehazed image. This represents the pixel mean of the real image corresponding to the dehazed image. This represents the standard deviation of pixel values ​​in the dehazed image. This represents the standard deviation of the pixel values ​​in the real image corresponding to the dehazed image. The table shows the pixel value covariance between the dehazed image and the corresponding real image. and Indicates a constant; The data processing procedure of the sliding window attention mechanism module is represented by the following formula: ， in, This indicates the relative position deviation term. Represents the query matrix. Represents the key matrix. Represents a value matrix, express of norm, Show of Norm, This represents the learnable scaling factor.

6. An image dehazing system based on an optical flow dynamic feature-enhanced attention mechanism network, used to implement the image dehazing method based on an optical flow dynamic feature-enhanced attention mechanism network as described in any one of claims 1-5, characterized in that, The system includes: First acquisition module: acquires multiple surveillance videos of the monitored scene, performs first preprocessing on the multiple surveillance videos, and obtains corresponding preprocessed surveillance videos; The second acquisition module: acquires a training set and a test set based on the multiple preprocessed surveillance videos, and performs a second preprocessing on the training set to obtain a preprocessed training set; First processing module: performs fogging processing on the multiple pre-processed surveillance video segments to construct a fogged image stream set based on the fogged surveillance video segments; Construction Module: Construct an optical flow dynamic feature enhanced attention mechanism network, which includes a background feature extraction module, a color feature extraction module, and a sliding window attention mechanism module; Training module: Constructs a composite loss function and trains the optical flow dynamic feature enhancement attention mechanism network based on the preprocessed training set, the fogged image stream set, and the composite loss function; Test module: Test the trained optical flow dynamic feature enhancement attention mechanism network according to the test set, input the image to be dehazed into the optical flow dynamic feature enhancement attention mechanism network that has passed the test, and output the dehazed image; The second processing module processes the dehazed image according to a progressive optimization algorithm to obtain the final target dehazed image.

7. An electronic device, characterized in that, The method includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network as described in any one of claims 1-5.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image dehazing method based on an optical flow dynamic feature enhanced attention mechanism network as described in any one of claims 1-5.

Images