Railway scene image rain and fog removing method, device and equipment and storage medium

Abstract

The present application provides a kind of image rain and fog removal method, device, equipment and storage medium of railway scene, the method comprises: obtaining the target image of railway scene under rain and fog to be removed;Based on the target image of rain and fog to be removed, the preprocessed image of target image is generated;The preprocessed image is input into target Unet network, and the high-resolution image of target image is encoded and decoded by target Unet network Process, and the target image of rain and fog removal is obtained by pixel by pixel inflation filtering and fusion operation;Wherein, target Unet network is obtained by training initial Unet network through paired training sample;Paired training sample includes multiple rain and fog free sample images under railway scene and the superimposed rain and fog image corresponding to each rain and fog free sample image generated based on image transformation method.The present application realizes that a large number of paired training samples are generated based on image transformation method, improves model training effect, and then improves the rain and fog removal effect of model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for removing rain and fog from railway scene images. Background Technology

[0002] In railway scenarios, rain and fog severely reduce the visibility of critical targets such as tracks, signal lights, and obstacles. De-raining and de-fogging technology can restore image details, helping drivers or autonomous driving systems quickly identify track conditions, pedestrians, vehicles, or other obstacles ahead, thus avoiding collisions. Furthermore, raindrops or fog may cause monitoring systems to mistake rain and fog noise for dangerous targets (e.g., falsely report obstacles) or mask real hazards (e.g., track breaks). De-raining and de-fogging processing can improve the image signal-to-noise ratio and reduce the false alarm rate.

[0003] In existing technologies, two photos of the same railway scene are captured in a natural environment, one clear and one obscured by rain and fog. These paired photos are then used as training samples to train a deep learning model. The trained model is then used to remove rain and fog from the image. However, capturing two photos of the same railway scene in a natural environment, one clear and one obscured by rain and fog, is quite difficult. This presents a significant challenge to training the deep learning model for rain and fog removal, resulting in poor performance. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for removing rain and fog from railway scene images. It enables the generation of a large number of paired training samples based on image transformation methods, thereby improving the training effect of the model and thus enhancing the rain and fog removal effect of the model.

[0005] In a first aspect, the present invention provides a method for removing rain and fog from railway scene images, the method comprising the following steps:

[0006] Acquire the target image of the railway scene from which rain and fog need to be removed;

[0007] Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated;

[0008] The preprocessed image of the target image is input into the target Unet network. The target Unet network encodes and decodes the high-resolution image of the target image, and obtains the de-rained target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training an initial Unet network with paired training samples. The paired training samples include multiple rain-free and fog-free sample images in a railway scene and superimposed rain-free and fog-free images corresponding to each rain-free and fog-free sample image generated based on an image transformation method.

[0009] According to the present invention, a method for removing rain and fog from railway scene images includes generating a preprocessed image of the target image based on the target image to be removed from, comprising:

[0010] The target image is decomposed into low-frequency and high-frequency components by discrete wavelet transform, resulting in decomposed high- and low-frequency components. The high-frequency components are used to characterize the overall structure of the target image, and the low-frequency components are used to characterize the detailed information of the target image.

[0011] The high and low frequency components after decomposition are subjected to frequency domain enhancement processing to obtain the input feature map of the target image;

[0012] Based on the input feature map of the target image, image reconstruction is guided by a similarity feature aggregation algorithm to obtain the reconstructed feature map of the target image;

[0013] The reconstructed feature map of the target image is subjected to multi-scale dilation convolution processing to extract multi-scale features under different dilation rates;

[0014] The multi-scale features under different expansion rates are adaptively fused using a weight generator to generate fusion weights for the multi-scale features.

[0015] The fusion weights based on the multi-scale features are used to recover a high-resolution image of the target image through deconvolution operation;

[0016] The high-resolution image information of the target image is converted from the frequency domain space to the pixel space by inverse wavelet transform operation to obtain the preprocessed image of the target image.

[0017] According to the present invention, a method for removing rain and fog from railway scene images includes obtaining a reconstructed feature map of the target image by guiding image reconstruction based on a similarity feature aggregation algorithm according to the input feature map of the target image.

[0018] The input feature map of the target image is segmented into multiple feature blocks, and the inter-block similarity matrix between the multiple feature blocks is calculated;

[0019] Perform a Sigmoid operation on the inter-block similarity matrix to generate an attention matrix;

[0020] The image patches are weighted and reconstructed based on the attention matrix to obtain the reconstructed feature map of the target image.

[0021] According to the present invention, a method for removing rain and fog from railway scene images, wherein the target Unet network is constructed through the following steps:

[0022] Obtain the original railway scene dataset, which includes the multiple rain-free and fog-free sample images;

[0023] Based on the image transformation method, rain and fog effects are simulated on the original railway scene dataset to generate superimposed rain and fog images corresponding to each of the rain-free sample images; the simulation operation includes rain effect simulation and fog effect simulation.

[0024] Based on the multiple rain-free and fog-free sample images and the superimposed rain-and-fog images corresponding to each rain-free and fog-free sample image, the paired training samples are determined.

[0025] Based on the superimposed rain and fog images corresponding to each of the rain-free sample images, a preprocessed image of each superimposed rain and fog image is generated;

[0026] The preprocessed images of each of the superimposed rain and fog images are input into the initial Unet network. The initial Unet network performs encoding and decoding processing on the preprocessed images of each of the superimposed rain and fog images. The de-rain and fog sample images corresponding to each of the superimposed rain and fog images are obtained through the pixel-by-pixel dilation filtering and fusion operation.

[0027] Based on the rain-de-fog sample images corresponding to each of the superimposed rain-fog images and each of the rain-free sample images, a first value of the loss function is determined.

[0028] Based on the first value of the loss function, the model parameters of the initial Unet network are optimized using the Adam optimization algorithm of adaptive moments estimation to obtain the optimal model parameters;

[0029] Based on the optimal model parameters, the target Unet network is determined.

[0030] According to the present invention, a method for removing rain and fog from railway scene images includes simulating rain and fog effects based on the image transformation method on the original railway scene dataset to generate superimposed rain and fog images corresponding to each rain-free sample image, comprising:

[0031] Raindrops are abstracted as line segments with specific lengths, angles, and transparency, and raindrop trajectory equations are generated based on the geometric characteristics of the raindrops.

[0032] The number of raindrops and their spatial distribution are determined by a random distribution function.

[0033] Based on the raindrop trajectory equation, the number of raindrops, and the spatial distribution of raindrops, a rainy day image is synthesized by combining a motion blur kernel and a brightness modulation formula.

[0034] Depth maps were constructed based on Koschmieder's atmospheric scattering law to simulate the distribution characteristics of fog concentration at different depths;

[0035] A gradient fog effect from the image center to the edge is achieved based on distance field theory, and the natural transition effect of the fog is enhanced by Gaussian filter to generate foggy day images;

[0036] Based on the rainy day image, the foggy day image, and each of the rain-free and fog-free sample images, a superimposed rain-fog image corresponding to each of the rain-free and fog-free sample images is obtained by fusing them together.

[0037] According to the present invention, a method for removing rain and fog from railway scene images, wherein obtaining the de-fog target image through pixel-by-pixel dilation filtering and fusion operations includes:

[0038] The output image is subjected to pixel-by-pixel dilation filtering using four scale filter kernels to obtain intermediate images at different scales; the dilation factors for each filter kernel are 1, 2, 3 and 4 respectively.

[0039] The intermediate images at different scales are fused to obtain the target image after rain and fog removal.

[0040] Secondly, the present invention also provides an image rain and fog removal device for railway scenes, the device comprising the following modules:

[0041] The preprocessing module is used to acquire the target image of the railway scene from which rain and fog need to be removed;

[0042] Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated;

[0043] The rain and fog removal module is used to input the preprocessed image of the target image into a target Unet network, and to perform encoding and decoding processing on the preprocessed image of the target image through the target Unet network, and to obtain the rain and fog removed target image through pixel-by-pixel dilation filtering and fusion operations; wherein, the target Unet network is obtained by training an initial Unet network with paired training samples; the paired training samples include multiple rain and fog-free sample images in a railway scene and superimposed rain and fog images corresponding to each rain and fog-free sample image generated based on an image transformation method.

[0044] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image rain and fog removal method for railway scenes as described above.

[0045] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image rain and fog removal method for railway scenes as described above.

[0046] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image rain and fog removal method for railway scenes as described above.

[0047] The present invention provides a method, apparatus, device, and storage medium for removing rain and fog from railway scene images. The method involves acquiring a target image of the railway scene from which rain and fog are to be removed, generating a preprocessed image of the target image based on the target image, inputting the preprocessed image into a target Unet network, encoding and decoding the preprocessed image through the target Unet network, and obtaining the de-fogging target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training an initial Unet network using paired training samples. These paired training samples include multiple rain-free and fog-free sample images of the railway scene and superimposed rain and fog images corresponding to each rain-free and fog-free sample image generated based on an image transformation method.

[0048] This invention generates superimposed rain and fog images corresponding to each rain-free and fog-free sample image based on an image transformation method. The rain-free and fog-free sample images and their corresponding superimposed rain and fog images are used as paired training samples to train a target Unet network. Then, the high-resolution image corresponding to the target image to be de-fogged is input into the target Unet network. The target Unet network performs encoding and decoding processing on the preprocessed target image, and obtains the de-fogged target image through pixel-by-pixel dilation filtering and fusion operations. This invention simulates realistic rain and fog effects on the original railway scene dataset based on an image transformation method and accurately superimposes them onto the original clean image, thereby generating a large number of paired training samples. This achieves the generation of a large number of paired training samples based on an image transformation method, improving the model's training effect based on a large number of paired training samples, and thus enhancing the model's de-fogging effect. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0050] Figure 1 This is one of the flowcharts illustrating the rain and fog removal method for railway scenes provided by the present invention.

[0051] Figure 2 This is the second flowchart illustrating the rain and fog removal method for railway scenes provided by this invention.

[0052] Figure 3 This is the third flowchart illustrating the rain and fog removal method for railway scenes provided by this invention.

[0053] Figure 4 This is a schematic diagram of the image rain and fog removal device for railway scenes provided by the present invention.

[0054] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, a first node can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0057] The following is combined Figures 1-5 The present invention describes a method, apparatus, device, and storage medium for removing rain and fog from railway scene images.

[0058] Figure 1 This is one of the flowcharts illustrating the image rain and fog removal method for railway scenes provided by the present invention, such as... Figure 1 As shown, the method includes the following:

[0059] Step 101: Obtain the target image of the railway scene from which rain and fog need to be removed.

[0060] First, it should be noted that the subject of this invention is an electronic device, which is used to generate a large number of paired training samples based on image transformation methods, improve the training effect of the model based on a large number of paired training samples, and thus improve the rain and fog removal effect of the model.

[0061] First, acquire the target image to be removed from the rain and fog in the railway scene. For example, use railway monitoring cameras to collect original images or videos in the rain and fog environment. If it is video, extract the target image to be removed from frame by frame.

[0062] Step 102: Based on the target image to be removed from rain and fog, generate a preprocessed image of the target image.

[0063] Specifically, after obtaining the original target image to be de-foged, in order to enable the target Unet network to better understand the input target image, the target image to be de-foged can be pre-processed to obtain the pre-processed image of the target image, that is, the high-resolution image of the target image.

[0064] Preprocessing includes, for example, decomposing the original image (the target image to be de-rained and de-fogged) into different frequency domains and then processing the information in different frequency domains to achieve image super-resolution reconstruction and denoising (e.g., enhancing expression through multi-head attention machines).

[0065] A similarity feature aggregation module can also be introduced to guide image reconstruction by utilizing the similarity between different pixels, and so on.

[0066] Step 103: Input the preprocessed image of the target image into the target Unet network. The target Unet network encodes and decodes the preprocessed image of the target image and obtains the de-rained target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training the initial Unet network with paired training samples. The paired training samples include multiple rain-free and fog-free sample images in the railway scene and superimposed rain-free and fog-free sample images corresponding to each rain-free and fog-free sample image generated based on the image transformation method.

[0067] Specifically, after obtaining a high-resolution image of the target image, the preprocessed image of the target image is input into the target Unet network for rain and fog removal processing.

[0068] The target Unet network comprises an encoder, a decoder, a skip connection module, and an output layer. The encoder extracts multi-scale features from the rain and fog image through multiple convolutions and downsampling. Low-level features (shallow layers) capture local textures of the rain and fog (such as rain streaks and fog density distribution), while high-level features (deep layers) understand the global structure (such as object contours and illumination distribution). The decoder gradually restores image resolution through upsampling, combines details passed through skip connections to reconstruct a clear image, and eliminates the rain and fog features extracted by the encoder, preserving essential object features (such as edges and colors). The skip connection module fuses low-level features from the encoder (such as rain streak positions) with high-level features from the decoder (such as object shapes), avoiding detail loss and addressing spatial information loss caused by pooling, thus improving edge sharpness in the de-fogging image. The output layer maps the high-dimensional features output by the decoder to the Red-Green-Blue (RGB) space, generating the final de-fogging image.

[0069] The target Unet network is obtained by training the initial Unet network with a large number of paired training samples. In this embodiment, the large number of paired training samples are obtained as follows: First, the original railway scene dataset is acquired, which includes multiple rain-free and fog-free sample images. Then, a realistic rain and fog effect is simulated using image transformation methods, and it is accurately superimposed onto the original clean image, thereby generating a large number of paired training samples.

[0070] The method provided in this embodiment acquires a target image of the railway scene to be de-rained and fogged, generates a high-resolution image of the target image based on the target image to be de-rained and fogged, and then inputs the pre-processed image of the target image into a target Unet network. The target Unet network encodes and decodes the pre-processed image of the target image, and obtains the de-rained and fogged target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training an initial Unet network with paired training samples. The paired training samples include multiple fog-free sample images of the railway scene and superimposed rain and fog images corresponding to each fog-free sample image generated based on the image transformation method.

[0071] This invention generates superimposed rain and fog images corresponding to each rain-free and fog-free sample image based on an image transformation method. The rain-free and fog-free sample images and their corresponding superimposed rain and fog images are used as paired training samples to train a target Unet network. Then, the high-resolution image corresponding to the target image to be de-fogged is input into the target Unet network. The target Unet network performs encoding and decoding processing on the high-resolution image of the target image, and obtains the de-fogged target image through pixel-by-pixel dilation filtering and fusion operations. This invention simulates realistic rain and fog effects on the original railway scene dataset based on an image transformation method and accurately superimposes them onto the original clean image, thereby generating a large number of paired training samples. This achieves the generation of a large number of paired training samples based on an image transformation method, improving the model's training effect based on a large number of paired training samples, and thus enhancing the model's de-fogging effect.

[0072] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0073] According to the present invention, a method for removing rain and fog from railway scene images generates a high-resolution image of the target image based on the target image to which rain and fog need to be removed, comprising:

[0074] The target image is decomposed into low-frequency and high-frequency components by discrete wavelet transform, resulting in high- and low-frequency components. The high-frequency components are used to characterize the overall structure of the target image, while the low-frequency components are used to characterize the detailed information of the target image.

[0075] Frequency domain enhancement processing is performed on the decomposed high and low frequency components to obtain the input feature map of the target image;

[0076] Based on the input feature map of the target image, the image reconstruction is guided by a similarity feature aggregation algorithm to obtain the reconstructed feature map of the target image;

[0077] Multi-scale dilation convolution processing is performed on the reconstructed feature map of the target image to extract multi-scale features under different dilation rates;

[0078] The fusion weights of multi-scale features are generated by adaptively fusing multi-scale features under different expansion rates through a weight generator.

[0079] The fusion weights based on multi-scale features are used to recover a high-resolution image of the target image through deconvolution operation;

[0080] The high-resolution image information of the target image is transformed from the frequency domain space to the pixel space by inverse wavelet transform, resulting in the preprocessed image of the target image.

[0081] Specifically, in some embodiments, the process of obtaining a high-resolution image through preprocessing in step 102 is as follows:

[0082] First, the target image is decomposed into low-frequency and high-frequency components using Discrete Wavelet Transform (DWT) to obtain the decomposed high and low frequency components.

[0083] {LL,LH,HL,HH}=DWT(x)

[0084] Where DWT represents Discrete Wavelet Transform, x represents the input target image, LL is the low-frequency component, and LH, HL and HH are the horizontal, vertical and diagonal high-frequency components, respectively.

[0085] In the wavelet domain, high-frequency components characterize the overall structure of the target image, while low-frequency components characterize its detailed information. For example, high-frequency components represent texture information such as edges, textures, and subtle color variations. Low-frequency components typically correspond to the general outline and overall structure of the image, capturing its overall trends and gentle features. In the wavelet domain, noise is usually distributed in the high-frequency components; therefore, noise can be suppressed by processing the high-frequency domain.

[0086] Furthermore, frequency domain enhancement processing is performed on the decomposed high and low frequency components to obtain the input feature map of the target image. This invention processes high and low frequency information separately, that is, it enhances the features of image information in different frequency domains through convolution and self-attention operations.

[0087]

[0088] In this process, each layer of self-attention acts on features at different scales, enhancing expressive power through a multi-head mechanism.

[0089] {LL′,LH′,HL′,HH′}= ({LL,LH,HL,HH})

[0090] Wherein, LL is the low-frequency component, LH, HL and HH are the horizontal, vertical and diagonal high-frequency components, respectively, and LL′, LH′, HL′ and HH′ are the components after enhanced expression.

[0091] Where n is the number of times the self-attention mechanism is applied to each subband, and in this invention, n=3. Then, the enhanced features are fused using a channel attention mechanism, and channel weights are generated through global average pooling and a fully connected layer:

[0092]

[0093] Wherein, GAP() is the global average pooling layer. , For the weights of the fully connected layer, It is the ReLU activation function. Here, s is the sigmoid function, and s is the channel scaling factor, which is also the channel weight.

[0094] Furthermore, based on the input feature map of the target image, image reconstruction is guided by a similarity feature aggregation algorithm to obtain the reconstructed feature map of the target image.

[0095] To address the issues of uneven noise and sharpness distribution during rain and fog removal, this invention employs a similarity feature aggregation algorithm. This algorithm uses the similarity between different regions of an image as prior knowledge to guide the image reconstruction process, which not only improves the accuracy and quality of reconstruction but also enhances the stability of the image generation process.

[0096] Furthermore, multi-scale dilation convolution is performed on the reconstructed feature map of the target image to extract multi-scale features at different dilation rates. For example, for the new feature map... (The reconstructed feature map of the target image) undergoes multi-scale aggregation. The input features are processed through dilated convolutions with different dilation rates to ultimately obtain features at different scales. , , , :

[0097]

[0098] in, Indicates the pixel coordinates before dilation. The matrix representing the convolution kernel, These are the pixel coordinates after dilation. This represents a convolution operation with an inflation rate of r. This represents the i-th feature block.

[0099] The formula uses convolution kernels with different dilation rates (r=1,2,3,4) to extract multi-scale features.

[0100]

[0101] in, This indicates that after a convolution operation with an inflation rate of r, the output feature map contains the feature value at position (x, y). This represents the convolution kernel weight matrix with an inflation rate of r, where m and n are the indexes of the weights, respectively. For input image patches The pixel coordinates are adjusted according to the dilation rate.

[0102] Furthermore, a weight generator adaptively fuses multi-scale features at different inflation rates to generate fusion weights for the multi-scale features. For example, a weight generator is used to fuse input features at different scales. , , , Four weights are generated respectively.

[0103]

[0104] in, For the fusion weights of the generated multi-scale features, The Sigmoid function is used. Concat() concatenates multi-scale features along the channel dimension, Conv() generates preliminary weights through convolutional layers, and finally the Sigmoid function compresses the weights to the [0,1] interval, allowing weights of different scales to be adjusted independently. This step can adaptively fuse multi-scale features and enhance the representation of important regions.

[0105] Furthermore, the fusion weights based on multi-scale features are used to recover a high-resolution image of the target image through deconvolution operations, while preserving detailed information:

[0106]

[0107] in A high-resolution image of the target image. This represents a convolution operation with an inflation rate of r. The fusion weights for the multi-scale features obtained in the previous step are: This is element-wise multiplication, meaning that each channel or pixel of each feature map is multiplied by its weights. Scaling. This is a deconvolution operation used to upsample the fused feature map to the target resolution. Its kernel weights... Learning through backpropagation, the formula is:

[0108]

[0109] in, This represents a high-resolution image of the target image, where m and n are the position indices of the weights, respectively. For deconvolution operation, ) represents the feature value of the input feature map at coordinate position (xm, yn).

[0110] Then, the image information is transformed from the frequency domain to the pixel space through the inverse wavelet transform operation.

[0111] The method provided in this embodiment decomposes the original target image into different frequency domains and then processes the information in different frequency domains to achieve image super-resolution reconstruction and denoising. It also introduces a global feature aggregation module to guide image reconstruction by utilizing the similarity between different pixels, enabling the model to better use the prior knowledge of the original image and improve the stability of the model.

[0112] According to the present invention, a method for removing rain and fog from railway scene images involves reconstructing the image based on a similarity feature aggregation algorithm using an input feature map of the target image to obtain a reconstructed feature map of the target image, including:

[0113] The input feature map of the target image is segmented into multiple feature blocks, and the inter-block similarity matrix between the multiple feature blocks is calculated.

[0114] Perform a sigmoid operation on the inter-block similarity matrix to generate an attention matrix;

[0115] The image patches are weighted and reconstructed based on the attention matrix to obtain the reconstructed feature map of the target image.

[0116] Specifically, in some embodiments, the specific implementation process of obtaining the reconstructed feature map of the target image by guiding image reconstruction based on the similarity feature aggregation algorithm according to the input feature map of the target image includes the following steps:

[0117] First, the input feature map of the target image is segmented into multiple feature blocks, and the inter-block similarity matrix between the multiple feature blocks is calculated.

[0118] For example, the input feature map F1 (the input feature map of the enhanced target image) is processed by a convolution operation to obtain feature F2. Then, F2 is divided into 3*3 feature blocks. After normalizing these feature blocks, an inner product operation is performed to obtain the inter-block similarity.

[0119]

[0120] in, This represents the similarity matrix between the i-th feature block and the j-th feature block. Represents the i-th feature block. Indicates the first Each feature block.

[0121] Furthermore, a sigmoid operation is performed on the inter-block similarity matrix to generate an attention matrix.

[0122] For example, the matrix can then be used to perform a Sigmoid operation on it in the column direction:

[0123]

[0124] in, Let represent the similarity matrix between the i-th and j-th feature blocks after normalization. Let represent the similarity matrix between the i-th feature block and the j-th feature block, and n represent the number of feature blocks.

[0125] Furthermore, the image patches are weighted and reconstructed based on the attention matrix to obtain the reconstructed feature map of the target image.

[0126] Specifically, the attention matrix obtained in the above formula is used to reconstruct all image patches. Each image patch is then reconstructed as a weighted sum of N image patches according to the attention matrix, i.e.

[0127]

[0128] in, Represents the total reconstructed image patch. Represents the j-th feature block. Let represent the similarity matrix between the i-th feature block and the j-th feature block, and n represent the total number of feature blocks.

[0129] Then, the re-acquired image patches are reconstructed to obtain new feature maps. That is, the reconstructed feature map of the target image.

[0130] The method provided in this embodiment first divides the input feature map of the target image into multiple feature blocks and calculates the inter-block similarity matrix between the multiple feature blocks; then, a sigmoid operation is performed on the inter-block similarity matrix to generate an attention matrix; subsequently, the image blocks are weighted and reconstructed based on the attention matrix to obtain the reconstructed feature map of the target image. Addressing the problems of uneven noise and sharpness distribution during rain and fog removal, this invention employs a similarity feature aggregation algorithm, using the similarity between different regions of the image as prior knowledge to guide the image reconstruction process. This not only improves the accuracy and quality of the reconstruction but also enhances the stability of the image generation process.

[0131] According to the present invention, a method for removing rain and fog from railway scene images, wherein the target Unet network is constructed through the following steps:

[0132] Obtain the original railway scene dataset, which includes multiple rain-free and fog-free sample images;

[0133] Based on the original railway scene dataset, rain and fog effects are simulated using image transformation methods to generate superimposed rain and fog images corresponding to each non-rain and fog sample image; the simulation operation includes rain effect simulation and fog effect simulation.

[0134] Based on multiple rain-free fog sample images and the superimposed rain-fog images corresponding to each rain-free fog sample image, pairs of training samples are determined.

[0135] Based on the superimposed rain and fog images corresponding to each rain-free sample image, generate preprocessed images of each superimposed rain and fog image;

[0136] The preprocessed images of each superimposed rain and fog image are input into the initial Unet network. The initial Unet network encodes and decodes the preprocessed images of each superimposed rain and fog image, and obtains the de-rain and fog sample images corresponding to each superimposed rain and fog image through pixel-by-pixel dilation filtering and fusion operations.

[0137] Based on the de-fog sample images corresponding to each superimposed rain and fog image and each rain-free sample image, the first value of the loss function is determined.

[0138] Based on the first value of the loss function, the model parameters of the initial Unet network are optimized using the Adam optimization algorithm of adaptive moments estimation to obtain the optimal model parameters;

[0139] The target Unet network is determined based on the optimal model parameters.

[0140] Specifically, in some embodiments, the training process of the target Unet network includes the following steps:

[0141] First, the original railway scene dataset is obtained, which includes multiple rain-free and fog-free sample images. Then, rain and fog effects are simulated on the original railway scene dataset using image transformation methods, generating superimposed rain and fog images corresponding to each rain-free and fog-free sample image. The simulation operations include rain effect simulation and fog effect simulation. Next, the multiple rain-free and fog-free sample images and their corresponding superimposed rain and fog images are determined as paired training samples.

[0142] Furthermore, based on the superimposed rain and fog images corresponding to each rain-free sample image, preprocessed images of each superimposed rain and fog image are generated. The preprocessing process for the superimposed rain and fog images corresponding to each rain-free sample image is similar to the steps for removing rain and fog from the target image, and will not be described in detail here.

[0143] Furthermore, the preprocessed images of each superimposed rain and fog image are input into the initial Unet network. The initial Unet network performs encoding and decoding processing on the preprocessed images of each superimposed rain and fog image, and obtains the de-fog sample image corresponding to each superimposed rain and fog image through pixel-by-pixel dilation filtering and fusion operations. The encoding and decoding process of the preprocessed images of each superimposed rain and fog image is similar to the steps of preprocessing the target image to be de-fog removed, and will not be described again here.

[0144] Furthermore, based on the de-fog sample images corresponding to each superimposed rain and fog image and each rain-free sample image, the first value of the loss function is determined. The design of the loss function includes pixel-level loss, adversarial loss, and combined loss, which can be determined according to training requirements.

[0145] Furthermore, based on the first value of the loss function, the Adam optimization algorithm using adaptive moments estimation is used to optimize the model parameters of the initial Unet network to obtain the optimal model parameters. The Adam optimizer employs a learning rate decay strategy, determining the model parameters at the end of the training iteration as the optimal model parameters. Then, based on the optimal model parameters, the target Unet network is determined, and model training is completed.

[0146] The method provided in this embodiment simulates a realistic rain and fog effect on the original railway scene dataset based on image transformation methods, and accurately superimposes it onto the original clean image, thereby generating a large number of paired training samples. Based on the large number of paired training samples, the initial Unet network is trained to obtain the target Unet network, and the target Unet network has a better defogging effect.

[0147] According to the present invention, a method for removing rain and fog from railway scene images simulates rain and fog effects based on an image transformation method on an original railway scene dataset, generating superimposed rain and fog images corresponding to each rain-free sample image, including:

[0148] Raindrops are abstracted as line segments with specific lengths, angles, and transparency, and raindrop trajectory equations are generated based on the geometric characteristics of raindrops.

[0149] The number of raindrops and their spatial distribution are determined by a random distribution function.

[0150] Rainy day images are synthesized by combining the raindrop trajectory equation, the number of raindrops, and the spatial distribution of raindrops with the motion blur kernel and the brightness modulation formula.

[0151] Depth maps were constructed based on Koschmieder's atmospheric scattering law to simulate the distribution characteristics of fog concentration at different depths;

[0152] A gradient fog effect from the image center to the edge is achieved based on distance field theory, and the natural transition effect of the fog is enhanced by Gaussian filter to generate foggy day images;

[0153] Based on rainy day images, foggy day images, and various rain-free and fog-free sample images, a superimposed rain-fog image corresponding to each rain-free and fog-free sample image is obtained by fusing them together.

[0154] Specifically, in some embodiments, the specific process of generating paired training samples is exemplified as follows:

[0155] The steps for generating rainy day images include the following:

[0156] First, raindrops are abstracted as line segments with specific lengths, angles, and transparency, and raindrop trajectory equations are generated based on the geometric characteristics of the raindrops.

[0157] The equation for the raindrop trajectory is:

[0158]

[0159] Where P0 is the initial position, and v is the velocity vector, satisfying |v|∈[v] m ᵢ n ,v max Subsequently, a random distribution function was used to generate the spatial distribution of raindrops, ensuring that the distribution of raindrops in the image conforms to the statistical characteristics of natural rainfall.

[0160] Furthermore, the number of raindrops and their spatial distribution are determined using a random distribution function.

[0161] The number of raindrops N satisfies:

[0162]

[0163] in ξ is the density coefficient, A is the image area, I is the intensity parameter, ξ~N(0,1) is a Gaussian random variable, and σ is the fluctuation coefficient.

[0164] Furthermore, rainy day images are synthesized by combining the raindrop trajectory equation, the number of raindrops, and the spatial distribution of raindrops with the motion blur kernel and the brightness modulation formula.

[0165] Motion blurring of raindrops is achieved through convolution operations, enhancing the dynamic visual characteristics of raindrops during their fall. The blur kernel K is defined as follows:

[0166]

[0167] Where L is the blur length, w is the width, and rect is the rectangular function. Finally, based on the principles of physical optics, the brightness of the raindrops is modulated and the image is fused to ensure that it can present a realistic visual effect under different backgrounds.

[0168]

[0169] in For the merged image, The original target image, This represents the raindrop brightness map, that is, the generated rain layer matrix, where α is the mixing coefficient.

[0170] The steps involved in generating foggy images are as follows:

[0171] First, a depth map was constructed based on Koschmieder's atmospheric scattering law to simulate the distribution characteristics of fog concentration at different depths.

[0172] This invention uses Koschmieder's atmospheric scattering law to simulate the attenuation and scattering phenomena of light propagating in a foggy atmosphere. Scattering equation:

[0173]

[0174] Where I0 is the original image (target image), β is the scattering coefficient, d is the depth, and A is the atmospheric light value. This is a depth map.

[0175] Depth maps simulate the distribution characteristics of fog at different depths, causing distant objects to be obscured by denser fog. The relevant formula is:

[0176]

[0177] Where (x0, y0) is the image center, η is the depth coefficient, and d max This represents the maximum depth value.

[0178] Furthermore, a gradual fog effect from the image center to the edge is achieved based on distance field theory, and the natural transition effect of the fog is enhanced by a Gaussian filter to generate a foggy image.

[0179] The fog density function is:

[0180]

[0181] Where ρ0 is the basic density and f is the density modulation function. To address the abrupt change at the fog boundary, a Gaussian filter is applied to enhance the natural transition effect of the fog. The Gaussian kernel is defined as follows:

[0182]

[0183] Where σ is the standard deviation, which controls the degree of fuzziness.

[0184] Finally, the fog image is fused with the original image. Then, the rainy image, foggy image, and each rainless / foggy sample image generated based on the image transformation method are fused to obtain a superimposed rain / foggy image corresponding to each rainless / foggy sample image.

[0185] The method provided in this embodiment simulates realistic rain and fog effects on the original railway scene dataset based on image transformation methods, and accurately superimposes them onto the original clean image, thereby generating a large number of paired training samples. In the process of synthesizing rainy weather images, this invention considers factors such as the size, direction, and density of raindrops; in the process of synthesizing foggy weather images, it considers factors such as the concentration and brightness of fog. The model samples are easy to obtain and are closer to the real situation, with high sample reliability.

[0186] According to the present invention, a method for removing rain and fog from railway scene images is provided, which obtains a de-rained target image through pixel-by-pixel dilation filtering and fusion operations, including:

[0187] The output image is subjected to pixel-by-pixel dilation filtering using four scale filter kernels to obtain intermediate images at different scales; the dilation factors for each filter kernel are 1, 2, 3 and 4 respectively.

[0188] By fusing intermediate images at different scales, a target image with rain and fog removed is obtained.

[0189] Specifically, by processing the input features through dilated convolutions with different dilation rates, features at different scales are obtained. , , , :

[0190]

[0191] in, Indicates the pixel coordinates before dilation. The matrix representing the convolution kernel, These are the pixel coordinates after dilation.

[0192] Then, intermediate images at different scales are fused to obtain the target image after rain and fog removal.

[0193] The method provided in this embodiment fuses the output results under different scale filtering kernels, thereby improving the stability of the model.

[0194] Figure 2 This is a second flowchart illustrating the image rain and fog removal method for railway scenes provided by the present invention. The method includes:

[0195] Input an original, rain-free, fog-free image of a railway scene;

[0196] Determine the generation task: generate a foggy image or generate a rainy image;

[0197] To generate a rainy image, create a rain layer matrix: draw raindrop trajectories, implement motion blur effects, and adjust brightness.

[0198] To generate a foggy image, create a fog layer matrix: construct a fog depth map, control fog density, and enhance filtering;

[0199] Image fusion: Combines rainy day images (foggy day images) with the original rain-free and fog-free images to output a rainy and foggy image.

[0200] Figure 3 This is the third flowchart of the image rain and fog removal method for railway scenes provided by the present invention. The method includes:

[0201] Input a rain and fog image.

[0202] Multi-band feature enhancement includes wavelet transform decomposition, multi-head attention enhancement, similarity feature aggregation, and multi-scale dilated convolution.

[0203] Feature fusion and reconstruction: multi-scale weight fusion, deconvolution sampling, and inverse wavelet transform.

[0204] The reconstructed image is input into the UNet network structure and fused with dilatation filtering to output a de-rained and de-fog image.

[0205] The image rain and fog removal device for railway scenes provided by the present invention is described below. The image rain and fog removal device for railway scenes described below can be referred to in correspondence with the image rain and fog removal method for railway scenes described above.

[0206] Figure 4 This is a schematic diagram of the image rain and fog removal device for railway scenes provided by the present invention, as shown below. Figure 4 As shown, the image rain and fog removal device 400 for the railway scene includes the following modules:

[0207] Preprocessing module 410 is used to acquire the target image of the railway scene to be free of rain and fog;

[0208] Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated;

[0209] The rain and fog removal module 420 is used to input the preprocessed image of the target image into the target Unet network, perform encoding and decoding processing on the preprocessed image of the target image through the target Unet network, and obtain the rain and fog removed target image through pixel-by-pixel dilation filtering and fusion operations; wherein, the target Unet network is obtained by training an initial Unet network with paired training samples; the paired training samples include multiple rain and fog-free sample images in the railway scene and superimposed rain and fog images corresponding to each rain and fog-free sample image generated based on the image transformation method.

[0210] The apparatus provided in this embodiment includes a preprocessing module 410, which acquires a target image of the railway scene to be de-rained and generates a high-resolution image of the target image based on the target image; and a rain and fog removal module 420, which inputs the preprocessed image of the target image into a target Unet network, performs encoding and decoding processing on the preprocessed image of the target image through the target Unet network, and obtains the de-rained target image through pixel-by-pixel dilation filtering and fusion operations; wherein, the target Unet network is obtained by training an initial Unet network with paired training samples, and the paired training samples include multiple rain-free sample images of the railway scene and superimposed rain and fog images corresponding to each rain-free sample image generated based on the image transformation method.

[0211] This invention generates superimposed rain and fog images corresponding to each rain-free and fog-free sample image based on an image transformation method. The rain-free and fog-free sample images and their corresponding superimposed rain and fog images are used as paired training samples to train a target Unet network. Then, the high-resolution image corresponding to the target image to be de-fogged is input into the target Unet network. The target Unet network performs encoding and decoding processing on the high-resolution image of the target image, and obtains the de-fogged target image through pixel-by-pixel dilation filtering and fusion operations. This invention simulates realistic rain and fog effects on the original railway scene dataset based on an image transformation method and accurately superimposes them onto the original clean image, thereby generating a large number of paired training samples. This achieves the generation of a large number of paired training samples based on an image transformation method, improving the model's training effect based on a large number of paired training samples, and thus enhancing the model's de-fogging effect.

[0212] According to the present invention, an image rain and fog removal device 400 for railway scenes is provided, wherein the preprocessing module 410 is specifically used for:

[0213] The target image is decomposed into low-frequency and high-frequency components by discrete wavelet transform, resulting in decomposed high- and low-frequency components. The high-frequency components are used to characterize the overall structure of the target image, and the low-frequency components are used to characterize the detailed information of the target image.

[0214] The high and low frequency components after decomposition are subjected to frequency domain enhancement processing to obtain the input feature map of the target image;

[0215] Based on the input feature map of the target image, image reconstruction is guided by a similarity feature aggregation algorithm to obtain the reconstructed feature map of the target image;

[0216] The reconstructed feature map of the target image is subjected to multi-scale dilation convolution processing to extract multi-scale features under different dilation rates;

[0217] The multi-scale features under different expansion rates are adaptively fused using a weight generator to generate fusion weights for the multi-scale features.

[0218] The fusion weights based on the multi-scale features are used to recover a high-resolution image of the target image through deconvolution operation;

[0219] The high-resolution image information of the target image is converted from the frequency domain space to the pixel space by inverse wavelet transform operation to obtain the preprocessed image of the target image.

[0220] According to the present invention, an image rain and fog removal device 400 for railway scenes is provided, wherein the preprocessing module 410 is further configured to:

[0221] The input feature map of the target image is segmented into multiple feature blocks, and the inter-block similarity matrix between the multiple feature blocks is calculated;

[0222] Perform a Sigmoid operation on the inter-block similarity matrix to generate an attention matrix;

[0223] The image patches are weighted and reconstructed based on the attention matrix to obtain the reconstructed feature map of the target image.

[0224] According to the present invention, an image rain and fog removal device 400 for railway scenes is provided, the device further comprising a model training module;

[0225] The model training module is used for:

[0226] Obtain the original railway scene dataset, which includes the multiple rain-free and fog-free sample images;

[0227] Based on the image transformation method, rain and fog effects are simulated on the original railway scene dataset to generate superimposed rain and fog images corresponding to each of the rain-free sample images; the simulation operation includes rain effect simulation and fog effect simulation.

[0228] Based on the multiple rain-free and fog-free sample images and the superimposed rain-and-fog images corresponding to each rain-free and fog-free sample image, the paired training samples are determined.

[0229] Based on the superimposed rain and fog images corresponding to each of the rain-free sample images, a preprocessed image of each superimposed rain and fog image is generated;

[0230] The preprocessed images of each of the superimposed rain and fog images are input into the initial Unet network. The initial Unet network performs encoding and decoding processing on the preprocessed images of each of the superimposed rain and fog images. The de-rain and fog sample images corresponding to each of the superimposed rain and fog images are obtained through the pixel-by-pixel dilation filtering and fusion operation.

[0231] Based on the rain-de-fog sample images corresponding to each of the superimposed rain-fog images and each of the rain-free sample images, a first value of the loss function is determined.

[0232] Based on the first value of the loss function, the model parameters of the initial Unet network are optimized using the Adam optimization algorithm of adaptive moments estimation to obtain the optimal model parameters;

[0233] Based on the optimal model parameters, the target Unet network is determined.

[0234] According to the present invention, an image rain and fog removal device 400 for railway scenes is provided, wherein the model training module is further configured to:

[0235] Raindrops are abstracted as line segments with specific lengths, angles, and transparency, and raindrop trajectory equations are generated based on the geometric characteristics of the raindrops.

[0236] The number of raindrops and their spatial distribution are determined by a random distribution function.

[0237] Based on the raindrop trajectory equation, the number of raindrops, and the spatial distribution of raindrops, a rainy day image is synthesized by combining a motion blur kernel and a brightness modulation formula.

[0238] Depth maps were constructed based on Koschmieder's atmospheric scattering law to simulate the distribution characteristics of fog concentration at different depths;

[0239] A gradient fog effect from the image center to the edge is achieved based on distance field theory, and the natural transition effect of the fog is enhanced by Gaussian filter to generate foggy day images;

[0240] Based on the rainy day image, the foggy day image, and each of the rain-free and fog-free sample images, a superimposed rain-fog image corresponding to each of the rain-free and fog-free sample images is obtained by fusing them together.

[0241] According to the present invention, an image rain and fog removal device 400 for railway scenes is provided, wherein the rain and fog removal module 420 is specifically used for:

[0242] The output image is subjected to pixel-by-pixel dilation filtering using four scale filter kernels to obtain intermediate images at different scales; the dilation factors for each filter kernel are 1, 2, 3 and 4 respectively.

[0243] The intermediate images at different scales are fused to obtain the target image after rain and fog removal.

[0244] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute an image rain and fog removal method for a railway scene, the method including:

[0245] Acquire the target image of the railway scene from which rain and fog need to be removed;

[0246] Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated;

[0247] The preprocessed image of the target image is input into the target Unet network. The target Unet network encodes and decodes the preprocessed image of the target image, and obtains the de-rained target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training an initial Unet network with paired training samples. The paired training samples include multiple rain-free and fog-free sample images in a railway scene and superimposed rain-free and fog-free images corresponding to each rain-free and fog-free sample image generated based on an image transformation method.

[0248] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0249] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the image rain and fog removal method for railway scenes provided by the above methods, the method comprising:

[0250] Acquire the target image of the railway scene from which rain and fog need to be removed;

[0251] Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated;

[0252] The preprocessed image of the target image is input into the target Unet network. The target Unet network encodes and decodes the preprocessed image of the target image, and obtains the de-rained target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training an initial Unet network with paired training samples. The paired training samples include multiple rain-free and fog-free sample images in a railway scene and superimposed rain-free and fog-free images corresponding to each rain-free and fog-free sample image generated based on an image transformation method.

[0253] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image rain and fog removal method for railway scenes provided by the methods described above, the method comprising:

[0254] Acquire the target image of the railway scene from which rain and fog need to be removed;

[0255] Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated;

[0256] The preprocessed image of the target image is input into the target Unet network. The target Unet network encodes and decodes the preprocessed image of the target image, and obtains the de-rained target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training an initial Unet network with paired training samples. The paired training samples include multiple rain-free and fog-free sample images in a railway scene and superimposed rain-free and fog-free images corresponding to each rain-free and fog-free sample image generated based on an image transformation method.

[0257] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0258] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0259] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for removing rain and fog from railway scene images, characterized in that, include: Acquire the target image of the railway scene from which rain and fog need to be removed; Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated; The step of generating a preprocessed image of the target image based on the target image to be degraded by removing rain and fog includes: decomposing the target image into low-frequency and high-frequency components using discrete wavelet transform to obtain decomposed high- and low-frequency components; the high-frequency components are used to characterize the overall structure of the target image, and the low-frequency components are used to characterize the detailed information of the target image; performing frequency domain enhancement processing on the decomposed high- and low-frequency components to obtain an input feature map of the target image; guiding image reconstruction based on a similarity feature aggregation algorithm according to the input feature map of the target image to obtain a reconstructed feature map of the target image; performing multi-scale dilation convolution processing on the reconstructed feature map of the target image to extract multi-scale features under different dilation rates; adaptively fusing the multi-scale features under different dilation rates through a weight generator to generate fusion weights for the multi-scale features; recovering the high-resolution image of the target image based on the fusion weights of the multi-scale features through deconvolution operation; and converting the high-resolution image information of the target image from the frequency domain space to the pixel space through inverse wavelet transform operation to obtain the preprocessed image of the target image. The preprocessed image of the target image is input into the target Unet network. The target Unet network encodes and decodes the preprocessed image of the target image, and obtains the de-rained target image through pixel-by-pixel dilation filtering and fusion operations. The target Unet network is obtained by training an initial Unet network with paired training samples. The paired training samples include multiple rain-free and fog-free sample images in a railway scene and superimposed rain-free and fog-free images corresponding to each rain-free and fog-free sample image generated based on an image transformation method.

2. The image rain and fog removal method for railway scenes according to claim 1, characterized in that, The step of obtaining the reconstructed feature map of the target image by guiding image reconstruction based on the input feature map of the target image using a similarity feature aggregation algorithm includes: The input feature map of the target image is segmented into multiple feature blocks, and the inter-block similarity matrix between the multiple feature blocks is calculated; Perform a Sigmoid operation on the inter-block similarity matrix to generate an attention matrix; The image patches are weighted and reconstructed based on the attention matrix to obtain the reconstructed feature map of the target image.

3. The image rain and fog removal method for railway scenes according to claim 1, characterized in that, The target Unet network is constructed through the following steps: Obtain the original railway scene dataset, which includes the multiple rain-free and fog-free sample images; Based on the image transformation method, rain and fog effects are simulated on the original railway scene dataset to generate superimposed rain and fog images corresponding to each of the rain-free sample images; the simulation operation includes rain effect simulation and fog effect simulation. Based on the multiple rain-free and fog-free sample images and the superimposed rain-and-fog images corresponding to each rain-free and fog-free sample image, the paired training samples are determined. Based on the superimposed rain and fog images corresponding to each of the rain-free sample images, a preprocessed image of each of the superimposed rain and fog images is generated; The preprocessed images of each of the superimposed rain and fog images are input into the initial Unet network. The high-resolution images of each of the superimposed rain and fog images are encoded and decoded by the initial Unet network. The de-rain and fog sample images corresponding to each of the superimposed rain and fog images are obtained by the pixel-by-pixel dilation filtering and fusion operation. Based on the rain-de-fog sample images corresponding to each of the superimposed rain-fog images and each of the rain-free sample images, a first value of the loss function is determined. Based on the first value of the loss function, the model parameters of the initial Unet network are optimized using the Adam optimization algorithm of adaptive moments estimation to obtain the optimal model parameters; Based on the optimal model parameters, the target Unet network is determined.

4. The image rain and fog removal method for railway scenes according to claim 3, characterized in that, The step of simulating rain and fog effects based on the image transformation method on the original railway scene dataset, and generating superimposed rain and fog images corresponding to each of the rain-free sample images, includes: Raindrops are abstracted as line segments with specific lengths, angles, and transparency, and raindrop trajectory equations are generated based on the geometric characteristics of the raindrops. The number of raindrops and their spatial distribution are determined by a random distribution function. Based on the raindrop trajectory equation, the number of raindrops, and the spatial distribution of raindrops, a rainy day image is synthesized by combining a motion blur kernel and a brightness modulation formula. Depth maps were constructed based on Koschmieder's atmospheric scattering law to simulate the distribution characteristics of fog concentration at different depths; A gradient fog effect from the image center to the edge is achieved based on distance field theory, and the natural transition effect of the fog is enhanced by Gaussian filter to generate foggy day images; Based on the rainy day image, the foggy day image, and each of the rain-free and fog-free sample images, a superimposed rain-fog image corresponding to each of the rain-free and fog-free sample images is obtained by fusing them together.

5. The image rain and fog removal method for railway scenes according to claim 1, characterized in that, The process of obtaining the de-rained and de-fog-laden target image through pixel-by-pixel dilation filtering and fusion operations includes: The output image is subjected to pixel-by-pixel dilation filtering using four scale filter kernels to obtain intermediate images at different scales; the dilation factors for each filter kernel are 1, 2, 3 and 4 respectively. The intermediate images at different scales are fused to obtain the target image after rain and fog removal.

6. A rain and fog removal device for railway scene images, characterized in that, include: The preprocessing module is used to acquire the target image of the railway scene from which rain and fog need to be removed; Based on the target image to be freed of rain and fog, a preprocessed image of the target image is generated; The preprocessing module is specifically used to decompose the target image into low-frequency components and high-frequency components through discrete wavelet transform to obtain the decomposed high- and low-frequency components; the high-frequency components are used to characterize the overall structure of the target image, and the low-frequency components are used to characterize the detailed information of the target image. The high and low frequency components after decomposition are subjected to frequency domain enhancement processing to obtain the input feature map of the target image; based on the input feature map of the target image, image reconstruction is guided by a similarity feature aggregation algorithm to obtain the reconstructed feature map of the target image; the reconstructed feature map of the target image is subjected to multi-scale dilation convolution processing to extract multi-scale features under different dilation rates; The multi-scale features under different expansion rates are adaptively fused using a weight generator to generate fusion weights for the multi-scale features. The fusion weights based on the multi-scale features are used to recover the high-resolution image of the target image through deconvolution operation; the high-resolution image information of the target image is converted from the frequency domain space to the pixel space through inverse wavelet transform operation to obtain the preprocessed image of the target image. The rain and fog removal module is used to input the preprocessed image of the target image into a target Unet network, and to perform encoding and decoding processing on the preprocessed image of the target image through the target Unet network, and to obtain the rain and fog removed target image through pixel-by-pixel dilation filtering and fusion operations; wherein, the target Unet network is obtained by training an initial Unet network with paired training samples; the paired training samples include multiple rain and fog-free sample images in a railway scene and superimposed rain and fog images corresponding to each rain and fog-free sample image generated based on an image transformation method.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image rain and fog removal method for railway scenes as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image rain and fog removal method for railway scenes as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image rain and fog removal method for railway scenes as described in any one of claims 1 to 5.

Images