A method and system for generating simulated rain field images
Patent Information
- Application Number
- CN202310403404.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-14
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2043-04-14
AI Technical Summary
Existing technologies are not effective in synthesizing realistic and natural rainy images, especially in the synthesis and restoration of realistic and natural rainy images.
By constructing a simulated rain field generation network, including a generator, discriminator, and encoder, the network is trained on the training set images using spectral normalization and self-attention mechanisms. The generator employs transposed convolution and ReLU activation functions, the discriminator uses three convolutional layers, and the encoder extracts latent feature vectors from a standard Gaussian distribution. The adversarial process between the generator and discriminator is optimized using self-attention mechanisms and optimized loss functions.
It generates more realistic and natural rainy images, improving the visual quality and visibility of the images, and is suitable for fields such as autonomous driving and video surveillance.
Smart Images
Figure CN116596743B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of rain image generation technology, and in particular to a method for generating simulated rain field images. Background Technology
[0002] Computer vision is a key technology for realizing functions such as autonomous driving and video surveillance, and its effectiveness is achieved while ensuring a certain level of image quality. In rainy weather, the captured images and videos are easily affected by raindrops, resulting in reduced visibility and blurred images, which affects the visual effect of outdoor imaging.
[0003] Currently, many solutions for rain removal have emerged in the restoration of rainy images, and the rain removal effect is constantly improving. Among image restoration algorithms, single-image rain removal methods have achieved excellent performance in image rain removal. However, current models are trained on rainy image datasets based on artificially synthesized images with regularly distributed rainy attributes. Compared to single-image rain removal, the restoration effect on realistic and natural rainy images is unsatisfactory. Therefore, how to synthesize realistic and natural rainy images is a problem that needs to be solved by those skilled in the art. Summary of the Invention
[0004] This invention provides a method and system for generating simulated rain field images, which are used to synthesize more natural and realistic rain maps.
[0005] To achieve the above effects, the technical solution of the present invention is as follows:
[0006] A method for generating simulated rain field images includes the following steps:
[0007] S1: Obtain the rainy image dataset and preprocess the rainy image dataset to obtain preprocessed images;
[0008] S2: Normalize the preprocessed image to obtain normalized samples, which are divided into training set images and test set images;
[0009] S3: Construct a simulated rain field generation network; wherein, the simulated rain field generation network includes: a generator: used to generate rainy images; a discriminator: used to distinguish between real rainy images in the test set and rainy images generated by the generator; and an encoder: used to extract latent feature vectors that conform to a standard Gaussian distribution.
[0010] S4: Train the simulated rain field generation network using training set images, and use spectral normalization and self-attention mechanism on the discriminator to improve training capability;
[0011] S5: Through extensive training and hyperparameter optimization, a trained simulated rain field generation network is obtained; the rainy image to be processed is input into the trained simulated rain field generation network to generate a simulated rain field image.
[0012] In the above scheme, the rainy image dataset consists of real, natural rainy images. Rain variables such as peak quantity, rain intensity, and rain density are used to generate randomly distributed rain fields with defined attributes through a simulated rain field generation network. The quality of the simulated rain field generated by the network is evaluated numerically and visually.
[0013] Further, step S1 preprocessing specifically involves: pixelating the rain density of the images in the rainy image dataset, where rain density represents the proportion of rainy pixels in the rainy image dataset; removing images in the rainy image dataset where the proportion of rain density is less than 0.1 to obtain the preprocessed image X.
[0014] Further, step S2 specifically involves sampling the preprocessed image X according to dimensions h = 32 and w = 32 to obtain sampled samples, where h is the height and w is the width; the sampled samples are then sorted according to the minimum value... Normalization process to obtain normalized samples The normalized samples are divided into training set images and test set images.
[0015] Furthermore, in step S3, the simulated rain field generation network introduces the rain intensity R. i Number of peak rainfall events N p As a controllable property of rainy images:
[0016] R i To add rain intensity to rainy images;
[0017] R d This represents the percentage of rainy pixels in the rainy image dataset.
[0018] N p Number of peak rainfall events;
[0019] Y = G(z, R)
[0020] In the formula, Y is the generated rain map, and R = (R i R d N p Let ) be a set of vectors containing fixed attributes of the simulated rain field added manually, and z be the latent feature vectors of the training set images extracted by the encoder that conform to a standard Gaussian distribution. z is expressed as:
[0021]
[0022] In the formula, Indicates the normalized sample The expectation is calculated, where α and β are the latent features of the normalized rain field, and N(0, I) represents the standard Gaussian distribution. After encoding, a latent feature vector that satisfies the standard Gaussian distribution is formed.
[0023] Furthermore, in step S3, both the encoder and discriminator employ three convolutional layers with a kernel size of 5×5 and output channels of 128, 256, and 512, respectively; the discriminator input consists of normalized samples. The sampled real rainy images and generated rainy images are output as a 1D vector to distinguish between real and generated rainy images. During the training process, the generator and discriminator continuously compete against each other until they reach a dynamic equilibrium.
[0024] The generator employs transposed convolution with a kernel size of 4×4 and output channels of 512, 256, and 128, with a stride of 2 for each. The ReLU activation function is used after each convolutional layer.
[0025] Furthermore, in step S4, the discriminator is subjected to spectral normalization and self-attention mechanism as follows:
[0026] Spectral normalization is performed on each layer of the discriminator;
[0027] To obtain global correlations in simulated rain scenes, a self-attention mechanism is added between the last two convolutional layers of the discriminator. This self-attention mechanism handles vector sequences of varying lengths while simultaneously producing an output vector sequence y of the same length. The latent feature vector z is used as the input vector sequence, and Q, K, and V in the formula represent the learnable vector sequences.
[0028] Q = ω q z
[0029] K = ω k z
[0030] V=ω v z
[0031] In the formula, ω q ω k ω v Given a learnable parameter matrix, the output vector sequence y is represented as:
[0032]
[0033] In the formula, γ is the dynamic weight generated by the self-attention mechanism, j∈(1,N) is the position index of the input and output vector sequence, and Softmax is the function for column-wise normalization.
[0034] Furthermore, the Q, K, and V vector sequences in the self-attention mechanism are all obtained using convolutional layers with a kernel size of 1 and channel numbers of 32 and 256, respectively. The self-attention mechanism has a dimension of 256 and uses the ReLU activation function.
[0035] Furthermore, in step S5, through extensive training and hyperparameter optimization, the trained simulated rain field generation network is obtained as follows: the loss function includes a first adversarial loss function and a second adversarial loss function. The first adversarial loss function is used to make the image generated by the generator approximate the real data distribution. The first adversarial loss function is expressed as follows:
[0036]
[0037] In the formula, L adv To counteract the loss, it consists of discriminator loss D and generator loss 1-D, with the sum of discriminator loss and generator loss being 1;
[0038] The second adversarial loss function is derived from the first adversarial loss function L. adv The improved second adversarial loss function is obtained through the following process: sampling images from the training set and adding noise during the sampling process; and setting noise weights λ to optimize training capability. t As time increases, the noise n gradually decreases, causing the JS divergence to quantify, and the discriminator loss to remain in a state of flux; the second adversarial loss function L + (G, D) is represented as follows:
[0039]
[0040] In the formula, It is an autoencoded latent vector;
[0041] The first adversarial loss function and the second adversarial loss function are added together to obtain the total loss function.
[0042] During training, the hyperparameters are continuously optimized using an optimizer based on the total loss function value. The optimizer chosen is the Adam optimizer, and the hyperparameter is the noise weight λ. t The system determines whether the loss value no longer changes. If it does, training stops; otherwise, training continues. After extensive training and optimization, when the generator and discriminator reach Nash equilibrium, the trained simulated rain field generation network is obtained.
[0043] Furthermore, after obtaining the trained simulated rain field generation network in step S5, the process also includes using a novel reparameterization technique, namely, mapping normalized samples to autoencoded latent vectors of the second adversarial loss function through the encoder. It belongs to the standard Gaussian distribution N(0, I); The definition is as follows:
[0044]
[0045] In the formula, μ, ψ 2 This is the output vector of the encoder.
[0046] A simulated rainfield image generation system, comprising:
[0047] The preprocessing module is used to acquire a dataset of rainy images and preprocess the dataset to obtain preprocessed images.
[0048] The normalization module is used to normalize the preprocessed image to obtain normalized samples, which are divided into training set images and test set images.
[0049] The network building block is used to build a simulated rain field generation network;
[0050] The training module is used to train the simulated rain field generation network using training set images, and uses spectral normalization and self-attention mechanism on the discriminator to improve training capability.
[0051] The generation module is used to obtain a trained simulated rain field generation network through extensive training and hyperparameter optimization; the image to be processed is input into the trained simulated rain field generation network to generate a rainy image.
[0052] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:
[0053] This invention constructs a simulated rain field generation network by preprocessing and normalizing a dataset of rainy images. To obtain the global correlation of the images, spectral normalization and self-attention mechanisms are used on the discriminator to improve training ability, so that the simulated rain field generation network can synthesize more realistic and natural rainy images. Attached Figure Description
[0054] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the invention. To better illustrate this embodiment, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understood by those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0055] Figure 1 This is a schematic diagram of the process for generating simulated rain field images according to an embodiment of the present invention;
[0056] Figure 2 This is a schematic diagram of the overall structure of the simulated rain field generation network according to an embodiment of the present invention;
[0057] Figure 3This is the pixelation process for rainy images according to an embodiment of the present invention;
[0058] Figure 4 This is a pixelated mapping of the number of peak values in a rainy image according to an embodiment of the present invention;
[0059] Figure 5 The FID score change during the training process in this embodiment of the invention.
[0060] Figure 6 The changes in the loss function curves of generator G and discriminator D during the training process of this embodiment of the invention;
[0061] Figure 7 For the modification of the embodiments of the present invention r d The obtained simulated rain field distribution image;
[0062] Figure 8 Change n to the embodiments of the present invention p The obtained simulated rain field distribution image;
[0063] Figure 9 The simulated rain field distribution obtained by adding a fixed attribute vector to the random sampling in the embodiments of the present invention. Detailed Implementation
[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0065] Before introducing the embodiments of this application, the relevant terms involved in the embodiments of this application are first explained as follows:
[0066] Divergence can be used to characterize the strength of the divergence of vector fields at points in space.
[0067] Example 1
[0068] For easier understanding, please refer to Figure 1 and Figure 2 An embodiment of a method for generating simulated rain field images provided by the present invention includes the following steps:
[0069] S1: Obtain the rainy image dataset and preprocess the rainy image dataset to obtain preprocessed images;
[0070] The preprocessing specifically involves selecting rainy images from the SMHI (the Swedish Meteorological and Hydrological Institute) obtained through radar mapping measurements. There are a total of 26,125 rainy images, including many images with no or little rain.
[0071] The rain density of the images in the rainy image dataset is pixelated. Rain density represents the proportion of rainy pixels in the rainy image dataset. Images with a rain density proportion of less than 0.1 in the rainy image dataset are removed to obtain the preprocessed image X.
[0072] S2: As Figure 3 As shown, the preprocessed image is normalized to obtain normalized samples, which are divided into training set images and test set images. Specifically, the preprocessed image X (Original radar MAP) is sampled according to dimensions h = 32 and w = 32 to obtain sampled samples (unfiltered pixels), where h is the height and w is the width; the sampled samples are then sorted according to the minimum value... Normalization (filtered pixels) yields normalized samples. (Normalized map) divides the normalized samples into training set images and test set images;
[0073] Max pooling is used during normalization to pixelate and extract features from the training image. Max pooling takes the maximum value of the feature points in the neighborhood as the peak value. Figure 4 As shown, the brighter the pixel, the larger the peak value.
[0074] S3: Construct a simulated rain field generation network; wherein, the simulated rain field generation network includes: a generator: used to generate rainy images, which performs an inverse transformation of the probability distribution through a parameterized probabilistic generation model to obtain a generation probability distribution; a discriminator: used to distinguish between real rainy images in the test set and rainy images generated by the generator; and an encoder: used to extract latent feature vectors that conform to a standard Gaussian distribution.
[0075] Reference Figure 2 In the simulated rain field generation network, the encoder, generator, and discriminator are used to obtain a real natural simulated rain field with fixed attributes. E is the encoder, G is the generator, D is the discriminator, and SAMPLE is a sample taken from a real rainy image. The discriminator D is used to distinguish between the image generated by the generator G and the image generated by the normalized sample. Rainy images obtained through sampling;
[0076] Introducing rainfall intensity R iNumber of peak rainfall events N p As a controllable property of rainy images:
[0077] R i To add rain intensity to rainy images;
[0078] R d This represents the percentage of rainy pixels in the rainy image dataset.
[0079] N p Number of peak rainfall events;
[0080] Y = G(z, R)
[0081] In the formula, Y is the generated rain map, and R = (R i R d N p Let ) be a set of vectors containing fixed attributes of the simulated rain field added manually, and z be the latent feature vectors of the training set images extracted by the encoder that conform to a standard Gaussian distribution. z is expressed as:
[0082]
[0083] In the formula, Indicates the normalized sample The expectation is calculated, where α and β are the latent features of the normalized rain field, and N(0, I) represents the standard Gaussian distribution. After encoding, a latent feature vector that satisfies the standard Gaussian distribution is formed.
[0084] The latent feature vector is the latent vector corresponding to the latent space; the latent space is the space composed of latent vectors, that is, the space between the encoder and the generator.
[0085] Specifically, the encoder encodes the training set images to form latent spatial vectors for training. In the simulated rain field generation network, the encoder extracts latent feature vectors that conform to a standard Gaussian distribution.
[0086] This invention aims to design and generate more realistic and natural simulated rain fields. The simulated rain field generation network used is a common GAN architecture. Both the encoder and discriminator employ three convolutional layers with a kernel size of 5×5 and output channels of 128, 256, and 512, respectively. The discriminator input consists of normalized samples. The sampled real rainy images and generated rainy images are output as a 1D vector to distinguish between real and generated rainy images. During the training process, the generator and discriminator continuously compete against each other until they reach a dynamic equilibrium.
[0087] The generator employs transposed convolution, which is used in the simulated rain field generation network to learn the distribution of real images, making the generated images more realistic and fooling the discriminator. The convolution kernel size is 4×4, and the number of output channels is 512, 256, and 128, respectively, with a stride of 2 for each. The ReLU activation function is used after each convolution layer.
[0088] S4: A new reparameterization technique is used to extract latent variables from the encoder to facilitate the generator to learn deeper levels. In order to obtain better global correlation of rainy images, the simulated rain field generation network is trained with training set images to learn the rain distribution pattern. Spectral normalization and self-attention mechanism are used on the discriminator to improve training ability.
[0089] Specifically, the discriminator uses spectral normalization and self-attention mechanisms as follows:
[0090] Spectral normalization is performed on each layer of the discriminator;
[0091] To obtain global correlations in simulated rain scenes, a self-attention mechanism is added between the last two convolutional layers of the discriminator. This self-attention mechanism handles vector sequences of varying lengths while simultaneously producing an output vector sequence y of the same length. The latent feature vector z is used as the input vector sequence, and Q, K, and V in the formula represent the learnable vector sequences.
[0092] Q = ω q z
[0093] K = ω k z
[0094] V=ω v z
[0095] In the formula, ω q ω k ω v Given a learnable parameter matrix, the output vector sequence y is represented as:
[0096]
[0097] In the formula, γ is the dynamic weight generated by the self-attention mechanism, j∈(1,N) is the position index of the input and output vector sequence, and Softmax is the function for column-wise normalization.
[0098] The Q, K, and V vector sequences in the self-attention mechanism are all obtained using convolutional layers with a kernel size of 1 and 32 and 256 channels, respectively. The self-attention mechanism has a dimension of 256 and uses the ReLU activation function.
[0099] S5: Through extensive training and hyperparameter optimization, a trained simulated rain field generation network is obtained; the rainy image to be processed is input into the trained simulated rain field generation network to generate a simulated rain field image.
[0100] Specifically, the simulated rain field generation network obtained through extensive training and hyperparameter optimization is as follows: the loss function includes a first adversarial loss function and a second adversarial loss function. The first adversarial loss function is used to make the image generated by the generator approximate the real data distribution. The first adversarial loss function is expressed as follows:
[0101]
[0102] In the formula, L adv To counteract the loss, it consists of discriminator loss D and generator loss 1-D, with the sum of discriminator loss and generator loss being 1;
[0103] During training, it was found that the loss function curve did not change significantly because D=0, the JS divergence could not be quantified, and model training stalled; the second adversarial loss function was modified by the first adversarial loss function L. adv The improved second adversarial loss function is obtained through the following process: sampling images from the training set and adding noise during the sampling process; and setting noise weights λ to optimize training capability. t As time increases, the noise n gradually decreases, causing the JS divergence to quantify, and the discriminator loss to remain in a state of flux; the second adversarial loss function L + (G, D) is represented as follows:
[0104]
[0105] In the formula, For the autoencoded latent vector, it represents the result of a repetitive parameterization technique;
[0106] The first adversarial loss function and the second adversarial loss function are added together to obtain the total loss function. The problem of optimizing the maximum and minimum of the loss function.
[0107]
[0108] During training, the hyperparameters are continuously optimized using an optimizer based on the total loss function value. The optimizer chosen is the Adam optimizer, and the hyperparameter is the noise weight λ. t The system determines whether the loss value no longer changes. If it does, training stops; otherwise, training continues. After extensive training and optimization, when the generator and discriminator reach Nash equilibrium, the trained simulated rain field generation network is obtained.
[0109] Specifically, after obtaining the trained simulated rain field generation network in step S5, the process also includes using a new reparameterization technique, namely, mapping normalized samples to autoencoded latent vectors of the second adversarial loss function through the encoder. It belongs to the standard Gaussian distribution N(0, I); The definition is as follows:
[0110]
[0111] In the formula, μ, ψ 2 This is the output vector of the encoder.
[0112] It should be noted that the present invention constructs an autoencoded latent vector by squaring the encoder output variables. Mapping to the generated simulated rain field network makes the loss function curve change more obvious, further affecting the loss curve of the discriminator, thereby improving training performance; by mapping normalized samples to the autoencoded latent vector of the second adversarial loss function through the encoder, the training performance of the model is improved.
[0113] This invention uses a novel reparameterization technique on a training dataset of rainy images to form an autoencoded latent vector and applies a standard Gaussian distribution to it. The resulting simulated rainfield images exhibit a more natural distribution.
[0114] Example 2
[0115] Specifically, based on Example 1, the solution will be described in conjunction with specific embodiments to further demonstrate its technical effects. Specifically:
[0116] A simulated rainfield image generation system, comprising:
[0117] The preprocessing module is used to acquire a dataset of rainy images and preprocess the dataset to obtain preprocessed images.
[0118] The normalization module is used to normalize the preprocessed image to obtain normalized samples, which are divided into training set images and test set images.
[0119] The network building block is used to build a simulated rain field generation network;
[0120] The training module is used to train the simulated rain field generation network using training set images, and uses spectral normalization and self-attention mechanism on the discriminator to improve training capability.
[0121] The generation module is used to obtain a trained simulated rain field generation network through extensive training and hyperparameter optimization; the image to be processed is input into the trained simulated rain field generation network to generate a rainy image.
[0122] Example 3
[0123] Specifically, based on Example 1, the solution will be described in conjunction with specific embodiments to further demonstrate its technical effects. Specifically:
[0124] Figures 3-4 , Figures 7-9 The horizontal and vertical axes in the image represent the size of the window image. This is a format output by the Python image library and has no special meaning.
[0125] To verify the performance of the simulated rain field generation network of this invention, the generator was trained for 25 steps (representing the number of training iterations), the training block size was 128, the seed was 1, and the learning rate was 2×10⁻⁶. -4 The Adam optimizer is used for optimization, with gradient descent set to a default factor of 10, and the autoencoder latent vector size is 128. Adversarial loss and improved adversarial loss are used to evaluate the model training process. To increase training stability, spectral normalization is applied to the discriminator, while the generator remains unchanged. The internationally recognized FID (Fréchet Inception Distance) score is used as the performance evaluation metric for the simulated rain field generator network.
[0126] Figure 5 The FID score curve demonstrates the ability of the simulated rainfield generator network (SRFGNet) of this invention to generate simulated rainfields. The lower the score, the better the generator performance and the more realistic and natural the obtained simulated rainfield distribution. Figure 6 These are the generator loss function and discriminator loss function curves, designed to obtain the minimum generator loss and the maximum discriminator loss. Figure 5 , Figure 6 The horizontal axis represents the number of training iterations, and the vertical axis represents the FID score.
[0127] During the training of the simulated rain field generation network, ensuring good generative performance and optimizing the training process inevitably presents a dilemma: if the sole objective is to reduce the generator loss, the generation performance of the simulated rain field generation network will inevitably decline, resulting in an increase in the FID score; conversely, if the goal is to achieve the lowest possible FID score, the training process cannot be guaranteed to proceed normally. As shown in Table 1, the selected simulated rain field generation network training settings are the optimal choice.
[0128] Table 1. Average values of various indicators during the training process.
[0129] SRFGNet (Simulated Rain Field Generation Network) Average value FID 0.035 Discriminator loss 1.386 Generator loss 1.402
[0130] In its implementation, this invention also visually evaluates the generated natural rain field. This is based on the manually added simulated rain field attribute vector R = [R...]. i R d N p ].like Figure 7 As shown, keep R i =1, N p =5 remains unchanged, R d = [0.1, 0.5, 0.9], comparing the impact of the proportion of rain density in the rain field on the generation performance of the simulated rain field generation network. Figure 8 Demonstrates keeping R d =0.4, R i =1 remains unchanged, N p When the value is [5, 15, 25], the impact of the number of rain peaks on the performance of the simulated rain field generation network. The performance impact refers to the quality of the generated rain map after adding the rain peak number attribute vector to the simulated rain field generation network; that is, the simulated rain field generation network can distribute rain more realistically based on this attribute vector; the principle is similar for other attribute vectors.
[0131] like Figure 9 As shown, a comparison is made based on the two attributes of rain density and rainfall amount; keeping R... i =1,R d =0.4, N p With the value 5 unchanged, a set of generated data is randomly sampled from the generated samples. Regarding the uniformity of rain distribution, only the fourth image in the first column, the first two images in the second column, the first image in the third column, and the last image in the fourth column show uneven distribution. In terms of rainfall amount, each sub-image exhibits varying rainfall amounts. The results indicate that, under the same attribute vector, the rain field generation network of this invention generates images that simulate rain field distribution more realistically and naturally.
[0132] In the specific implementation process, the present invention also conducted a subjective evaluation, comparing the generated image with the randomly sampled normalized sample image, and selecting 20 participants to score whether the generated simulated rain field distribution was natural and realistic, with a full score of 100 points, and taking the average score. As shown in Table 2, the simulated rain field distribution generated by the simulated rain field generation network of the present invention is the most realistic and natural.
[0133] Table 2 Subjective Evaluation Score Table
[0134] Image Average score Generate image 93±0.3 Original image 89±0.1
[0135] In summary, the FID score and loss function curve were used to evaluate the model, and user subjective evaluation was used as the evaluation criterion for generating simulated rain fields. The results show that the simulated rain field generation network of this invention generates more realistic and natural simulated rain field distribution images compared to other methods. The simulated rain field generation network achieved the lowest FID score and best performance, producing more natural simulated rain field image distributions. This invention has good application prospects. In the future, this simulated rain field generation network will be combined with single-image rain removal tasks to achieve better rain removal performance, and applied to generate more realistic rainy images. In the civilian field, it can also be used in transportation systems to facilitate timely detection of traffic violations; in the military field, it can be used for UAV detection to make observations clearer and more accurate.
[0136] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A method for generating simulated rain field images, characterized in that, Includes the following steps: S1: Obtain the rainy image dataset and preprocess the rainy image dataset to obtain preprocessed images; S2: Normalize the preprocessed image to obtain normalized samples, which are divided into training set images and test set images; S3: Construct a simulated rain field generation network; wherein, the simulated rain field generation network includes: a generator: used to generate rainy images; a discriminator: used to distinguish between real rainy images in the test set and rainy images generated by the generator; and an encoder: used to extract latent feature vectors that conform to a standard Gaussian distribution. In step S3, the simulated rain field generation network introduces rainfall intensity. and peak rainfall As a controllable property of rainy images: To add rain intensity to rainy images; This represents the percentage of rainy pixels in the rainy image dataset. Number of peak rainfall events; In the formula, For the generated rain map, R= ( R i ,R d ,N p () is a set of vectors containing manually added fixed properties of the simulated rain field. The encoder extracts latent feature vectors that conform to a standard Gaussian distribution from the training set images. Expressed as: In the formula, Indicates the normalized sample Seeking expectations, As a potential characteristic of the normalized rainfall field, It represents a standard Gaussian distribution, which, after encoding, forms a latent feature vector that satisfies the standard Gaussian distribution. S4: Train the simulated rain field generation network using training set images, and use spectral normalization and self-attention mechanism on the discriminator to improve training ability; S5: Through extensive training and hyperparameter optimization, a trained simulated rain field generation network is obtained; the rainy image to be processed is input into the trained simulated rain field generation network to generate a simulated rain field image; In step S5, through extensive training and hyperparameter optimization, the trained simulated rain field generation network is obtained as follows: The loss function includes a first adversarial loss function and a second adversarial loss function. The first adversarial loss function is used to make the image generated by the generator approximate the real data distribution. The first adversarial loss function is expressed as follows: In the formula, To combat the loss, discriminator loss is used. and generator loss The sum of the discriminator loss and the generator loss is 1. The second adversarial loss function is derived from the first adversarial loss function. The improved second adversarial loss function is obtained through the following process: sampling images from the training set and adding noise during the sampling process; and setting noise weights for optimal training capability. As time goes on, noise... Gradually decrease, making Divergence quantization, discriminator loss is in a changing state; second adversarial loss function L + ( G,D The following is represented: In the formula, It is an autoencoded latent vector; The first adversarial loss function and the second adversarial loss function are added together to obtain the total loss function. , = L adv （ G,D ）+ L + （ G,D ）； During training, the hyperparameters are continuously optimized using an optimizer based on the total loss function value. The optimizer chosen is the Adam optimizer, and the hyperparameters are noise weights. The system determines whether the loss value no longer changes. If it does, training stops; otherwise, training continues. After extensive training and optimization, when the generator and discriminator reach Nash equilibrium, the trained simulated rain field generation network is obtained. After obtaining the trained simulated rain field generation network in step S5, the process also includes using a novel reparameterization technique, namely, mapping normalized samples to autoencoded latent vectors of the second adversarial loss function through the encoder. , It belongs to the standard Gaussian distribution ; The definition is as follows: In the formula, This is the output vector of the encoder.
2. The method for generating simulated rain field images according to claim 1, characterized in that, Step S1 preprocessing specifically involves: pixelating the rain density of the images in the rainy image dataset, where rain density represents the proportion of rainy pixels in the dataset; and removing images where the rain density proportion is less than 0.1 to obtain the preprocessed image. .
3. The method for generating simulated rain field images according to claim 2, characterized in that, Step S2 specifically involves processing the preprocessed image. According to size The sampling obtained is a sample, of which For height, Width; sample the data according to the minimum value Normalization process to obtain normalized samples The normalized samples are divided into training set images and test set images.
4. The method for generating a simulated rain field image according to claim 1, characterized in that, The encoder and discriminator described in step S3 both employ three convolutional layers, with a kernel size of [missing value]. The number of output channels are 128, 256, and 512, respectively; the discriminator input is composed of normalized samples. The sampled real rainy images and generated rainy images are output as a 1D vector to distinguish between real and generated rainy images. During the training process, the generator and the discriminator continuously compete against each other, and eventually the generator and the discriminator reach a dynamic equilibrium. The generator employs a transposed convolution with a kernel size of [size missing]. The number of output channels are 512, 256 and 128 respectively, and the stride is set to 2 for each; the ReLU activation function is used after each convolution.
5. The method for generating simulated rain field images according to claim 1, characterized in that, In step S4, the discriminator is subjected to spectral normalization and self-attention mechanism as follows: Spectral normalization is performed on each layer of the discriminator; To capture global correlations in simulated rain scenes, a self-attention mechanism is added between the last two convolutional layers of the discriminator. This self-attention mechanism handles vector sequences of varying lengths while simultaneously producing an output vector sequence of the same length. , latent feature vector As the input vector sequence, in the formula As a learnable sequence of vectors: In the formula, q , k , v Given a learnable parameter matrix, the output vector sequence Represented as: In the formula, Dynamic weights generated for the self-attention mechanism. represents the position index of the input and output vector sequences; Softmax is a function that normalizes the vectors column-wise.
6. The method for generating a simulated rain field image according to claim 5, characterized in that, The self-attention mechanism The vector sequences were obtained using convolutional layers with a kernel size of 1 and 32 and 256 channels, respectively. The self-attention mechanism had a dimension of 256 and used the ReLU activation function.
7. A simulated rainfield image generation system, used to implement the simulated rainfield image generation method described in claim 1, characterized in that, include: The preprocessing module is used to acquire a dataset of rainy images and preprocess the dataset to obtain preprocessed images. The normalization module is used to normalize the preprocessed image to obtain normalized samples, which are divided into training set images and test set images. The network building block is used to build a simulated rain field generation network; The training module is used to train the simulated rain field generation network using training set images, and uses spectral normalization and self-attention mechanism on the discriminator to improve training capability. The generation module is used to obtain the trained simulated rain field generation network through extensive training and hyperparameter optimization; The image to be processed is input into the trained simulated rain field generation network to generate a rainy image.