A method for augmenting traffic flow dataset in rainy and foggy weather based on image style transfer

Abstract

The present invention provides a method for augmenting a rainy and foggy weather traffic flow dataset based on image style transfer. The method obtains traffic flow datasets for normal weather and rainy and foggy weather, as well as a traffic flow image sequence for a target road section. A generative adversarial network is constructed to transform the scenes in the normal weather dataset into rainy and foggy weather scenes. The method first separates the background domain from the target domain in the dataset, then feeds the target domain samples and the entire image samples into the adversarial network for style transfer learning. The background domain images are then fed into a classifier to obtain specific scene labels, completing the augmentation of the rainy and foggy weather scene dataset. Finally, the trained network model is introduced into the target road section image sequence to obtain an augmented dataset of the target road section scene in rainy and foggy weather. The present invention improves the imaging quality of existing style-transferred images, increases the clarity and recognizability of targets in the transferred images, and features a stable algorithm training process with strong portability, making it suitable for rainy and foggy weather data and augmentation for various traffic scenarios.

Description

Technical Field

[0001] The present invention relates to transfer learning and adversarial network technology, and in particular to a method for amplifying a traffic flow dataset in rainy and foggy weather based on image style transfer. Background Art

[0002] Sufficient and well-annotated datasets are the foundation for advancing computer vision research. However, creating such datasets requires a lot of manpower and resources. In the field of transportation, NGSIM (Next Generation Simulation) is the most widely used image-based vehicle dataset. However, many studies have shown that NGSIM data has irreparable errors, and real datasets of comparable size have not been released in recent years. Therefore, synthetic datasets have become the focus of attention in the field of image processing. However, datasets based solely on synthetic data, such as 3D rendering and 3D modeling, are usually not suitable for real images. The solution to this problem is to use style transfer to transfer the objects and features of a certain scene in the real dataset to another scene, that is, to transfer the knowledge learned in the source domain of labeled data to the target domain of unlabeled data.

[0003] Rainy and foggy weather conditions affect the quality of visible light images, and further impact the clarity and recognizability of objects in the images. Traffic object detection and recognition algorithms used in normal weather conditions often perform poorly in rainy and foggy weather. Therefore, it is necessary to create a traffic flow dataset for rainy and foggy weather to train traffic information perception algorithms such as detection and tracking in these environments.

[0004] Dataset augmentation based on style transfer is a feasible and effective image processing technology. Existing research includes Chinese patent CN202011517955.6, which discloses a method for amplifying silk artifact images based on style transfer, and Chinese patent CN202111005177.7, which discloses a method for data augmentation of small sample scenes. These methods employ the WCT algorithm for style transfer calculations. However, these existing methods suffer from unstable training iterations, taking a long time to generate a suitable model for a specific scene, and significantly degrade image edge quality after transfer. The target structure and contours in the image differ significantly from those in the source image, making them unsuitable for subsequent traffic information perception and processing.

[0005] Traffic information perception in rainy and foggy weather is a feasible and effective algorithm for intelligent transportation systems. In existing studies, Chinese patent CN202010191052.7 proposed a method for vehicle detection using infrared images in rainy and foggy weather, and Chinese patent CN201410387383.2 disclosed a vehicle detection method for all-weather traffic environments based on multi-feature fusion, in which virtual coil data is used to capture vehicle information. In general, most studies consider replacing visible light data sources to solve the problem of traffic target perception in rainy and foggy weather. However, visible light data has intuitive imaging, rich semantic features, and low deployment cost. It is the most ideal perception data type in intelligent transportation systems. It is necessary to consider making full use of existing normal weather traffic data sets to provide support for traffic target perception in rainy and foggy weather.

[0006] In view of this, it is necessary to provide a new method to solve at least part of the above problems. Summary of the Invention

[0007] In order to overcome the shortcomings of the existing technology, the present invention proposes a method for augmenting a rainy and foggy weather traffic flow dataset based on image style transfer. The technology first obtains normal and rainy and foggy weather traffic flow datasets and a target road section image sequence. Next, a generative adversarial network is constructed to convert the normal weather dataset scene into rainy and foggy weather. The background domain and target domain in the dataset are first segmented. Then, the target domain samples and the whole image samples are respectively sent to the adversarial network for style transfer learning. Then, the background domain image is sent to the classifier to obtain a specific scene label to complete the rainy and foggy weather scene dataset augmentation. Finally, the trained network model is brought into the target road section image sequence to obtain the target road section scene rainy and foggy weather amplified dataset. The present invention improves the imaging quality of existing style transfer images, increases the clarity and recognizability of targets in the transferred images, and the algorithm training process is stable and highly portable. It is suitable for rainy and foggy weather data and augmentation of various traffic scenarios.

[0008] Technical solution: To solve the above technical problems, the present invention adopts the following technical solution:

[0009] A method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer, comprising:

[0010] S1: Obtain a traffic flow image dataset D under normal weather conditions n With label L n , obtain the traffic flow image dataset D under rainy and foggy weather conditions a With label L a , obtain a traffic flow video image sequence D of a target road section t ;

[0011] S2: Label L of traffic flow images under normal weather conditions nFrom the traffic flow image dataset D under normal weather conditions n The target domain image samples and background domain image samples are segmented, and the label L of the traffic flow picture under rainy and foggy weather conditions is obtained. a From the traffic flow image dataset D under rainy and foggy weather conditions a Segmenting target domain image samples and background domain image samples, wherein the background domain image samples are image samples in the traffic flow image dataset stripped of target domain image samples;

[0012] S3: Construct an unsupervised generative adversarial network, and send the target domain image samples and the complete image samples into the unsupervised generative adversarial network for style transfer learning to obtain the target domain D after transfer. t,f Compared with the complete image D after migration t,p ;

[0013] S4: Traffic flow image dataset D under normal weather conditions n Traffic flow image dataset D under rainy and foggy weather conditions a The background domain image samples are fed into the classifier T in the unsupervised generative adversarial network. bg , get the specific scene label Based on the target domain D after migration tf , the complete image after migration D t and scene-specific tags Form a style transfer model for traffic flow image datasets from normal weather conditions to rainy and foggy weather conditions;

[0014] S5: The traffic flow video image sequence D of the target road section t Input classifier T in the style transfer model bg , get the traffic flow background label of the target road section And generate the corresponding background, and randomly transfer the target domain D t,f Mapping to the above background, we get the traffic flow dataset D of the target road section under rainy and foggy weather conditions. ft With label L ft .

[0015] Furthermore, the method for expanding the traffic flow dataset in rainy and foggy weather based on image style transfer of the present invention includes the traffic flow image dataset D under normal weather conditions in step S1. n The acquisition angle of the labeled image and the traffic flow video image sequence D of the target road section t The traffic flow image dataset D under rainy and foggy weather conditions has the same or similar collection angle. a The labeled images include the traffic flow video image sequence D of the target road section t The scene of the acquisition perspective.

[0016] Furthermore, in the method for expanding the rainy and foggy weather traffic flow dataset based on image style transfer of the present invention, the label L of the traffic flow image under normal weather conditions in step S1 is n , Label of traffic flow images under rainy and foggy weather conditions Label L a , Traffic flow label L of the target road section under rainy and foggy weather conditions ft The type of is bounding box or target mask, and the three types are the same.

[0017] Furthermore, the method for expanding the traffic flow dataset in rainy and foggy weather based on image style transfer of the present invention includes the traffic flow image dataset D under normal weather conditions in step S1. n Contains label L n The total number of images is greater than 5000, and the traffic flow image dataset D under rainy and foggy weather conditions a The total number of labeled images is greater than 100.

[0018] Furthermore, in the method for expanding the rainy and foggy weather traffic flow dataset based on image style transfer of the present invention, the generative adversarial network structure for style transfer learning in step S3 includes a generator G, a discriminator D, and a classifier T. bg With stabilizer St, where:

[0019] The input of the generator G includes a traffic flow image dataset D under normal weather conditions n With random noise, the output of generator G is rain and fog migration image D t ;

[0020] The input of the discriminator D is the rain and fog migration image D t Traffic flow image dataset D under rainy and foggy weather conditions a , the output of the discriminator D is the rain and fog migration image D t Traffic flow image dataset D under rainy and foggy weather conditions a The discrimination results and the adjustment strategy of the generator G;

[0021] Classifier T bg The input is the traffic flow image dataset D under normal weather conditions n Traffic flow image dataset D under rainy and foggy weather conditions a The background image of the target domain sample is stripped, and the classifier T bg The output is a specific scene label

[0022] The input of the stabilizer St is the target domain result D after migration t,f Label L for traffic flow images under rainy and foggy weather conditions a The corresponding entire image result after migration is D t,p, the output of the stabilizer St is the adjusted complete rain and fog weather dataset D tj With the corresponding label L tj .

[0023] Furthermore, in the method for expanding the rainy and foggy weather traffic flow dataset based on image style transfer of the present invention, the structure of the generator G is a convolutional neural network including a residual module, wherein the random noise is specialized through a fully connected layer and compared with the traffic flow image dataset D under normal weather conditions. n The image is merged, and then downsampled by a convolution layer with a step size of 1 and a dimension of 64 and the activation function relu. Then, the residual features are extracted by a 5-layer residual module, and upsampled by a deconvolution layer with a step size of 1 and a dimension of 3. Finally, the rain and fog migration image D is output after the activation function tanh. t .

[0024] Furthermore, in the method for amplifying a traffic flow dataset in rainy and foggy weather based on image style transfer of the present invention, the discriminator D uses a four-layer convolutional layer to extract features, and normalizes the features through batch normalization BN and activation function lrelu, and finally outputs the probability that the corresponding image is true through the activation function tanh.

[0025] Furthermore, in the method for expanding the rainy and foggy weather traffic flow dataset based on image style transfer of the present invention, the classifier T bg It includes two 4*4 downsampling layers. After the two downsampling layers, the background image still retains key scene information such as road range, lighting conditions, and acquisition perspective on the basis of compressed size.

[0026] Furthermore, in the method for augmenting a rainy and foggy weather traffic flow dataset based on image style transfer of the present invention, in the style transfer adversarial network of step S3, the loss function of the generator G is as follows:

[0027]

[0028] in, Represents the generator adversarial loss, given by the discriminator in the adversarial network, L con is the content loss, which is used to describe the underlying information difference between the generated image and the source image. t is the classifier loss, which is used to distinguish the differences in the generated images in a specific scene given by the classifier. λ1, λ2, and λ3 are weight coefficients for adjusting the importance of the three losses;

[0029] Fighting Losses Take the cross entropy of the discriminator D on the pseudo fusion image, that is:

[0030]

[0031] Where D(G(v,i)) represents the probability that the discriminator D judges the generated image as true;

[0032] Content loss L con The definition is as follows:

[0033]

[0034] Among them, L SSIM Represents the structural loss calculated by the structural similarity index SSIM, which is used to constrain the structural similarity of the generated image. PMSE Represents the loss of pixel-to-pixel mean square error MSE, which is used to constrain the edge similarity of the generated image. δ1 and δ2 are used to adjust L SSIM and L PMSE Importance weight parameter.

[0035] Furthermore, in the method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer of the present invention, in the style transfer adversarial network of step S3, the loss function of the discriminator D is as follows:

[0036] L D =p1E cross [-logD(D n )]+p2E cross [-log(1-D(G(D n ,z)))

[0037] Among them, p1 and p2 are weight coefficients reflecting the amount of information of the source image and the generated image, and the calculation method is the same as the weight calculation method in the generator SSIM loss function.

[0038] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0039] 1. The present invention's method for augmenting a rainy and foggy weather traffic flow dataset based on image style transfer employs a generative adversarial network (GAN) for image style transfer to address the problem of augmenting a rainy and foggy weather traffic flow dataset. First, the scene background domain and target domain are divided based on existing datasets. Next, a large number of normal weather datasets and a small number of rainy and foggy weather datasets are fed into the GAN, synthesizing the rainy and foggy weather into the normal weather scene. The background domain is then fed into a classifier to obtain scene-specific background label information. Finally, the trained GAN model is introduced into the target image sequence to complete the augmentation of the rainy and foggy weather dataset for the target scene. This method improves the imaging quality of existing style-transferred images, increasing the clarity and recognizability of targets in the transferred images. The algorithm training process is stable and highly portable, making it applicable to rainy and foggy weather data and augmentation for various traffic scenarios. This provides an excellent data source for traffic scenario applications such as monitoring data collection and vehicle safe driving in rainy and foggy weather.

[0040] 2. The method for augmenting the rainy and foggy weather traffic flow dataset based on image style transfer of the present invention designs a rainy and foggy weather dataset augmentation adversarial network generator loss function that includes content loss. The content loss is defined as the weighted sum of SSIM structural loss and PMSE pixel loss. The structural information and texture information of the source image are retained in the migrated image, taking into account the content consistency and style variability of the style transfer algorithm, effectively improving the imaging quality of the migrated image and the clarity of the target in the image.

[0041] 3. The method for augmenting the rainy and foggy weather traffic flow dataset based on image style transfer of the present invention designs a background representation capability for the classifier training model, and outputs the feature map after downsampling the source image as the background label of a specific scene. On the basis of compressing the background image size, it can still retain key scene information such as road range, lighting conditions, and acquisition perspective, which is conducive to extracting data scene information, enabling the algorithm to generate training set images of similar scenes according to the background label, expand the scope of application of the target scene, enhance the versatility of the model and the portability of the augmented dataset. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide a further understanding of the present invention and, together with the description, to explain the embodiments of the present invention, but do not constitute a limitation of the present invention. In the accompanying drawings:

[0043] Figure 1 This is a technical flow chart of the method for augmenting traffic flow dataset in rainy and foggy weather based on image style transfer proposed by the present invention.

[0044] Figure 2 This is a schematic diagram of the overall network structure of the method for augmenting traffic flow datasets in rainy and foggy weather based on image style transfer proposed in this invention. DETAILED DESCRIPTION

[0045] In order to further understand the present invention, preferred embodiments of the present invention are described below in conjunction with examples. However, it should be understood that these descriptions are only for further illustrating the features and advantages of the present invention, rather than limiting the claims of the present invention.

[0046] The description in this section is based on typical embodiments only, and the present invention is not limited to the scope of the embodiments described. Combinations of different embodiments, replacement of certain technical features in different embodiments, and replacement of certain technical features in the embodiments with the same or similar prior art methods are also within the scope of the present invention.

[0047] like Figure 1 The figure shows a flow chart of a method for augmenting a rainy and foggy weather traffic flow dataset based on video image style transfer. The specific steps are as follows:

[0048] Step 1: Obtain a large number of traffic flow image datasets D under normal weather conditions n With label L n , obtain a small amount of traffic flow image dataset D under rainy and foggy weather conditions a With label L a , obtain a target road section traffic flow video image sequence D t Among them, the traffic flow image dataset D under normal weather conditions n The total number of labeled images in the video is more than 5,000, and the video image sequence D of the traffic flow of the target road section is collected from the viewing angle. t The collection angle is similar, label L n It can be an external frame or target mask, but the traffic flow label L under normal weather conditions n , Traffic flow label L under rainy and foggy weather conditions a Traffic flow label L of the target road section under rainy and foggy weather conditions ft The type is consistent, traffic flow image dataset D under rainy and foggy weather conditions a The total number of labeled images in the dataset is more than 100, and includes the target road section traffic flow video image sequence D t Capture the scene from the perspective.

[0049] Step 2: Traffic flow label L under normal weather conditions n Traffic flow label L under rainy and foggy weather conditions a The target domain source images and background domain source images are segmented from their corresponding datasets.

[0050] Step 3: Construct an unsupervised generative adversarial network to train the traffic flow image dataset D under normal weather conditions n The scene is transformed into rainy and foggy weather conditions.

[0051] The target domain source image and the complete source image are respectively sent to the adversarial network for style transfer learning to obtain the target domain result D after transfer. t,f Compared with the complete image result D t,p The adversarial network structure of the style transfer is as follows:

[0052] 1) The overall structure of the adversarial network is as follows Figure 2 As shown, it includes generator G, discriminator D, and classifier T bg With the stabilizer St. where the generator input includes D n With random noise, the generator outputs the rain and fog migration image D t ; The discriminator input is the rain and fog migration image D t Traffic flow image dataset D under rainy and foggy weather conditions a , the output is the rain and fog migration image D t Traffic flow image dataset D under rainy and foggy weather conditions aTrue and false discrimination results and adjustment strategy of generator G; classifier T bg The input is a traffic flow image dataset D under normal weather conditions. n Traffic flow image dataset D under rainy and foggy weather conditions a The background image of the target domain sample is stripped and the output is the specific scene sample label The input of the stabilizer St is the target domain D after migration t,f Traffic flow images with rain and fog weather conditions label L a Corresponding background D t,p , the output is the adjusted complete rain and fog weather dataset D tj With the corresponding label L tj .

[0053] 2) The structure of the adversarial network generator G is a convolutional neural network with residual connections, in which the noise is first specialized through a fully connected layer and merged with the image. It is then downsampled through a convolutional layer with a step size of 1 and a dimension of 64 and a relu activation function. After that, the residual features are extracted through 5 layers of residual blocks and upsampled through a deconvolution layer with a step size of 1 and a dimension of 3. Finally, the rain and fog generated image D is output through a tanh function. t .

[0054] 3) The adversarial network discriminator D uses four convolutional layers to extract features, and normalizes the features through BN (batch normolization) and lrelu (Leaky ReLU) to alleviate the network gradient vanishing problem. Finally, the tanh activation function is used to output the probability that the corresponding image is real.

[0055] 4) Classifier T bg The structure consists of two 4*4 downsampling layers. After the two downsampling layers, the feature map can still retain key scene information such as road range, lighting conditions, and acquisition viewing angle on the basis of compressing the background image size. This is conducive to extracting data scene information, enabling the algorithm to generate training set images of similar scenes based on background labels, expanding the scope of application of target scenes and enhancing the versatility of the training set.

[0056] In the style transfer adversarial network, the loss function of each network is calculated as follows:

[0057] 5) The generator loss function is expressed as follows:

[0058]

[0059] in, Represents the generator adversarial loss, given by the discriminator D in the adversarial network; L con is the content loss, which is used to describe the difference in underlying information between the generated image and the source image; L tis the classifier loss, which is used to distinguish the differences in the generated images in a specific scene given by the classifier; λ1, λ2 and λ3 are weight coefficients for adjusting the importance of the three losses.

[0060] Adversarial loss is one of the ways in which the generator G interacts with the discriminator D. It is also the channel for the adversarial network to implement the "adversarial" operation. Optimizing the adversarial loss can improve the ability of the generator G to generate images with better fusion effects, and can also increase the source image information in the fused image. In this invention, the adversarial loss takes the cross entropy of the discriminator D's discrimination results on the pseudo-fused image, that is:

[0061]

[0062] Among them, D(G(v,i)) represents the probability that the discriminator judges the generated image to be true.

[0063] The content loss defines the correlation between the pseudo-fused image generated by the generator G and the input source image, and determines which source image features will be retained in the fused image. It is necessary to take into account both the consistency of content and the variability of style. Structural features that are invariant to both image source and lighting changes are the key components of the content loss function. At the same time, in order to enhance the clarity of the target domain content outline, additional low-level edge features should be added to improve the clarity and position accuracy of the perceived target in the fused image. Taking the above into consideration, the content loss of the adversarial network of the present invention is defined as follows:

[0064]

[0065] Among them, L SSIM Represents SSIM (Structure Similarity Index Measure) structural loss, which is used to constrain the structural similarity of generated images. PMSE Represents the pixel pair MSE (Pairwise Mean Squared Error) loss, which is used to constrain the edge similarity of the generated image. δ1 and δ2 are weight parameters for adjusting the importance of the two losses.

[0066] Specifically, in the SSIM network, the input image can be represented by three features: structure, brightness, and contrast:

[0067]

[0068] Among them, ||.|| represents the L2 norm calculation, Representative I k The pixel mean, Represents the difference between the original image and the mean pixel, k∈{i,v}, indicating whether the original image type is visible light or infrared. Contrast c kTo a certain extent, it reflects the quality of the image, so the maximum contrast is used to obtain the expected contrast.

[0069]

[0070] Structural features k It is the key feature in the SSIM loss function. Taking into account the influence of dual-source images, the expected structural features are calculated in a weighted manner.

[0071]

[0072]

[0073] For the weight parameter w(I k ), when the image feature s k When the difference is large, it is likely that the information content of a certain source is small and the image quality is poor. In this case, the contrast c k The higher the source image structure features are, the higher the weight is, so that the fused image can obtain more details from the source image with better quality; when the image feature s k When they are close, it means that the information content of the dual-source images is equivalent. At this time, without increasing the network parameters and accelerating the network convergence speed, equal weights are given to the dual-source structural features.

[0074] For the brightness feature l k ,Since it varies greatly in the source image and cannot represent stable image features well, it is not considered when calculating the loss function.

[0075] According to the above discussion, the fused image is expected to It can be expressed as:

[0076]

[0077] For the pseudo-fused image I f , the SSIM loss of pixel p can be expressed as:

[0078]

[0079] in represents the expected image variance, Represents the covariance between the expected image and the pseudo fused image.

[0080] Total SSIM loss function L SSIM It can be expressed as:

[0081]

[0082] Pixel-pair MSE loss is used to generate the similarity of corresponding pixels between image pairs rather than the similarity of the entire image. This loss allows the model to focus on the target domain area with more details without spending extra computing power on distinguishing irrelevant details such as color and pixel intensity. PMSE is calculated as follows:

[0083]

[0084] Among them, D n is the source image of the dataset, G(D n ,z) is the generated image, k represents the number of pixels in the source image, ||.|| 2 represents the two-norm, is the Hadamard product operation, and m is the target domain mask.

[0085] 6) In the adversarial network, the discriminator D needs to distinguish the authenticity of the source image and the generated image. At the same time, since there is no true reference image as input in the discriminator, the loss function of the discriminator D should only be composed of the generated image discrimination result and the source image discrimination result. The loss of the adversarial network discriminator D is expressed as follows:

[0086] L D =p1E cross [-logD(D n )]+p2E cross [-log(1-D(G(D n ,z)))

[0087] Among them, p1 and p2 are weight coefficients reflecting the amount of information of the source image and the generated image, and the calculation method is the same as the weight calculation method in the generator SSIM loss function.

[0088] 7) Classifier T bg The role of is to extract data scene information, so that the algorithm can generate training set images of similar scenes according to the background label, expand the scope of application of the target scene, and enhance the versatility of the training set. Therefore, the texture similarity between the backgrounds should be more reflected in the loss, so the TV norm is used as the classifier T bg The loss function is:

[0089] L t =E cross [|||G(D n ,z)-D n || TV ]

[0090] Step 4: Collect a large number of traffic flow image datasets D under normal weather conditions n A small amount of traffic flow image dataset D under rainy and foggy weather conditions a The background image of the target domain sample is stripped and fed into the classifier Tbg , get the label containing specific background information That is, the migration of the traffic flow dataset from normal weather conditions to rainy and foggy weather conditions is completed.

[0091] Step 5: Import the target road section traffic flow video image sequence D t To classifier T bg , obtain the target road section traffic flow background label Random D t,f Mapped to the label Generated background D t,p In the process, we obtain the traffic flow dataset D under rainy and foggy weather conditions on the target road section. ft With label L ft .

[0092] The description and application of the present invention here are illustrative and are not intended to limit the scope of the present invention to the above-mentioned embodiments. The relevant descriptions of the effects or advantages involved in the specification may not be reflected in the actual experimental examples due to the uncertainty of specific condition parameters or other factors, and the relevant descriptions of the effects or advantages are not used to limit the scope of the invention. Variations and changes to the embodiments disclosed here are possible, and the replacement of the embodiments and various equivalent components are well known to those of ordinary skill in the art. It should be clear to those skilled in the art that, without departing from the spirit or essential characteristics of the present invention, the present invention can be implemented in other forms, structures, arrangements, proportions, and with other components, materials and parts. Without departing from the scope and spirit of the present invention, other variations and changes can be made to the embodiments disclosed here.

Claims

1. A method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer, characterized in that: include: S1: Obtain a traffic flow image dataset D under normal weather conditions n With label L n , obtain the traffic flow image dataset D under rainy and foggy weather conditions a With label L a , obtain a traffic flow video image sequence D of a target road section t ; S2: Label L of traffic flow images under normal weather conditions n From the traffic flow image dataset D under normal weather conditions n The target domain image samples and background domain image samples are segmented, and the label L of the traffic flow picture under rainy and foggy weather conditions is obtained. a From the traffic flow image dataset D under rainy and foggy weather conditions a Segmenting target domain image samples and background domain image samples, wherein the background domain image samples are image samples in the traffic flow image dataset stripped of target domain image samples; S3: Construct an unsupervised generative adversarial network, and send the target domain image samples and the complete image samples into the unsupervised generative adversarial network for style transfer learning to obtain the target domain D after transfer. t,f Compared with the complete image D after migration t,p ; S4: Traffic flow image dataset D under normal weather conditions n Traffic flow image dataset D under rainy and foggy weather conditions a The background domain image samples are fed into the classifier T in the unsupervised generative adversarial network. bg , get the specific scene label Based on the target domain D after migration t,f , the complete image after migration D t,p and scene-specific tags Form a style transfer model for traffic flow image datasets from normal weather conditions to rainy and foggy weather conditions; S5: The traffic flow video image sequence D of the target road section t Input classifier T in the style transfer model bg , get the traffic flow background label of the target road section And generate the corresponding background, and randomly transfer the target domain D t,f Mapping to the above background, we get the traffic flow dataset D of the target road section under rainy and foggy weather conditions. ft With label L ft .

2. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 1 is characterized in that: The traffic flow image dataset D under normal weather conditions in step S1 n The acquisition angle of the labeled image and the traffic flow video image sequence D of the target road section t The traffic flow image dataset D under rainy and foggy weather conditions has the same or similar collection angle. a The labeled images include the traffic flow video image sequence D of the target road section t The scene of the acquisition perspective.

3. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 1 is characterized in that: The label L of the traffic flow picture under normal weather conditions in step S1 n , Label of traffic flow images under rainy and foggy weather conditions Label L a , Traffic flow label L of the target road section under rainy and foggy weather conditions ft The type of is bounding box or target mask, and the three types are the same.

4. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 1 is characterized in that: The traffic flow image dataset D under normal weather conditions in step S1 n Contains label L n The total number of images is greater than 5000, and the traffic flow image dataset D under rainy and foggy weather conditions a The total number of labeled images is greater than 100.

5. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 1, characterized in that: The generative adversarial network structure for style transfer learning in step S3 includes a generator G, a discriminator D, and a classifier T. bg With stabilizer St, where: The input of the generator G includes a traffic flow image dataset D under normal weather conditions n With random noise, the output of generator G is rain and fog migration image D t ; The input of the discriminator D is the rain and fog migration image D t Traffic flow image dataset D under rainy and foggy weather conditions a , the output of the discriminator D is the rain and fog migration image D t Traffic flow image dataset D under rainy and foggy weather conditions a The discrimination results and the adjustment strategy of the generator G; Classifier T bg The input is the traffic flow image dataset D under normal weather conditions n Traffic flow image dataset D under rainy and foggy weather conditions a The background image of the target domain sample is stripped, and the classifier T bg The output is a specific scene label The input of the stabilizer St is the target domain result D after migration t,f Label L for traffic flow images under rainy and foggy weather conditions a The corresponding entire image result after migration is D t,p , the output of the stabilizer St is the adjusted complete rain and fog weather dataset D tj With the corresponding label L tj .

6. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 5, characterized in that: The structure of the generator G is a convolutional neural network containing a residual module, in which random noise is specialized through a fully connected layer and compared with the traffic flow image dataset D under normal weather conditions. n The image is merged, and then downsampled by a convolution layer with a step size of 1 and a dimension of 64 and the activation function relu. Then, the residual features are extracted by a 5-layer residual module, and upsampled by a deconvolution layer with a step size of 1 and a dimension of 3. Finally, the rain and fog migration image D is output after the activation function tanh. t .

7. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 5, characterized in that: The discriminator D uses four convolutional layers to extract features, normalizes the features through batch normalization (BN) and activation function lrelu, and finally outputs the probability that the corresponding image is true through activation function tanh.

8. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 5, characterized in that: The classifier T bg It includes two 4*4 downsampling layers. After the two downsampling layers, the background image still retains key scene information such as road range, lighting conditions, and acquisition perspective on the basis of compressed size.

9. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 1, characterized in that: In the style transfer adversarial network of step S3, the loss function of the generator G is as follows: in, Represents the generator adversarial loss, given by the discriminator in the adversarial network, L con is the content loss, which is used to describe the underlying information difference between the generated image and the source image. t is the classifier loss, which is used to distinguish the differences in the generated images in a specific scene given by the classifier. λ1, λ2, and λ3 are weight coefficients for adjusting the importance of the three losses; Fighting Losses Take the cross entropy of the discriminator D on the pseudo fusion image, that is: Where D(G(v,i)) represents the probability that the discriminator D judges the generated image as true; Content loss L con The definition is as follows: Among them, L SSIM Represents the structural loss calculated by the structural similarity index SSIM, which is used to constrain the structural similarity of the generated image. PMSE Represents the loss of pixel-to-pixel mean square error MSE, which is used to constrain the edge similarity of the generated image. δ1 and δ2 are used to adjust L SSIM and L PMSE Importance weight parameter.

10. The method for augmenting a traffic flow dataset in rainy and foggy weather based on image style transfer according to claim 1, characterized in that: In the style transfer adversarial network of step S3, the loss function of the discriminator D is as follows: L D =p1E cross [-logD(D n )]+p2E cross [-log(1-D(G(D n ,z))) Among them, p1 and p2 are weight coefficients reflecting the amount of information of the source image and the generated image, and the calculation method is the same as the weight calculation method in the generator SSIM loss function.

Images