A night bridge traffic detection data enhancement method based on CycleGAN

Abstract

The application discloses a night bridge traffic detection data enhancement method based on CycleGAN. The method comprises the following steps: complete labeling of daytime traffic picture data to obtain true daytime traffic picture data with complete labels, and partial labeling of night traffic picture data to obtain true night traffic picture data with partial labels; constructing a night bridge traffic detection data enhancement model based on CycleGAN, and training the night bridge traffic detection data enhancement model; multi-scale screening of the trained night bridge traffic detection data enhancement model; and obtaining enhanced night bridge traffic detection data according to the true daytime traffic picture data with complete labels and the final night bridge traffic detection data enhancement model. The application not only improves the recognition accuracy and generalization ability of the YOLOv5 target detection model, achieves the effect of generative data enhancement, and reduces the labor cost of night traffic data labeling.

Description

Technical Field

[0001] This invention relates to the field of bridge traffic load identification, specifically to a method for enhancing bridge traffic detection data at night based on CycleGAN. Background Technology

[0002] Currently, significant progress has been made in computer vision-based bridge traffic load recognition methods. However, most detection models are trained and validated primarily based on daytime scene data, resulting in significant limitations in their applicability to nighttime scenarios. Traffic flow on bridges exhibits different characteristics at different times of day, especially at night when lighting conditions are poor, and vehicle headlights further complicate detection due to interference with cameras. Since object detection technology is data-driven, the accuracy of detection models is highly dependent on the training dataset. When the training dataset mainly covers daytime scenes, its detection performance in nighttime scenes often drops significantly. The most direct way to improve the accuracy of nighttime traffic flow recognition is to increase the proportion of nighttime data in the training set, but this raises the issue of high manpower costs for data annotation.

[0003] While some open-source datasets provide a certain amount of nighttime data in the transportation field, these datasets are not entirely suitable for bridge scenarios. To obtain more accurate bridge traffic object detection models, it is often necessary to collect and label additional nighttime bridge traffic data. However, traditional data augmentation methods such as random pruning and flipping can increase the diversity of training data to some extent, but they do not consider the complex lighting and color changes at night, thus limiting the effectiveness of daytime pre-trained models in nighttime scenarios. Furthermore, labeling data is both time-consuming and expensive, especially in nighttime environments where poor lighting conditions and camera interference further increase the difficulty of labeling.

[0004] While some automated annotation techniques utilizing machine learning for feature extraction exist, their annotation quality is generally inferior to that of manual annotation. The increased cost of manual annotation indirectly impacts the aforementioned accuracy issues. Even when researchers attempt to fine-tune weights pre-trained from daytime images to improve nighttime detection accuracy without adding additional nighttime labeled data, the results are inconsistent. Therefore, enhancing nighttime bridge traffic detection data while effectively reducing the cost and difficulty of data annotation has become a pressing problem in this field. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides a method for enhancing nighttime bridge traffic detection data based on CycleGAN.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0007] A method for augmenting nighttime bridge traffic detection data based on CycleGAN includes the following steps:

[0008] S1. Collect daytime and nighttime traffic flow image data of bridge traffic, fully annotate the daytime traffic flow image data to obtain true daytime traffic flow image data with complete labels, and partially annotate the nighttime traffic flow image data to obtain true nighttime traffic flow image data with partial labels.

[0009] S2. Construct a nighttime bridge traffic detection data augmentation model based on CycleGAN. Train the nighttime bridge traffic detection data augmentation model using real daytime traffic flow image data with complete labels and real nighttime traffic flow image data with partial labels to obtain the trained nighttime bridge traffic detection data augmentation model.

[0010] S3. Perform multi-scale screening on the trained nighttime bridge traffic detection data augmentation model to obtain the final nighttime bridge traffic detection data augmentation model.

[0011] S4. Based on real daytime traffic flow image data with complete labels and the final nighttime bridge traffic detection data enhancement model, obtain enhanced nighttime bridge traffic detection data.

[0012] Further, in step S2, the nighttime bridge traffic detection data augmentation model includes a first generator, a second generator, a first discriminator, and a second discriminator; the first generator is used to convert true daytime traffic flow image data from the first independent training domain into false nighttime traffic flow image data, and to convert holiday traffic flow image data generated by the second generator into false nighttime traffic flow image data; the second generator is used to convert true nighttime traffic flow image data from the second independent training domain into holiday traffic flow image data, and to convert false nighttime traffic flow image data generated by the first generator into holiday traffic flow image data; the first discriminator is used to distinguish between true nighttime traffic flow image data from the second independent training domain and false nighttime traffic flow image data generated by the first generator; the second discriminator is used to distinguish between true daytime traffic flow image data from the first independent training domain and holiday traffic flow image data generated by the second generator.

[0013] Furthermore, the nighttime bridge traffic detection data augmentation model is trained using fully labeled daytime traffic flow image data and partially labeled nighttime traffic flow image data, resulting in the trained nighttime bridge traffic detection data augmentation model. This process includes the following steps:

[0014] S21. Construct the first independent training domain and the second independent training domain;

[0015] S22. Input the true daytime traffic flow image data with complete labels into the first independent training domain, and input the true nighttime traffic flow image data with partial labels into the second independent training domain to train the nighttime bridge traffic detection data augmentation model and obtain the trained nighttime bridge traffic detection data augmentation model.

[0016] Furthermore, step S3 includes the following steps:

[0017] S31. Use a visual semantic information comparison method to perform preliminary screening of the trained nighttime bridge traffic detection data augmentation models in order to obtain an initial combination of nighttime bridge traffic detection data augmentation models.

[0018] S32. Construct a loss function model for nighttime bridge traffic detection data enhancement;

[0019] S33. The initial combination of nighttime bridge traffic detection data augmentation models is finally screened using the FID method and the loss function model for nighttime bridge traffic detection data augmentation to obtain the final nighttime bridge traffic detection data augmentation model.

[0020] Further, in step S31, a preliminary screening of the trained nighttime bridge traffic detection data augmentation model is performed using a visual semantic information comparison method. Specifically, the visual semantic information is determined, including vehicle appearance type, relative distribution of traffic flow position, streetlights and median strips. The consistency of semantic information between the fake nighttime traffic flow image data generated by the trained nighttime bridge traffic detection data augmentation model and the real daytime traffic flow image data before generation is compared, so as to perform a preliminary screening of the trained nighttime bridge traffic detection data augmentation model.

[0021] Furthermore, in step S32, the loss function model for nighttime bridge traffic detection data augmentation includes the least squares loss function, the cycle consistency loss function, and the identity mapping function for nighttime bridge traffic detection data augmentation.

[0022] Furthermore, the least squares loss function for nighttime bridge traffic detection data enhancement is expressed as:

[0023]

[0024] Where: G is the first generator, D Y Let X be the first discriminator, Y be the first independent training domain, E be the expectation of a certain distribution, and y ~ p. data (y) represents the distribution of nighttime traffic flow image data p. data Sampling is performed in (y), x~p data (x) represents the distribution of daytime traffic flow image data p. data Sampling is performed in (x), DY (y) is the output value of the first discriminator for the true nighttime traffic flow image data y. The closer the value is to 1, the better the first discriminator D... Y The higher the probability that y is a real sample, the better D Y (G(x)) is the output value of the first discriminator on the fake nighttime traffic flow image data G(x), F is the second generator, and D X For the second discriminator, D x (x) is the output value of the second discriminator for the true daytime traffic flow image data x, D x (F(y)) is the output value of the second discriminator for the holiday traffic flow image data F(y).

[0025] Furthermore, the enhanced cyclic consistency loss function for nighttime bridge traffic monitoring data is expressed as:

[0026]

[0027] Where: G is the first generator, F is the second generator, E is the expectation of a certain distribution, and x ~ p data (x) represents the distribution of daytime traffic flow image data p. data Sampling is performed in (x), || ||1 is the L1 norm, y~p data (y) represents the distribution of nighttime traffic flow image data p. data Sampling is performed in (y), F(G(x)) is the output value of the second generator for the fake nighttime traffic flow image data G(x), x is the real daytime traffic flow image data, G(F(y)) is the output value of the first generator for the holiday traffic flow image data F(y), and y is the real nighttime traffic flow image data.

[0028] Furthermore, the identity mapping function for enhancing nighttime bridge traffic detection data is expressed as:

[0029]

[0030] Where: G is the first generator, F is the second generator, E is the expectation of a certain distribution, and y ~ p data (y) represents the distribution of nighttime traffic flow image data p. data Sampling is performed in (y), || ||1 is the L1 norm, x~p data (x) represents the distribution of daytime traffic flow image data p. data Sampling is performed in (x), where G(y) is the output value of the first generator for the true nighttime traffic flow image data y, and y is the true nighttime traffic flow image data. F(x) is the output value of the second generator for the true daytime traffic flow image data x, and x is the true daytime traffic flow image data.

[0031] Further, in step S33, the initial combination of nighttime bridge traffic detection data augmentation models is finally screened using the loss function model of nighttime bridge traffic detection data augmentation. The specific process is as follows: based on the loss function model of nighttime bridge traffic detection data augmentation, the loss function curve of each model in the initial combination of nighttime bridge traffic detection data augmentation models is plotted, and models whose four loss function values ​​tend to be stable within the range of the loss function curve and have no fluctuations exceeding 0.3 are screened to finally screen the initial combination of nighttime bridge traffic detection data augmentation models.

[0032] The present invention has the following beneficial effects:

[0033] This invention constructs a nighttime bridge traffic detection data augmentation model based on CycleGAN. It uses unpaired real daytime and nighttime traffic flow data for training and employs a multi-scale screening strategy combining visualized training results, a loss function model for nighttime bridge traffic detection data augmentation, and FID values. This comprehensive analysis, from qualitative to quantitative, yields the final nighttime bridge traffic detection data augmentation model. Then, this model is used to perform domain transformation on labeled real daytime traffic flow images and transfer the label data as a whole to generate a training set of semantically consistent fake nighttime traffic flow images. This results in enhanced nighttime bridge traffic detection data, improving the recognition accuracy and generalization ability of the YOLOv5 object detection model, achieving generative data augmentation. Furthermore, the overall transfer of labeled data reduces the manpower cost of labeling nighttime traffic flow data. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of a nighttime bridge traffic detection data augmentation method based on CycleGAN.

[0035] Figure 2 This is a schematic diagram of the nighttime bridge traffic detection data enhancement model structure of the present invention;

[0036] Figure 3 A schematic diagram of the process for conducting simulation experiments using the method provided by this invention;

[0037] Figure 4 This is a graph of the loss function of the present invention;

[0038] Figure 5 This is a graph showing the FID value results of the present invention;

[0039] Figure 6 The simulation results are shown in the figure with validation set A as the validation set.

[0040] Figure 7 The simulation results are shown in the figure with validation set B as the validation set.

[0041] Figure 8 The simulation results are shown in the figure with the validation set C as the validation set. Detailed Implementation

[0042] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0043] like Figure 1 As shown, a method for augmenting bridge traffic detection data at night based on CycleGAN includes steps S1-S4, as detailed below:

[0044] S1. Collect daytime and nighttime traffic flow image data for bridge traffic. Completely annotate the daytime traffic flow image data to obtain true daytime traffic flow image data with complete labels, and partially annotate the nighttime traffic flow image data to obtain true nighttime traffic flow image data with partial labels.

[0045] In an optional embodiment of the present invention, the present invention collects daytime and nighttime traffic flow image data of bridge traffic, and fully annotates the daytime traffic flow image data to obtain true daytime traffic flow image data with complete labels. However, due to the difficulty of annotating nighttime data in reality, the present invention simulates real-world conditions by partially annotating the nighttime traffic flow image data to obtain true nighttime traffic flow image data with partial labels.

[0046] S2. Construct a nighttime bridge traffic detection data augmentation model based on CycleGAN. Train the nighttime bridge traffic detection data augmentation model using real daytime traffic flow image data with complete labels and real nighttime traffic flow image data with partial labels to obtain the trained nighttime bridge traffic detection data augmentation model.

[0047] In an optional embodiment of the present invention, the nighttime bridge traffic detection data augmentation model includes a first generator, a second generator, a first discriminator, and a second discriminator, such as... Figure 2As shown. A generator is used to convert true daytime traffic flow image data from a first independent training domain into false nighttime traffic flow image data, and to convert holiday traffic flow image data generated by a second generator into false nighttime traffic flow image data; a second generator is used to convert true nighttime traffic flow image data from a second independent training domain into holiday traffic flow image data, and to convert false nighttime traffic flow image data generated by the first generator into holiday traffic flow image data; a first discriminator is used to distinguish between true nighttime traffic flow image data from the second independent training domain and false nighttime traffic flow image data generated by the first generator; a second discriminator is used to distinguish between true daytime traffic flow image data from the first independent training domain and holiday traffic flow image data generated by the second generator.

[0048] This invention utilizes fully labeled daytime traffic flow image data and partially labeled nighttime traffic flow image data to train a nighttime bridge traffic detection data augmentation model, obtaining the trained nighttime bridge traffic detection data augmentation model, including the following steps:

[0049] S21. Construct the first independent training domain X and the second independent training domain Y.

[0050] This invention constructs a first independent training domain X and a second independent training domain Y, then the true daytime traffic flow image data x∈X and the true nighttime traffic flow image data y∈Y.

[0051] S22. Input the true daytime traffic flow image data with complete labels into the first independent training domain, and input the true nighttime traffic flow image data with partial labels into the second independent training domain to train the nighttime bridge traffic detection data augmentation model and obtain the trained nighttime bridge traffic detection data augmentation model.

[0052] S3. Perform multi-scale screening on the trained nighttime bridge traffic detection data augmentation model to obtain the final nighttime bridge traffic detection data augmentation model.

[0053] In an optional embodiment of the present invention, step S3 includes the following steps:

[0054] S31. The trained nighttime bridge traffic detection data augmentation models are initially screened using a visual semantic information comparison method to obtain an initial combination of nighttime bridge traffic detection data augmentation models.

[0055] This invention utilizes a visual semantic information comparison method to perform preliminary screening of the trained nighttime bridge traffic detection data augmentation model. The specific process is as follows: determine the visual semantic information, including vehicle appearance type, relative distribution of traffic flow position, streetlights and median strips; compare the consistency of semantic information between the fake nighttime traffic flow image data generated by the trained nighttime bridge traffic detection data augmentation model and the real daytime traffic flow image data before generation, so as to perform preliminary screening of the trained nighttime bridge traffic detection data augmentation model.

[0056] S32. Construct a loss function model to enhance nighttime bridge traffic detection data.

[0057] The loss function model for nighttime bridge traffic detection data augmentation includes the least squares loss function, the cycle consistency loss function, and the identity mapping function.

[0058] When training with traditional cross-entropy loss, gradient vanishing may occur, especially when the generator's samples closely resemble real samples. In this case, the discriminator's feedback becomes weak, thus affecting the generator's learning. Therefore, for... Figure 2 In terms of adversarial loss, this invention uses the same least squares loss function as LSGAN, which has significant advantages.

[0059] This invention addresses the inadequacy of adversarial loss alone. Adversarial loss is only used to train the generator and discriminator, aiming to make the generated image distribution as close as possible to the real data distribution of the target domain. This optimization path only requires the generated images to be more realistic, without guaranteeing a clear correspondence between input and output. For example, given an input image x in a domain X containing several cars, the generator's output G(x) might map to multiple different samples in domain Y. As long as these samples conform to the statistical distribution of domain Y, even if G(x) contains no cars, it still meets the training expectation of adversarial loss. In this case, the training of generators G and F is unstable, leading to pattern collapse or semantic inconsistencies between samples. Therefore, this invention introduces a cycle consistency loss, such as... Figure 2 As shown, images x and y are output as F(G(x)) and G(F(y)) respectively after a series of generation processes. In the training process, this invention aims to achieve the following effect: x≈F(G(x)) and y≈G(F(y)). That is, this invention aims to change the distribution of the domain without changing the content and semantic information of the sample after the image goes through a cycle. This invention achieves the above constraints through the cycle consistency loss function to force the generators G and F to retain the key features of the input sample during the training process, ensuring semantic consistency and the uniqueness of the mapping.

[0060] Furthermore, this invention introduces an identity mapping function to ensure that generators G and F do not perform unnecessary transformations on images that already belong to the target domain.

[0061] The least squares loss function for nighttime bridge traffic monitoring data augmentation is expressed as:

[0062]

[0063] Where: G is the first generator, D Y Let X be the first discriminator, Y be the first independent training domain, E be the expectation of a certain distribution, and y ~ p. data (y) represents the distribution of nighttime traffic flow image data p. data Sampling is performed in (y), x~p data (x) represents the distribution of daytime traffic flow image data p. data Sampling is performed in (x), D Y (y) is the output value of the first discriminator for the true nighttime traffic flow image data y. The closer the value is to 1, the better the first discriminator D... Y The higher the probability that y is a real sample, the better D Y (G(x)) is the output value of the first discriminator on the fake nighttime traffic flow image data G(x), F is the second generator, and D X For the second discriminator, D x (x) is the output value of the second discriminator for the true daytime traffic flow image data x, D x (F(y)) is the output value of the second discriminator for the holiday traffic flow image data F(y).

[0064] Compared to traditional cross-entropy loss, the least squares loss enhanced by nighttime bridge traffic detection data can make the fine-tuning of discriminator parameters smoother, providing a more reasonable optimization path, thus avoiding the gradient vanishing problem to some extent.

[0065] The enhanced cyclic consistency loss function for nighttime bridge traffic monitoring data is expressed as:

[0066]

[0067] Where: G is the first generator, F is the second generator, E is the expectation of a certain distribution, and x ~ p data (x) represents the distribution of daytime traffic flow image data p. data Sampling is performed in (x), || ||1 is the L1 norm, y~p data (y) represents the distribution of nighttime traffic flow image data p. dataSampling is performed in (y), F(G(x)) is the output value of the second generator for the fake nighttime traffic flow image data G(x), x is the real daytime traffic flow image data, G(F(y)) is the output value of the first generator for the holiday traffic flow image data F(y), and y is the real nighttime traffic flow image data.

[0068] The identity mapping function for enhancing nighttime bridge traffic monitoring data is expressed as:

[0069]

[0070] Where: G is the first generator, F is the second generator, E is the expectation of a certain distribution, and y ~ p data (y) represents the distribution of nighttime traffic flow image data p. data Sampling is performed in (y), || ||1 is the L1 norm, x~p data (x) represents the distribution of daytime traffic flow image data p. data Sampling is performed in (x), where G(y) is the output value of the first generator for the true nighttime traffic flow image data y, and y is the true nighttime traffic flow image data. F(x) is the output value of the second generator for the true daytime traffic flow image data x, and x is the true daytime traffic flow image data.

[0071] This invention makes the generator's mapping between the two domains smoother by constraining G(y)≈y and F(x)≈x, ensuring that only the data that needs to be transformed is transformed, while the data that is already in the target domain is directly output, thus avoiding meaningless mapping.

[0072] S33. The initial combination of nighttime bridge traffic detection data augmentation models is finally screened using the FID method and the loss function model for nighttime bridge traffic detection data augmentation to obtain the final nighttime bridge traffic detection data augmentation model.

[0073] FID (Fréchet Inception Distance, an evaluation criterion for diffusion models) calculates the Fréchet distance between two multidimensional Gaussian distributions, representing the feature distributions of the real and generated images, respectively. This invention calculates the FID value, expressed as:

[0074] FID = ||μ real -μ fake || 2 +Tr(Σ real +Σ fake -2(Σ real Σ fake ) 1 / 2 )

[0075] Where: μ real and μ fakeLet ∑ be the mean of the features of the real image and the generated image, respectively. real and ∑ fake Let be the covariance matrices of the features of the real image and the generated image, respectively, and Tr be the trace of the matrix.

[0076] The smaller the calculated FID value, the smaller the Fréchet distance between the generated data and the real data, and thus the higher the generation quality of the model. This invention uses this method to screen the initial combination of augmentation models for nighttime bridge traffic detection data.

[0077] This invention utilizes a loss function model for nighttime bridge traffic detection data augmentation to perform final screening of the initial combination of nighttime bridge traffic detection data augmentation models. The specific process is as follows: Based on the loss function model for nighttime bridge traffic detection data augmentation, the loss function curve of each model in the initial combination of nighttime bridge traffic detection data augmentation models is plotted. Models whose four loss function values ​​tend to be stable within the range of the loss function curve and have no fluctuations exceeding 0.3 are selected for final screening of the initial combination of nighttime bridge traffic detection data augmentation models.

[0078] S4. Based on real daytime traffic flow image data with complete labels and the final nighttime bridge traffic detection data enhancement model, obtain enhanced nighttime bridge traffic detection data.

[0079] In an optional embodiment of the present invention, the present invention utilizes the final nighttime bridge traffic detection data enhancement model that has been selected to transform real daytime traffic flow image data with complete labels. Without changing the shape, position and relative layout of vehicles, the image background is transformed into a nighttime environment. Based on this, the generated fake nighttime traffic flow image data can be matched one by one with the label files of the daytime source data to complete the overall label migration and obtain enhanced nighttime bridge traffic detection data.

[0080] Simulation experiment:

[0081] like Figure 3 As shown, this invention uses the YOLOv5 object detection model to evaluate the effectiveness of the proposed method in enhancing traffic detection data on bridges. Specifically, this invention uses a pre-trained model based on the open-source YOLOv5 model, and then uses different training sets created with real daytime traffic flow image data and fake nighttime traffic flow image data to train the YOLOv5 object detection model. The recognition accuracy of each training set on different validation sets is then calculated to verify the effectiveness of the data enhancement.

[0082] This invention first uses the method described above to select the final nighttime bridge traffic detection data augmentation model. For example... Figure 4As shown, this invention records the specific values ​​of various loss functions during the training process. As can be seen above, the nighttime bridge traffic detection data augmentation model constructed by this invention exhibits a cyclical structure. Specifically, there are two main directions in the training process of the nighttime bridge traffic detection data augmentation model, corresponding to... Figure 3 The invention focuses on the conversion from daytime traffic flow image data to nighttime traffic flow image data, as it primarily analyzes data using actual daytime traffic flow image data as input. Figure 3 The left half, referred to here as the XY part, yields four loss function values ​​through the calculation of the loss function model for nighttime bridge traffic detection data enhancement: the discriminator loss DX-Y in the least squares loss function, the generator loss GX-Y, the cycle consistency loss cycleX-Y in the cycle consistency loss function, and the identity mapping loss idtX-Y in the identity mapping function. This invention... Figure 7 As can be seen, all four loss functions show a gradual decreasing trend before epoch (iteration number) reaches 50, indicating that the model has not converged. This corresponds to the poor performance of the generated data at epoch 10. When the epoch is between 50 and 130, the identity mapping loss value idtX-Y is in the range of 0.1 to 0.2, with no obvious abrupt change and a slight decreasing trend, which is already at a relatively good level. However, the other three loss function curves do not show an obvious decreasing trend, but instead exhibit fluctuations around a certain value. Among them, the generator loss GX-Y and cycle consistency loss cycleX-Y fluctuate the most dramatically, even showing a sudden change of more than 0.5 in value. Combining the results of the visualization semantic information comparison method, this invention can conclude that within this epoch range, the model's fluctuations around a certain benchmark value reflect better stability than models with epochs less than 50, but it has not yet reached the optimal level. Similar fluctuations are extremely common in deep learning, and the most likely reason is that the model has difficulty learning the complex details of the samples. When the epoch is between 130 and 135, this invention can observe that all four loss functions change uniformly, which is very similar to the loss function change characteristics before common model convergence. When the epoch is greater than 135, this invention observes... Figure 7It was found that the values ​​of the four loss functions have basically stabilized. The identity mapping loss, discriminator loss, and cycle consistency loss have stabilized at around 0.1, 0.2, and 0.4 respectively, with no significant abrupt changes. While the generator loss has shown a stabilizing trend, it still exhibits small abrupt changes, though these are much smaller than those observed when the epoch is between 50 and 130. Therefore, this invention concludes that the model has essentially converged at epoch 135. Thus, the optimal generative model should be selected from the epoch range of 135 to 200. Even though we consider the model to be nearly converged at this point, there are still slight differences in the generated data under different weights. Therefore, this invention further uses the FID metric, specifically used to evaluate the quality of generative models, to further screen for the final nighttime bridge traffic detection data augmentation model. This invention uses 2000 real daytime traffic flow images as input data. It performs domain transformation on the daytime traffic flow images using model weights within the epoch range of 135 to 200. Then, it calculates the FID value between the generated data at each epoch and the 2000 real nighttime traffic flow images. The results are as follows: Figure 5 As shown. By Figure 5 It was found that when the epoch was between 135 and 165, the FID value fluctuated slightly. When the epoch was greater than 165, the FID value was basically stable, with the maximum FID value difference being only 1.751. Looking back at this point... Figure 4 The generator loss function value GX-Y was used in this invention. The invention found that the mutations in the epoch range of 165 to 200 were slightly smaller than those in the epoch range of 135 to 165, further reflecting that the model had converged at this stage. Therefore, this invention selected the weighted model with the lowest FID value (18.264) in the 200th epoch as the final nighttime bridge traffic detection data augmentation model.

[0083] Next, this invention utilizes the final nighttime bridge traffic detection data to augment the model weights and perform domain transformation on the fully labeled true daytime traffic flow images, obtaining 2000 fully labeled fake nighttime traffic flow images. Since the CycleGAN-based domain transformation model does not change the semantic information of vehicle type, quantity, and relative position distribution within the original daytime traffic flow data after style transfer, this invention directly transfers the labels of the true daytime traffic flow images as a whole. After simple correction of some invalid labels that are detrimental to YOLOv5 training, they are directly used as fully labeled fake nighttime traffic flow images, greatly saving the manual annotation cost of nighttime traffic flow data. Based on the above data, this invention creates three training sets, named the true daytime training dataset, the fake nighttime training set, and a hybrid set of the true daytime and fake nighttime training sets, with total data volumes of 2000, 2000, and 4000, respectively.

[0084] This invention randomly selected 200 traffic flow images from both daytime and nighttime traffic flow image datasets, and used them to create three validation sets (Validation Set A, Validation Set B, and Validation Set C) to verify the accuracy of the YOLOv5 model. Validation Set A consisted of 200 nighttime traffic flow images, Validation Set B consisted of a mixed dataset of 400 daytime and nighttime traffic flow images, and Validation Set C consisted of 200 daytime traffic flow images, with all daytime traffic flow images in Validation Set C derived from Validation Set B). The maximum mAP50 value was recorded in real-time during the training process. When the mAP50 value for the next 100 epochs did not exceed the current maximum mAP50 value, this invention considered YOLOv5 to have converged and recorded the accuracy evaluation index value for each condition at this point, such as... Figure 6 , Figure 7 and Figure 8 As shown.

[0085] This invention focuses on the F1 score and mAP50 score of each training set under different validation sets. In the comparison, this invention considers a model to be superior when both scores are higher than the other. Figure 6It can be seen that when using the real daytime traffic flow image dataset as the training set and validation set A as the validation set, the F1 score and mAP50 score are 0.707 and 0.721 respectively, which are at a low level. This reflects the low recognition performance of the pre-trained model using real daytime traffic flow data as the training set in nighttime scenes, further illustrating the necessity of data augmentation. When using the fake nighttime traffic flow dataset and a mixed set of the real daytime training dataset and the fake nighttime training set as the training set, the F1 score and mAP50 score are 0.772 / 0.805 and 0.811 / 0.835 respectively, which are 9.2% / 13.9% and 12.5% / 15.8% higher than the accuracy when using the real daytime traffic flow image dataset as the training set. Therefore, we conclude that the fake nighttime traffic flow image data after domain transformation using the method provided in this invention provides more valuable feature information that conforms to the real nighttime traffic flow scene than the real daytime traffic flow data, and can effectively improve the recognition accuracy of the object detection model in nighttime scenes. Furthermore, training the model by mixing real daytime traffic flow image data and fake nighttime traffic flow image data can further improve detection accuracy.

[0086] However, in reality, it's impossible to focus solely on the accuracy of nighttime traffic flow recognition. Therefore, this invention also uses validation set B as the validation set, which is more challenging for object detection models, and trains YOLOv5 using the three training sets mentioned above. Figure 7 It can be seen that when using a mixed set of the real daytime training dataset and the fake nighttime training dataset as the training set, the F1 score and mAP50 score are 0.835 and 0.859, respectively, which are 1.7% and 2.3% higher than when using the real daytime training dataset as the training set, indicating that data augmentation still exists when using a mixed set of the real daytime training dataset and the fake nighttime training dataset as the training set. However, when using the fake nighttime training set as the training set, the F1 score and mAP50 score are 0.779 and 0.796, respectively, which are 5.1% and 5.2% lower than when using the real daytime training dataset as the training set. Since the model's recognition accuracy is significantly improved on the fake nighttime training set when using validation set A (F1: 9.2% and mAP50: 13.9%), but shows a performance decline on validation set B, this is because using the fake nighttime training set as the training set greatly sacrifices the recognition accuracy on the real daytime training dataset to improve the accuracy of nighttime traffic scenes. This phenomenon is normal because day and night scenes have huge differences in features. When using real daytime image data as a training set to verify nighttime scenes and when using fake nighttime image data as a training set to verify daytime scenes, the decrease in the accuracy of the object detection model is inevitable.

[0087] To further verify the above conjecture, this invention extracted 200 daytime traffic flow data images from validation set B to create validation set C, which consists entirely of daytime scenes. YOLOv5 was then trained again using the three training sets. Figure 8 It can be seen that when using the fake nighttime training set as the training set, the F1 score and mAP50 score are 0.793 and 0.814, respectively, which are 8.4% and 8.8% lower than when using the real daytime training set, thus verifying the above conjecture. However, when using a mixed set of the real daytime and fake nighttime training sets as the training set, the F1 score and mAP50 score are 0.866 and 0.895, respectively, showing almost no change compared to when using the real daytime training set. This indicates that using a mixed set of the real daytime and fake nighttime training sets as the training set does not affect the model's accuracy in daytime traffic scenarios.

[0088] In summary, regarding the feasibility of data augmentation, this invention draws the following conclusions: (1) The method provided by this invention successfully preserves key semantic information such as vehicle type, quantity, and relative position when generating fake nighttime traffic flow image data. This ensures that the annotations of the original daytime dataset can be directly adjusted with only minor corrections, thereby significantly reducing the manual labeling cost of nighttime traffic flow image data. (2) When training the YOLOv5 model on the fake nighttime training set or a hybrid set and validating it on the validation set A, both F1 and mAP50 show significant improvements. Specifically, the hybrid set achieves a maximum improvement of 13.9% on F1 and a maximum improvement of 15.8% on mAP50, highlighting the value of the augmented data obtained through the method of this invention in improving the accuracy of nighttime scene models. (3) When the target domain involved includes daytime traffic flow scenarios, training with only the fake nighttime training set will reduce the recognition accuracy of target detection. However, when training with the hybrid set, not only can the recognition accuracy be guaranteed to remain unaffected in daytime traffic flow scenarios, but also the F1 value and mAP50 value can be improved by 1.7% and 2.3% respectively on the validation set B, demonstrating strong generalization ability. (4) Therefore, although the augmented data obtained through this invention performs well in specific domains (nighttime) enhancement, the hybrid set can ensure robust performance in different scenarios.

[0089] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

[0093] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for augmenting nighttime bridge traffic detection data based on CycleGAN, characterized in that, Includes the following steps: S1. Collect daytime and nighttime traffic flow image data of bridge traffic, fully annotate the daytime traffic flow image data to obtain true daytime traffic flow image data with complete labels, and partially annotate the nighttime traffic flow image data to obtain true nighttime traffic flow image data with partial labels. S2. Construct a nighttime bridge traffic detection data augmentation model based on CycleGAN. Train the nighttime bridge traffic detection data augmentation model using real daytime traffic flow image data with complete labels and real nighttime traffic flow image data with partial labels to obtain the trained nighttime bridge traffic detection data augmentation model. S3. Perform multi-scale screening on the trained nighttime bridge traffic detection data augmentation model to obtain the final nighttime bridge traffic detection data augmentation model. S4. Based on real daytime traffic flow image data with complete labels and the final nighttime bridge traffic detection data enhancement model, obtain enhanced nighttime bridge traffic detection data; Step S3 includes the following steps: S31. Use a visual semantic information comparison method to perform preliminary screening of the trained nighttime bridge traffic detection data augmentation models in order to obtain an initial combination of nighttime bridge traffic detection data augmentation models. S32. Construct a loss function model for nighttime bridge traffic detection data enhancement; S33. The initial combination of nighttime bridge traffic detection data augmentation models is finally screened using the FID method and the loss function model for nighttime bridge traffic detection data augmentation to obtain the final nighttime bridge traffic detection data augmentation model. In step S32, the loss function model for nighttime bridge traffic detection data augmentation includes the least squares loss function, the cycle consistency loss function, and the identity mapping function for nighttime bridge traffic detection data augmentation. In step S33, the initial combination of nighttime bridge traffic detection data augmentation models is finally screened using the loss function model of nighttime bridge traffic detection data augmentation. The specific process is as follows: Based on the loss function model of nighttime bridge traffic detection data augmentation, the loss function curve of each model in the initial combination of nighttime bridge traffic detection data augmentation models is plotted. Models whose four loss function values ​​in the loss function curve tend to be stable within the interval and have no fluctuations above 0.3 are selected for the final screening of the initial combination of nighttime bridge traffic detection data augmentation models. The four loss function values ​​are the discriminator loss DX-Y, generator loss GX-Y in the least squares loss function, the cycle consistency loss cycleX-Y in the cycle consistency loss function, and the identity mapping loss idtX-Y in the identity mapping function.

2. The method for nighttime bridge traffic detection data augmentation based on CycleGAN according to claim 1, characterized in that, In step S2, the nighttime bridge traffic detection data augmentation model includes a first generator, a second generator, a first discriminator, and a second discriminator; the first generator is used to convert the true daytime traffic flow image data of the first independent training domain into false nighttime traffic flow image data, and to convert the holiday traffic flow image data generated by the second generator into false nighttime traffic flow image data. The second generator is used to convert the real nighttime traffic flow image data of the second independent training domain into holiday traffic flow image data, and to convert the fake nighttime traffic flow image data generated by the first generator into holiday traffic flow image data; the first discriminator is used to distinguish between the real nighttime traffic flow image data of the second independent training domain and the fake nighttime traffic flow image data generated by the first generator. The second discriminator is used to distinguish between the true daytime traffic flow image data of the first independent training domain and the holiday traffic flow image data generated by the second generator.

3. The method for nighttime bridge traffic detection data augmentation based on CycleGAN according to claim 1, characterized in that, The nighttime bridge traffic detection data augmentation model is trained using fully labeled daytime traffic flow images and partially labeled nighttime traffic flow images. The trained nighttime bridge traffic detection data augmentation model is obtained through the following steps: S21. Construct the first independent training domain and the second independent training domain; S22. Input the true daytime traffic flow image data with complete labels into the first independent training domain, and input the true nighttime traffic flow image data with partial labels into the second independent training domain to train the nighttime bridge traffic detection data augmentation model and obtain the trained nighttime bridge traffic detection data augmentation model.

4. The method for nighttime bridge traffic detection data augmentation based on CycleGAN according to claim 1, characterized in that, In step S31, a preliminary screening of the trained nighttime bridge traffic detection data augmentation model is performed using a visual semantic information comparison method. Specifically, the visual semantic information is determined, including vehicle appearance type, relative distribution of traffic flow position, streetlights, and median strips. The consistency of semantic information between the fake nighttime traffic flow image data generated by the trained nighttime bridge traffic detection data augmentation model and the real daytime traffic flow image data before generation is compared, so as to perform a preliminary screening of the trained nighttime bridge traffic detection data augmentation model.

5. The method for nighttime bridge traffic detection data augmentation based on CycleGAN according to claim 1, characterized in that, The least squares loss function for nighttime bridge traffic monitoring data augmentation is expressed as: in: G For the first generator, D Y As the first discriminator, X As the first independent training domain, Y As a second independent training domain, For the expectation of a certain distribution, y ~ p data ( y )for y Distribution of nighttime traffic flow image data p data ( y Sampling is performed in ) x ~ p data ( x )for x Distribution of real daytime traffic flow image data p data ( x Sampling is performed in ) D Y ( y The first discriminator analyzes the nighttime traffic flow image data. y The output value, the closer the value is to 1, the more it indicates the first discriminator. D Y think y The higher the probability that it is a real sample, D Y ( G ( x The first discriminator analyzes the fake nighttime traffic flow image data. G ( x The output value of ) F For the second generator, D X For the second discriminator, D x (x) represents the second discriminator's assessment of true daytime traffic flow image data. x The output value, D x ( F ( y The second discriminator analyzes holiday traffic flow image data. F ( y The output value of ).

6. The method for nighttime bridge traffic detection data augmentation based on CycleGAN according to claim 1, characterized in that, The enhanced cyclic consistency loss function for nighttime bridge traffic monitoring data is expressed as: in: G For the first generator, F For the second generator, For the expectation of a certain distribution, x ~ p data ( x )for x Distribution of real daytime traffic flow image data p data ( x Sampling is performed in ) It is an L1 norm. y ~ p data ( y )for y Distribution of nighttime traffic flow image data p data ( y Sampling is performed in ) F ( G ( x The second generator generates fake nighttime traffic flow image data. G ( x The output value of ) x This is real daytime traffic flow image data. G ( F ( y The first generator generates holiday traffic flow image data. F ( y The output value of ) y This is real nighttime traffic flow image data.

7. The method for nighttime bridge traffic detection data augmentation based on CycleGAN according to claim 1, characterized in that, The identity mapping function for enhancing nighttime bridge traffic monitoring data is expressed as: in: G For the first generator, F For the second generator, For the expectation of a certain distribution, y ~ p data ( y )for y Distribution of nighttime traffic flow image data p data ( y Sampling is performed in ) It is an L1 norm. x ~ p data ( x )for x Distribution of real daytime traffic flow image data p data ( x Sampling is performed in ) G ( y (The first generator generates images of real-night traffic flow.) y The output value, y This is real nighttime traffic flow image data. F ( x (This refers to the second generator generating real daytime traffic flow image data) x The output value, x These are real daytime traffic flow images.

Images