A plug-and-play dual data enhancement system and method

Abstract

This invention discloses a plug-and-play dual data augmentation system and method, comprising at least a color perturbation module and a dual-style storage module. The color perturbation module combats overfitting caused by different color features in different scenes through color perturbation. The dual-style storage module mines different style information in the feature space and stores the style information in dual memories. The dual memories store target style information and background style information respectively, and diversity is increased by switching the styles of the target and background. This invention combines image-level and feature-level augmentation methods to solve single-domain object detection tasks in urban scenes, effectively addressing the problem of high costs in sample collection and labeling in such problems, and improving the robustness and effectiveness of the model in detecting objects in invisible areas.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent transportation technology, and mainly relates to a plug-and-play dual data augmentation system and method. Background Technology

[0002] Object detection is a key technology in intelligent transportation systems. In urban scenarios, accurate target localization is crucial for improving the safety of autonomous driving systems. With the development of deep learning, significant breakthroughs have been achieved in typical object detection tasks within the field of computer vision. This development has also effectively supported object detection in urban scenarios.

[0003] In urban scenes, image styles often vary with weather and time of day. For example, images collected during the day differ from those collected at night, resulting in different data distributions across different scenarios. If an object detector is trained using images collected during the day, it may not generalize well to images collected in other scenarios. Therefore, directly applying typical object detection methods to urban scenes requires collecting images from different scenarios. However, in real-world applications, collecting images from all possible scenarios is extremely difficult because unseen scenes are unpredictable. Furthermore, even if we could collect images from all possible scenarios, labeling them remains a costly and time-consuming task. While some unsupervised adaptive methods have been proposed to address this issue, data collection from the target scene is still necessary.

[0004] To address the single-domain generalizable object detection task in urban scenarios, most existing methods rely on adversarial learning-based reinforcement techniques, primarily used in image classification. Meanwhile, several different methods have been proposed for single-domain object detection, such as self-supervised methods and self-distillation methods. Specifically, these methods extract domain-invariant features through cyclic decoupling and self-distillation, separating the domain-invariant representation from the domain-specific representation in the absence of domain labels. However, these methods exhibit poor generalization performance in single-domain object detection. The poor generalization performance of some existing techniques can have practical drawbacks. Specifically, poor generalization performance means that the model performs well on the training set but poorly on new datasets, affecting its practical value. For example, in the field of autonomous driving within urban object detection scenarios, poorly generalized object detection algorithms can negatively impact the autonomous driving system's ability to identify and judge obstacles. Especially in weather conditions such as rain, fog, or insufficient lighting, autonomous vehicles may fail to accurately identify pedestrians, vehicles, and other obstacles, increasing the risk of traffic accidents and posing potential safety risks. Furthermore, object detection algorithms with poor generalization performance may lead to false positives and false negatives in autonomous vehicles, causing the autonomous driving system to fail to react correctly and severely affecting its robustness and reliability. Therefore, there is an urgent need for a new method with better generalization performance to solve single-domain object detection tasks in urban scenarios. Summary of the Invention

[0005] This invention addresses the poor generalization performance of existing single-domain object detection models by providing a plug-and-play dual data augmentation system and method. It includes at least a color perturbation module and a dual-style storage module. The color perturbation module combats overfitting caused by different color features in different scenes through color perturbation. The dual-style storage module mines different style information in the feature space and stores this information in two separate memories: one for the target style and the other for the background style. Variety is increased by switching between the target and background styles. This system and method combine image-level and feature-level augmentation methods to solve single-domain object detection tasks in urban scenes, effectively addressing the high cost of sample collection and labeling in such problems, and improving the robustness and effectiveness of the model in detecting objects in invisible areas.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is: a plug-and-play dual data enhancement system, comprising at least a color perturbation module and a dual-style storage module.

[0007] The color perturbation module: uses color perturbation to combat overfitting caused by different color features in different scenarios;

[0008] The dual-style storage module: mines different style information in the feature space and stores the style information in dual memory. The dual memory stores target style information and background style information respectively. The diversity is increased by switching the style of the target and the background.

[0009] As an improvement of the present invention, in the color perturbation module, the RGB channels are randomly swapped, transforming the channel order of [R,G,B] into [R,B,G], [G,R,B], [G,B,R], [B,R,G], and [B,G,R]. During the training phase, an image with a randomized color channel order is used to train the system against the original image.

[0010] As an improvement to the present invention, in the dual-style storage module, for N o The input image of an object, whose intermediate feature maps based on convolutional layers are defined as f, is divided into different blocks according to a benchmark. and The feature map is used to represent the background and target sets, where C represents the number of channels, and A b and A oi Let μ represent the background region and the i-th object in the spatial dimension, respectively; for the first object, the style information can be represented as μ. o1 =[μ1 o1 ,…,μ C o1 ] and σ o1 =[σ1 o1 ,…,σ C o1 Extract style information from the i-th channel:

[0011]

[0012]

[0013] Similarly, style information for the background and all targets can be calculated, and then the style information (μ) can be... o1 ,σ o1 Save to the corresponding background style storage module M back Within this, different stylistic information can be extracted:

[0014]

[0015] Among them, M back [r] σ and M back [r] μ It is in M back The r-th style information is randomly selected from the data.

[0016] As another improvement of the present invention, the color perturbation module and the dual-style storage module use Faster R-CNN as the object detector. A fixed-size image is input into the detector, and a feature map is extracted through a convolutional layer. Then, a region proposal network (RPN) is used to obtain region proposal boxes. The feature map and the region proposal boxes are input into the ROI Pooling layer to obtain the proposed feature map. Finally, the proposed feature map is fed into the softmax classification layer and the bounding box regression layer, respectively, so as to obtain the category of the detected object and the final precise location of the detection box.

[0017] To achieve the above objectives, the present invention also adopts the following technical solution: a plug-and-play dual data augmentation method, comprising the following steps:

[0018] S1: Input a fixed-size image and put the input image into the color perturbation module;

[0019] S2: Enhance color information by interfering with the RGB channels in the image space by generating color perturbations;

[0020] S3: The original image and the color-perturbed image are fed into the task network, and style swapping is performed between the shallow and deep networks;

[0021] S4: Mine different style information in the feature space. The style information includes at least target style information and background style information. Increase diversity by switching the style of the target and the background.

[0022] S5: Obtain the category of the detected object in the image and the precise location of the detection box.

[0023] Compared with existing technologies, the present invention has the following advantages: It proposes a plug-and-play dual data augmentation system, introduces two components, a color perturbation module and a dual-style storage module, and provides a comprehensive data augmentation strategy. At the same time, it utilizes style information from the entire training set to improve the robustness and effectiveness of the target detection model in the invisible domain. The generalization performance of the method of the present invention is better than the state-of-the-art method at present, which can improve the correctness and reliability of artificial intelligence application systems, thereby avoiding some problems caused by insufficient algorithm generalization ability. The present invention also has the following three characteristics: (1) Wide applicability: The method of the present invention can be extended to any application scenario using target detection, including but not limited to street scene target detection, street scene lane line recognition, license plate recognition, autonomous driving and other general target detection scenarios; (2) Simple and easy to use: In any existing target detection algorithm application, inserting the dual augmentation method of the present invention can improve the algorithm's generalization ability. The present invention is a plug-and-play algorithm; (3) Low cost and high efficiency: For some existing research methods, including but not limited to directly deepening and widening the neural network layers, adding auxiliary networks, adversarial methods, etc., the above methods often double the training time of the improved algorithm. This invention focuses on the key point and simplifies the module that improves generalization ability. Because it neither increases the number of neural network parameters nor the amount of computation in the neural network inference process, the training time is not significantly increased. Compared with other methods, the method of this invention is low-cost and efficient. Attached Figure Description

[0024] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0025] Figure 2 This is a flowchart of step S1 of the method of the present invention;

[0026] Figure 3 This is a flowchart of step S2 of the method of the present invention;

[0027] Figure 4 This is a schematic diagram of the forward process of the algorithm in step S2 of the method of the present invention;

[0028] Figure 5 This is a schematic diagram of the method of the present invention;

[0029] Figure 6 This is a schematic diagram showing the location of the dual-style storage module in the Faster RCNN backbone network structure in the method of the present invention;

[0030] Figure 7 This is a schematic diagram showing the location of the dual-style storage module in the YOLOv5 backbone network structure in the method of this invention. Detailed Implementation

[0031] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0032] Example 1

[0033] Before describing the specific steps of this method, we first give the relevant definitions and representations:

[0034] (a) Color Perturbation (CP)

[0035] (b) Dual-Style Memory (DSM)

[0036] (c) Image Style Transfer Algorithm (AdaIN): Adaptive Instance Normalization

[0037] (d) A normalization method (IN) for improving image style transformation: InstanceNormalization

[0038] (e) Image Representation

[0039] (f) Color Jitter

[0040] (g) Faster R-CNN, a deep learning model for object detection.

[0041] (h) A deep learning framework for object detection: YOLOv5

[0042] (i) Region Poposal Network (RPN)

[0043] (j) Candidate Region of Interest Pooling (ROI Pooling) identified from the target image

[0044] (k) An activation function that can be used as the last layer of a neural network for multi-class classification output: Softmax

[0045] (l) Bounding Box

[0046] (m) Manually annotated real data: Ground-truth

[0047] (n) image patch

[0048] (o) Cross Entropy Loss (CLS LOSS)

[0049] (p) Bounding Box Loss (BBX LOSS)

[0050] (q) Semantic style mask, used to locate and distinguish different objects, backgrounds, or parts in an image: mask

[0051] (r) A deep residual network designed to address the degradation problem during deep neural network training: ResNet

[0052] (s) An image style manipulation method: MixStyle

[0053] A plug-and-play dual data augmentation system includes at least a color perturbation module and a dual-style storage module. The color perturbation module combats overfitting caused by different color features in different scenes through color perturbation. The dual-style storage module mines different style information in the feature space and stores this style information in dual storage units. The dual storage units respectively store target style information and background style information, increasing diversity by switching between target and background styles. In the color perturbation module, RGB channels are randomly swapped, transforming the channel order [R,G,B] into [R,B,G], [G,R,B], [G,B,R], [B,R,G], and [B,G,R]. Image color channel perturbation introduces different information, effectively mitigating overfitting. Then, during the training phase, an image with a randomized color channel order is used to train the system alongside the original image. Furthermore, the color perturbation module in this system can be combined with other data augmentation methods in YOLOv5 (such as mosaic and blending) to further enhance the model's generalization ability.

[0054] In a dual-style storage module, for N o The input image of an object, whose intermediate feature maps based on convolutional layers are defined as f, is divided into different blocks according to a benchmark. and The feature map is used to represent the background and target sets, where C represents the number of channels, and A b and A oi Let μ represent the background region and the i-th object in the spatial dimension, respectively; for the first object, the style information can be represented as μ. o1 =[μ1 o1 ,…,μ C o1 ] and σ o1 =[σ1 o1 ,…,σ Co1 Extract style information from the i-th channel:

[0055]

[0056]

[0057] Similarly, style information for the background and all targets can be calculated, and then the style information (μ) can be... o1 ,σ o1 Save to the corresponding background style storage module M back Within this, different stylistic information can be extracted:

[0058]

[0059] Among them, M back [r] σ and M back [r] μ It is in M back The r-th style information is randomly selected from the data.

[0060] In the dual-style storage module, this invention uses fixed-length queues, which does not require a large amount of memory. Since the training set needs to be shuffled during each iteration, the available dual-style storage module for a specific sample changes with each iteration, allowing this invention to extract more diverse style information from other samples. Furthermore, we use shared memory in our experiments to save all style information and access it from the corresponding style storage, further emphasizing the importance of these steps in the proposed method. Specifically, the dual-style storage module method in this invention uses four fixed-length queues to store the mean and variance of the foreground and the mean and variance of the background, respectively. After each iteration, a set of style information is stored, and simultaneously, a new set of style information is randomly selected from the style queues to perform style transformation on the image. When the queue storage length reaches its limit, the oldest stored style information (the style information at the head of the queue) is deleted, and the new style information is stored at the tail of the queue.

[0061] Since the style storage module in this invention is plug-and-play, it can be placed in a shallow network structure for most object detection networks. This is because shallow network structures tend to retain more image detail features, while deep network structures retain more semantic features. Figure 6 As shown, a schematic diagram of the backbone network structure of Faster R-CNN is presented. The style storage module (DSM) achieves the best results after being inserted into the first feature extraction module of ResNet. For example... Figure 7As shown, a schematic diagram of the backbone network structure of YOLOv5 is given. Similarly, the insertion position of the Style Storage Module (DSM) in the YOLOv5 network structure is the same, and the effect is best after inserting it into the structure after the first downsampling.

[0062] This invention uses Faster R-CNN as the object detector. First, a fixed-size image is input into the detector, and feature maps are extracted through convolutional layers. These feature maps are then passed through a Region Proposal Network (RPN) to obtain region proposals. Next, the feature maps and region proposals are input into a Region of Interest (ROI) Pooling layer to obtain proposed feature maps. Finally, the proposed feature maps are fed into a softmax classification layer and a bounding box regression layer, respectively, to obtain the category of the detected object and the final precise location of the detection box.

[0063] Example 2

[0064] A plug-and-play dual data augmentation method, primarily targeting single-domain object detection tasks in urban scenarios, such as... Figure 1 As shown, the specific steps include the following:

[0065] 1) First, input images of a fixed size and put these inputs into the color perturbation module.

[0066] 2) Improve model performance by generating color perturbations, i.e., interfering with the RGB channels in the image space to enhance color information. For example... Figure 2 As shown.

[0067] 3) The original image and the color-perturbed image are put into the task network, and style swapping is performed between the shallow and deep networks.

[0068] 4) Improve the robustness of object detection models in invisible domains by mining different style information in the feature space. For example... Figure 3 As shown.

[0069] 5) Finally, obtain the category of the detected object in the image and the precise location of the detection box.

[0070] In single-domain urban object detection tasks, a lack of diversity in the training set can lead to overfitting of the model or training data. Step 2) utilizes color perturbation to enhance the color information in the training set, i.e., randomly swapping the RGB channels. First, a random number P is generated and its value is checked against 0.5. If the random number is greater than 0.5, the image is directly input into the task network for training. Otherwise, the channel order of the input image is randomly generated. Specifically, the [R,G,B] channel order can be transformed into [R,B,G], [G,R,B], [G,B,R], [B,R,G], and [B,G,R]. Finally, the image with the altered channel order is fed into the task network along with the original input image for training.

[0071] Image-level data augmentation is widely used in computer vision, with ColorJitter being a popular method. This method randomly adjusts the brightness, contrast, and saturation of an image and has proven its effectiveness in domain-generalized classification tasks. However, the objects used in image classification tasks differ from those captured in small object images in urban environments. Using ColorJitter for data augmentation may result in the loss of important information about small objects.

[0072] AdaIN can transfer style information from one image to another. Specifically, the statistics of the feature maps represent the style information of the image. In step 4), the intermediate result features of the task network are first obtained, the image is divided into N blocks, and the style features of each block are calculated. Then, it is checked whether the foreground and background storage modules are full. For the foreground style storage module, if it is full, the earliest foreground style feature stored in the foreground storage module is removed, and the current foreground style feature is stored in the foreground module. Then, background style features are randomly obtained, and the foreground of each image is re-stylized. For the background style storage module, if it is full, the earliest background style feature stored in the background storage module is removed, and the current background style feature is stored in the background module. Then, foreground style features are randomly obtained, and the foreground of each image is re-stylized. Finally, the foreground and background features are integrated.

[0073] The Ada IN method is used to exchange style information between the target and the background. The feature mapping of the two sets of images is defined as follows: C represents the number of features, and H and W are the height and width of the feature map, respectively. We assume the goal is to extract style information. Transfer to f, specifically defined as follows:

[0074]

[0075] in This represents the mean and standard deviation of the channel. The channel of the i-th layer is represented as:

[0076]

[0077]

[0078] Where ò is a constant for numerical stability. Similarly, we can also obtain... Statistical data This refers to the image style information mentioned earlier.

[0079] To enhance the model's generalization ability, we generate enhanced features to increase the diversity of abstract styles in the feature space. In the single-domain urban object detection task with limited style information, our goal is to extract style information from the entire training data. Furthermore, style differences may exist between local objects and the background.

[0080] Therefore, this invention achieves this goal through dual-style storage, which stores style information in two memories. Specifically, two style memories are first generated to store the target style information and the background style information, respectively. Here, we use M... obj and M back Let M represent the storage used to hold style information for the target and background. obj and M back Some style information has been saved, such as Figure 4 As shown, for N o The input image of an object has an intermediate feature map based on a convolutional layer, which is defined as f. We divide f into different blocks according to a benchmark. and This is used to represent the feature map of the background and target sets. Here, C represents the number of channels, and A... b and A oi Let μ represent the background region and the i-th object in the spatial dimension, respectively. For the first object, the style information can be represented as μ. o1 =[μ1 o1 ,…,μ C o1 ] and σ o1 =[σ1 o1 ,…,σ C o1 Therefore, we can extract style information from the i-th layer channel.

[0081]

[0082]

[0083] Similarly, style information for the background and all targets can be calculated and then saved to the corresponding style storage module. For example, (μ o1 ,σ o1 Save to M obj In China, based on M obj and M back We can uncover different style information.

[0084]

[0085] Among them, M back [r] σ and Mback [r] O It is in M back The r-th style information is randomly selected from the data. Simultaneously, we use the same scheme to enhance style diversity across all targets and backgrounds.

[0086] For the dual-style storage module, we use a fixed-length queue, which does not require a large amount of memory. Since the training set needs to be shuffled during each iteration, the available dual-style storage module for a specific sample changes with each iteration, allowing us to extract more diverse style information from other samples. Specifically, we take feature maps as input and divide them into different patches based on ground truth. For each iteration, we first compute the style information of the target and background in the image. If the target style or background style in the memory exceeds the maximum length of the memory, the earliest style information stored in the memory is removed. Then, the style information is stored in the corresponding memory, followed by random selection of style information from the two style memories. Finally, we normalize all patches using the AdaIN method. Furthermore, we experiment with shared memory to store all style information and access it from the corresponding style stores. This further emphasizes the importance of these steps in our method.

[0087] In summary, this invention achieves better generalization in single-domain object detection scenarios through a color perturbation module and a dual-style storage module. On one hand, this invention perturbs the color of the input original image. By randomly swapping the RGB channels of the original image, the color distribution of the image is changed and the training set is expanded, allowing the model to better adapt to input data with various color distributions. On the other hand, this invention increases diversity by randomly swapping the style information of the target and background in different images. Specifically, the style features of the target and background in different images are stored in two separate modules. Then, target style features and background style features are randomly selected from the two modules respectively. Finally, by aligning the mean and standard deviation of the target style features with the mean and standard deviation of the background style features, a new image is synthesized, increasing diversity. Furthermore, as... Figure 5 As shown, this invention also predicts object categories using cross-entropy loss and uses bounding box loss to calculate the difference between the bounding boxes predicted by the model and the true bounding boxes, guiding the model to learn more accurate bounding box predictions.

[0088] Test case

[0089] We utilize the domain generalization error bound to further demonstrate the effectiveness of our proposed method. Definition Define P from the convex hull Λ t X Minimum value π * as well as definition As the diameter of the convex hull. Therefore, we can obtain...

[0090]

[0091] in, It is the target domain and has the same characteristics as the target domain (P). X * The joint risk of the training domains with the most similar distributions.

[0092] In the theorem, the first term aims to minimize the empirical error of the training set, which can be achieved using a general loss function for object detection. The last term can be considered a constant. For the second term in the theorem, γ represents the difference between all combinations of training and target domains.

[0093] This invention generates different style information based on multiple different distributions, which can be considered as different domains. Therefore, introducing different style information based on CP and DSM helps reduce overfitting to the original single training set and effectively mitigates the aforementioned risks. ρ represents the maximum distance between γ and ρ. The method of this invention uses DSM to extract different style information from the training data itself, while color perturbation only involves switching RGB channels. This shows that generating different style information in our method does not introduce a large domain gap between training samples. The method of this invention has advantages in reducing the generalization error range of γ and ρ. In this section, we use Faster R-CNN as a baseline and conduct experiments to fully verify the effectiveness of the CP and DSM modules in this invention on the Diversity-Weather and SIM10k2Cityscapes datasets. The experimental results are shown in Tables 1 and 2. As can be seen from these two tables, the color perturbation (CP) and dual-style storage (DSM) proposed in this invention can improve the model's generalization ability on both datasets. For example, on Diversity-Weather, CP and DSM outperform the baseline by +1.18% (35.82 vs. 34.64) and +2.12% (36.76 vs. 34.64), respectively, confirming the effectiveness of these proposed modules. Furthermore, combining CP with DSM yields even better performance. We also observed significant improvements on SIM10k2Cityscapes when using our method, due to the large domain gap between the virtual data (SIM10k) and the real-world data (Cityscapes). Therefore, our method can achieve substantial performance gains when the invisible domain differs significantly from the training set.

[0094] Table 1 shows the module ablation experiments on the Diversity-Weather dataset.

[0095]

[0096] Table 2 shows the module ablation experiments on the SIM10k dataset.

[0097]

[0098] Comparison of the color perturbation method in this invention with ColorJitter: ColorJitter is an image data augmentation method that randomly alters the brightness, contrast, and saturation of an image, and it has been widely used in computer vision. In this experiment, the proposed color perturbation method is compared with its performance on the Diverse-Weather and SIM10k2Cityscapes datasets. The experimental results are shown in Tables 3 and 4. These tables show that the proposed color perturbation method achieves better performance than ColorJitter, for example, on diversity-weather and sim10k2cityscape, the performance is improved by +0.48% (37.45 vs. 36.97) and +1.21% (56.38 vs. 55.17), respectively. The main reason is that small objects in urban scene images may be blurry when using ColorJitter.

[0099] Table 3 Comparison of Color Perturbation (CP) and Color Jitter (CJ) on the Diverse-Weather dataset

[0100]

[0101] Table 4 Comparison of Color Perturbation (CP) and Color Jitter (CJ) on the SIM10k dataset

[0102]

[0103] A comparison of the Dual Style Storage (DSM) method and MixStyle in this invention: MixStyle is an enhancement method for batch mixing style information of images. Since the DSM method of this invention does not introduce additional information (i.e., it only mines style information from the training set), comparing them is fair. Experimental results are shown in Table 5. The experimental results show that DSM outperforms MixStyle by 0.92 (36.76 vs. 35.84) on the Diverse-Weather dataset.

[0104] Table 5 compares DSM and MixStyle on the Diverse-Weather dataset.

[0105]

[0106] In summary, this invention provides a simple and effective method to solve the single-domain generalization problem in object detection tasks in urban scenes. It mainly comprises two modules: an image-level color perturbation module and a feature-level dual-style storage module. The CP module randomly shuffles the RGB channels to generate different color information, while the DSM module utilizes object and background style memories to store and extract different style information throughout the dataset. This effectively solves the problem of high costs associated with sample collection and labeling in such cases, improving the robustness and effectiveness of the model in detecting objects in invisible regions.

[0107] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. A plug-and-play dual data augmentation system, characterized in that... It includes at least a color perturbation module and a dual-style storage module. The color perturbation module: uses color perturbation to combat overfitting caused by different color features in different scenarios; The dual-style storage module: mines different style information in the feature space and stores the style information in dual memory. The dual memory stores target style information and background style information respectively. The diversity is increased by switching the style of the target and the background. In a dual-style storage module, for those with The input image of an object, whose intermediate feature map based on the convolutional layer is defined as f, is used to determine the benchmark. Divided into different blocks, and The feature map is used to represent the background and target sets, where C represents the number of channels, and and Let represent the background region and the i-th object in the spatial dimension, respectively; for the first object, the style information can be represented as... and Extract style information from the i-th channel: ； ； Similarly, style information for the background and all targets can be calculated, and then the style information... Save to the corresponding background style storage module Within this, different stylistic information can be extracted: ； in, and Is The r-th style information is randomly selected from the data.

2. The plug-and-play dual data augmentation system as described in claim 1, characterized in that: In the color perturbation module, the RGB channels are randomly swapped, and the channel order of [R,G,B] is transformed into [R, B, G], [G, R, B], [G, B,R], [B, R, G] and [B, G, R]. During the training phase, an image with a random color channel order is used to train the original image.

3. The plug-and-play dual data augmentation system as described in claim 2, characterized in that: The color perturbation module and the dual-style storage module use Faster RCNN as the object detector. A fixed-size image is input into the detector, and the feature map is extracted through the convolutional layer. Then, the region generation network RPN is used to obtain the region proposal box. The feature map and region proposal boxes are input into the ROI Pooling layer to obtain the proposed feature map. Finally, the proposed feature map is fed into the softmax classification layer and the bounding box regression layer to obtain the category of the detected object and the final precise location of the detection box.

4. A plug-and-play dual data augmentation method using the system as described in claim 1, characterized in that, Includes the following steps: S1: Input a fixed-size image and put the input image into the color perturbation module; S2: Enhance color information by interfering with the RGB channels in the image space by generating color perturbations; S3: The original image and the color-perturbed image are fed into the task network, and style swapping is performed between the shallow and deep networks; S4: Mine different style information in the feature space, wherein the style information includes at least target style information and background style information, and increase diversity by switching the style of the target and the background; S5: Obtain the category of the detected object in the image and the precise location of the detection box.

5. The plug-and-play dual data augmentation method as described in claim 4, characterized in that: Step S2 specifically includes: S21: Generate a random number P and determine whether the random number is greater than 0.

5. If the random number is greater than 0.5, proceed to step S12; otherwise, proceed to step S13. S22: Directly input the image into the task network for training; S23: Randomly generate the channel order of the input image. The [R,G,B] channel order is transformed into [R, B, G], [G, R,B], [G, B, R], [B, R, G], and [B, G, R]. S24: Feed the image with the changed channel order along with the original input image into the task network for training.

6. The plug-and-play dual data augmentation method as described in claim 4, characterized in that: Step S4 specifically includes: S41: Obtain the intermediate result features of the task network, divide the image into N blocks, and calculate the style features of each block. S42: Check whether the foreground storage module and background storage module are full; S43: For the foreground style storage module, if the foreground style storage is full, remove the foreground style feature that was stored first in the foreground storage module, store the current foreground style feature in the foreground module, then randomly obtain the background storage style feature, and re-stylize the foreground of each image. S44: For background style storage, if the background style storage is full, remove the background style feature that was stored first in the background storage module, store the current background style feature in the background module, then randomly obtain the foreground style feature stored in the foreground, and re-stylize the foreground of each image. S45: Integrating foreground and background features.

7. The plug-and-play dual data augmentation method as described in claim 6, characterized in that: Step S45 specifically includes: S451: Interpolate the semantic style mask using the nearest neighbor method to make its size the same as each image block; S452: Multiply the input features by the corresponding background mask to obtain the mean and variance of the background style features; multiply the input features by the corresponding foreground mask to obtain the mean and variance of the foreground style features. S453: AdaIN is used to normalize the background style features, and the background features are restored by the mean and variance of the foreground style features; IN is used to normalize the foreground style features, and the foreground features are restored by the mean and variance of the background style features. The mean and variance of the activated channels of the foreground features are aligned to match the mean and variance learned from the background features.

Images