A data and structure optimization and improvement method and processing terminal for YOLOv5

Abstract

This invention discloses a data and structure optimization and improvement method and processing terminal for YOLOv5. The method includes: improving the head structure of the original YOLOv5 model using a head-shared convolution approach; performing data augmentation through lossless mosaic, removing labels whose truncation ratio does not meet requirements and setting the corresponding regions to background pixel values ​​of 114, thus ensuring consistency between target features and target labels. This invention proposes a data and structure optimization and improvement method and processing terminal for YOLOv5, aiming to address the scale imbalance problem in the YOLOv5 model and the inconsistency between target labels and target features generated by the original mosaic data augmentation algorithm.

Description

Technical Field

[0001] This invention belongs to the field of target detection and relates to a data and structure optimization and improvement method and processing terminal for YOLOv5. Background Technology

[0002] With the increasing number of cars in my country and the growing traffic congestion, nighttime driving safety has become a major focus of social concern and research. Effective detection of vehicles at night, acquiring comprehensive information about vehicles ahead through sensors to achieve early collision warnings and intelligent high-beam illumination, is a fundamental prerequisite for ensuring nighttime driving safety. However, different sensors have their own performance characteristics, and using a single type of sensor cannot fully meet the needs of environmental perception. Currently, vehicle detection schemes based on information fusion generally first use millimeter-wave radar to detect vehicles, generating regions of interest (ROIs), and then use vision-based vehicle detection algorithms to further detect ROIs. This detection process is prone to missed detections due to errors in millimeter-wave radar detection. Furthermore, visual detection within ROIs lacks a global perspective, resulting in the loss of contextual information and potential false detections. Therefore, using individual sensors for initial vehicle detection and then fusing the results at the decision-making level is a more robust and effective choice, leveraging the complementary strengths of each sensor. Furthermore, most current vehicle detection algorithms based on information fusion aim to provide ADAS systems with position, speed, and category information. However, intelligent and safe high beam illumination is also crucial for safe driving at night. Vision-based vehicle detection methods extract vehicle features from images using image processing and computer vision techniques to detect the position and category of vehicles ahead. Vision-based vehicle detection methods can be divided into traditional methods and deep learning methods. Traditional methods manually extract vehicle features from images, such as vehicle edges, symmetry, and undercarriage shadows, and then use these features for vehicle detection. In contrast, deep learning methods automatically extract features from images using neural networks, offering greater robustness. Currently, the deep learning methods widely used in vehicle detection mainly fall into two categories: two-stage detection algorithms and single-stage detection algorithms. The YOLOv5 model serves as the benchmark object detection model. Good object detection results can provide the intelligent high beam system with the correct non-high beam illumination area, reducing the risk of glare for other nighttime drivers.

[0003] The YOLOv5 model suffers from scale imbalance and inconsistencies between the target labels and target features generated by the original mosaic data augmentation algorithm. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and to propose a data and structure optimization improvement method and processing terminal for YOLOv5, which aims to address the problem of scale imbalance in the YOLOv5 model and the inconsistency between the target labels and target features generated by the original mosaic data augmentation algorithm.

[0005] To this end, the present invention discloses a data and structure optimization and improvement method for YOLOv5, the method comprising: improving the header structure of the original YOLOv5 by adopting a header-shared convolution method;

[0006] Data augmentation is performed using lossless mosaic, removing labels that do not meet the cropping requirements and setting the corresponding regions to a background pixel value of 114, so that the target features are consistent with the target labels.

[0007] Preferably, the improvement of the original YOLOv5 header structure using a shared convolution method includes: in YOLOv5, the number of channels in the input header feature map are 256, 512, and 1024 respectively. Since the shared convolution uses the same convolution in different headers,

[0008] The feature maps need to have the same number of channels, so change the number of channels to 512.

[0009] At the same time, the convolutions of different output headers are replaced with the same convolution.

[0010] Preferably, the improvement of the original YOLOv5 head structure using head-shared convolution includes: dynamically setting the area ratio threshold area_thr in an increasing trend during training. Specifically, the total training period is set to 300 epochs, with area_thr set to 0.3 in epochs 0-99, 0.6 in epochs 100-199, and 0.9 in epochs 200-299.

[0011] Preferably, before improving the header structure of the original YOLOv5 by adopting a header-shared convolution method, the method further includes: processing the dataset using an intelligent data resampling algorithm.

[0012] Preferably, the processing of the dataset using the intelligent data resampling algorithm includes:

[0013] Set the minimum expected number of targets for each category to Num;

[0014] Calculate the quantity matrix;

[0015] Based on the total number of target samples in each category of the dataset, iterate through each category from smallest to largest, calculating the ratio (ratio) between the set minimum expected sample size Num and the current total number of samples in that category. If the ratio > 1, select images containing that category and duplicate them by the ratio. During duplication, three data augmentation methods are randomly applied: Gaussian blur, adding salt-and-pepper noise, and slight rotation. Images used for duplication are not included in the data resampling process. This process is repeated until all categories have been traversed.

[0016] Preferably, the step of setting the minimum expected target number for each category to Num further includes:

[0017] The minimum expected target quantity Num for each category was set using a simulation calculation method, which involved setting a series of Num values.

[0018] The proposed data resampling process is simulated, and then the target number ratio R between the largest and smallest sample classes after data resampling is recorded to obtain a series of (Num, R) points, which are then plotted on a two-dimensional coordinate system.

[0019] The optimal Num value is selected through a series of (Num, R) points.

[0020] Preferably, the step of selecting the optimal Num value through a series of (Num, R) points includes:

[0021] Calculate the percentage decrease in R value for each Num relative to the previous R value. R descenet R can be Calculate; based on a series of (Num, descenet) R This can plot the curve, where Num = 2500, and the corresponding descenet. R The first time it approaches zero indicates that when the Num value increases from 2000 to 2500, the effect of narrowing the gap between the target number of large sample categories and small sample categories is weak. From Num=2000 onwards, the effect is already very low. Therefore, taking Num=2000 is the minimum expected target number for each category, and at this time the cost-effectiveness of data resampling is the highest.

[0022] The second objective of this invention can be achieved by adopting the following technical solution:

[0023] A processing terminal includes a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, the processing terminal implements the aforementioned data and structure optimization improvement method for YOLOv5.

[0024] This invention designs a vision-based target detection algorithm. Considering the varying impacts of high beam illumination on vehicles in different states and the different levels of danger, a nighttime driving road dataset containing 12 target categories was constructed from 139 consecutive scenes. For real-time performance considerations, YOLOv5 was selected as the baseline target detection network. To further improve target detection performance, an intelligent data resampling method was used to augment the training data and address the class imbalance problem in the self-built dataset. Shared convolution was used to improve the YOLOv5 head, solving the problem of insufficient training of some parts of the head due to data scale imbalance. Simultaneously, to address the inconsistency between target labels and target features caused by the original mosaic data augmentation algorithm, a lossless mosaic data augmentation algorithm was proposed. Combining these methods, the improved YOLOv5 achieved a 5.7% improvement in mAP@0.5 on the validation set, effectively enhancing the vision-based vehicle detection performance. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a data and structure optimization improvement method for YOLOv5 according to the present invention;

[0028] Figure 2 This invention provides the overall implementation flow of an intelligent data resampling algorithm for improving data and structure optimization methods for YOLOv5.

[0029] Figure 3 This invention provides a quantitative matrix for an intelligent data resampling algorithm that improves data and structure optimization methods for YOLOv5.

[0030] Figure 4 The graph shows the variation of the ratio R between the maximum and minimum number of samples and the variation of the decrease in the R value in the data and structure optimization improvement method for YOLOv5 according to the present invention.

[0031] Figure 5 This is a schematic diagram illustrating the implementation of a header-shared convolution in a data and structure optimization improvement method for YOLOv5 according to the present invention.

[0032] Figure 6 This is a flowchart of a lossless mosaic data augmentation method for improving data and structure in YOLOv5 according to the present invention.

[0033] Figure 7 This invention provides a visualization effect of lossless mosaic under a reasonable threshold for a data and structure optimization improvement method for YOLOv5.

[0034] Figure 8 This is a lossless mosaic effect diagram under different area ratio thresholds (area_thr) for a data and structure optimization improvement method for YOLOv5 according to the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0036] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0037] Furthermore, the use of terms such as "first" and "second" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. When the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed by this invention.

[0038] This invention provides a data and structure optimization and improvement method for YOLOv5, with reference to... Figure 1-8 The method includes:

[0039] Step 100: The header structure of the original YOLOv5 was improved by using a header-shared convolution method.

[0040] Step 200: Data augmentation is performed using lossless mosaic, removing labels whose cropping ratio does not meet the requirements and setting the corresponding regions to background pixel values ​​of 114, so that the target features are consistent with the target labels.

[0041] This invention designs a vision-based target detection algorithm. Considering the varying impacts of high beam illumination on vehicles in different states and the different levels of danger, a nighttime driving road dataset containing 12 target categories was constructed from 139 consecutive scenes. For real-time performance considerations, YOLOv5 was selected as the baseline target detection network. To further improve target detection performance, an intelligent data resampling method was used to augment the training data and address the class imbalance problem in the self-built dataset. Shared convolution was used to improve the YOLOv5 head, solving the problem of insufficient training of some parts of the head due to data scale imbalance. Simultaneously, to address the inconsistency between target labels and target features caused by the original mosaic data augmentation algorithm, a lossless mosaic data augmentation algorithm was proposed. Combining these methods, the improved YOLOv5 achieved a 5.7% improvement in mAP@0.5 on the validation set, effectively enhancing the vision-based vehicle detection performance.

[0042] Preferably, in step 100, the header structure of the original YOLOv5 is improved by using a shared convolution method, including: in YOLOv5, the number of channels in the feature map of the input Head are 256, 512, and 1024, respectively. Since the shared convolution uses the same convolution in different Heads,

[0043] The feature maps need to have the same number of channels, so change the number of channels to 512.

[0044] At the same time, the convolutions of different output headers are replaced with the same convolution.

[0045] Preferably, step 200 involves data augmentation using lossless mosaic, removing labels whose cropping ratio does not meet the requirements and setting the corresponding regions to background pixel values ​​of 114, so that the target features are consistent with the target labels. This includes: during training, dynamically setting the area ratio threshold area_thr in an increasing trend. Specifically, the total training period is set to 300 epochs, with area_thr set to 0.3 in epochs 0-99, 0.6 in epochs 100-199, and 0.9 in epochs 200-299.

[0046] Here, a reasonable area ratio threshold, area_thr, is set, and an image from the texturing process is selected to visualize the above operation, achieving the effect shown below. Figure 7-8 As shown, from Figure 7-8As can be seen from the gray squares, the proposed lossless data augmentation not only removes target labels that do not meet the requirements of the area ratio before and after the cropping, but also sets the corresponding area in the image to the background pixel value of 114, thereby making the target features consistent with the label.

[0047] Because image region cropping in mosaic data augmentation is random, a fixed area ratio threshold (area_thr) cannot effectively ensure consistency between target features and target labels. If area_thr is set too low, small parts of the vehicle body may be mistaken for foreground; if it is set too high, features that meet foreground requirements may be mistaken for background. Therefore, during training, the area_thr threshold is dynamically set in an increasing trend. Specifically, the total training period is set to 300 epochs. Area_thr is set to 0.3 in epochs 0-99, 0.6 in epochs 100-199, and 0.9 in epochs 200-299. The lossless mosaic data augmentation effect produced by different threshold settings is shown below. Figure 8 As shown in the first column, when the threshold is set low, lossless mosaic data augmentation can effectively filter out irrelevant labels and cover small features as background color. As shown in the second and third columns, when the threshold is set high, lossless mosaic data augmentation, although sacrificing some positive samples, can effectively avoid the situation where the target is treated as background, which occurs in the original mosaic data augmentation.

[0048] Preferably, before step 100, which improves the header structure of the original YOLOv5 by using a header-shared convolution, the method further includes step 100A, which processes the dataset using an intelligent data resampling algorithm.

[0049] Preferably, step 100A, processing the dataset using the intelligent data resampling algorithm, includes:

[0050] Step 101A: Set the minimum expected number of targets for each category to Num;

[0051] Step 102A: Calculate the quantity matrix;

[0052] Specifically, a quantity matrix is ​​generated, and the contents of the matrix are as follows: Figure 3 As shown, the last column represents the image ID, and the preceding column numbers represent the category. Therefore, the elements other than the ID represent the target quantity of a certain category on the image with the corresponding ID, such as... Figure 4 In the image with id 00513, there are 3 targets of category Cls_1.

[0053] Step 103A: Based on the total number of target samples in each category of the dataset, iterate through each category from smallest to largest, calculating the ratio (ratio) between the set minimum expected sample size Num and the current total sample size for that category. If the ratio > 1, select images containing that category and duplicate them by the ratio. During duplication, randomly apply three data augmentation methods: Gaussian blur, adding salt-and-pepper noise, and slight rotation. The images used for duplication are not used in the data resampling process. Repeat this process until all categories have been traversed.

[0054] Specifically, based on the total number of target samples in each category of the dataset, each category is iterated from smallest to largest. The ratio (ratio) is calculated between the set minimum expected sample size Num and the current total number of samples in that category. If the ratio > 1, images containing that category are selected and copied by the ratio (rounded to the nearest integer). During the copying process, three data augmentation methods are randomly applied: Gaussian blur, adding salt-and-pepper noise, and slight rotation. The images used for copying are not included in the data resampling process. This process is repeated until all categories have been traversed.

[0055] To determine the minimum expected target number Num for each category, a simulation method was used. This involved setting a series of Num values, simulating the proposed data resampling process, and then recording the ratio R of the target number between the largest and smallest sample categories after data resampling. This resulted in a series of (Num, R) points, which were then plotted on a two-dimensional coordinate system based on a self-built dataset. Figure 4 .from Figure 4 It can be seen that as the set Num value becomes larger, the target ratio R between the largest and smallest sample classes after data resampling gradually decreases, and the class imbalance is gradually improved.

[0056] Step 101A, which sets the minimum expected target number for each category to Num, further includes:

[0057] The minimum expected target quantity Num for each category was set using a simulation calculation method, which involved setting a series of Num values.

[0058] The proposed data resampling process is simulated, and then the target number ratio R between the largest and smallest sample classes after data resampling is recorded to obtain a series of (Num, R) points, which are then plotted on a two-dimensional coordinate system.

[0059] The optimal Num value is selected through a series of (Num, R) points.

[0060] Preferably, the optimal Num value is selected through a series of (Num, R) points, including:

[0061] Calculate the percentage decrease in R value for each Num relative to the previous R value. R descenet R can be Calculate; based on a series of (Num, descenet) R This can plot the curve, where Num = 2500, and the corresponding descenet. R The first time it approaches zero indicates that when the Num value increases from 2000 to 2500, the effect of narrowing the gap between the target number of large sample categories and small sample categories is weak. From Num=2000 onwards, the effect is already very low. Therefore, taking Num=2000 is the minimum expected target number for each category, and at this time the cost-effectiveness of data resampling is the highest.

[0062] Specifically, although from Figure 4 The curve trend shows that continuing to increase the Num value can continue to improve the imbalance between the number of foreground categories. However, after data resampling, the rate of decrease of the target number ratio R between the largest and smallest sample categories gradually slows down, and the balance between the number of foreground categories no longer improves significantly. At the same time, if the Num value is too large, it will cause the training set to expand greatly, reducing training efficiency.

[0063] Therefore, a suitable Num value needs to be determined to balance the training set size and the foreground class balance. Starting from the magnitude of the R-value decrease, based on... Figure 4 Calculate the percentage decrease in R value for each Num relative to the previous R value. R descenet R can be The result is obtained. Based on a series of (Num, descenet) R ) can be drawn Figure 4 You can see descenet R As the Num value increases, it generally shows a decreasing trend; that is, as the Num value increases, the rate at which the difference in the number of targets between large and small sample categories narrows slows down. For example... Figure 4 As shown in the red circle, Num = 2500 at this time, corresponding to descenet R The first time it approaches zero indicates that when the Num value increases from 2000 to 2500, the effect of narrowing the gap in the number of target samples between large and small sample categories is already very weak. Taking values ​​after Num=2000 has a very low effect. Therefore, taking Num=2000 is the minimum expected number of target samples for each category, and at this time the cost-effectiveness of data resampling is the highest.

[0064] This invention designs data and structure optimization improvement methods for YOLOv5. An intelligent data resampling algorithm is proposed to alleviate the class imbalance problem in the dataset. A head-shared convolution method is used to improve the scale imbalance problem in the dataset. To address the inconsistency between target labels and target features caused by the original mosaic algorithm, a lossless mosaic data augmentation algorithm is proposed.

[0065] Example 2

[0066] This embodiment provides a processing terminal, including a processor and a memory for storing processor-executable programs. When the processor executes the program stored in the memory, the processing terminal implements a data and structure optimization improvement method for YOLOv5 as described in Embodiment 1 above.

[0067] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data and structure optimization and improvement method for YOLOv5, characterized in that, The method includes: The header structure of the original YOLOv5 was improved by adopting a header-shared convolution method; Data augmentation is performed using lossless mosaic, removing labels that do not meet the cropping requirements and setting the corresponding regions to background pixel values ​​of 114, so that the target features are consistent with the target labels. The improvement to the original YOLOv5 header structure employs a shared convolution approach. Specifically, in YOLOv5, the number of channels in the input header feature map is 256, 512, and 1024. Because the shared convolution uses the same convolution across different headers, The feature maps need to have the same number of channels, so change the number of channels to 512. At the same time, the convolutions of different output headers are replaced with the same convolution.

2. The data and structure optimization and improvement method for YOLOv5 according to claim 1, characterized in that, The improvement of the original YOLOv5 head structure by using head-shared convolution includes: dynamically setting the area ratio threshold area_thr in an increasing trend during training. Specifically, the total training period is set to 300 epochs, with area_thr set to 0.3 in epochs 0-99, 0.6 in epochs 100-199, and 0.9 in epochs 200-299.

3. The data and structure optimization and improvement method for YOLOv5 according to claim 1, characterized in that, Before improving the header structure of the original YOLOv5 by using a head-shared convolution method, the method also included processing the dataset through an intelligent data resampling algorithm.

4. The data and structure optimization and improvement method for YOLOv5 according to claim 1, characterized in that, The processing of the dataset using the intelligent data resampling algorithm includes: Set the minimum expected number of targets for each category to Num; Calculate the quantity matrix; Based on the total number of target samples in each category in the dataset, iterate through each category from smallest to largest, calculate the ratio of the set minimum expected sample size Num to the current total number of samples in that category. If the ratio > 1, select the image containing that category and copy it by the ratio. During the copying process, three data augmentation methods are randomly applied: Gaussian blur, adding salt and pepper noise, and slight rotation. The copied image is not used in the data resampling process. Then repeat this process until all categories have been traversed.

5. The data and structure optimization and improvement method for YOLOv5 according to claim 4, characterized in that, The minimum expected number of targets for each category is set to Num, and the following is also included: The minimum expected target quantity Num for each category was set using a simulation calculation method, which involved setting a series of Num values. The proposed data resampling process is simulated, and then the target number ratio R between the largest and smallest sample classes after data resampling is recorded to obtain a series of (Num, R) points, which are then plotted on a two-dimensional coordinate system. The optimal Num value is selected through a series of (Num, R) points.

6. A processing terminal, characterized in that, The device includes a processor and a memory for storing a processor-executable program. When the processor executes the program stored in the memory, the processing terminal performs the method as described in any one of claims 1 to 5.

Images