A complex road scene target detection method based on YOLOX model

Abstract

The application discloses a complex road scene target detection method based on a YOLOX model, relates to the technical field of road target detection, and aims at the problems of missed detection and low detection accuracy of a target detection algorithm under the conditions of long target distance, serious vehicle and pedestrian shielding and uneven light, designs a high-performance down-sampling module, retains high-quality fused semantic features, replaces a detection head with a decoupling task alignment detection head, and improves the interactivity of network output.In addition, an SGIOU loss function is designed to improve the attention of the network to the shape of a boundary box in the training process, and a dynamic cost function is used to reasonably plan the attention points of the network in different training stages.The method significantly reduces the network operation cost and the training parameter amount, and improves the detection performance of the algorithm for targets in a complex road scene.

Description

Technical Field

[0001] The present invention relates to the technical field of road target detection, and in particular to a method for detecting targets in complex road scenes based on a YOLOX model. Background Art

[0002] In today's society, science and technology and economy are constantly developing. People's demand for travel has driven the vigorous development of the automobile industry. The number of cars in my country has also continued to grow. However, it has also led to many road problems such as major traffic accidents that endanger travel safety and road congestion that affects travel quality.

[0003] Driving is a complex process, requiring drivers to accurately discern the background from key road objects to avoid collisions and even more serious accidents. During daily driving, drivers must maintain adequate distance from vehicles ahead and behind to ensure sufficient reaction time. However, complex road scenarios can easily lead to driver distraction and poor judgment, creating potential safety hazards.

[0004] Object detection, a classic task in computer vision, primarily distinguishes foreground and foreground objects in an image, identifies their categories, and outputs their location information. This is particularly important for vehicles to perceive and make decisions about external objects. Given the ease of technical implementation and subsequent deployment, monocular vision-based object detection algorithms are commonly used for external perception. This process can be summarized as two steps: feature extraction and object localization. With the rapid advancement of hardware technology and datasets, deep convolutional neural networks are widely used in various object classification networks, enabling them to fully learn the feature information in images. For subsequent object localization, two common approaches exist: setting up a separate network to detect objects, or adding a detection head after the feature extraction network. Due to the high real-time requirements for road object detection, end-to-end models are generally adopted. These models balance detection accuracy and efficiency, but suffer from limited detection capabilities for small objects and struggle to cope with complex scenes. Therefore, improving the detection capabilities of fast detection algorithms in complex scenes is a current research hotspot. Summary of the Invention

[0005] In response to the above problems and technical requirements, the inventors have proposed a method for detecting complex road scene targets based on the YOLOX model. The technical solution of the present invention is as follows:

[0006] A method for detecting complex road scene objects based on the YOLOX model includes the following steps:

[0007] Road images from different scenarios are acquired, uniformly scaled, and pixel values ​​normalized to construct a dataset. The dataset is divided into training and test sets according to a predetermined ratio. A TA-YOLOX model is constructed, and a high-performance downsampling module (Efficient Maxpooling, EMP) is designed. A decoupled task alignment detection head is introduced to improve the interactivity of the classification and localization tasks at the network output. The training set images are input into the improved TA-YOLOX model for training. Finally, the trained TA-YOLOX model is output to achieve cross-scale detection of complex road targets.

[0008] The beneficial technical effects of the present invention are:

[0009] 1) The complex road scene target detection method based on the YOLOX model proposed in this application adopts the SGIOU loss function to improve the convergence effect of the network during training and the detection ability of occluded targets;

[0010] 2) To reduce the loss of feature information during feature fusion, this application designs the EMP module to replace the Maxpooling layer in the neck and improve the network's high- and low-dimensional feature fusion capabilities;

[0011] 3) This application improves the positive and negative sample allocation rules, allowing the network to dynamically adjust the allocation benchmark based on the number of training iterations, so that the network can more reasonably allocate positive and negative samples in different training stages;

[0012] 4) To improve the misalignment between the detection head's positioning and classification tasks, this application introduces an improved task-aligned head to enhance the matching of the outputs of different network branches. Through the aforementioned network structure, positive and negative sample matching, and improvements to the loss function, the improved YOLOX anchor-free detection algorithm achieves excellent detection performance in complex road scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of the complex road scene target detection method provided by this application.

[0014] Figure 2 Schematic diagram of the network structure of the TA-YOLOX model provided in this application.

[0015] Figure 3 This is a schematic diagram of the structure of the decoupled task alignment detection head provided in this application.

[0016] Figure 4 This is a comparison chart of the detection effects of the TA-YOLOX model provided in this application and the traditional YOLOX model. DETAILED DESCRIPTION

[0017] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0018] Please refer to Figure 1 As shown, this application provides a complex road scene target detection method based on the YOLOX model, which specifically includes the following steps:

[0019] Step 1: Use a camera fixed on the front of a smart car to obtain road images in different scenarios. All images are uniformly scaled to 512*512 size and the pixel values ​​are normalized to construct a dataset.

[0020] Step 2: Divide the dataset into training set and test set in a ratio of 4:1.

[0021] Step 3: Construct TA-YOLOX model. Figure 2 As shown in Figure 1, the TA-YOLOX model includes a feature extraction network, a feature fusion network, and a decoupled task alignment detection head, where:

[0022] 1) The feature extraction network uses the DarkNet53 network, which contains 53 convolutional layers, each followed by a batch normalization layer (BN) and a Silu activation function. The combination of these three forms the CBS module. Residual connections, similar to those in ResNet50, are used to avoid vanishing gradients. Note that the DarkNet53 feature extraction network uses three and nine Bottleneck1 layers in Dark2 and Dark3, respectively, while nine and three Bottleneck2 layers are used in Dark4 and Dark5, respectively.

[0023] 2) The feature fusion network uses a feature pyramid network to fuse high- and low-dimensional features, and employs the EMP module as a downsampling module. The EMP module consists of two branches: one with a maximum pooling module and the other with a CBS module. After tensor concatenation, the two branches enter the CBS module, preserving high-quality fused semantic features.

[0024] 3) The decoupled task alignment detection head uses the ESE module to learn the features output by the feature fusion network and learn the fine-grained features therein, and then outputs them to the localization task output layer and the classification task output layer respectively. Figure 3 As shown in the figure, the ESE module passes through a global average pooling layer and then a fully connected layer FC, and finally the result after the Sigmoid activation function is multiplied element-wise with the feature map input to the ESE module to improve the matching degree of the output results of the positioning task and the classification task.

[0025] Step 4: Input the training set into the TA-YOLOX model for training.

[0026] The loss function Loss of the TA-YOLOX model includes classification loss, positioning loss, and obj loss. The calculation expression is:

[0027]

[0028] Where, L cls and L obj Represent the classification and confidence loss functions respectively, and both are calculated using the cross entropy loss function. reg is the positioning loss function, which is calculated using the SGIOU loss function. λ is the balance coefficient of the positioning loss function, which is set to 5 in this embodiment. N postive is the number of cells assigned as positive samples on the input feature map.

[0029] Since the traditional IOU loss function only considers the overlap between the ground-truth box and the predicted box and uses it to guide network convergence, in complex road target detection tasks, the shapes of objects vary greatly, such as pedestrians and vehicles. Therefore, this application designs the SGIOU loss function to improve positioning performance. Its calculation expression is:

[0030] SGIOU=IOU-α·Ω-β·Y (2)

[0031]

[0032]

[0033]

[0034]

[0035] Where Ω and Υ are shape loss and space distance loss respectively, α and β are balance coefficients, both of which are 0.5, w, h, w gt 、h gt are the width and height of the prediction box and the GT box respectively, θ is the degree of attention to shape loss, and the optimal value calculated by the genetic algorithm is 4. IOU represents the ratio of the intersection area of ​​the prediction box and the GT box to the union area, C is the area of ​​the minimum enclosing rectangle composed of the GT box and the prediction box, A is the area of ​​the GT box, and B is the area of ​​the prediction box.

[0036] The specific training steps of the TA-YOLOX model are as follows:

[0037] 1) Load the pre-trained weights of the feature extraction network and fix the weights of darknet53, initialize the parameters of the feature fusion network and the decoupled task alignment detection head, and then input the training set images into the TA-YOLOX model in batches.

[0038] 2) The predicted bounding box output by the regression prediction is scaled proportionally, and then non-maximum suppression is performed on all predicted bounding boxes to remove redundant bounding boxes. A dynamic cost function is used for positive and negative sample allocation, and the network's focus at different training stages is rationally planned. Specifically, the positioning loss and classification loss values ​​for each positive sample and the true box are calculated separately, and the sum of the positioning loss and classification loss values ​​is used as the cost function for positive and negative sample allocation. Considering that the object type of the positioning box needs to be quickly determined in the early stage of model training and that the accuracy of the positioning box should be paid more attention in the later stage of training, a dynamically changing coefficient lg(epoch) is added before the positioning loss in the cost function according to the changes in the epoch of model training. This makes the positioning loss term close to 0 in the early stage of training and gradually increases with the increase in training rounds, thereby increasing the model's focus on the classification task in the early stage of training and allowing the model to pay more attention to the positioning accuracy of the predicted box in the later stage of training, which improves the learning efficiency of the network to a certain extent.

[0039] 3) Using Loss as the loss function, the loss function value is calculated and the weights of the unfrozen network are updated. To improve network training efficiency, a cosine annealing learning rate is used for training. The learning rate changes according to a cosine curve with half the total number of iterations as a cycle. Using a cosine annealing strategy for the learning rate facilitates network convergence and avoids being trapped in a local optimum that is difficult to escape. In addition, the cosine annealing strategy allows for smooth changes in the learning rate, facilitating network learning.

[0040] Repeat steps 1 to 3 until the total number of iterations is reached.

[0041] Step 5: Output the trained TA-YOLOX model for road object detection and test the model using the test set to obtain the average accuracy, recall rate, and number of detections per second, ultimately achieving cross-scale detection of complex road objects.

[0042] To demonstrate the effectiveness of the present invention's "YOLOX-based Road Object Detection," this example uses the publicly available KITTI autonomous driving dataset. Its data acquisition equipment primarily includes a grayscale camera, a color camera, Edmund Optical lenses, a rotating 3D laser scanner, and inertial and GPS navigation systems. The KITTI dataset captures scenes ranging from rural areas to urban areas and highways, primarily targeting cars, trucks, pedestrians, and cyclists. Some targets are labeled "undetermined" due to their distance from the camera or significant overlap. These labels are not input into network training, nor do they appear in the prediction results. A single image in the dataset contains up to 15 vehicles and 30 pedestrians. The official dataset consists of a training set and a test set. However, since the test set labels are not publicly available, the effectiveness of the present invention cannot be verified. Therefore, the 7,481 training set images were divided into training and test sets in a 4:1 ratio, with 6,000 training samples and 1,481 test samples.

[0043] Build the TA-YOLOX model, load pre-trained weights into the DarkNet53 backbone network and freeze them, and initialize the parameters of the feature fusion network and detection head. Input the training set images into the TA-YOLOX model for training. During model training, 32 samples are used per batch, with an initial learning rate of 0.01 and a final learning rate of 0.00001, varying by half a cosine period. During training, four images are randomly concatenated to enhance the network's ability to distinguish foreground from background. Every 10 iterations, the input image size is randomly resized to 67%-150% of the original size. The number of iterations is set to 300.

[0044] After training, the test set was used to verify the detection effect of the model. In addition, in order to verify the superiority of this method in the road object detection task, the TA-YOLOX model was compared with a variety of other models. Each method was trained with the same training set and tested with the same test set. The evaluation indicators of the test results were the number of parameters for large, medium and small objects, the amount of computation GFLOPS, the average accuracy mAP, and the detection time for a single image (including prediction and post-processing). The test results are shown in Table 1, where mAP@0.50 represents an IOU threshold of 0.50.

[0045] Table 1 Evaluation indicators of detection results of each algorithm

[0046]

[0047]

[0048] As can be seen from the data in the table, the detection performance indicators mAP@0.50 and mAP@0.50:0.95 of the algorithm proposed in this application are 2.9% and 4.1% higher than the baseline network respectively, while significantly reducing the network size and computational complexity, which is of great significance for the subsequent deployment of the network. Although the detection speed of the network has been slightly reduced, it has no effect on the real-time detection of the network. Compared with other mainstream algorithms, it can be seen that the mAP@0.50 and mAP@0.50:0.95 of the TA-YOLOX model always maintain their lead, with only the detection speed slightly lower than YOLOX-m, Fcos, YOLOv7-l and YOLOv8-l. However, for daily use, 30 to 40FPS can already meet the needs of real-time detection. Therefore, the method proposed in this application achieves a good balance between detection accuracy and detection speed, and still has a great advantage in deployment and application.

[0049] In order to further compare the method proposed in this application with the original YOLOX-1 algorithm, the detection effects of three difficult-to-detect images in the KITTI dataset are compared. Figure 4 As shown in the figure, from top to bottom are the original image in the KITTI dataset, the detection effect of YOLOX-1, and the detection effect of the TA-YOLOX model. In the original image, the distant or overlapping areas are magnified, and it can be seen that some targets in the original image are far away, the targets overlap severely, and the lighting is uneven. Comparing the detection results, it can be seen that the method proposed in this application can effectively detect small targets, and the detection effect is good in the case of overlap and poor lighting, which proves that the improved algorithm effectively solves the problems of missed target detection and inaccurate positioning.

[0050] From the above, it can be seen that the TA-YOLOX model provided by this application improves the YOLOX algorithm to address the above-mentioned issues. In response to the problem that the algorithm has low detection accuracy for small targets and targets with uneven lighting, an EMP downsampling module is designed to reduce the loss of semantic features during feature fusion. In response to the problem that the algorithm is prone to missed detections and false detections when the target is severely occluded, an SGIOU loss function is designed to make the network pay more attention to the shape of the bounding box during training. In response to the problem that the decoupling head proposed by the original YOLOX algorithm is prone to output low-quality detection frames, resulting in inaccurate positioning when the target is far away, a decoupling task alignment detection head is introduced. The positioning task and the classification task guide each other in the final output of the network, and a dynamic sample matching strategy is designed to improve the network's focus on classification information in the early stage, and to focus more on improving the quality of the positioning frame in the middle and late stages of training.

[0051] The above description is only a preferred embodiment of the present application, and the present invention is not limited to the above embodiment. It is understood that other improvements and variations directly derived or imagined by those skilled in the art without departing from the spirit and concept of the present invention should be considered to be included in the scope of protection of the present invention.

Claims

1. A target detection method for complex road scenes based on the YOLOX model, characterized in that: The method comprises: Step 1: Obtain road images in different scenarios, uniformly scale the images and normalize the pixel values ​​to construct a dataset. Step 2: Divide the data set into a training set and a test set according to a predetermined ratio; Step 3: Construct a TA-YOLOX model, which includes a feature extraction network, a feature fusion network, and a decoupled task alignment detection head. The feature extraction network uses the darknet53 network and adopts the residual connection method. The feature fusion network adopts the feature pyramid network to fuse high- and low-dimensional features, and uses the EMP module as the downsampling module. The decoupled task alignment detection head uses the ESE module to learn the features output by the feature fusion network and the fine-grained features therein, and then outputs them to the localization task output layer and the classification task output layer respectively. The EMP module includes two branches, one branch is provided with a maximum pooling layer, and the other branch is provided with a CBS module. The two branches enter the CBS module after tensor splicing; the CBS module includes a convolutional layer, a batch normalization layer and a Silu activation function connected in sequence; The ESE module passes through a global average pooling layer and then a fully connected layer, and finally the result after the Sigmoid activation function is multiplied element-by-element with the feature map input to the ESE module to improve the matching degree of the output results of the positioning task and the classification task; Step 4: Input the training set into the TA-YOLOX model for training, including: Load the pre-trained weights of the feature extraction network and fix the weights of darknet53, initialize the parameters of the feature fusion network and the decoupled task alignment detection head, and then input the training set images into the TA-YOLOX model in batches; The predicted bounding boxes output by the regression prediction are scaled proportionally, and then all predicted bounding boxes are subjected to non-maximum suppression to remove redundant bounding boxes. A dynamic cost function is used to allocate positive and negative samples, and the focus of the network is rationally planned at different training stages. by Calculate the loss function value for the loss function and update the weights of the unfrozen network part. Use cosine annealing learning rate for training. The learning rate changes according to the cosine curve with half a cycle of the total number of iterations. Re-performing the step of loading the pre-trained weights of the feature extraction network until the total number of iterations is reached; Step 5: Output the trained TA-YOLOX model for road object detection and use it to test the model using the test set to obtain the average accuracy, recall rate, and number of detections per second, ultimately achieving cross-scale detection of complex road objects.

2. The complex road scene target detection method based on the YOLOX model according to claim 1 is characterized in that: The use of a dynamic cost function to distribute positive and negative samples includes: Calculate the positioning loss value and classification loss value of each positive sample and the real frame respectively, and add the positioning loss value and classification loss value together as the cost function for the positive and negative sample allocation. epoch The change of the positioning loss in the cost function is preceded by a dynamically changing coefficient , making the positioning loss term close to 0 in the early stage of training and gradually increasing with the increase of training rounds, so as to improve the focus of the model on the classification task in the early stage of training and let the model pay more attention to the positioning accuracy of the prediction box in the later stage of training.

3. The complex road scene target detection method based on the YOLOX model according to claim 1 is characterized in that: The loss function of the TA-YOLOX model Contains classification loss, positioning loss and obj The loss is calculated as: ； Where, and Represent the classification and confidence loss functions respectively, and both are calculated using the cross entropy loss function. As the positioning loss function, we use SGIOU The loss function is calculated. is the balance coefficient of the positioning loss function, is the number of cells assigned as positive samples on the input feature map; described SGIOU The calculation expression of the loss function is: ； ； ； ； ； Where, and are shape loss and spatial distance loss respectively, and is the balance coefficient, w 、 h 、 、 are the width and height of the prediction box and GT box respectively, is the degree of concern for shape loss, IOU Represents the ratio of the intersection area and the union area of ​​the prediction box and the GT box, is the area of ​​the minimum enclosing rectangle formed by the GT box and the prediction box, is the area of ​​the GT box, is the area of ​​the prediction box.

Images