Target detection method and system under roadside view angle

Abstract

The invention discloses a target detection method and system under a roadside view angle. The method comprises the following steps: acquiring a data set; constructing a roadside image detection improved model; training and evaluating the road side image detection improved model based on the data set; acquiring a roadside image; and inputting the roadside image into a trained roadside image detection improved model for identification, and realizing target detection under a roadside view angle. According to the method, feature fusion is reconstructed on the basis of YOLOv5s, so that small target information can be more effectively captured, and the small target feature extraction capability is improved; secondly, the GFPN is introduced for multi-scale feature fusion, so that the fusion efficiency of feature information with different resolutions is improved; in addition, a CA attention mechanism is introduced, so that the robustness of the model is improved; and finally, replacing a Focal-EIOU loss function with a loss function of the algorithm model, and improving the positioning speed while improving the bounding box regression speed.

Description

[0001] The present application is a divisional application of the patent application entitled "A New Roadside Perspective Target Detection System and Method", the original application date is August 15, 2022, and the application number is 202210985807.X. TECHNICAL FIELD

[0002] The present application relates to the field of intelligent transportation vision technology, in particular to a roadside perspective target detection method and system. BACKGROUND

[0003] Intelligent perception system is an important prerequisite for realizing automatic driving. The current perception system mainly equips vehicles with laser radar or camera sensors. Due to the movement of the vehicle, the vehicle perception system cannot observe the surrounding environment for a long time. The vehicle-mounted perspective focuses more on the front area, while the roadside camera observes the scene for a long time and in a wide range. The vehicle-mounted field of view is easily blocked by objects in front of the vehicle, while the roadside field of view is located at a higher position on the road for monitoring, which can provide more rich road condition information according to the actual intersection, and can effectively reduce the occurrence of traffic accidents. The current intelligent perception system mainly detects based on the vehicle-mounted perspective, and achieves high detection accuracy, but there are few application researches on roadside perspective target detection.

[0004] In recent years, with the development of deep learning and the emergence of high-performance image processors, using deep learning convolutional neural network to detect targets has been used by almost all computer vision researchers. Typical two-stage detection algorithms include R-CNN, Fast R-CNN, and Faster R-CNN. However, due to the structural limitations, the detection speed cannot meet the real-time requirements. YOLO (You Only Look Once) was first proposed by Redmon et al. in 2015. As a one-stage detection algorithm, it performs classification and regression tasks while generating frames, and the detection rate is significantly improved. With the development of machine vision, the latest YOLOv5s algorithm performs more outstandingly in model parameter quantity and floating point operation quantity.

[0005] The perspective of the roadside end has a wider perception range. The target information obtained by the visual sensor is mainly small targets. Due to the lack of information extracted by the shallow layer of the detection model network, it is difficult to accurately classify and accurately locate small-scale targets. At the same time, in complex road conditions, the target information is more complex, and there are situations where the detection target and the background information are mixed together or multiple targets overlap. In addition, the information perceived by the roadside end also needs to be transmitted to the vehicle for decision and control through wireless communication technology, so there are high requirements for the real-time performance and easy deployment of the detection algorithm. SUMMARY

[0006] The purpose of the present application is to provide a roadside perspective target detection method and system to solve the above-mentioned problems existing in the prior art.

[0007] To achieve the above-mentioned purpose, the present application provides a roadside perspective target detection method, comprising:

[0008] Obtaining a data set;

[0009] Constructing a roadside image detection improved model; the roadside image detection improved model is obtained by removing the top feature extraction layer based on the YOLOv5s algorithm model, integrating the CA attention mechanism at the end of the backbone network, then using the generalized feature pyramid network structure to connect the features of the previous layer and the current layer in a cross-scale manner and establish a skip layer connection, finally inputting the output of the generalized feature pyramid network structure into the detection layer to generate a prediction box, and then using the Focal-EIOU loss function to obtain the prediction result;

[0010] Training and evaluating the roadside image detection improved model based on the data set;

[0011] Obtaining a roadside image;

[0012] Inputting the roadside image into the trained roadside image detection improved model for recognition to realize target detection under the roadside perspective.

[0013] Optionally, the Focal-EIOU loss function includes overlap loss, center distance loss and width-height loss.

[0014] Optionally, before training and evaluating the roadside image detection improved model based on the data set, the method further comprises:

[0015] Dividing the data set into different sub-data sets based on lighting conditions, weather conditions, road scenes and camera specifications, and dividing them into training sets and validation sets.

[0016] The present application also provides a roadside perspective target detection system, comprising:

[0017] A data acquisition module, a model construction module and a loss function module;

[0018] The data acquisition module is used to obtain roadside image data sets and training data sets;

[0019] The model construction module includes an information extraction module and an information fusion module, and the information fusion module is used to fuse the feature information;

[0020] The information extraction module is used to extract the feature information in the roadside image data set and the training data set.

[0021] Optionally, the information extraction module comprises a feature extraction module and an attention module; the attention module is used for position information embedding and coordinate information feature map generation, and enhances important channels and spatial feature information in the features; the feature extraction module is obtained by removing a top feature extraction layer of a backbone network feature extraction module of a YOLOv5s algorithm model, and is used for extracting feature information.

[0022] Optionally, the information fusion module adopts a generalized feature pyramid network structure and a cross-scale feature fusion method to fuse feature information, wherein the cross-scale feature fusion method comprises two modes: feature fusion of three nodes, i.e., a node above, an oblique upper node and an oblique lower node of each scale, and fusion of features of a previous layer and a current layer and establishment of a skip layer connection.

[0023] Optionally, the loss function module comprises an EIOULoss and a FocalLoss; the EIOULoss is used for optimizing convergence speed and regression accuracy, and the FocalLoss is used for optimizing a sample imbalance problem in a bounding box regression task.

[0024] Optionally, the information fusion module adopts a feature fusion mode starting from a bottom feature extraction layer to fuse information.

[0025] The technical effects of the present application are as follows:

[0026] 1. On the basis of studying unique features of roadside perspective images, an improved YOLOv5s-based target detection algorithm under roadside perspective is proposed by optimizing a network, channels and parameters. In view of the characteristics of a complex roadside environment and small target image size, a feature fusion layer of an original network is reconstructed, and a feature space perception field is increased, so that it is more conducive to feature extraction and recognition of small targets.

[0027] 2. Since the feature fusion layer reconstruction deletes a high-multiplication down-sampling feature fusion layer, in view of the loss caused by the reduction of deep feature fusion, the feature pyramid network structure of the Neck part of the YOLOv5s network is improved, the ability of aggregating multi-scale feature information is enhanced, so that it can more effectively fuse feature information of different resolutions, improve the multi-scale feature fusion efficiency of the model, and further improve the accuracy of target detection.

[0028] 3. In view of the influence of different weather environments and other interference factors on the detection effect, an attention mechanism is introduced in the Backbone part of the YOLOv5s network, the perception field of the network is improved, the model can more accurately locate and recognize important information, and the robustness of the model is improved.

[0029] 4. In order to improve the convergence speed and regression accuracy, we improved the loss function in the detection head part of YOLOv5s network, accelerated the learning of high confidence targets, and improved the training speed and anchor positioning accuracy.

[0030] 5. Reduce the weight of the model, which is more conducive to the deployment of edge devices. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor under the premise of the drawings.

[0032] Figure 1 YOLOv5s network structure diagram in the embodiments of the present application;

[0033] Figure 2 Feature fusion layer reconstruction network structure diagram in the embodiments of the present application;

[0034] Figure 3 FPN, PANet, BiFPN and GFPN network structure in the embodiments of the present application;

[0035] Figure 4 Queen-Fusion queen fusion structure diagram in the embodiments of the present application;

[0036] Figure 5 log 2n-link cross-layer connection structure diagram in the embodiments of the present application;

[0037] Figure 6 CA structure encoding attention process diagram in the embodiments of the present application;

[0038] Figure 7 Improved diagram of the backbone network in the embodiments of the present application;

[0039] Figure 8 Method flowchart in the embodiments of the present application;

[0040] Figure 9 Improved YOLOv5 network structure diagram in the embodiments of the present application. DETAILED DESCRIPTION

[0041] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0042] Embodiment one

[0043] The perspective of the roadside end has a wider perception range, and the target information collected is usually small in scale. In complex road conditions, the target information is more complex and the overlap phenomenon is serious. The traditional deep learning method is difficult to achieve good detection accuracy, and the missed detection rate is high. In addition, due to the high requirements of roadside devices for the real-time performance and easy deployment of detection algorithms. Therefore, the target detection algorithm based on deep learning gradually shows its advantages. Typical Two-stage detection algorithms include R-CNN, Fast R-CNN, and Faster R-CNN. However, due to the structural limitations, the detection speed cannot meet the real-time requirements. YOLO (You Only Look Once) was first proposed by Redmon et al. in 2015. As a One-stage detection algorithm, it has significantly improved detection speed. With the development of machine vision, the latest YOLOv5s algorithm performs more outstanding in model parameter quantity and floating point operation quantity.

[0044] Therefore, the present application improves the YOLOv5s algorithm and proposes a target detection system and method under the perspective of the roadside.

[0045] As shown in Figures 1-9 The present embodiment provides a target detection system and method under the perspective of the roadside, which includes:

[0046] Obtain a data set; construct a roadside image detection improved model; train and evaluate the roadside image detection improved model based on the data set; obtain a roadside image; input the roadside image into the trained roadside image detection improved model for recognition, and realize target detection under the perspective of the roadside.

[0047] The structure of YOLOv5s is as shown in Figure 1As shown, the Input input end, Backbone main network, Neck network and Head output end four parts are composed. The Input input end has the functions of Mosaic data enhancement, adaptive anchor box calculation and adaptive picture scaling. The Backbone main network includes Focus structure, C3 structure and pyramid pooling SPP structure, and different levels of features in the image are extracted through deep convolution operation. The Neck network is composed of pyramid FPN and path aggregation network PAN. The Head is the last detection, which predicts different sizes of targets on different size feature maps, and finally processes the predicted anchor box through NMS (Non Maximum Suppression).

[0048] The Backbone part of the YOLOv5s network is responsible for extracting image feature information. Through multiple downsampling operations on the input image, different scale feature information is obtained. The bottom layer feature has higher resolution and contains more location and detail information, but the semantic information is low and the noise is more. The high-level feature has more semantic information, but the resolution is low and the detail perception ability is poor. Effective feature fusion of the extracted feature information is the key to improve the detection performance of the model. The Neck of the YOLOv5s network is composed of pyramid FPN and path aggregation network PAN. FPN transfers the deep semantic features to the shallow layer to enhance the semantic expression of multiple scales, and PAN transfers the positioning information of the shallow layer to the deep layer to enhance the positioning ability of multiple scales.

[0049] For the difficulties of complex, small size and serious coverage of roadside detection targets, the Backbone network feature extraction module and Neck network feature fusion module of the YOLOv5s network are restructured to obtain the feature fusion layer restructured network structure diagram as shown in Figure 2 In the main network, because multiple downsampling is used for feature extraction, the feature information of small weak targets is continuously reduced or even disappeared with the increase of feature level. Therefore, the top feature extraction layer is removed based on the YOLOv5s main network, which reduces the network complexity and reduces the invalid information entering the next stage for feature fusion. In the feature fusion module, in order to capture more effective information of small weak targets, feature fusion is started from the bottom feature extraction layer. Because the large size feature map has a large spatial receptive field and contains more location and detail information, it can accurately detect small weak targets in the image.

[0050] The receptive field is wider under the roadside perspective, and the target image information collected in different scenes is often inconsistent in size, especially in complex road conditions, there are complex target information and serious target overlap. Therefore, different sizes of target detection need to be considered in the task of target detection.

[0051] Feature fusion is currently the main method for solving multi-scale difference problems. Representative algorithms include FPN, PANet, and BiFPN. Their core idea is to extract features of different resolutions through convolutional modules in the backbone network and then fuse these features. However, these FPN architectures primarily focus on feature fusion but neglect intra-block connections. GFPN (Generalized Feature Pyramid Network) fuses features from the previous and current layers through cross-scale connections and establishes skip connections, providing more efficient information transmission and allowing for expansion into deeper networks. Under different FLOPs (Floating-point Operations per Second) performance balances, GFPN exhibits superior performance, surpassing other state-of-the-art (SOTA) solutions, such as... Figure 3 As shown in part (d).

[0052] Sufficient information exchange should include not only skip-level connections but also cross-scale connections to overcome large-scale variations; GFPN proposes a Queen-Fusion structure. For example... Figure 4 As shown, nodes at each scale simultaneously perform feature fusion on their previous, diagonally above, and diagonally below nodes. In scenarios with large scale variations, this facilitates effective information transmission and feature fusion across different scales for target image features. Furthermore, using concat instead of sum for feature fusion effectively reduces feature fusion loss. Compared to other feature pyramid network structures, the GFPN structure has higher complexity. To avoid gradient vanishing due to increased computation, GFPN proposes log 2n-link cross-scale feature fusion to increase feature reuse. This not only improves the depth of GFPN extensions but also preserves effective feature reuse, such as... Figure 5 As shown.

[0053] In summary, GFPN modifies the Neck part of YOLOv5s to improve the model's multi-scale feature fusion capability.

[0054] During information extraction in the backbone network, the receptive field of the high-level feature map is relatively large due to multiple downsampling steps, but much detailed information is lost, especially small targets, which may be completely lost. To enhance the detection accuracy of the network, this application introduces a coordinate attention mechanism and constructs a hierarchical attention structure similar to human perception to enhance the network's extraction capabilities.

[0055] The main idea of ​​the CA attention mechanism is to embed location information into channel attention. Accurate location information encoding channel relationships and long-range dependencies is more conducive to the network extracting important information from feature images. The CA attention mechanism mainly consists of two steps: location information embedding and coordinate information feature map generation. The specific steps are as follows: Figure 6 As shown, input image information X is subjected to average pooling along the horizontal and vertical coordinate directions using pooling kernels of size (1×W) and (H×1) respectively, to obtain perceptual attention maps z in two independent directions, horizontal and vertical. w With z h Their sizes are (C×1×W) and (C×H×1) respectively. Next, z... h A dimension transpose is performed to obtain a feature map of size (C×1×H), which is then concatenated with a feature map of size (C×1×W). This concatenation is then fed into a shared (1×1) convolutional module and a non-linear activation function to obtain the process feature map f∈R. C / r×1×(H+W) , where r is the channel downsampling ratio in the convolution. Then, f is split into two feature vectors f_i in the channel dimension. h f w And adjust the size of the vector in the second and third dimensions so that their sizes are f∈R. C / r×1×W and f∈R C / r×H×1 Channel transformation is performed using (1×1) convolution kernels, making f w f h The sizes are f∈R C ×1×W and f∈R C×H×1 Finally, the attention weight map g in two spatial directions is obtained by activating the Sigmoid(x) function. h and g w Each attention weight map carries the long-range dependency of the feature map along a specific direction. Finally, the input feature map is multiplied by the two weights, thereby enhancing the expressive power of the feature map.

[0056] To address the issues of large discrepancies in target detection and low detection rate for small targets in roadside images, the original YOLOv5s model algorithm tends to lose feature information of small targets during convolutional sampling, resulting in poor detection performance for small targets. Therefore, as... Figure 7 As shown, a CA attention mechanism was added to the Backbone part of the YOLOv5s network to enhance the important channel and spatial feature information in the features, namely the "key points" of the learning target, thereby effectively improving the accuracy of locating small targets.

[0057] The Detect part of YOLOv5s is composed of a loss function and a non-maximum suppression. The loss function is used to measure the degree of overlap between the predicted bounding box generated by the model and the true box. The non-maximum suppression is used for the screening of the target box, suppressing non-maximum elements, searching for local maximum values, removing redundant detection boxes, and obtaining the final prediction result. The loss function used in the YOLOv5s model is CIOU Loss, which takes into account the overlap area, center distance, and aspect ratio of the boundary box regression. However, the width and height used by CIOU Loss are a relative proportion, not the actual width and height values, which has some ambiguity, so it sometimes hinders the effective optimization of the model similarity. To address this issue, Zhang et al. proposed EIOU Loss by separating the aspect ratio based on CIOU, and added Focal focusing on high-quality anchor boxes. The principles are shown in equations (1) and (2).

[0058]

[0059] Focal-EIOU Loss is composed of three parts: overlap loss, center distance loss, and width-height loss. The first two parts continue the method of CIOU Loss, separating the loss term of aspect ratio into the difference between the predicted width and height and the minimum enclosing width and height, which accelerates the convergence of the predicted box and improves the regression accuracy of the predicted box. However, the width-height loss directly minimizes the difference between the width and height of the target box and the anchor box, which makes the convergence faster. Where C ω and C h are the width and height of the minimum enclosing box covering two boxes. Considering the problem of training imbalance in boundary box regression, the number of high-quality anchor boxes with small regression error is much smaller than that of low-quality samples with large error in an image, and poor-quality samples will produce excessive gradient to affect the training process.

[0060] To address the problem of training imbalance in boundary box regression, the number of high-quality anchor boxes with small regression error is much smaller than that of low-quality samples with large error in an image, and poor-quality samples will produce excessive gradient to affect the training process, FocalLoss is introduced to optimize the sample imbalance problem in the boundary box regression task, separating high-quality anchor boxes from low-quality anchor boxes, and focusing on high-quality anchor boxes in the regression process. The principle is shown in equation (3).

[0061] L Focal-EIOU = IOU γ L EIOU (3)

[0062] The present application trains and tests the improved network using the public roadside dataset Rope3D. The feature fusion layer of the original YOLOv5s model is restructured to filter out redundant features and retain important features, improving the accuracy of small target detection. Then, the GFPN feature pyramid structure is used for feature fusion to improve the speed and efficiency of feature fusion, and the CA attention mechanism is integrated to enhance the robustness of the model. Finally, the Focal-EIOU loss function is used to improve the regression rate of the bounding box and the positioning accuracy of the anchor box.

[0063] For the above improvement strategy, an improved YOLOv5 target detection algorithm based on roadside perspective is proposed, and its general network structure diagram is as shown in Figure 9 .

[0064] In order to improve the small target detection rate, as shown by the two dashed boxes in Figure 9 , the 32 times down-sampling feature fusion layer is deleted, the 4 times down-sampling feature fusion layer is added, the ability of shallow feature information extraction is enhanced, and the problem of continuous reduction or even disappearance of weak small target feature information with the increase of feature level is effectively prevented. The attention mechanism is added at the end of the backbone network, as shown by A in Figure 9 , which enhances the important channels and spatial feature information in the feature and improves the ability to locate small targets. Then, the multi-scale feature information collected by the Backbone part is input into the Neck part for feature fusion. It contains two information transmission paths, top-down and bottom-up. In the former path, the deep features are passed down to provide semantic guidance for shallow features. In the latter path, the shallow features are passed up to supplement the details of the deep features. The fused features have both abstract semantic information and rich detail information, which is beneficial to improve the positioning accuracy and classification accuracy. At the same time, the output nodes of the same layer are connected to each other, which can fuse more feature information without increasing additional consumption. In addition, each scale node simultaneously fuses features from the three nodes above, diagonally above and diagonally below, which is helpful for effective information transmission and feature fusion of different scales in the scene with large scale changes. Finally, the output of GFPN is input into the detection layer to generate prediction boxes, and then NMS operation is performed to filter out redundant prediction boxes in local regions and obtain the final prediction results.

[0065] The Rope3D public data set was selected for experimental prediction. Due to the YOLOv5s configuration, the data set's annotation file was preprocessed, and a 2D box was used for target labeling. The Rope3D data set is challenging because the roadside camera is installed on the roadside pole rather than the top of the car. There are different camera configuration conditions, such as camera internal parameters, pitch angle height, and installation height, so the detection algorithm needs to have a certain robustness. In addition, because the perception range is larger from the roadside perspective, more objects can be observed, which also increases the difficulty of the perception system detection.

[0066] The data set is divided into training and validation sets, containing 50k images and over 1.5M objects, collected under various lighting conditions (day / night / sunset), different weather conditions (rain / sunny / cloudy), different road scenes, and different camera specifications such as focal length and viewpoint. To facilitate comparative experimental analysis in the paper, the Rope3D data set is divided into different sub-data sets according to the scene, as shown in Table 1. These are the cloudy, night, sunny, and rainy data sets.

[0067] Table 1

[0068]

[0069] The experiment was conducted under the Windows 10 operating system, with an Intel(R) Xeon(R) Silver 4210 CPU and an NVIDIA TITAN RTX GPU. The development environment was Python 3.8, Pytorch 1.6, and CUDA 10.1. To ensure the accuracy of the training results, the participating algorithms were trained and tested under the same training parameters, with model parameter values set as shown in Table 2.

[0070] Table 2

[0071]

[0072] To more accurately analyze the performance of the improved YOLOv5s network, it is essential to use appropriate evaluation metrics. Frame Per Second (FPS) and Mean Average Precision (mAP) are used as evaluation metrics for the algorithm model in this experiment. Each metric reflects the performance of the detection algorithm in different aspects to some extent. First, the precision and recall should be calculated. Precision is the proportion of correctly predicted targets in all predicted targets, highlighting the relevance of the results. Recall, also known as recall, is the proportion of correctly predicted targets in all real (positive) targets. The calculation formulas are shown in equations (4) and (5):

[0073]

[0074]

[0075] Wherein, TP represents the number of samples actually positive and classified as positive examples; FP represents the number of samples actually negative but classified as positive examples; FN represents the number of samples actually positive but classified as negative examples.

[0076] The average precision (AP) is to average the precision of a class in the data set, as shown in equation (6), P and R represent the precision and recall respectively, and AP is the area contained by the curve. The average precision mean (mAP) refers to the average of all class APs, and its calculation formula is shown in equation (6) and equation (7).

[0077]

[0078] The model is performed on the Rope3D dataset. First, the configuration file is changed according to the training requirements of YOLOv5s, and the experimental parameter values are set, including the decay coefficient, epoch value, learning rate, batch size, momentum, etc. During the experiment training process, it can be found by checking the training log that when the training reaches about 150 times, the loss function is basically not decreasing and gradually tending to be stable, so the training is terminated when the training reaches 150 times.

[0079] According to the characteristics of lightweight deployment of edge devices, YOLOv5s model is used for training and prediction, and lightweight optimization and improvement are carried out. In order to verify the influence of single structure change on network performance, a total of 3 kinds of models are designed for comparison with the original YOLOv5s model, which are:

[0080] 1) YOLOv5s-T: The feature fusion layer of YOLOv5s network is reconstructed, the 32 times down-sampling feature fusion layer is deleted, and the 4 times down-sampling feature fusion layer is added.

[0081] 2) YOLOv5s-CA: The attention mechanism CA module is added after the spatial pyramid pooling module of the Backbone part of YOLOv5s network.

[0082] 3) YOLOv5s-G: The GFPN structure is introduced to modify the feature pyramid structure of the Neck part of YOLOv5s network.

[0083] YOLOv5s, YOLOv5s-T, YOLOv5s-G and YOLOv5s-CA are trained and tested respectively, and the complexity and performance of the four models are shown in Table 3 and Table 4.

[0084] Table 3

[0085]

[0086] Table 4

[0087]

[0088] From the analysis of the experimental results, the experimental results of the original YOLOv5s model on the data set show that the mAP value is 53.6%, and the detection rate is 71.4. Compared with the original YOLOv5s, the parameter quantity of YOLOv5s-T model is reduced by 75.2%, the operation quantity is reduced by 16.5%, the detection rate is increased by 14.8, the model size is reduced by 71.1%, and the mAP value is increased by 3.5%. This is due to the use of feature fusion layer at 32 times, which effectively reduces the high-dimensional calculation and feature fusion overhead caused by deep features. YOLOv5s-G uses GFPN structure, compared with the original YOLOv5s model, the parameter quantity increases more, the calculation quantity increases, which leads to the detection rate decreases by 25.2, but the mAP value increases by 3.0%. Compared with the original YOLOv5s model, YOLOv5s-CA adds attention mechanism CA module, although the parameter quantity and calculation quantity of the model increase slightly, the detection rate is basically unchanged and the mAP value increases by 0.5%, which shows the effectiveness of introducing attention mechanism.

[0089] To verify the effectiveness of the improved loss function, a loss function comparison experiment was conducted. The epoch value set in the experimental parameters was 150. During the training process, it was found that when the training reached about 150 times, the loss function tended to be stable, so the training was terminated.

[0090] The improved loss function of YOLOv5 has a more obvious downward trend, indicating the effectiveness of the improved algorithm.

[0091] To further verify the effectiveness of the improved loss function, the results of the improved loss function and the original algorithm loss function on the algorithm performance are compared, as shown in Table 5.

[0092] Table 5

[0093]

[0094] In order to verify the influence of the improved loss function on the performance of the algorithm, the loss function in the algorithm is replaced separately in the experiment, and the Focal-EIOU Loss and the original algorithm CIOU Loss are compared horizontally. The experimental results show that although the mAP50 value of the loss function as EIOU does not change much, the average precision of the prediction confidence from 0.5 to 0.95 is improved by 0.5%, and the FPS is improved by 13.3, so it can be concluded that the proposed Focal-EIOU Loss has better effect on the algorithm.

[0095] In order to better verify the effectiveness of the four improvement strategies in this experiment, an ablation experiment is carried out on the Rope3D dataset to analyze the effectiveness of the improved algorithm. The down-sampling feature fusion layer, GFPN module, CA module and loss function are reconstructed in turn on the basis of YOLOv5s. In order to ensure the accuracy of the experiment, the same parameter configuration is used for training, the decay coefficient is 0.0005, the learning rate is 0.01, the batch is set to 8, and the momentum is set to 0.937. The experimental results are shown in Table 6.

[0096] Table 6

[0097]

[0098] As can be seen from Table 6, experiment 1 is the training result of the original YOLOv5s network on the Rope3D dataset, mAP50 is 53.6%, and FPS is 71.4. Experiment 2 reconstructs the feature fusion layer based on experiment 1, and mAP50 and FPS are improved by 3.5% and 14.8 compared with experiment 1. It can be seen that the feature fusion layer reconstruction increases the network feature space perception field, which can be more conducive to small target feature extraction and recognition, and effectively improves the network detection performance for small targets. Experiment 3 introduces GFPN structure in the Neck part of YOLOv5s network based on experiment 2, mAP50 and FPS reach 58.3% and 71.4, mAP50 is improved by 4.7% compared with experiment 1, and FPS is basically unchanged, indicating that the GFPN structure enhances the ability of aggregating multi-scale feature information, which can more effectively fuse feature information of different resolutions, improve the multi-scale feature fusion efficiency of the model, and further improve the target detection accuracy. Experiment 4 introduces the attention mechanism CA module in the Backbone part of YOLOv5s network based on experiment 3, which improves the perception field of the network, so that the model can more accurately locate and identify important information, and mAP50 is improved by 5.2% compared with experiment 1. However, due to the superposition of the module, the model complexity increases, and the FPS decreases by 5.6. Considering the convergence speed and inaccurate regression results in the detection of roadside small targets, experiment 5 introduces Focal-EIOU Loss in the detection head part of YOLOv5s network based on experiment 4, which accelerates the learning of high-confidence targets, improves the bounding box regression rate and anchor frame positioning accuracy, and mAP50 and FPS are improved by 0.3% and 6.1% compared with experiment 9, respectively, and 5.5% compared with experiment 1. And the FPS remains at 71.9, indicating that the model still has high detection speed, which meets the real-time detection of roadside devices.

[0099] The present application mainly aims at the problem that the detection target is small, the background is complex, and the feature extraction is limited in the roadside target detection, which leads to the problem that the detection speed and accuracy of road condition information cannot be improved simultaneously. The algorithm reconstructs the feature fusion module based on YOLOv5s, which can more effectively capture small target information and improve the small target feature extraction capability. Secondly, GFPN is introduced for multi-scale feature fusion, which improves the fusion efficiency of different resolution feature information. In addition, the CA module is introduced to improve the robustness of the model. Finally, Focal-EIOU Loss is replaced as the loss function of the algorithm model, which improves the bounding box regression speed and positioning speed. Based on the Rope3D dataset, a new dataset is selected for experiment. The experimental results show that the improved algorithm has stronger adaptability in complex environment, the accuracy is improved by 5.5% based on the original algorithm, and the detection speed is basically unchanged. In addition, the weight of the model is reduced by 60.4%, which is more conducive to the deployment of edge devices. However, compared with the ideal detection requirements, our network is prone to some fuzzy targets. This problem will be our main research direction in the future.

[0100] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0101] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between each embodiment can be referred to each other.

[0102] The principles and implementation modes of the present application are described by applying specific examples in this paper. The above description of the embodiments is only used to help understand the method and its core idea of the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In conclusion, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A roadside target detection method, characterized in that, The target detection method under the roadside view angle comprises the following steps: obtaining a data set; constructing a roadside image detection improved model; the roadside image detection improved model is obtained by removing the top feature extraction layer of a YOLOv5s algorithm model, integrating a CA attention mechanism at the end of a backbone network, then using a generalized feature pyramid network structure to fuse the features of the previous layer and the current layer in a cross-scale connection manner and establishing a skip layer connection, finally inputting the output of the generalized feature pyramid network structure into a detection layer to generate a prediction box, and then using a Focal-EIOU loss function to obtain a prediction result; training and evaluating the roadside image detection improved model based on the data set; obtaining a roadside image; inputting the roadside image into the trained roadside image detection improved model for recognition to realize target detection under the roadside view angle. 2.The roadside target detection method of claim 1, wherein, The Focal-EIOU loss function comprises an overlap loss, a center distance loss and a width-height loss. 3.The roadside target detection method of claim 1, wherein, Before the training and evaluation of the roadside image detection improved model based on the data set, the following steps are further included: dividing the data set into different sub-data sets based on lighting conditions, weather conditions, road scenes and camera specifications, and dividing them into a training set and a validation set.

4. A roadside target detection system under a visual angle, characterized in that, comprises: a data acquisition module, a model construction module and a loss function module; the data acquisition module is used to obtain a roadside image data set and a training data set; the model construction module comprises an information extraction module and an information fusion module, and the information fusion module is used to fuse the feature information; the information extraction module is used to extract feature information in the roadside image data set and the training data set.

5. The roadside target detection system of claim 4, wherein, The information extraction module comprises a feature extraction module and an attention module; wherein the attention module is used for position information embedding and coordinate information feature map generation to enhance important channels and spatial feature information in the features; the feature extraction module is obtained by removing the top feature extraction layer of the backbone network feature extraction module of the YOLOv5s algorithm model, and is used to extract feature information.

6. The roadside target detection system of claim 4, wherein, The information fusion module uses a generalized feature pyramid network structure and a cross-scale feature fusion method to fuse the feature information, wherein the cross-scale feature fusion method comprises two ways: each scale node simultaneously fuses the features of the three nodes above it, diagonally above and diagonally below, and fuses the features of the previous layer and the current layer and establishes a skip layer connection.

7. The roadside under-target detection system of claim 4, wherein, The loss function module comprises EIOULoss and FocalLoss; the EIOULoss is used to optimize the convergence speed and regression accuracy, and the FocalLoss is used to optimize the sample imbalance problem in the bounding box regression task.

8. The roadside under-target detection system of claim 4, wherein, The information fusion module adopts a feature fusion method starting from the bottom feature extraction layer for information fusion.

Images