Pole-shaped obstacle detection method for driver assistance

Abstract

This invention relates to the field of assisted driving technology, specifically to a method for detecting pole-shaped obstacles in assisted driving, comprising the following steps: creating a self-made pole-shaped obstacle dataset; labeling the acquired images using data annotation software, and processing the labeled files to divide them into training, validation, and test sets according to proportions; using an improved loss function based on the YOLOv5s network to backfeed network parameters; introducing the MK function into the original loss function; adding a dual-channel hybrid attention mechanism (CBAM) after the residual module; introducing an improved multi-scale feature fusion network structure into the neck network; training the model; and applying the trained model to pole-shaped obstacle detection. This method not only improves the recall and precision for detecting obstacles in the complex and ever-changing traffic environment of urban roads, but also solves the problem of variations in the size and shape of pole-shaped obstacles at different scales.

Description

Technical Field

[0001] This invention relates to the field of driver assistance technology, specifically to a method for detecting pole-shaped obstacles for driver assistance. Background Technology

[0002] With the rapid development of computer vision technology and deep learning algorithms, obstacle detection has become a crucial technology in assisted driving and even autonomous driving systems. Currently, deep learning-based obstacle detection methods have become mainstream, with commonly used methods including R-CNN, Fast R-CNN, Faster R-CNN, SSD, and YOLO. Among these, the YOLO series of algorithms performs better in terms of speed and accuracy, making it one of the most widely used obstacle detection algorithms. Existing pole obstacle detection systems face challenges such as a lack of basic datasets and uneven dataset quality, which limit algorithm performance and lead to low detection accuracy. Furthermore, the complex and variable traffic environment on urban roads, such as at night, in rain, or in snow, reduces visibility, further decreasing the accuracy of pole obstacle detection. Moreover, existing YOLOv5 series algorithms are prone to false positives and false negatives in complex and variable environments. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method for detecting pole-shaped obstacles for assisted driving.

[0004] To achieve the above objectives, the present invention provides a method for detecting pole-shaped obstacles for assisted driving, comprising the following steps:

[0005] S01, Self-made rod-shaped obstacle dataset;

[0006] S02: Label the acquired images using data annotation software, process the labeled files, and divide them into training, validation, and test sets according to the proportions;

[0007] S03: Based on the YOLOv5s network, an improved loss function is used to backfeed back the network parameters;

[0008] S04: Introduce the Monge-Kantorovich function into the original loss function;

[0009] S05: Add a dual-channel hybrid attention mechanism CBAM after the residual block in the YOLOv5s neck network;

[0010] S06: Embedding an improved multi-scale feature fusion architecture MFFNA in the neck network;

[0011] S07: Train the MFMAM-YOLOv5s algorithm and apply the trained model to rod obstacle detection.

[0012] Preferably, in S01, the dataset consists of images taken under complex weather conditions, such as sunny, cloudy, rainy, and snowy weather, and at different times of day, including morning, noon, afternoon, and evening.

[0013] Preferably, step S04 specifically comprises:

[0014] The mathematical definition of the Munger problem introduced into the mathematical MK function is: Given two metric spaces X and Y and two probability measures μ∈P(X), ν∈P(Y), the cost loss function is: c∶X×Y→R∪{+∞}:

[0015] (MP):=inf{∫ X c(x,T(x))dμ(x)|T:X→Y and T #μ＝v}

[0016] Where c(x,T(x)) represents the loss from x to T(x), X→Y and T #μ＝v These are respectively measurable mapping and transport mapping;

[0017] The Kantorovich Problem is an extension of the Monge Problem. Kantorovich generalized the transport mapping to a transport scheme. Given two probability measures μ∈P(X), ν∈P(Y) and a cost loss function c∶X×Y→{0,+∞}, the Kantorovich Problem can be defined as follows:

[0018] (KP):=inf{∫ X×Y c(x,y)dγ(x,y)|γ∈∏(μ,ν)}

[0019] However, the joint probability measure γ belongs to the transmission scheme space:

[0020] ∏(μ,ν)={γ∈P(X×Y)|π X#γ =μ,π Y#γ =ν}

[0021] Where, π X#γ :=∫πd y ,π X#γ :=∫πd x It represents a marginal probability measure.

[0022] Regarding IoU loss functions, such as GIoU, DIoU, and CIoU, they do not consider the orientation between the ground truth bounding box and the predicted bounding box, resulting in slow convergence. To address this, the SIoU loss function was introduced, redefining related loss functions, specifically including angular loss, distance loss, shape loss, and optimized IoU loss. However, IoU loss is the most commonly used loss function in object detection, representing the intersection-union ratio (IoU) between the ground truth and predicted bounding boxes. The IoU loss optimized using the MK function shows better convergence than the original IoU loss.

[0023] Preferably, in S05, the CBAM module first executes the channel attention mechanism, and then executes the spatial attention mechanism.

[0024] Preferably, in S05, global pooling is first used to compress the dimension from H×W×C to 1×1×C, that is, to compress H×W into a 1×1 dimension Z∈R. H×W×C :

[0025]

[0026] Among them, v c ∈R H×W×C The dimension of the element feature map representing the input is H×W×C, F sq (·) indicates a squeeze mapping;

[0027] Then, the activation module incorporates the 1×1×C dimension obtained from the compressed part into the fully connected layer, predicts the importance of each channel, and then activates the corresponding channels of the previous feature map for operation; its expression is:

[0028] s = F ex (Z,W)=σ(W1δ(W0Z))

[0029] Among them, F ex (·,W) represents the activation mapping. and δ is the ReLU function, and δ and σ are the ReLU function and sigmoid function, respectively;

[0030] Finally, by rescaling x and activation s, the output β = [U1, U2, ..., U] of SEN is obtained in the feature space. c ]∈R H×W×C :

[0031] U c =F scale (x c ,s c )=x c ×s c

[0032] Among them, F scaleThis represents the channel attention weighted mapping function.

[0033] In S05, the spatial attention mechanism uses a set of convolutional kernels to perform convolution operations on the feature map to learn the weight coefficients at each location. Assuming a feature map of H×W×C, max pooling and average pooling are performed on each channel to obtain a 2×H×W×1 channel description. These two channel descriptions are then concatenated. Next, a 7×7 convolutional layer and the sigmoid activation function are applied to obtain the weight coefficients. Finally, these weight coefficients are used to perform a weighted summation of the features at each location, generating a weighted spatial feature map. The expression is as follows:

[0034]

[0035] in, and F represents the feature maps after global average pooling and global max pooling. 7×7 This represents a convolution operation with a kernel size of 7×7, represented by the sigmoid function.

[0036] The beneficial effects of this invention are: while ensuring detection accuracy, it greatly improves inference speed; the inference time of this model is 1.4ms, and the mAP value is 96.6%. This method not only improves the recall and precision of detection in the complex and ever-changing traffic environment of urban roads, but also solves the problem of the size and shape changes of rod-shaped obstacles at different scales. Attached Figure Description

[0037] Figure 1 This is a flowchart of the present invention;

[0038] Figure 2a The actual bounding box A in this invention gttb And predicted bounding box A pbb ;

[0039] Figure 2b This represents the intersection of the true bounding box and the predicted bounding box in this invention.

[0040] Figure 2c This is the union of the true bounding box and the predicted bounding box in this invention;

[0041] Figure 3 This is a structural diagram of the cascaded and fused hybrid domain attention mechanism in this invention;

[0042] Figure 4 This is an architecture diagram of the MFMAM-YOLOv5s network in this invention;

[0043] Figure 5 This is a partial result of the road test using this method. Detailed Implementation

[0044] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0045] The following is combined Figure 1 The algorithm process and its principles are described.

[0046] A method for detecting pole-shaped obstacles for assisted driving includes the following steps:

[0047] S01. Self-made rod-shaped obstacle dataset. The main idea is to take images under complex weather conditions, such as sunny, cloudy, rainy and snowy days, and at different times of the day, including morning, noon, afternoon and evening.

[0048] S02. Label the collected images using data annotation software, process the labeled files, and divide them into training set, validation set, and test set according to the proportion.

[0049] S03. Based on the YOLOv5s network, an improved loss function is used for backfeedback of network parameters to reduce the overall loss of object detection. Since the loss function is used to evaluate the difference between the model's prediction and the true target, the closer the prediction is to the true target, the smaller the loss function value. However, the YOLOv5s loss function mainly includes three aspects: box loss (box_loss), confidence loss (obj_loss), and classification loss (cls_loss). The total loss function of the YOLOv5s network is:

[0050] Total_loss=obj_scale*obj_loss+cls_scale*cls_loss+box_scale*box_loss

[0051] Here, obj_scale, cls_scale, and box_scale correspond to the weight coefficients of the three losses, and obj_scale = 1, cls_scale = 0.5, and box_scale = 0.1.

[0052] S04. We introduce the Monge-Kantorovich (MK) function into the original loss function. By introducing the MK function, the model can pay more attention to the spatial relationships between objects. This solves the problem of overlapping predicted boxes in YOLOv5 and improves detection accuracy. At the same time, the MK loss function can regularize the model and prevent overfitting. We replace the original CIoU function with the SIoU loss function.

[0053] S05. This invention adds a dual-channel hybrid attention mechanism (CBAM) after the residual block in the YOLOv5s neck network. Specifically, the CBAM module first performs a channel attention mechanism, and then a spatial attention mechanism, to enhance useful spatial and channel information in the feature map. This better handles targets with small objects and other complex situations in the image, helps the model focus more on important features when processing images, and suppresses irrelevant background information, thereby improving detection accuracy and recall.

[0054] S06. This invention employs a feature fusion target detection algorithm based on a multi-scale feature pyramid in the original pooling layer structures SPP and PAN. By introducing bottom-up and top-down cross-layer connections into the efficient feature pyramid network, efficient feature fusion and optimization are achieved. It can effectively solve problems such as target scale variations, target overlap, and irregular target shapes, and has better robustness and detection performance compared to SPP and PAN structures.

[0055] For further optimization, step S04 specifically includes:

[0056] The mathematical definition of the Munger problem introduced into the mathematical MK function is: Given two metric spaces X and Y and two probability measures μ∈P(X), ν∈P(Y), the cost loss function is: c∶X×Y→R∪{+∞}:

[0057] (MP):=inf{∫ x c(x,T(x))dμ(x)|T:X→Y and T #μ＝ν}

[0058] Where c(x,T(x)) represents the loss from x to T(x), X→Y and T #μ＝ν These are measurable mapping and transport mapping, respectively.

[0059] The Kantorovich Problem is an extension of the Monge Problem; Kantorovich generalized transport mappings to transport schemes. Given two probability measures μ∈P(X), ν∈P(Y) and a cost loss function c∶X×Y→{0,+∞}, the Kantorovich Problem can be defined as follows:

[0060] (KP):=inf{∫ X×Y c(x,y)dγ(x,y)|γ∈∏(μ,ν)}

[0061] However, the joint probability measure γ belongs to the transmission scheme space:

[0062] ∏(μ,ν)={γ∈P(X×Y)|πX#γ =μ,π Y#γ =ν}

[0063] Where, π X#γ :=∫πd y ,π X#γ :=∫πd x It represents a marginal probability measure.

[0064] Regarding IoU loss functions, such as GIoU, DIoU, and CIoU, they do not consider the orientation between the ground truth bounding box and the predicted bounding box, resulting in slow convergence. To address this, the SIoU loss function was introduced, redefining related loss functions, specifically including angular loss, distance loss, shape loss, and optimized IoU loss. However, IoU loss is the most commonly used loss function in object detection, representing the intersection-union ratio (IoU) between the ground truth and predicted bounding boxes. The IoU loss optimized using the MK function shows better convergence than the original IoU loss.

[0065] like Figures 2a to 2c The diagram illustrates the relationship between the ground truth bounding boxes, the predicted bounding boxes, and the intersection-union ratio (IU).

[0066] Figure 2a For the true bounding box A gttb And predicted bounding box A pbb , Figure 2b The intersection of the ground truth bounding box and the predicted bounding box. Figure 2c It is the union of the true bounding box and the predicted bounding box.

[0067]

[0068] IoU cost is used to calculate the overlap between the detected bounding boxes. IoU loss ensures that the overlap between the predicted bounding box and the ground truth is maximized, thus improving detection accuracy. Its expression is as follows:

[0069] L IoU Cost =1-IoU

[0070] The final SIoU loss function expression is as follows:

[0071]

[0072] For further optimization, step S05 specifically includes:

[0073] In object detection, the smaller the resolution of the base feature image, the more channels it contains. Therefore, as the model becomes deeper, the low-resolution base feature map contains a large amount of channel information. Consequently, it becomes difficult for the object detector to filter out the important channel information from this vast amount of information. However, channel attention mechanisms effectively solve this problem, enabling the model to better focus on useful channel feature information, thereby improving model performance. Squeeze-and-Excitation Networks are a typical example of channel attention mechanisms. They first use global pooling to compress the dimension from H×W×C to 1×1×C, that is, compressing H×W into a 1×1 dimension Z∈R. H×W×C :

[0074]

[0075] Among them, v c ∈R H×W×C The dimension of the element feature map representing the input is H×W×C, F sq (·) indicates a squeeze mapping.

[0076] Then, the activation module incorporates the 1×1×C dimension obtained from the compressed part into the fully connected layer, predicts the importance of each channel, and then activates the corresponding channels of the previous feature map for operation. Its expression is:

[0077] s = F ex (Z,W)=σ(W1δ(W0Z))

[0078] Among them, F ex (·,W) represents the activation mapping. and δ is the ReLU function, and δ and σ are the ReLU function and sigmoid function, respectively.

[0079] Finally, by rescaling x and activation s, the output β = [U1, U2, ..., U] of SEN is obtained in the feature space. c ]∈R H×W×C :

[0080] U c =F scale (x c ,s c )=x c ×s c

[0081] Among them, F scale This represents the channel attention weighted mapping function.

[0082] The spatial attention mechanism uses a set of convolutional kernels to convolve the feature map to learn the weight coefficients at each location. Assuming a feature map of size H×W×C, max pooling and average pooling are performed along one channel dimension to obtain a 2×H×W×1 channel descriptor. These two descriptor channels are then concatenated. Next, a 7×7 convolutional layer and the sigmoid activation function are applied to obtain the weight coefficients. Finally, these weight coefficients are used to perform a weighted summation of the features at each location, generating a weighted spatial feature map. The expression is as follows:

[0083]

[0084] in, and F represents the feature maps after global average pooling and global max pooling. 7×7 This represents a convolution operation with a kernel size of 7×7, represented by the sigmoid function.

[0085] Currently, in deep learning, there are two main methods for connecting these two attention mechanisms: cascaded methods and parallel methods. Parallel attention mechanisms have a relatively simple structure but may not fully utilize the correlations between different feature subspaces, resulting in insufficient feature representation capabilities. Therefore, more layers need to be stacked. Cascaded attention mechanisms, on the other hand, can utilize the outputs of multiple attention modules for cascading, thereby better capturing the spatial and channel correlations of features and enhancing feature representation capabilities. Furthermore, cascaded structures can also perform attention fusion on feature maps of different scales, better adapting to objects of different scales. The cascaded structure is shown below. Figure 3 As shown.

[0086] For further optimization, step S06 specifically includes:

[0087] This invention proposes a novel multi-scale fusion network architecture, MFFNA, which improves model accuracy and efficiency while reducing computational cost, memory consumption, and power consumption. First, convolutional fusion is performed using an improved CSPStage, followed by structural reparameterization, specifically fusion of convective 2D and Batch Normalization (BN). Therefore, the expression for BN in the ith channel of the feature map is as follows.

[0088]

[0089] Where μ represents the mean, σ 2 Let represent variance, γr represent weight, β represent bias, and ∈ is a very small constant whose purpose is to prevent the denominator of the formula from being zero.

[0090] In summary, this invention successfully applied the MFMAM-YOLOv5s algorithm to complex road surfaces, detecting rod-shaped obstacles. The architecture of the MFMAM-YOLOv5s network is as follows: Figure 4 As shown.

[0091] Here, the original C3 convolutional layer is replaced with a C3_Faster convolutional layer based on a lightweight Faster neural network. A convolutional layer is added at layer 10 to reduce the dimensionality of the previous convolutional layer, thus reducing the number of parameters and computational cost. Then, an improved multi-scale feature fusion module (MFFNA) is added. Finally, a dual-channel hybrid attention mechanism (CBAM) is introduced into the convolutional layers of the neural network.

[0092] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting pole-shaped obstacles for assisted driving, characterized in that: Includes the following steps: S01, Self-made rod-shaped obstacle dataset; S02: Label the acquired images using data annotation software, process the labeled files, and divide them into training, validation, and test sets according to the proportions; S03: Based on the YOLOv5s network, an improved loss function is used to backfeed back the network parameters; S04: Introduce the Monge-Kantorovich function into the original loss function; S05: Add a dual-channel hybrid attention mechanism CBAM after the residual block in the YOLOv5s neck network; S06: Embedding an improved multi-scale fusion network architecture, MFFNA, in the neck network; S07: Train the MFMAM-YOLOv5s algorithm and apply the trained model to rod obstacle detection; Step S04 specifically involves: The mathematical definition of the Monge problem, introduced into the mathematical Monge-Kantorovich function, is as follows: Given two metric spaces X and Y and two probability measures... Cost loss function: : ； Where c(x,T(x)) represents the loss from x to T(x), These are respectively measurable mapping and transport mapping; The Kantorovich problem is an extension of the Monge problem; Kantorovich generalized transport mappings to transport schemes. Given two probability measures... and a cost loss function Therefore, the Kantorovich problem can be defined as: ； However, joint probability measure Belongs to the transmission scheme space: ； in, , Represents a marginal probability measure; The Monge-Kantorovich function is used as a regularization term to constrain the model, and the original CIoU function is replaced with the SIoU loss function. IoU cost is used to calculate the overlapping portion of the detection boxes, and its expression is as follows: ； The SIoU loss function expression is as follows: 。 2. The method for detecting pole-shaped obstacles for assisted driving according to claim 1, characterized in that: In S01, the dataset consists of images taken under complex weather conditions at different times of day, including morning, noon, afternoon, and evening.

3. The method for detecting pole-shaped obstacles for assisted driving according to claim 1, characterized in that: In S05, the CBAM module first executes the channel attention mechanism, and then the spatial attention mechanism to suppress interference factors in complex traffic environments and focus on the feature information of pole-shaped obstacles.

4. The method for detecting pole-shaped obstacles for assisted driving according to claim 1, characterized in that: In S05, global pooling is first used to compress the dimension from H×W×C to 1×1×C, that is, to compress H×W into 1×1 dimension. : ； in, The dimension of the feature map representing the input elements is H×W×C. Indicates extrusion mapping; Then, the activation module incorporates the 1×1×C dimension obtained from the compressed part into the fully connected layer, predicts the importance of each channel, and then activates the corresponding channels of the previous feature map for operation; its expression is: ； in, Represents the incentive mapping, . Here, δ and σ represent the ReLU function and the sigmoid function, respectively. Finally, by rescaling x and the activation value s, the output of SEN is obtained in the feature space. : ； in, This represents the channel attention weighted mapping function.

5. The method for detecting pole-shaped obstacles for assisted driving according to claim 3, characterized in that: In S05, the spatial attention mechanism uses a set of convolutional kernels to perform convolution operations on the feature map to learn the weight coefficients at each position. Assuming a feature map of H×W×C, max pooling and average pooling are performed on each channel to obtain a 2×H×W×1 channel descriptor. These two descriptor channels are then concatenated. Afterward, a 7×7 convolutional layer and the sigmoid activation function are applied to obtain the weight coefficients. Finally, these weight coefficients are used to perform a weighted summation of the features at each position, generating a weighted spatial feature map, the expression of which is as follows: ； in, and This represents the feature maps after global average pooling and global max pooling. This represents a convolution operation with a kernel size of 7 × 7. This is the sigmoid function.

Images