Deep learning-based intersection traffic flow detection method and system in intelligent network connection vehicle

Abstract

The invention relates to the technical field of deep learning and intelligent network connection, and discloses an intersection traffic flow detection method and system based on deep learning in an intelligent network connection vehicle. Preprocessing the frame image; the preprocessed image passes through a Backbone module integrated with an RFCBAMConv attention mechanism, and rich features in the image are extracted; carrying out classification and regression tasks on the extracted features through a DFMhead detection head, further determining a detected target category, estimating a bounding box position of the detected target category, and outputting a final detection result; sIWL is introduced into the loss calculation module to serve as a loss function, so that the detection capability of a small target is enhanced; by improving a traffic flow detection algorithm based on deep learning and applying the algorithm to the intelligent networked vehicle, the detection accuracy and stability of small targets and shielded targets are improved, so that the intelligent networked vehicle can re-plan a path in advance to avoid a congested road section when encountering long-distance intersection congestion in a planned path.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and intelligent network technology, and more specifically, to a method and system for detecting traffic flow at intersections based on deep learning in intelligent networked vehicles. Background Art

[0002] At present, the research of intelligent networked vehicles mainly focuses on self-sensing and self-control, and they cannot perceive the traffic flow at long-distance intersections. Intelligent networked vehicles face the problem of congestion at long-distance intersections where the planned route has been arranged, and they cannot re-plan the route in advance to avoid congested intersections.

[0003] In recent years, deep learning-based traffic flow detection algorithms have been widely used at intersections. At the same time, deep learning-based traffic flow detection algorithms also face some problems. First, the current traffic intersections are very wide, and the cameras at the traffic intersections are far away from the vehicles at the intersections, so there are many small target vehicles in the photos taken by the cameras. Secondly, the vehicles at the intersections are close to each other, and there will be mutual occlusion between the vehicles, so there are many occluding vehicles in the photos taken by the cameras.

[0004] Therefore, the present invention provides a method and system for detecting traffic flow at intersections based on deep learning in intelligent connected vehicles, which improves the above-mentioned technical problems. Summary of the invention

[0005] The disclosed embodiments aim to address the deficiencies of the prior art and provide a method and system for detecting traffic flow at intersections based on deep learning in intelligent connected vehicles. The present invention improves the traffic flow detection algorithm based on deep learning and applies it to intelligent connected vehicles to improve the detection accuracy and stability of small targets and occluded targets, so that when the intelligent connected vehicle encounters congestion at a long-distance intersection on a planned route, it can re-plan the route in advance to avoid the congested section.

[0006] The above technical objectives of the present invention are achieved through the following technical solutions: A method and system for detecting traffic flow at an intersection based on deep learning in an intelligent connected vehicle, comprising the following steps: S1, obtaining a real-time frame of a camera at an intersection;

[0007] S2, preprocess each captured real-time frame image to match the model's input specification; the preprocessed image is then passed through the Backbone module that integrates the RFCBAMConv attention mechanism to extract rich features in the image;

[0008] S3. After feature extraction, the Neck module is used to fuse feature maps at different levels to enhance the model’s ability to detect objects of various scales. The DFMhead detection head is then used to perform classification and regression tasks on the extracted features to further determine the detected target category and estimate its bounding box position, and output the final detection result.

[0009] S4. In order to further optimize the detection results, SIWL is introduced as the loss function in the loss calculation module; the SIWL can calculate the positive and negative sample losses more accurately, which is used to enhance the detection capability of small targets.

[0010] As a preferred technical solution of the present invention, frame sequence data of traffic flow is obtained in real time through cameras at intersections; the data will be used as a basic input for target detection to monitor vehicles, pedestrians and other traffic elements at intersections.

[0011] As a preferred technical solution of the present invention, the Backbone module integrating the RFCBAMConv attention mechanism includes: RFAConv and CBAM;

[0012] The RFAConv introduces a receptive field feature attention mechanism; the RFAConv workflow includes: a receptive field feature attention map generation stage, a receptive field feature map generation stage, a receptive field feature map and attention weight combination stage, and a feature extraction stage;

[0013] The calculation formula of RFAConv can be expressed as:

[0014] F = Softmax(g 1x1 (AvgPool(X)))x

[0015] ReLU(Norm(g kxk (X)))=

[0016] A rf xF rf

[0017] Among them, g i×i represents a grouped convolution of size i×i, k represents the size of the convolution kernel, Norm represents normalization, X represents the input feature map, and F is obtained by converting the attention map A rf And the receptive field spatial feature map F rf The feature map obtained by multiplication;

[0018] The CBAM dynamically adjusts the network's attention to the channel and spatial position in the feature map through the channel and spatial attention mechanism. The CBAM workflow includes: a channel attention map generation stage, a spatial attention map generation stage, a channel and spatial attention combination stage, and an adaptive adjustment stage;

[0019] Given an intermediate feature F∈R C×H×W As input, CBAM sequentially derives a 1D channel attention map M c ∈R Cx1x1 and 2D spatial attention map Ms ∈R 1×H×W ;

[0020] The calculation formula of CBAM can be expressed as:

[0021] M c (F) = Sigmoid(Conv2d

[0022] (AvgPool(F))+

[0023] Conv2d(MaxPool(F)))

[0024] M s (F) = Sigmoid(Conv2dx

[0025] ([max(F):mean(F)]))

[0026]

[0027] in, It represents element-by-element multiplication, F1 represents the result of input feature map F after channel attention, and F2 represents the result of F1 after spatial attention map.

[0028] As a preferred technical solution of the present invention, the calculation formula of the Backbone module integrating the RFCBAMConv attention mechanism is:

[0029]

[0030] Sigmoid(GConv2d(F))

[0031]

[0032] Among them, F3 is the result of input feature map F after receptive field attention and path attention, and F4 is the result of F3 after spatial attention.

[0033] As a preferred technical solution of the present invention, the DFMhead detection head is composed of the following modules: a scale perception module, a space perception module and a task perception attention module;

[0034] The scale-aware module is deployed on feature maps of different levels to learn the relative importance of different semantic information on each feature map to appropriately enhance its features. Given a feature tensor F∈R L×C×H×W , L represents the number of feature maps, C represents the number of feature map channels, H and W represent the height and width of the feature map; the calculation formula of the scale perception module is:

[0035]

[0036] where f(·) is a linear function approximated by a 1×1 convolutional layer,

[0037] The spatial perception module focuses on the discriminative area that is consistent between spatial position and feature level based on the spatial attention module of fusion features; the spatial perception module is divided into two steps: first, the features of objects of different scales in space are learned by using deformable convolution, and then the features across levels are aggregated at the same spatial position; the calculation formula of the spatial perception module is:

[0038]

[0039] Where K is the number of deformable convolution sampling locations, p k +Δp k is the sampling position p k Shifted by Δp k , to focus on the discrimination area, Δm k is the position p k The importance scalar of self-learning at ;

[0040] The task-aware attention module is deployed on the channel, and guides different feature channels to support different tasks according to the different convolution kernel responses of the object; the calculation formula of the task-aware attention module is:

[0041] π C (F)·F=

[0042] max(α 1 (F)·F C +β 1 (F),

[0043] α 2 (F)·F C +β 2 (F)

[0044] Among them, F C is the characteristic slice of the Cth channel, θ(·)=[α 1 ,α 2 ,β 1 ,β 2 ] T is a hyperparameter that is learned to control the activation threshold; the θ(·) function first performs global average pooling on L×H×W dimensions to reduce the dimensionality, then uses two fully connected layers and a normalization layer, and finally applies an activation function to normalize the output to [-1, 1].

[0045] As a preferred technical solution of the present invention, the workflow of the DFMhead detection head is as follows: first, after passing through the scale-aware attention module, the feature map becomes more sensitive to the scale difference of the foreground object; after further passing through the spatial-aware attention module, the feature map becomes sparser and focuses on the discriminative spatial position of the foreground object; finally, after passing through the task-aware attention module, the feature map re-forms different activations according to the requirements of different downstream tasks;

[0046] The calculation formula of DFMhead detection head is:

[0047] W(F)=π C (π S (π L (F)·F)·F)·F

[0048] Among them, π L (·),π S (·),π c (·) are scale attention function, spatial attention function, and task attention function respectively.

[0049] As a preferred technical solution of the present invention, SIWL is introduced as a loss function, and the process of accurately calculating the loss of positive and negative samples is as follows: first, Gaussian modeling of the bounding box is performed; then, the Gaussian Wasserstein distance is normalized, and the Wasserstein distance is used to measure the similarity of the derived Gaussian distribution; finally, the loss of positive and negative samples is calculated by the SIWL loss function;

[0050] The Gaussian modeling process of the bounding box is as follows: for a horizontal bounding box R=(cx, cy, w, h), where (cx, cy), w and h represent the center coordinates, width and height respectively; the equation of the inscribed ellipse can be expressed as:

[0051]

[0052] Where (μ x ,μ y ) is the center coordinate of the ellipse, σ x ,σ y is the semi-axis length along the x and y axes; therefore, μ x =cx,μ y =cy,σ x =w / 2,σ y =h / 2; the probability density function of the two-dimensional Gaussian distribution is given by:

[0053]

[0054] Where x, μ and Σ represent the coordinates (x, y), mean vector and covariance matrix of the Gaussian distribution; the following constraints are met:

[0055] (X-μ) T ∑ -1 (X-μ)=1

[0056]

[0057] The process of normalizing the Gaussian Wasserstein distance is as follows: for two two-dimensional Gaussian distributions μ1=N(m1,Σ1) and μ2=N(m2,Σ2), the second-order Wasserstein distance between μ1 and μ2 is defined as:

[0058]

[0059] where ||·|| F is the Frobenius norm;

[0060] For the Gaussian distributions Na and Nb modeled from bounding boxes A = (cxa, cya, wa, ha) and B = (cxb, cyb, wb, hb), it can be further simplified to:

[0061]

[0062] We further normalize it using its exponential form and obtain a new metric called Normalized Wasserstein Distance (NWD):

[0063]

[0064] Where C is a constant that is closely related to the data set;

[0065] The calculation formula of the SIWL loss function is:

[0066] l SIWL =1-SIWL(Ν p ,N g )

[0067] Where Np is the Gaussian distribution model of the prediction box P, and Ng is the Gaussian distribution model of the gt box G.

[0068] As a preferred technical solution of the present invention, a deep learning-based intersection traffic flow detection system in an intelligent networked vehicle comprises: an edge detection client, a cloud server and an intelligent networked vehicle client;

[0069] The edge detection client is on the embedded platform development board, and an improved deep learning-based vehicle flow detection algorithm is deployed to monitor the vehicle flow at the intersection in real time. At the same time, an MQTT client communication program is deployed on the development board. The MQTT client establishes a connection with the cloud server and publishes vehicle flow data to a predefined MQTT topic. By utilizing the high efficiency and lightweight characteristics of MQTT, the vehicle flow is sent to the MQTT server of the cloud server in real time.

[0070] The cloud server builds an MQTT server program to manage the publish and subscribe relationship between the edge detection client and the intelligent connected vehicle client, and receives edge detection client information and sends it to the intelligent connected vehicle client;

[0071] The intelligent connected vehicle client deploys the ROS operating system on the embedded development board and configures the MQTT client communication program; it is used to receive the long-distance intersection traffic flow information sent by the cloud server and use this data to re-plan the vehicle's route.

[0072] In summary, the present invention has the following beneficial effects:

[0073] First, intelligent connected vehicles can obtain real-time traffic flow at long-distance intersections, re-plan new routes, and avoid congested intersections.

[0074] Improve the deep learning-based traffic flow detection algorithm and apply it to intelligent connected vehicles to improve the detection accuracy and stability of small targets and occluded targets, so that when the intelligent connected vehicles encounter congestion at long-distance intersections on the planned path, they can re-plan the path in advance to avoid congested roads.

[0075] Second, the Backbone module based on the deep learning target detection algorithm extracts richer features.

[0076] The c2f feature extraction network module used in the original backbone of YOLOv8 has a small receptive field and loses semantic information and spatial information. The present invention adopts attention convolution (RFCBAMConv) to expand the receptive field of the network and enhance the semantic and spatial information of the network for small targets and occluded targets.

[0077] Third, the head module based on deep learning target detection algorithm can better complete regression and detection tasks.

[0078] The head of YOLOv8 uses simple convolutional layers and fully connected layers to perform regression and classification tasks, but does not effectively utilize the close connection between the characteristics of the fusion of large, medium and small features provided by the Neck module. This makes it difficult for the model to effectively notice small targets and occluded targets. In contrast, dyhead introduces a scale-aware attention module, which makes the feature map more sensitive to the scale changes of foreground objects. Then, the spatial-aware attention module further processes the feature map to make it more focused on the discrimination of the spatial position of the foreground object. Finally, the task-aware attention module adjusts the activation state of the feature map according to the specific downstream task requirements. In this way, the present invention uses dyhead to effectively improve the performance of the network in detecting small targets and occluded targets, and realizes the unified optimization of scale, space and task attention mechanisms.

[0079] Fourth, the NWD loss function module based on the deep learning target detection algorithm can better detect small targets.

[0080] In YOLOv8, the shortcomings of the loss function mainly include the failure to effectively handle the training bias caused by the imbalance of positive and negative samples, and the failure to capture the relationship between objects and contextual information in complex scenes. These factors limit the performance of the model in small target detection and occluded target processing. To solve these problems, the present invention proposes a new loss function NWD, which effectively handles the loss calculation between positive and negative samples by modeling the bounding box as a two-dimensional Gaussian distribution and using the Wasserstein distance to measure the similarity between the derived Gaussian distributions. In particular, it is insensitive to objects of different scales, thereby improving the performance of the network in small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 A flow chart of a method for detecting traffic flow at an intersection based on deep learning in an intelligent connected vehicle provided by an embodiment of the present invention;

[0082] Figure 2 Framework diagram of the deep learning-based intersection traffic flow detection system in the intelligent connected vehicle provided by the embodiment of the present invention

[0083] Figure 3 A path planning diagram for an intelligent connected vehicle provided by an embodiment of the present invention;

[0084] Figure 4 A traffic flow detection diagram for a long-distance congested intersection provided by an embodiment of the present invention;

[0085] Figure 5 A route map for replanning an intelligent networked vehicle provided in an embodiment of the present invention;

[0086] Figure 6The structure diagram of the improved target detection attention convolution RFCBAMConv provided by the embodiment of the present invention;

[0087] Figure 7 A structural diagram of an improved target detection head dyhead provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0088] The present application is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements can also be made without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0089] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0090] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. In addition, the words "first", "second", "third", etc. used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0091] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in this specification and in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0092] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0093] The disclosed embodiments are intended to solve the problem that when an intelligent connected vehicle encounters congestion at a long-distance intersection in a planned route, it cannot re-plan the route in advance to avoid the congested section. In view of this, the disclosed embodiments propose a method and system for detecting traffic flow at an intersection based on deep learning in an intelligent connected vehicle, which uses deep learning technology to accurately detect the traffic flow at the intersection and applies it to the intelligent connected vehicle; so that the intelligent connected vehicle can obtain and analyze the traffic flow data of the long-distance intersection in real time, and can obtain the congestion situation in time, and re-plan the route accordingly to avoid the congested section. This application not only gives full play to the advantages of deep learning in image processing and pattern recognition, but also effectively solves the problem that the intelligent connected vehicle cannot flexibly adjust the route when it encounters congestion at the intersection. At the same time, the present invention continuously optimizes the deep learning algorithm, which can improve its recognition ability of small targets and occluded targets, thereby ensuring that the intelligent connected vehicle accurately obtains the traffic flow at complex intersections.

[0094] Please refer to Figure 1 , Figure 1 The flowchart of the method and system for detecting traffic flow at intersections based on deep learning in the intelligent connected vehicle according to the embodiment of the present disclosure is shown. The overall process mainly includes the following 4 steps:

[0095] S1: Get the real-time frame of the intersection camera.

[0096] The frame sequence data of traffic flow is obtained in real time through the camera at the intersection. These image data will be used as the basic input for target detection to monitor vehicles, pedestrians and other traffic elements at the intersection.

[0097] S2: Frame image preprocessing and feature extraction.

[0098] First, each captured frame is preprocessed, including image resizing and normalization operations, to match the model's input specifications. The preprocessed image is then passed through the Backbone module that integrates the RFCBAMConv attention mechanism to extract rich features in the image, thereby increasing the focus on important details and providing high-quality features for subsequent detection tasks.

[0099] Specifically, in the Backbone module of YOLOV8, attention convolution (RFCBAMConv) is used to replace traditional convolution, and the ordinary convolution in the feature fusion module (C2f) is replaced by RFCBAMConv. RFCBAMConv uses RFAConv to effectively learn information at different positions in the receptive field, improve the network's perception of global and local features, and uses CBAM to dynamically adjust the attention of channels and spatial positions, so that the network can capture important features more specifically and improve the adaptability of the model.

[0100] RFCBAMConv is a fusion of Receptive Field Attention Convolutional (RFAConv) and Convolutional Block Attention Module (CBAM).

[0101] The workflow of RFCBAMConv is divided into the following two steps: RFAConv stage and CBAM stage.

[0102] Embodiment 1, RFAConv improves the network's attention to specific areas of the receptive field by introducing the receptive field feature attention mechanism, thereby enhancing the feature extraction capability. The RFAConv workflow includes the receptive field feature attention map generation stage, the receptive field feature map generation stage, the receptive field feature map and attention weight combination stage, and the feature extraction stage.

[0103] Generally, the RFAConv calculation formula can be expressed as:

[0104] F = Softmax(g 1x1 (AvgPool(X)))x

[0105] ReLU(Norm(g kxk (X)))=

[0106] A rf xF rf

[0107] Among them, g i×i represents a grouped convolution of size i×i, k represents the size of the convolution kernel, Norm represents normalization, X represents the input feature map, and F is obtained by converting the attention map A rf And the receptive field spatial feature map F rf The feature map obtained by multiplication.

[0108] Embodiment 2: CBAM is a simple and effective feedforward convolutional neural network attention module. CBAM dynamically adjusts the network's attention to the channel and spatial position in the feature map through the channel and spatial attention mechanism, thereby improving the network's ability to express features at different levels. The CBAM workflow includes the channel attention map generation stage, the spatial attention map generation stage, the channel and spatial attention combination stage, and the adaptive adjustment stage.

[0109] Given an intermediate feature F∈R C×H×W As input, CBAM sequentially derives a 1D channel attention map M c ∈R Cx1x1 and 2D spatial attention map M s ∈R1×H×W Generally, the CBAM calculation formula can be expressed as:

[0110] M c (F) = Sigmoid(Conv2d

[0111] (AvgPool(F))+

[0112] Conv2d(MaxPool(F)))

[0113] M s (F) = Sigmoid(Conv2dx

[0114] ([max(F):mean(F)]))

[0115]

[0116] in, It represents element-by-element multiplication, F1 represents the result of input feature map F after channel attention, and F2 represents the result of F1 after spatial attention map.

[0117] In summary, RFCBAMConv uses RFAConv to effectively learn information at different positions in the receptive field, improve the network's ability to perceive global and local features, and uses CBAM to dynamically adjust the attention of channels and spatial positions, so that the network can capture important features more specifically and improve the adaptability of the model. Figure 2 The calculation formula of this module is:

[0118]

[0119] Sigmoid(GConv2d(F))

[0120]

[0121] Among them, F3 is the result of input feature map F after receptive field attention and path attention, and F4 is the result of F3 after spatial attention.

[0122] S3: Feature fusion and detection head.

[0123] After feature extraction, the Neck module is used to fuse feature maps at different levels to enhance the model's detection capabilities for targets of various scales and improve its accuracy and robustness. Subsequently, the DFMhead detection head module of the present invention performs classification and regression tasks on the extracted features, further determines the detected target category and estimates its bounding box position, and outputs the final detection result.

[0124] Specifically, in the Head module, the detection head (Yolo-head) in the YOLOv8 network is replaced with a unified attention mechanism detection head DFMhead). DFMhead is a dynamic target detection head that integrates attention. It consists of three modules: scale perception, space perception, and task perception.

[0125] Embodiment 3: Scale-aware module: The scale-aware module is deployed on feature maps of different levels. It learns the relative importance of different semantic information on each feature map to appropriately enhance its features. Given a feature tensor F∈R L×C×H×W , L represents the number of feature maps, C represents the number of feature map channels, H and W represent the height and width of the feature map, and the calculation formula of this module is:

[0126]

[0127] where f(·) is a linear function approximated by a 1×1 convolutional layer,

[0128] Embodiment 4, spatial perception module: The spatial perception module focuses on the discriminant area that is consistent between spatial position and feature level based on the spatial attention module of fusion features. This module is divided into two steps. First, the features of objects of different scales in space are learned by using deformable convolution, and then the features across levels are aggregated at the same spatial position. The calculation formula of this module is:

[0129]

[0130] Where K is the number of deformable convolution sampling locations, p k +Δp k is the sampling position p k Shifted by Δp k , to focus on the discrimination area, Δm k is the position p k The self-learning importance scalar at .

[0131] Example 5, Task Perception Module: The task perception attention module is deployed on the channel. It guides different feature channels to support different tasks according to the different convolution kernel responses of the object. The calculation formula of this module is:

[0132] π C (F)·F=

[0133] max(α 1 (F)·F C +β 1 (F),

[0134] α 2 (F)·F C +β2 (F)

[0135] Among them, F C is the characteristic slice of the Cth channel, θ(·)=[α 1 ,α 2 ,β 1 ,β 2 ] T is a hyperparameter that is learned to control the activation threshold. The θ(·) function first performs global average pooling on the L×H×W dimensions to reduce the dimensionality, then uses two fully connected layers and a normalization layer, and finally applies an activation function to normalize the output to [-1, 1].

[0136] Example 6: The DFMhead workflow is as follows, and the structure diagram is as follows: Figure 3 As shown in Figure 1. After first passing through the scale-aware attention module, the feature map becomes more sensitive to the scale differences of the foreground objects. After further passing through the space-aware attention module, the feature map becomes sparser and focuses on the discriminative spatial position of the foreground objects. Finally, after passing through the task-aware attention module, the feature map reforms different activations according to the needs of different downstream tasks.

[0137] The DFMhead calculation formula is:

[0138] W(F)=π C (π S (π L (F)·F)·F)·F

[0139] Among them, π L (·),π S (·),π c (·) are scale attention function, spatial attention function, and task attention function respectively.

[0140] S4: SIWL loss calculation module.

[0141] In order to further optimize the intersection traffic flow detection method based on deep learning, the SIWL of the present invention is introduced as a loss function in the loss calculation module. SIWL can more accurately calculate the loss of positive and negative samples and enhance the model's ability to detect small targets.

[0142] Specifically, the Scale-Invariant Wasserstein Loss (SIWL) is used in the loss calculation module for loss calculation. SIWL can better calculate the loss of positive and negative samples. First, the bounding box is modeled as a two-dimensional Gaussian distribution, and then the Wasserstein distance is used to measure the similarity of the derived Gaussian distribution. Finally, the loss of positive and negative samples is calculated. The main advantage of the Wasserstein distance is that it can measure distribution similarity even if there is no overlap or the overlap is negligible. In addition, SIWL is insensitive to objects of different scales, so it is more suitable for measuring the similarity between tiny objects. SIWL is divided into three steps: Gaussian modeling of the bounding box, normalized Gaussian Wasserstein distance, and SIWL loss function.

[0143] Example 7: Gaussian modeling of bounding box: For a horizontal bounding box R = (cx, cy, w, h), where (cx, cy), w and h represent the center coordinates, width and height respectively. The equation of the inscribed ellipse can be expressed as:

[0144]

[0145] Where (μ x ,μ y ) is the center coordinate of the ellipse, σ x ,σ y is the semi-axis length along the x and y axes. Therefore, μ x =cx,μ y =cy,σ x =w / 2,σ y =h / 2. The probability density function of the two-dimensional Gaussian distribution is given by:

[0146]

[0147] Where x, μ and Σ represent the coordinates (x, y), mean vector and covariance matrix of the Gaussian distribution. The following constraints are met:

[0148] (X-μ) T ∑ -1 (X-μ)=1

[0149]

[0150] Example 8. Normalized Gaussian Wasserstein distance: SIWL uses the Wasserstein distance from optimal transmission theory to calculate the distribution distance. For two two-dimensional Gaussian distributions μ1 = N(m1,Σ1) and μ2 = N(m2,Σ2), the second-order Wasserstein distance between μ1 and μ2 is defined as:

[0151]

[0152] where ||·|| F is the Frobenius norm. In addition, for Gaussian distributions Na and Nb modeled from bounding boxes A = (cxa, cya, wa, ha) and B = (cxb, cyb, wb, hb), Equation 6 can be further simplified to:

[0153]

[0154] We further normalize it using its exponential form and obtain a new metric called Normalized Wasserstein Distance (NWD):

[0155]

[0156] Where C is a constant that is closely related to the dataset.

[0157] Embodiment 9, SIWL loss function:

[0158] l SIWL =1-SIWL(Ν p ,N g )

[0159] Where Np is the Gaussian distribution model of the prediction box P, and Ng is the Gaussian distribution model of the gt box G.

[0160] The disclosed embodiment also proposes a deep learning-based intersection traffic flow detection system in an intelligent connected vehicle, the structure of which is as follows: Figure 2 As shown, it includes: edge detection client, cloud server and intelligent connected vehicle client.

[0161] The edge detection client deploys the improved deep learning-based vehicle flow detection algorithm of the present invention on the embedded platform development board to monitor the vehicle flow at the intersection in real time. At the same time, the MQTT client communication program is deployed on the development board. The MQTT client establishes a connection with the cloud server and publishes the vehicle flow data to the predefined MQTT topic. By utilizing the high efficiency and lightweight characteristics of MQTT, the vehicle flow is sent to the MQTT server of the cloud server in real time.

[0162] An MQTT server program is built on the cloud server to manage the publish and subscribe relationship between the edge detection client and the intelligent connected vehicle client, and to receive edge detection client information and send it to the intelligent connected vehicle client.

[0163] The intelligent connected vehicle client deploys the ROS operating system on the embedded development board and configures the MQTT client communication program. The system receives the long-distance intersection traffic flow information sent by the cloud server and uses this data to re-plan the vehicle's route.

[0164] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting traffic flow at intersections based on deep learning in intelligent connected vehicles, characterized in that: The method comprises the following steps: S1, obtain the real-time frame of the intersection camera; S2, preprocess each captured real-time frame image to match the model's input specification; the preprocessed image is then passed through the Backbone module that integrates the RFCBAMConv attention mechanism to extract rich features in the image; S3. After feature extraction, the Neck module is used to fuse feature maps at different levels to enhance the model’s ability to detect objects of various scales. The DFMhead detection head is then used to perform classification and regression tasks on the extracted features to further determine the detected target category and estimate its bounding box position, and output the final detection result. S4. In order to further optimize the detection results, SIWL is introduced as the loss function in the loss calculation module; the SIWL can calculate the positive and negative sample losses more accurately, which is used to enhance the detection capability of small targets.

2. The method for detecting traffic flow at intersections based on deep learning in an intelligent connected vehicle according to claim 1, characterized in that: The frame sequence data of traffic flow is obtained in real time through the camera at the intersection; the data will be used as the basic input for target detection to monitor vehicles, pedestrians and other traffic elements at the intersection.

3. The method for detecting traffic flow at intersections based on deep learning in an intelligent connected vehicle according to claim 1, characterized in that: The Backbone modules that integrate the RFCBAMConv attention mechanism include: RFAConv and CBAM; The RFAConv introduces a receptive field feature attention mechanism; the RFAConv workflow includes: a receptive field feature attention map generation stage, a receptive field feature map generation stage, a receptive field feature map and attention weight combination stage, and a feature extraction stage; The calculation formula of RFAConv can be expressed as: F=Softmax(g 1x1 (AvgPool(X)))x ReLU(Norm(g kxk (X)))= A rf xF rf Among them, g i×i represents a grouped convolution of size i×i, k represents the size of the convolution kernel, Norm represents normalization, X represents the input feature map, and F is obtained by converting the attention map A rf And the receptive field spatial feature map F rf The feature map obtained by multiplication; The CBAM dynamically adjusts the network's attention to the channel and spatial position in the feature map through the channel and spatial attention mechanism. The CBAM workflow includes: a channel attention map generation stage, a spatial attention map generation stage, a channel and spatial attention combination stage, and an adaptive adjustment stage; Given an intermediate feature F∈R C×H×W As input, CBAM sequentially derives a 1D channel attention map M c ∈R Cx1x1 and 2D spatial attention map M s ∈R 1×H×W ; The calculation formula of CBAM can be expressed as: M c (F)=Sigmoid(Conv2d (AvgPool(F))+ Conv2d(MaxPool(F))) M s (F)=Sigmoid(Conv2dx ([max(F):mean(F)])) in, It represents element-by-element multiplication, F1 represents the result of input feature map F after channel attention, and F2 represents the result of F1 after spatial attention map.

4. The method for detecting traffic flow at intersections based on deep learning in an intelligent connected vehicle according to claim 3, characterized in that: The calculation formula of the Backbone module integrating the RFCBAMConv attention mechanism is: Among them, F3 is the result of input feature map F after receptive field attention and path attention, and F4 is the result of F3 after spatial attention.

5. The method for detecting traffic flow at intersections based on deep learning in an intelligent connected vehicle according to claim 1, characterized in that: The DFMhead detection head consists of the following modules: scale perception module, space perception module and task perception attention module; The scale perception module is deployed on feature maps of different levels, and learns the relative importance of different semantic information on each feature map to appropriately enhance its features; Given a feature tensor F∈R L×C×H×W , L represents the number of feature maps, C represents the number of feature map channels, H and W represent the height and width of the feature map; the calculation formula of the scale perception module is: where f(·) is a linear function approximated by a 1×1 convolutional layer, The spatial perception module focuses on the discriminative area that is consistent between spatial position and feature level based on the spatial attention module of fusion features; the spatial perception module is divided into two steps: first, the features of objects of different scales in space are learned by using deformable convolution, and then the features across levels are aggregated at the same spatial position; the calculation formula of the spatial perception module is: Where K is the number of deformable convolution sampling locations, p k +Δp k is the sampling position p k Shifted by Δp k , to focus on the discrimination area, Δm k is the position p k The importance scalar of self-learning at ; The task-aware attention module is deployed on the channel, and guides different feature channels to support different tasks according to the different convolution kernel responses of the object; the calculation formula of the task-aware attention module is: p C (F)·F= max(a 1 (F)·F C +b 1 (F), a 2 (F)·F C +b 2 (F)) Among them, F C is the characteristic slice of the Cth channel, θ(·)=[α 1 ,α 2 ,β 1 ,β 2 ] T is a hyperparameter that is learned to control the activation threshold; the θ(·) function first performs global average pooling on L×H×W dimensions to reduce the dimensionality, then uses two fully connected layers and a normalization layer, and finally applies an activation function to normalize the output to [-1, 1].

6. The method for detecting traffic flow at intersections based on deep learning in an intelligent connected vehicle according to claim 5, characterized in that: The workflow of the DFMhead detection head is as follows: first, after passing through the scale-aware attention module, the feature map becomes more sensitive to the scale difference of the foreground object. After further passing through the spatial-aware attention module, the feature map becomes sparser and focuses on the discriminative spatial position of the foreground object. Finally, after passing through the task-aware attention module, the feature map re-forms different activations according to the needs of different downstream tasks. The calculation formula of DFMhead detection head is: W(F)=π C (p S (p L (F)·F)·F)·F Among them, π L (·),π S (·),π c (·) are scale attention function, spatial attention function, and task attention function respectively.

7. The method for detecting traffic flow at intersections based on deep learning in an intelligent connected vehicle according to claim 1, characterized in that: SIWL is introduced as the loss function. The process of accurately calculating the loss of positive and negative samples is as follows: first, Gaussian modeling of the bounding box is performed; then the Gaussian Wasserstein distance is normalized and the Wasserstein distance is used to measure the similarity of the derived Gaussian distribution; finally, the loss of positive and negative samples is calculated using the SIWL loss function; The Gaussian modeling process of the bounding box is as follows: for a horizontal bounding box R=(cx, cy, w, h), where (cx, cy), w and h represent the center coordinates, width and height respectively; the equation of the inscribed ellipse can be expressed as: Where (μ x ,μ y ) is the center coordinate of the ellipse, σ x ,σ y is the semi-axis length along the x and y axes; therefore, μ x =cx,μ y =cy,σ x =w / 2,σ y =h / 2; the probability density function of the two-dimensional Gaussian distribution is given by: Where x, μ and Σ represent the coordinates (x, y), mean vector and covariance matrix of the Gaussian distribution; the following constraints are met: (X-μ) T ∑ -1 (X-μ)=1 The process of normalizing the Gaussian Wasserstein distance is as follows: for two two-dimensional Gaussian distributions μ1=N(m1,Σ1) and μ2=N(m2,Σ2), the second-order Wasserstein distance between μ1 and μ2 is defined as: where ||·|| F is the Frobenius norm; For the Gaussian distributions Na and Nb modeled from bounding boxes A = (cxa, cya, wa, ha) and B = (cxb, cyb, wb, hb), it can be further simplified to: We further normalize it using its exponential form and obtain a new metric called Normalized Wasserstein Distance (NWD): Where C is a constant that is closely related to the data set; The calculation formula of the SIWL loss function is: l SIWL =1-SIWL(N p ,N g ) Where Np is the Gaussian distribution model of the prediction box P, and Ng is the Gaussian distribution model of the gt box G.

8. The deep learning-based intersection traffic flow detection system in intelligent connected vehicles is characterized by: The system is used to implement the deep learning-based intersection traffic flow detection method in an intelligent connected vehicle according to any one of claims 1 to 7, and the system includes: an edge detection client, a cloud server, and an intelligent connected vehicle client; The edge detection client is on the embedded platform development board, and an improved deep learning-based vehicle flow detection algorithm is deployed to monitor the vehicle flow at the intersection in real time. At the same time, an MQTT client communication program is deployed on the development board. The MQTT client establishes a connection with the cloud server and publishes vehicle flow data to a predefined MQTT topic. By utilizing the high efficiency and lightweight characteristics of MQTT, the vehicle flow is sent to the MQTT server of the cloud server in real time. The cloud server builds an MQTT server program to manage the publish and subscribe relationship between the edge detection client and the intelligent connected vehicle client, and receives edge detection client information and sends it to the intelligent connected vehicle client; The intelligent connected vehicle client deploys the ROS operating system on the embedded development board and configures the MQTT client communication program; it is used to receive the long-distance intersection traffic flow information sent by the cloud server and use this data to re-plan the vehicle's route.

Images