Unmanned vehicle control method based on deep learning

Abstract

The invention relates to the technical field of vehicle control, in particular to an unmanned vehicle control method based on deep learning, and the method comprises the following steps: collecting the multi-modal data of weak participants in a road environment in real time, and recognizing the weak participants; the behaviors of the weak participants at the next moment are predicted through multi-dimensional information fusion, and behavior intention prediction of the weak participants is carried out; the path planning and control strategy of the vehicle is adjusted in real time through the self-adaptive decision control module; and when the predicted behavior intention of the weak participant has high uncertainty or potential risk, an emergency avoidance mechanism is automatically triggered, the risk level of the current traffic environment is dynamically evaluated, an optimal avoidance path is generated, and emergency stop or detour operation is executed. According to the method, double guarantees of high-dimensional analysis of behavior intentions of participants and real-time control of vehicles are combined, and the driving safety in a complex traffic scene is ensured.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to an unmanned vehicle control method based on deep learning. Background Art

[0002] With the rapid development of artificial intelligence technology, driverless vehicles have gradually become an important part of the intelligent transportation system. At present, most of the driverless technologies available on the market rely on the fusion processing of multi-sensor data and traditional path planning and decision-making control algorithms. However, in complex urban traffic environments, especially when facing weak participants such as pedestrians, bicycles, and electric bicycles, existing technologies still face many challenges in detection accuracy, behavior prediction, and real-time decision-making.

[0003] In practical applications, small participants are often small in size, move at irregular speeds, and are often in complex backgrounds or occluded, which makes existing computer vision algorithms prone to false detection or missed detection. Although deep learning models such as convolutional neural networks (CNNs) have been introduced to improve detection accuracy, the recognition effect of small targets in complex environments is still unsatisfactory, making it difficult to meet the requirements of unmanned vehicles for high-precision, real-time detection of small targets.

[0004] Traditional behavior prediction methods are mostly based on simple kinematic models or short-term trajectory predictions, which cannot accurately capture the behavioral intentions of weak participants in complex traffic environments, such as suddenly crossing the road, changing lanes, etc. This lack of prediction ability seriously affects the safety and decision-making efficiency of driverless vehicles. Although some existing deep learning models can predict the short-term behavior of weak participants to a certain extent, they lack comprehensive consideration of environmental context and multi-dimensional information, resulting in limited accuracy and reliability of the prediction results.

[0005] In the face of high uncertainty or potential risks, existing decision-making control algorithms often show delayed response or are too conservative, making it difficult to adjust the vehicle's path planning and control strategy in a timely manner, which can easily lead to safety hazards. Especially in the case of frequent and unpredictable changes in the behavior of weak participants, existing algorithms often lack sufficient adaptability and cannot make the best decision based on real-time risk assessment. In addition, emergency avoidance measures usually rely on preset rules and thresholds, lack the ability to dynamically respond to real-time environmental changes, and result in insufficient handling capabilities in emergencies. Summary of the invention

[0006] The present invention provides an unmanned vehicle control method based on deep learning.

[0007] A method for controlling an unmanned vehicle based on deep learning comprises the following steps:

[0008] S1, collects multimodal data of weak participants in the road environment in real time through multiple sensors of the vehicle; uses an improved convolutional neural network (CNN) to detect objects in the collected data, identifies weak participants, and enhances the detection accuracy of the model for weak participants through the attention mechanism, ensuring high-precision recognition of small targets in complex backgrounds. Weak participants include pedestrians, bicycles, and electric bicycles;

[0009] S2, based on the identified weak participants, introduces a dynamic path convolution module into the pre-built spatiotemporal graph convolutional neural network ST-GCN, considers the real-time position and speed information of the participants, and introduces the environmental context and scene understanding module. Through multi-dimensional information fusion, the behavior of the weak participants at the next moment is predicted, and the behavioral intention of the weak participants is predicted;

[0010] S3, based on the prediction results of the behavior intention of the weak participants, the path planning and control strategy of the vehicle are adjusted in real time through the adaptive decision control module; the adaptive decision control module adopts the deep reinforcement learning (DRL) algorithm to optimize the vehicle's avoidance path and speed control strategy by continuously simulating and learning multiple potential risks in complex scenarios;

[0011] S4, when the predicted behavior intention of the weak participant is highly uncertain or has potential risks, the emergency avoidance mechanism is automatically triggered to dynamically evaluate the risk level of the current traffic environment, generate the best avoidance path, and perform emergency parking or detour operations.

[0012] Optionally, the multiple sensors in S1 include laser radar, camera, millimeter wave radar, wherein;

[0013] The LiDAR performs a 360-degree full-scale scan of the vehicle’s surroundings at a fixed scanning frequency, generating three-dimensional point cloud data for detecting objects on the road and their spatial positions, and distinguishing small participants from other larger targets (vehicles, buildings) through point cloud density analysis;

[0014] Equipped with multi-angle cameras, including front, side and rear cameras, to capture two-dimensional image data of the road environment in real time;

[0015] The long-range detection capability of millimeter-wave radar is utilized to capture the movement speed and direction of small participants.

[0016] Optionally, the improved convolutional neural network is used in S1 to perform object detection, and the detection accuracy of the model for weak participants is enhanced by the attention mechanism, specifically including:

[0017] S11, multi-scale feature extraction: The improved convolutional neural network adopts a multi-scale feature extraction module to capture target features of different sizes and scales, including distinguishing weak participants from background debris through layer-by-layer convolution operations in complex backgrounds and occluded environments;

[0018] S12, Feature enhancement based on attention mechanism: An attention mechanism module is introduced after the feature extraction layer of the improved convolutional neural network. The weights of each position and channel in the feature map are dynamically allocated through the spatial attention mechanism and the channel attention mechanism to enhance the feature attention of weak participants. The spatial attention mechanism enables the model to identify the specific position of weak participants by strengthening the spatial features of the target area; the channel attention mechanism highlights the feature expression of small targets by enhancing the significant features on the specific channel, thereby reducing the impact of background interference on the detection results;

[0019] S13, the improved convolutional neural network and the attention mechanism are integrated end-to-end, and the convolution kernel parameters of the improved convolutional neural network and the weight distribution of the attention module are simultaneously optimized through the back-propagation algorithm to ensure the high-precision detection and recognition of weak targets in a variety of complex scenarios.

[0020] Optionally, the improved convolutional neural network captures target features of different scales through a multi-scale feature extraction module, expressed as: F i (x) = Conv(x,W i )+b i , where x represents the input feature map, W i represents the weight matrix of the i-th layer convolution kernel, b i represents the bias of the i-th layer, F i (x) represents the feature map after the i-th convolution layer;

[0021] In multi-scale feature extraction, convolution kernels W of different sizes are used i To extract multi-scale features, by combining features of different scales, the detailed features of weak participants in different environments can be captured.

[0022] Optionally, the S2 specifically includes:

[0023] S21, construct a spatiotemporal graph convolutional neural network ST-GCN, where the node represents the state of each weak participant, the edge represents the relationship between participants or the connectivity of the motion trajectory, the node of the time dimension is used to capture the state changes of the participants at different times, and the edge of the space dimension is used to capture the spatial relationship between participants;

[0024] S22, introduction of dynamic path convolution module: a dynamic path convolution module is introduced in ST-GCN, which dynamically extracts the position and speed information of participants at different time nodes through convolution operations;

[0025] S23, introduction of environmental context and scene understanding module: the output features of the environmental context and scene understanding module are fused with the output of the dynamic path convolution module through the information fusion mechanism to perform multi-dimensional feature fusion. The output features include road type, weather conditions, and traffic flow.

[0026] S24, weak participant behavior prediction: Based on the fused multi-dimensional features, the behavior of the weak participant at the next moment is predicted through the fully connected layer and the Softmax layer to generate the prediction result

[0027] Optionally, the spatiotemporal graph convolutional neural network includes construction of a spatiotemporal graph, the spatiotemporal graph is represented as: G = (V, E), consisting of a node set V and an edge set E;

[0028] Node V: Each node v∈V represents the state of a weak participant at a specific moment. The node features include the location information of the participant. Speed ​​information and environmental characteristics

[0029] Edge E: Edges connect related nodes at different times and spatial locations.

[0030] In the graph convolution layer, the feature of node v at time t is Update by aggregating the features of neighboring nodes: in, is the updated feature of node v at time t+1, represents the neighboring nodes of node v, W t is the weight matrix of the graph convolution layer, σ is the activation function;

[0031] Modeling the time dimension: In order to capture dynamic changes in time, ST-GCN models the behavioral changes of weak participants by introducing nodes and edges in the time dimension:

[0032] Time dimension node: Each node has a representation at different time points;

[0033] Time dimension edges: add edges between the same participant nodes at different times.

[0034] Optionally, the S3 specifically includes:

[0035] S31, prediction result input: the prediction result of weak participant behavior output by spatiotemporal graph convolutional neural network As input, it is passed to the adaptive decision control module to evaluate the potential risks of the current traffic environment;

[0036] S32, construction of a deep reinforcement learning model: In the adaptive decision control module, a deep reinforcement learning algorithm is used to train the decision control model, and a deep reinforcement learning model based on a state-action-reward (SAR) structure is constructed, wherein;

[0037] State St: includes the current vehicle position, speed, direction, and the behavior prediction results of the weak participants

[0038] Action At: includes the vehicle's steering angle adjustment, acceleration or deceleration command, and avoidance path planning selection;

[0039] Reward Rt: A reward function is designed based on vehicle safety, ride comfort, and driving time factors to guide the vehicle's optimal path selection and control strategy;

[0040] S33, Simulation and Learning: Through the policy gradient algorithm, we continuously simulate and learn various potential risks in complex traffic scenarios in a virtual environment, including the sudden appearance of pedestrians and emergency braking of vehicles, to train and optimize the performance of the decision-making control model and update the policy function π θ (S t ) makes the action A chosen in a given state t Can maximize expected reward:

[0041] Among them, θ represents the parameters of the deep neural network, and γ is the discount factor, which measures the impact of future rewards.

[0042] S34, real-time adjustment: During actual driving, the vehicle’s current state and environmental changes are monitored in real time, and the vehicle’s path planning and control strategies are dynamically adjusted based on the trained deep reinforcement learning model to ensure that it can respond when encountering behavioral changes of weak participants.

[0043] Optionally, the response in S34 includes:

[0044] Adjust the vehicle's steering angle to avoid potential collision paths;

[0045] Control your speed to avoid conflicts with smaller participants.

[0046] Optionally, the S4 specifically includes:

[0047] S41, uncertainty assessment: After receiving the prediction results of the weak participant behavior output by the spatiotemporal graph convolutional neural network, the confidence of the prediction results is evaluated through the uncertainty quantification module. If the confidence is lower than the preset threshold τ or high-risk behaviors (such as suddenly crossing the road, changing lanes quickly, etc.) are detected, the current scene is considered to have high uncertainty or potential risks;

[0048] S42, triggering of emergency avoidance mechanism: after detecting high uncertainty or potential risk, the emergency avoidance mechanism is automatically triggered to enter the high-priority risk assessment process;

[0049] S43, Dynamic risk level assessment: By comprehensively analyzing the real-time data of the current traffic environment, including vehicle speed, road conditions, the location and speed of surrounding vehicles and pedestrians, and weather conditions, the risk level R(t) of the current environment is dynamically assessed. The assessment process is based on the following risk assessment formula: Among them, x i Represents the value of each risk factor (such as distance, speed difference, road slipperiness, etc.), β i is the weight coefficient of each risk factor, λ i It is a parameter used to adjust the impact of risk factors on the overall risk level. R(t) is the calculated risk level. The higher the value, the greater the risk.

[0050] S44, optimal avoidance path generation: Based on the current risk level R(t) and the relative position of the vehicle and surrounding obstacles, an optimal avoidance path is generated through a path planning algorithm to ensure that the vehicle can avoid the risk area with minimal steering or acceleration.

[0051] Optionally, the path planning is expressed as:

[0052] in, represents the set of all possible avoidance paths, c j represents the risk cost of each node on the path, Represents the distance from the current node to the target node. According to the generated optimal avoidance path, the vehicle is instructed to perform emergency operations.

[0053] Beneficial effects of the present invention:

[0054] The present invention, by constructing a multimodal data collection and processing system and introducing a dynamic path convolution module in the spatiotemporal graph convolutional neural network (ST-GCN), effectively improves the detection accuracy and behavior prediction ability of weak participants. By combining the improved convolutional neural network (CNN) with the spatial and channel attention mechanism, the model significantly enhances the recognition effect of small targets in complex backgrounds, reducing false detection and missed detection. In addition, the dynamic path convolution module realizes efficient prediction of the next moment behavior of weak participants by real-time analysis of the location information, speed information and environmental context characteristics of the participants, especially in complex traffic scenes, and can identify potential dangerous behaviors in advance, providing a reliable basis for subsequent decision-making and control.

[0055] The present invention introduces a deep reinforcement learning (DRL) algorithm in the adaptive decision-making control module, which can dynamically optimize the vehicle's path planning and control strategy by continuously simulating and learning various potential risks in complex traffic scenarios. The module uses the behavior prediction results of weak participants and combines the real-time status information of the vehicle to generate the optimal avoidance path and speed adjustment strategy, so that the vehicle can respond quickly and accurately in a complex and changeable traffic environment. Through continuous online learning and strategy updates, it ensures that the vehicle can quickly adapt to environmental changes when responding to emergencies, thereby improving the overall driving safety and stability.

[0056] The present invention designs an automatically triggered emergency avoidance mechanism when there is a high degree of uncertainty in the prediction of the behavior of the weak participant or when potential risks are detected. The mechanism can dynamically evaluate the risk level of the current traffic environment and generate the best avoidance path in real time. Through comprehensive analysis of real-time environmental data, the mechanism can accurately judge the degree of risk and perform emergency parking or detour operations in high-risk situations, thereby avoiding potential collision risks. The mechanism is particularly suitable for complex urban traffic environments and significantly improves the safety and decision-making ability of unmanned vehicles when facing unpredictable situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 A schematic flow chart of a vehicle control method according to an embodiment of the present invention;

[0059] Figure 2 Schematic diagram of behavioral intention prediction of a weak participant according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it is explained here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments, and those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments, and are not intended to specifically limit the present invention.

[0061] It should be noted that the references to "one embodiment", "an embodiment", "an exemplary embodiment", "some embodiments" and the like in the specification indicate that the embodiments described may include specific features, structures or characteristics, but not every embodiment may include the specific features, structures or characteristics. In addition, when a specific feature, structure or characteristic is described in conjunction with an embodiment, it should be within the knowledge of a person skilled in the art to implement such feature, structure or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0062] In general, a term can be understood, at least in part, from its use in context. For example, depending, at least in part, on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in the singular sense, or can be used to describe a combination of features, structures, or characteristics in the plural sense. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but can instead, depending, at least in part, on the context, allow for the presence of other factors that are not necessarily explicitly described.

[0063] like Figure 1 As shown, a method for controlling an unmanned vehicle based on deep learning comprises the following steps:

[0064] S1, collects multimodal data of weak participants in the road environment in real time through multiple sensors (lidar, camera, millimeter wave radar) of the vehicle; uses an improved convolutional neural network (CNN) to detect objects in the collected data, identifies weak participants, and enhances the detection accuracy of the model for weak participants through the attention mechanism, ensuring high-precision recognition of small targets in complex backgrounds. Weak participants include pedestrians, bicycles, and electric bicycles;

[0065] S2, based on the identified weak participants, introduces a dynamic path convolution module into the pre-built spatiotemporal graph convolutional neural network ST-GCN, considers the real-time position and speed information of the participants, and introduces the environmental context and scene understanding module. Through multi-dimensional information fusion, the behavior of the weak participants at the next moment (such as suddenly crossing the road, temporary parking, etc.) is predicted, and the behavioral intention of the weak participants is predicted;

[0066] S3, based on the prediction results of the behavior intention of weak participants, the adaptive decision control module adjusts the path planning and control strategy of the vehicle in real time; the adaptive decision control module adopts the deep reinforcement learning (DRL) algorithm to optimize the vehicle's avoidance path and speed control strategy by continuously simulating and learning multiple potential risks in complex scenarios, ensuring that the vehicle can make timely and accurate responses to the behavior changes of weak participants in complex urban traffic environments;

[0067] S4, when the predicted behavior intention of the weak participant is highly uncertain or has potential risks, the emergency avoidance mechanism is automatically triggered to dynamically evaluate the risk level of the current traffic environment, generate the best avoidance path and perform emergency parking or detour operations. This mechanism combines the dual guarantees of high-dimensional analysis of the participant's behavior intention and real-time vehicle control to ensure driving safety in complex traffic scenarios.

[0068] The various sensors in the S1 include LiDAR, cameras, and millimeter-wave radar, among which;

[0069] The LiDAR performs a 360-degree full-scale scan of the vehicle’s surroundings at a fixed scanning frequency, generating three-dimensional point cloud data for detecting objects on the road and their spatial positions, and distinguishing small participants from other larger targets (vehicles, buildings) through point cloud density analysis;

[0070] Equipped with multi-angle cameras, including front, side and rear cameras, to capture two-dimensional image data of the road environment in real time;

[0071] The long-range detection capability of millimeter-wave radar is utilized to capture the movement speed and direction of weak participants, especially in bad weather (such as rain, fog) or low light conditions. The highly penetrating radar signal ensures stable tracking of the target and updates its velocity vector information in real time.

[0072] S1 uses an improved convolutional neural network for object detection and uses an attention mechanism to enhance the model’s detection accuracy for weak participants. Specifically:

[0073] S11, multi-scale feature extraction: The improved convolutional neural network adopts a multi-scale feature extraction module to capture target features of different sizes and scales, including distinguishing weak participants from background clutter through layer-by-layer convolution operations in complex backgrounds and occluded environments;

[0074] S12, Feature enhancement based on attention mechanism: The attention mechanism module is introduced after the feature extraction layer of the improved convolutional neural network. The weights of each position and channel in the feature map are dynamically allocated through the spatial attention mechanism and the channel attention mechanism to improve the feature attention of weak participants. The spatial attention mechanism strengthens the spatial features of the target area, enabling the model to identify the specific location of the weak participants; the channel attention mechanism enhances the significant features on the specific channel, highlights the feature expression of small targets, and reduces the impact of background interference on the detection results;

[0075] S13, the improved convolutional neural network and the attention mechanism are integrated end-to-end, and the convolution kernel parameters of the improved convolutional neural network and the weight distribution of the attention module are simultaneously optimized through the back-propagation algorithm to ensure the high-precision detection and recognition of weak targets in a variety of complex scenarios.

[0076] The improved convolutional neural network captures target features of different scales through a multi-scale feature extraction module, expressed as: F i (x) = Conv(x,W i )+b i , where x represents the input feature map, W i represents the weight matrix of the i-th layer convolution kernel, b i represents the bias of the i-th layer, F i (x) represents the feature map after the i-th convolution layer;

[0077] In multi-scale feature extraction, convolution kernels W of different sizes are used i To extract multi-scale features, such as 3×3, 5×5, and 7×7, by combining features of different scales, the detailed features of weak participants in different environments can be captured.

[0078] The spatial attention mechanism enhances the features of key positions by calculating the attention weights of each position in the feature map, which can be expressed as:

[0079] Attention s (x)=σ(Conv(MaxPool(x)⊕AvgPool(x),W s ));

[0080] Among them, MaxPool(x) and AvggPool(x) represent the maximum pooling and average pooling operations of the input feature map x respectively. The results obtained represent the global maximum and average features. ⊕ represents the concatenation of the results of the two pooling operations. s is the convolution kernel used to calculate spatial attention, σ is the Sigmoid activation function, which is used to normalize the attention weight. Finally, the calculated spatial attention weight Attention s(x) is multiplied by the original feature map x to obtain the enhanced feature map: F s (x) = x × Attention s (x); Through this mechanism, the model can spatially highlight the location of weak participants and reduce the interference of background noise.

[0081] The channel attention mechanism increases the focus on specific features by calculating the importance weight of each channel in the feature map, which is expressed as:

[0082]

[0083] Among them, AvgPool(x) is a global average pooling operation on the input feature map x, and the result obtained represents the global average value of each channel. and They are the weight matrices used to calculate channel attention, ReLU is the ReLU activation function, which adds nonlinearity, and σ is the Sigmoid activation function, which is used to normalize the channel attention weight.

[0084] Finally, the channel attention weight Attention c (x) is multiplied by each channel of the input feature map to obtain the enhanced channel feature map: F c (x) = x × Attention c (x), through this mechanism, the model is able to enhance the feature expression related to weak participants on the channel and reduce the impact of irrelevant information.

[0085] In the process of identifying weak participants, due to the small size of these targets and the complex background, single-scale feature extraction may not be able to effectively capture their details. Therefore, the use of a multi-scale feature extraction module can significantly improve the recognition accuracy. The spatial attention mechanism is particularly important for identifying weak participants by enhancing the features of key positions. Since the area of ​​weak participants in the image is relatively small, the spatial attention mechanism can increase the attention to these small targets, thereby improving the detection accuracy. The feature map processed by multi-scale feature extraction and spatial attention mechanism will be sent to the subsequent classification and positioning network for target recognition, and the processed feature map will be used for target classification and positioning to ensure accurate recognition of weak participants. For example, a convolutional layer is used for further feature extraction, and then a fully connected layer is used for classification.

[0086] S2 specifically includes:

[0087] S21, construct a spatiotemporal graph convolutional neural network ST-GCN, where the node represents the state of each weak participant, the edge represents the relationship between participants or the connectivity of the motion trajectory, the node of the time dimension is used to capture the state changes of the participants at different times, and the edge of the space dimension is used to capture the spatial relationship between participants;

[0088] S22, introduction of dynamic path convolution module: a dynamic path convolution module is introduced in ST-GCN, which dynamically extracts the position and speed information of participants at different time nodes through convolution operations;

[0089] S23, introduction of environmental context and scene understanding module: the output features of the environmental context and scene understanding module are fused with the output of the dynamic path convolution module through the information fusion mechanism. The output features include road type, weather conditions, and traffic flow. The context information extracted from the external environment includes: road type c road 、Weather conditions weather Traffic flow traffic These features are extracted by the environment context module and combined into the feature update of ST-GCN. Fused into the output of the dynamic path convolution module: in, is the node feature updated by the dynamic path convolution module, It is the environmental context feature extracted at time t, and Concat is a feature concatenation operation that combines multi-dimensional information together;

[0090] S24, weak participant behavior prediction: Based on the fused multi-dimensional features, the behavior of the weak participant at the next moment is predicted through the fully connected layer and the Softmax layer to generate the prediction result represents the behavior of the participant at time t+1 (such as crossing the road, turning, slowing down, etc.),

[0091] The spatiotemporal graph convolutional neural network includes the construction of a spatiotemporal graph, which is represented by: G = (V, E), consisting of a node set V and an edge set E;

[0092] Node V: Each node v∈V represents the state of a weak participant at a specific moment. The node features include the location information of the participant. Speed ​​information and environmental characteristics

[0093] Edge E: Edges connect nodes that are related at different times and spatial positions. For example, edge e = (v,u)∈E can represent the position change of the same participant in adjacent time frames, or the spatial relationship between different participants in the same time frame.

[0094] In the graph convolution layer, the feature of node v at time t is Update by aggregating the features of neighboring nodes: in, is the updated feature of node v at time t+1, represents the neighboring nodes of node v, W t is the weight matrix of the graph convolution layer, σ is the activation function;

[0095] Modeling the time dimension: In order to capture dynamic changes in time, ST-GCN models the behavioral changes of weak participants by introducing nodes and edges in the time dimension:

[0096] Time dimension node: Each node has a representation at different time points, such as v (t) and v (t+1) Respectively represent the state of the same participant at time t and t+1;

[0097] Time dimension edge: add edges between the same participant nodes at different times, for example (v (t) ,v (t+1) ), used to model the continuity of participants in time.

[0098] After introducing the dynamic path convolution module, the update is as follows:

[0099] in, is the attention weight in the dynamic path convolution module, which is used to measure the influence of node u on node v at time t, expressed as:

[0100] W p and W s is the weight matrix of position and velocity information.

[0101] a is a trainable vector used to calculate the attention weights.

[0102] ‖ represents the concatenation operation of vectors.

[0103] is a dynamic convolution weight matrix that is dynamically adjusted over time to capture different motion patterns.

[0104] S3 specifically includes:

[0105] S31, prediction result input: the prediction result of weak participant behavior output by spatiotemporal graph convolutional neural network As input, it is passed to the adaptive decision control module to evaluate the potential risks of the current traffic environment;

[0106] S32, construction of a deep reinforcement learning model: In the adaptive decision control module, a deep reinforcement learning algorithm is used to train the decision control model, and a deep reinforcement learning model based on a state-action-reward (SAR) structure is constructed, wherein;

[0107] State St: includes the current vehicle position, speed, direction, and the behavior prediction results of the weak participants

[0108] Action At: includes the vehicle's steering angle adjustment, acceleration or deceleration command, and avoidance path planning selection;

[0109] Reward Rt: A reward function is designed based on vehicle safety, ride comfort, and driving time factors to guide the vehicle's optimal path selection and control strategy;

[0110] S33, Simulation and Learning: Through the policy gradient algorithm, we continuously simulate and learn various potential risks in complex traffic scenarios in a virtual environment, including the sudden appearance of pedestrians and emergency braking of vehicles, to train and optimize the performance of the decision-making control model and update the policy function π θ (S t ) makes the action A chosen in a given state t Can maximize expected reward:

[0111] Among them, θ represents the parameters of the deep neural network, and γ is the discount factor, which measures the impact of future rewards.

[0112] S34, real-time adjustment: During actual driving, the vehicle’s current state and environmental changes are monitored in real time, and the vehicle’s path planning and control strategies are dynamically adjusted based on the trained deep reinforcement learning model to ensure that it can respond when encountering behavioral changes of weak participants.

[0113] The response in S34 includes:

[0114] Adjust the vehicle's steering angle to avoid potential collision paths;

[0115] Control your speed to avoid conflicts with smaller participants.

[0116] S4 specifically includes:

[0117] S41, uncertainty assessment: After receiving the prediction results of the weak participant behavior output by the spatiotemporal graph convolutional neural network, the confidence of the prediction results is evaluated through the uncertainty quantification module. If the confidence is lower than the preset threshold τ or high-risk behaviors (such as suddenly crossing the road, changing lanes quickly, etc.) are detected, the current scene is considered to have high uncertainty or potential risks;

[0118] S42, triggering of emergency avoidance mechanism: after detecting high uncertainty or potential risk, the emergency avoidance mechanism is automatically triggered to enter the high-priority risk assessment process;

[0119] S43, Dynamic risk level assessment: By comprehensively analyzing the real-time data of the current traffic environment, including vehicle speed, road conditions, the location and speed of surrounding vehicles and pedestrians, and weather conditions, the risk level R(t) of the current environment is dynamically assessed. The assessment process is based on the following risk assessment formula: Among them, x i Represents the value of each risk factor (such as distance, speed difference, road slipperiness, etc.), β i is the weight coefficient of each risk factor, λ i It is a parameter used to adjust the impact of risk factors on the overall risk level. R(t) is the calculated risk level. The higher the value, the greater the risk.

[0120] S44, optimal avoidance path generation: Based on the current risk level R(t) and the relative position of the vehicle and surrounding obstacles, an optimal avoidance path is generated through a path planning algorithm to ensure that the vehicle can avoid the risk area with minimal steering or acceleration.

[0121] The path planning is expressed as:

[0122] in, represents the set of all possible avoidance paths, c j represents the risk cost of each node on the path, Indicates the distance from the current node to the target node. Based on the generated optimal avoidance path, instruct the vehicle to perform emergency operations, including:

[0123] Emergency stop: If the risk level R(t) exceeds the preset threshold and the evasive maneuver cannot be completed within the safe distance, the vehicle will perform an emergency stop to avoid a potential collision;

[0124] Detour: If the risk level allows and there is a safe avoidance path, the vehicle will perform a detour to avoid the risk area by adjusting the steering angle and speed.

[0125] The present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention. In order to make the public have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention, but those skilled in the art can fully understand the present invention without the description of these details. In addition, in order to avoid unnecessary confusion about the essence of the present invention, well-known methods, processes, procedures, components and circuits are not described in detail.

[0126] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for controlling an unmanned vehicle based on deep learning, characterized in that: The following steps are involved: S1, collects multimodal data of weak participants in the road environment in real time through multiple sensors of the vehicle; in the collected data, uses an improved convolutional neural network for object detection, identifies weak participants, and enhances the detection accuracy of the model for weak participants through the attention mechanism, ensuring high-precision recognition of small targets in complex backgrounds. Weak participants include pedestrians, bicycles, and electric bicycles; S2, based on the identified weak participants, introduces a dynamic path convolution module into the pre-built spatiotemporal graph convolutional neural network ST-GCN, considers the real-time position and speed information of the participants, and introduces the environmental context and scene understanding module. Through multi-dimensional information fusion, the behavior of the weak participants at the next moment is predicted, and the behavioral intention of the weak participants is predicted; S3, based on the prediction results of the behavior intention of the weak participants, the path planning and control strategy of the vehicle are adjusted in real time through the adaptive decision control module; among them, the adaptive decision control module adopts a deep reinforcement learning algorithm to optimize the vehicle's avoidance path and speed control strategy by continuously simulating and learning multiple potential risks in complex scenarios; S4, when there is uncertainty or potential risk in the predicted behavior intention of the weak participant, the emergency avoidance mechanism is automatically triggered to dynamically evaluate the risk level of the current traffic environment, generate the best avoidance path and perform emergency parking or detour operations.

2. The unmanned vehicle control method based on deep learning according to claim 1, characterized in that: The various sensors in S1 include laser radar, camera, millimeter wave radar, among which; The LiDAR performs a 360-degree full-scale scan of the vehicle’s surroundings at a fixed scanning frequency, generating three-dimensional point cloud data for detecting objects on the road and their spatial positions, and distinguishing weak participants from other targets through point cloud density analysis; Equipped with multi-angle cameras, including front, side and rear cameras, to capture two-dimensional image data of the road environment in real time; The long-range detection capability of millimeter-wave radar is utilized to capture the movement speed and direction of small participants.

3. The unmanned vehicle control method based on deep learning according to claim 1, characterized in that: In S1, an improved convolutional neural network is used for object detection, and the detection accuracy of the model for weak participants is enhanced through the attention mechanism. Specifically, the following are included: S11, multi-scale feature extraction: The improved convolutional neural network adopts a multi-scale feature extraction module to capture target features of different sizes and scales, including distinguishing weak participants from background debris through layer-by-layer convolution operations in complex backgrounds and occluded environments; S12, Feature enhancement based on attention mechanism: An attention mechanism module is introduced after the feature extraction layer of the improved convolutional neural network. The weights of each position and channel in the feature map are dynamically allocated through the spatial attention mechanism and the channel attention mechanism to enhance the feature attention of weak participants. The spatial attention mechanism enables the model to identify the specific position of weak participants by strengthening the spatial features of the target area; the channel attention mechanism highlights the feature expression of small targets by enhancing the significant features on the specific channel, thereby reducing the impact of background interference on the detection results; S13, integrates the improved convolutional neural network with the attention mechanism end-to-end, and optimizes the convolution kernel parameters of the improved convolutional neural network and the weight distribution of the attention module through the back-propagation algorithm, ensuring the high-precision detection and recognition of small targets in a variety of complex scenarios.

4. The unmanned vehicle control method based on deep learning according to claim 3 is characterized in that: The improved convolutional neural network captures target features of different scales through a multi-scale feature extraction module, which is expressed as: i (x) = Conv(x,W i )+b i , where x represents the input feature map, W i represents the weight matrix of the i-th layer convolution kernel, b i represents the bias of the i-th layer, F i (x) represents the feature map after the i-th convolution layer; In multi-scale feature extraction, convolution kernels W of different sizes are used i To extract multi-scale features, by combining features of different scales, the detailed features of weak participants in different environments can be captured.

5. The unmanned vehicle control method based on deep learning according to claim 1, characterized in that: The S2 specifically includes: S21, construct a spatiotemporal graph convolutional neural network ST-GCN, where the node represents the state of each weak participant, the edge represents the relationship between participants or the connectivity of the motion trajectory, the node of the time dimension is used to capture the state changes of the participants at different times, and the edge of the space dimension is used to capture the spatial relationship between participants; S22, introduction of dynamic path convolution module: a dynamic path convolution module is introduced in ST-GCN, which dynamically extracts the position and speed information of participants at different time nodes through convolution operations; S23, introduction of environmental context and scene understanding module: the output features of the environmental context and scene understanding module are fused with the output of the dynamic path convolution module through the information fusion mechanism to perform multi-dimensional feature fusion. The output features include road type, weather conditions, and traffic flow. S24, weak participant behavior prediction: Based on the fused multi-dimensional features, the behavior of the weak participant at the next moment is predicted through the fully connected layer and the Softmax layer to generate the prediction result 6. The unmanned vehicle control method based on deep learning according to claim 5, characterized in that: The spatiotemporal graph convolutional neural network includes the construction of a spatiotemporal graph, which is represented by: G = (V, E), consisting of a node set V and an edge set E; Node V: Each node v∈V represents the state of a weak participant at a specific moment. The node features include the location information of the participant. Speed ​​information and environmental characteristics Edge E: Edges connect related nodes at different times and spatial locations; In the graph convolution layer, the feature of node v at time t is Update by aggregating the features of neighboring nodes: in, is the updated feature of node v at time t+1, represents the neighboring nodes of node v, W t is the weight matrix of the graph convolution layer, σ is the activation function; Modeling the time dimension: In order to capture dynamic changes in time, ST-GCN models the behavioral changes of weak participants by introducing nodes and edges in the time dimension: Time dimension node: Each node has a representation at different time points; Time dimension edges: add edges between the same participant nodes at different times.

7. The unmanned vehicle control method based on deep learning according to claim 1, characterized in that: The S3 specifically includes: S31, prediction result input: the prediction result of weak participant behavior output by spatiotemporal graph convolutional neural network As input, it is passed to the adaptive decision control module to evaluate the potential risks of the current traffic environment; S32, construction of a deep reinforcement learning model: In the adaptive decision control module, a deep reinforcement learning algorithm is used to train the decision control model, and a deep reinforcement learning model based on a state-action-reward structure is constructed, wherein; State St: includes the current vehicle position, speed, direction, and the behavior prediction results of the weak participants Action At: includes the vehicle's steering angle adjustment, acceleration or deceleration command, and avoidance path planning selection; Reward Rt: A reward function is designed based on vehicle safety, ride comfort, and driving time factors to guide the vehicle's optimal path selection and control strategy; S33, Simulation and Learning: Through the policy gradient algorithm, we continuously simulate and learn various potential risks in complex traffic scenarios in a virtual environment, including the sudden appearance of pedestrians and emergency braking of vehicles, to train and optimize the performance of the decision-making control model and update the policy function π θ (S t ) makes the action A chosen in a given state t Can maximize expected reward: Among them, θ represents the parameters of the deep neural network, and γ is the discount factor, which measures the impact of future rewards; S34, real-time adjustment: During actual driving, the vehicle’s current status and environmental changes are monitored in real time, and the vehicle’s path planning and control strategies are dynamically adjusted based on the trained deep reinforcement learning model to ensure that it can respond when encountering behavioral changes of weak participants.

8. The unmanned vehicle control method based on deep learning according to claim 7, characterized in that: The response in S34 includes: Adjust the vehicle's steering angle to avoid potential collision paths; Control your speed to avoid conflicts with smaller participants.

9. The unmanned vehicle control method based on deep learning according to claim 1, characterized in that: The S4 specifically includes: S41, uncertainty assessment: After receiving the prediction results of the weak participant behavior output by the spatiotemporal graph convolutional neural network, the confidence of the prediction results is evaluated through the uncertainty quantification module. If the confidence is lower than the preset threshold τ or a high-risk behavior is detected, it is considered that the current scene has a high degree of uncertainty or potential risk; S42, triggering of emergency avoidance mechanism: after detecting high uncertainty or potential risk, the emergency avoidance mechanism is automatically triggered to enter the high-priority risk assessment process; S43, Dynamic risk level assessment: By comprehensively analyzing the real-time data of the current traffic environment, including vehicle speed, road conditions, the location and speed of surrounding vehicles and pedestrians, and weather conditions, the risk level R(t) of the current environment is dynamically assessed. The assessment process is based on the following risk assessment formula: Among them, x i Represents the value of each risk factor, β i is the weight coefficient of each risk factor, λ i It is a parameter used to adjust the impact of risk factors on the overall risk level. R(t) is the calculated risk level. The higher the value, the greater the risk. S44, optimal avoidance path generation: Based on the current risk level R(t) and the relative position of the vehicle and surrounding obstacles, an optimal avoidance path is generated through a path planning algorithm to ensure that the vehicle can avoid the risk area with minimal steering or acceleration.

10. The unmanned vehicle control method based on deep learning according to claim 9, characterized in that: The path planning is expressed as: in, represents the set of all possible avoidance paths, c j represents the risk cost of each node on the path, Represents the distance from the current node to the target node. According to the generated optimal avoidance path, the vehicle is instructed to perform emergency operations.

Images