Automatic driving path planning method and system based on large model analysis

Abstract

The application discloses an automatic driving path planning method and system based on large model analysis, relates to the technical field of path planning control, and comprises the following steps: acquiring motion state data of a vehicle and obtaining multi-modal perception data by perceiving a surrounding environment; jointly analyzing semantics of the two types of data; and constructing a heterogeneous graph neural network according to an analysis result. In combination with a social influence theory analysis network, a probabilistic prediction trajectory and a behavior intention are obtained. A driving route is adjusted to complete dynamic obstacle avoidance, a target planning path is generated, and finally, an instruction is issued based on the path to guide the vehicle to realize automatic driving. The application solves the technical problems that a traditional automatic driving path planning method is difficult to consider mutual influences and behavior changes of road participants, risk determination is not objective enough, and driving deviations cannot be corrected in time, and achieves the technical effects that driving trends are predicted in combination with interactive laws of road participants, driving routes are flexibly adjusted in combination with a judgment result, and driving states are corrected in real time.

Description

Technical Field

[0001] This invention relates to the field of path planning and control technology, and in particular to an autonomous driving path planning method and system based on large model analysis. Background Technology

[0002] Vehicle trajectory and position control are widely used in autonomous driving. Their implementation heavily relies on the processing and analysis of multi-source environmental and vehicle operation data. The control effect directly impacts driving safety, traffic efficiency, and vehicle stability. Accurate path planning data processing capabilities are crucial for ensuring the reliable operation of autonomous driving. Current path planning methods primarily employ conventional perception and modeling for data computation, outputting basic trajectory and position control data in scenarios with simple road conditions and stable traffic flow. However, real-world road environments are complex, with dynamic interactions among traffic participants, and massive volumes of multi-source heterogeneous perception data with complex spatiotemporal relationships. Traditional data processing methods are increasingly showing their limitations. Traditional data processing solutions cannot perform deep joint analysis of multimodal perception data, making it difficult to accurately uncover the correlation characteristics and behavioral intentions of different traffic participants. They cannot achieve real-time path adjustment driven by dynamic data across the entire domain. The resulting motion-related data processing results are incomplete and lack precision, leading to poor vehicle trajectory and position control performance and failing to meet the actual data processing needs for precise driving and safety management in complex road conditions. Summary of the Invention

[0003] This application provides an autonomous driving path planning method and system based on large model analysis, which solves the technical problems of traditional autonomous driving path planning, which makes it difficult to consider the mutual influence and behavioral changes of traffic participants on the road, the risk assessment is not objective enough, and the driving deviation cannot be corrected in a timely manner.

[0004] The first aspect of this application provides an autonomous driving path planning method based on large model analysis. The method includes: acquiring target motion state data of a target autonomous vehicle and perceiving the surrounding environment of the target autonomous vehicle to obtain target multimodal perception data; performing joint semantic parsing on the target motion state data and the target multimodal perception data to obtain a target parsing result, and constructing a heterogeneous graph neural network based on the target parsing result; introducing social influence theory to analyze the heterogeneous graph neural network to obtain multimodal probabilistic predicted trajectories and behavioral intentions; dynamically adjusting the predetermined autonomous driving route of the target autonomous vehicle using the multimodal probabilistic predicted trajectories and behavioral intentions as constraints to obtain a target planned path; and guiding the target autonomous vehicle to perform autonomous driving execution using the target planned path as the underlying control command.

[0005] A second aspect of this application provides an autonomous driving path planning system based on large model analysis. The system includes: a multimodal perception data acquisition module, used to acquire target motion state data of a target autonomous driving vehicle and perceive the surrounding environment of the target autonomous driving vehicle to obtain target multimodal perception data; a heterogeneous graph neural network construction module, used to perform joint semantic parsing of the target motion state data and the target multimodal perception data to obtain target parsing results, and construct a heterogeneous graph neural network based on the target parsing results; a predicted trajectory and intent acquisition module, used to introduce social influence theory to analyze the heterogeneous graph neural network to obtain multimodal probabilistic predicted trajectories and behavioral intentions; a target planning path acquisition module, used to dynamically adjust the predetermined autonomous driving route of the target autonomous driving vehicle using the multimodal probabilistic predicted trajectory and behavioral intentions as constraints to obtain a target planned path; and a low-level control command execution module, used to guide the target autonomous driving vehicle to perform autonomous driving execution using the target planned path as the low-level control command.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application collects motion state data of autonomous vehicles and surrounding multimodal perception data. Through joint semantic parsing, construction of heterogeneous graph neural networks, and analysis combined with social impact theory, it obtains trajectory and behavioral intention information of traffic participants, judges the motion status of surrounding entities, and uses the prediction results as constraints to adjust the route, thereby completing dynamic path planning and driving guidance for autonomous vehicles. This makes the vehicle trajectory and position control more accurate and stable, achieving the technical effect of predicting driving trends based on the interaction patterns of road participants, flexibly adjusting driving routes based on the evaluation results, and correcting driving status in real time. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a flowchart illustrating the autonomous driving path planning method based on large model analysis provided in the embodiments of this application.

[0009] Figure 2 This is a schematic diagram of the structure of an autonomous driving path planning system based on large model analysis provided in an embodiment of this application.

[0010] Figure labeling: 1. Multimodal perception data acquisition module; 2. Heterogeneous graph neural network construction module; 3. Predicted trajectory and intent acquisition module; 4. Target planning path acquisition module; 5. Low-level control command execution module. Detailed Implementation

[0011] This application provides an autonomous driving path planning method and system based on large model analysis, which solves the technical problems of traditional autonomous driving path planning, which makes it difficult to consider the mutual influence and behavioral changes of traffic participants on the road, the risk assessment is not objective enough, and the driving deviation cannot be corrected in a timely manner.

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0014] Example 1, as Figure 1 As shown, an autonomous driving path planning method based on large model analysis is described, wherein the method includes: The target motion state data of the target autonomous vehicle is acquired, and the surrounding environment of the target autonomous vehicle is perceived to obtain target multimodal perception data.

[0015] Specifically, the target motion state data of the autonomous vehicle is collected using conventional onboard sensing equipment: For example, a global positioning module (GPS) and an inertial measurement unit (IMU) are integrated and arranged in the vehicle's central control area, while wheel speed sensors are installed at the four wheels. During operation, the GPS module continuously operates at a sampling frequency of 10Hz to 50Hz, combining satellite signals with local data to calculate and output the vehicle's current position coordinates in real time. The IMU simultaneously collects vehicle attitude change information and calculates the vehicle's heading angle, with an accuracy of no more than 0.1°. The wheel speed sensors detect the wheel rotation frequency in real time and calculate the vehicle's real-time speed using preset wheel circumference parameters. These three types of target motion state data are simultaneously transmitted to the onboard computing unit, completing the acquisition of the vehicle's own operational data.

[0016] Image data of the vehicle's surrounding environment is collected by onboard high-definition vision cameras, following the conventional deployment method of autonomous driving perception equipment. For example, vision cameras are installed at the front, left and right sides, and rear of the vehicle, with each group of cameras working together to form a surround-view visual monitoring field of view. The vision cameras continuously capture images at a frame rate of 15fps to 30fps, covering the road area, traffic facilities, and stationary objects along the vehicle's route. The raw image data is transmitted back to the onboard computing unit in real time for temporary storage, thereby obtaining image information that reflects the static environment of the road.

[0017] Environmental point cloud data is collected by a vehicle-mounted LiDAR system. A multi-line LiDAR device can be used, fixedly installed on the vehicle's roof and outer front. The device scans the 360° space around the vehicle at a constant rotation speed. The LiDAR emits a detection beam and receives the echo signals reflected from the targets. Using a time-of-flight algorithm, it calculates the spatial distance and 3D contours of surrounding objects, generating continuous point cloud data. This data can accurately record the shape and spatial position of various dynamic traffic participants on the road, and the sampling density can be flexibly adjusted according to the actual monitoring scenario.

[0018] Road surface condition data can be directly acquired using road surface condition sensors or indirectly estimated using vision cameras or LiDAR. As a specific implementation, road surface condition sensors can be optical or ultrasonic non-contact sensors, installed on the vehicle chassis near the wheels. They use non-contact detection to perceive actual road conditions, identifying real-time information such as road surface wetness and smoothness, and then using built-in computational logic to calculate the road adhesion coefficient. Regardless of the acquisition method, the sampling frequency of road surface condition data is consistent with that of the vision camera and LiDAR, ensuring that road surface condition data is output synchronously with other environmental perception data, fully reflecting the actual conditions of the road surface on which the vehicle is traveling.

[0019] All sensing devices are connected to the vehicle's onboard controller local area network bus or onboard Ethernet bus. The system uses the second pulse signal of the Global Positioning System as a unified time reference and adds a global synchronization timestamp to each sensing data. This ensures that the vehicle motion status data and various environmental perception data are strictly consistent in the timing of data collection, effectively avoiding information deviation problems caused by asynchronous data collection from different devices.

[0020] By relying on vehicle-mounted sensing devices and optional alternative sensing methods to achieve synchronous acquisition of information across the entire domain, a comprehensive and time-consistent raw data foundation can be provided for subsequent multimodal data analysis and processing.

[0021] Joint semantic parsing is performed on the target motion state data and the target multimodal perception data to obtain the target parsing result, and a heterogeneous graph neural network is constructed based on the target parsing result.

[0022] In this embodiment, the heterogeneous graph neural network is a deep learning model that can handle multiple types of nodes and diverse connection relationships, and can uncover complex correlation features such as space and speed among dynamic traffic participants in the road.

[0023] Optionally, the image data is first semantically segmented, the point cloud data is dynamically detected and tracked for traffic participants, the road surface state data is estimated for the adhesion coefficient, and the vehicle motion state data is extracted to form four semantic layers. Then, the timestamps of each semantic layer are aligned to complete the joint semantic parsing and obtain the target parsing result. The specific implementation process of this step will be described in detail later.

[0024] Using dynamic traffic participants as nodes and spatial distance and speed difference between participants as edge attributes, a location graph, a speed graph, and a direction graph are constructed. The features of the three graphs are spliced ​​together to obtain an initial graph structure. The initial graph structure is then input into a graph convolutional network to extract node features, thereby obtaining a heterogeneous graph neural network. The specific implementation process of this step will also be described in detail later.

[0025] By introducing social influence theory to analyze the heterogeneous graph neural network, we can obtain multimodal probabilistic prediction trajectories and behavioral intentions.

[0026] In this embodiment of the application, social influence theory is a theory used to measure the strength of interaction between individuals. In the autonomous driving scenario, the degree of interaction between dynamic traffic participants on the road can be quantified by relying on relative distance, meeting time, and the angle between the directions of movement.

[0027] In one embodiment of this application, the first and second participants in the dynamic traffic participants are extracted. Based on social influence theory, the corresponding dimensions are quantified by combining relative distance, meeting time, and the angle between the directions of movement, and the interaction weights of the two are calculated. The interaction weights are input into a graph attention network to complete feature aggregation and update to obtain the interaction feature vector. Then, the historical trajectory sequences of the two participants are jointly decoded to obtain their respective probabilistic predicted trajectories and behavioral intentions. After integration, a multimodal probabilistic predicted trajectory and behavioral intention are obtained. The specific implementation process of this step will be described in detail later.

[0028] The predetermined autonomous driving route of the target autonomous vehicle is dynamically adjusted for obstacle avoidance based on the multimodal probabilistic predicted trajectory and behavioral intent, thereby obtaining the target planned path.

[0029] In this embodiment of the application, the predetermined autonomous driving route is a complete path trajectory and passing points pre-planned by the autonomous vehicle for initial reference driving.

[0030] Specifically, using multimodal probabilistic predicted trajectories and behavioral intentions as constraints, a comprehensive threat coefficient is calculated by weighting the relative distance, relative speed, and angle of motion between dynamic traffic participants and autonomous vehicles. At the same time, the corresponding threat coefficient is matched according to the category attributes of static obstacles. The two types of coefficients are used as weights of the predetermined safe distance constraint to adjust the predetermined autonomous driving route, complete dynamic obstacle avoidance, and obtain the target planning path. The specific implementation process of this step will also be described in detail later.

[0031] The target autonomous driving vehicle is guided to perform autonomous driving execution using the target planned path as the underlying control command.

[0032] In this embodiment, the underlying control commands are derived from path information and are directly used to control the specific actions of the vehicle's steering, acceleration, and braking components.

[0033] Specifically, a target control sequence containing steering angle, acceleration, and braking commands is generated based on the target planning path. This sequence is then transmitted via the vehicle controller local area network bus to the autonomous driving execution component, which consists of the steering actuator, drive motor, and brake, to guide the vehicle to complete autonomous driving according to the target planning path. The specific implementation process of this step will also be described in detail later.

[0034] Furthermore, the method provided in this application embodiment includes: The target motion state data includes vehicle speed, heading angle, and position coordinates, and the target multimodal perception data includes image data, point cloud data, and road surface state data.

[0035] Specifically, target motion state data is used to characterize the real-time operating state of the autonomous vehicle, including three core parameters: vehicle speed, heading angle, and position coordinates. Vehicle speed refers to the distance traveled by the vehicle per unit time, used to quantify the vehicle's speed and providing fundamental data for subsequent calculations of the relative speed between the vehicle and surrounding road users, and for assessing collision risk. Heading angle is the angle between the vehicle's direction of travel (i.e., the direction its longitudinal axis points) and geographic north, used to accurately describe the vehicle's direction of travel, providing directional basis for judging the vehicle's travel trend and planning steering actions. Position coordinates refer to the vehicle's spatial position information in a global geographic coordinate system, such as planar coordinates under the UTM coordinate system or the Gauss-Kruger projection coordinate system, used to determine the vehicle's specific position on the road, providing a positioning reference for path planning and spatial distance calculation.

[0036] Multimodal perception data is used to comprehensively perceive the road environment information surrounding the vehicle, including three core parameters: image data, point cloud data, and road surface condition data. Image data refers to the two-dimensional pixel information of the road scene collected by the vehicle-mounted vision camera, which can clearly present the texture, color, and contour features of the road, providing visual basis for identifying road boundaries, lane lines, traffic signs, and static obstacles. Point cloud data refers to the set of three-dimensional spatial coordinates of the road scene collected by the vehicle-mounted LiDAR, which can accurately reflect the three-dimensional shape and spatial position of objects, providing three-dimensional data support for detecting and tracking dynamic traffic participants and constructing spatial topological relationships. Road surface condition data refers to the road surface condition information collected by the road surface condition sensor, which can reflect the dryness / wetness, smoothness, and friction characteristics of the road surface, providing basic data for estimating the road surface adhesion coefficient and assessing the road safety conditions for vehicle operation.

[0037] Furthermore, the method provided in this application embodiment includes: The image data is semantically segmented to obtain a first semantic layer, consisting of road boundaries, lane lines, traffic signs, and static obstacles. Dynamic traffic participants in the point cloud data are detected and tracked to obtain arbitrary category identifiers, arbitrary 3D bounding boxes, and arbitrary historical trajectory sequences for any participant, forming a second semantic layer. The road surface state data is subjected to adhesion coefficient estimation to obtain a third semantic layer. The vehicle speed, heading angle, and position coordinates are used as a fourth semantic layer. The first, second, third, and fourth semantic layers are timestamped to obtain the target parsing result.

[0038] In this embodiment, static obstacles are various objects that are fixed in position and cannot move autonomously within the autonomous driving road and its surrounding area. Dynamic traffic participants are entities such as vehicles, non-motorized vehicles, and pedestrians that can move autonomously in the road scene. Any participant refers to each individual dynamic traffic participant detected and identified.

[0039] Optionally, semantic segmentation can be performed on the image data first, for example, using the U-Net semantic segmentation model to complete pixel-level classification tasks. The U-Net model adopts an encoder-decoder symmetric architecture. The encoder part contains four downsampling modules, each consisting of two 3×3 convolutional layers, one ReLU activation function layer, and one 2×2 max pooling layer. Each downsampling reduces the feature map size to half its original size and doubles the number of channels. The decoder part contains four upsampling modules, each consisting of one 2×2 deconvolutional layer, one skip connection layer, and two 3×3 convolutional layers. The skip connection layer concatenates the feature map of the corresponding layer of the encoder with the upsampled feature map of the decoder in terms of channel dimension. Finally, a 1×1 convolutional layer maps the number of channels of the feature map to the number of categories, outputting a semantic segmentation map with the same size as the input image. The model input is an RGB three-channel image resized to 512×512, and the output is a 512×512 single-channel semantic segmentation map. Each pixel value corresponds to a category label from 0 to 5, representing background, road boundary, lane lines, traffic signs, and static obstacles, respectively. It should be understood that other semantic segmentation networks such as DeepLab and PSPNet can also be used instead of U-Net; this embodiment is only for illustrative purposes.

[0040] The training process of the U-Net model was performed using the publicly available Cityscapes autonomous driving scene dataset. First, the dataset was preprocessed by uniformly cropping the original images to 512×512 pixels and performing class mapping on the labeled images, merging the 19 categories in the original annotations into the aforementioned 5 target categories. During training, data augmentation methods such as random horizontal flipping, random brightness adjustment, and random cropping were used to enhance the model's generalization ability. The model used the Adam optimizer for parameter updates, with an initial learning rate of 0.0001, a batch size of 8, and the cross-entropy loss function. The model weights were saved every 100 iterations during training. Training was stopped when the average intersection-union of the validation set improved by less than 0.001 over 10 consecutive iterations, or when the total number of iterations reached 10,000, resulting in the final pre-trained model. Next, the original images captured by the aforementioned vehicle-mounted vision camera are resized to 512×512 using bilinear interpolation and then input into the pre-trained U-Net model. The model outputs a semantic segmentation map, extracting pixel coordinates and category information for road boundaries, lane lines, traffic signs, and static obstacles based on pixel category labels. Static obstacles refer to objects in the road environment whose positions remain unchanged, including road guardrails, traffic cones, parked vehicles, roadside buildings, trees, and traffic signs. The information for each type of target is then organized according to a unified data format to obtain the first semantic layer.

[0041] Next, the point cloud data is preprocessed. First, a random sample consensus algorithm is used for ground point cloud segmentation. Three non-collinear points are randomly selected from the original point cloud, and a plane equation is fitted. The vertical distance from all points to this plane is calculated, and points with a distance less than 0.2m are marked as inliers, and the number of inliers is counted. This process is repeated 100 times, and the plane with the most inliers is selected as the optimal ground plane. All inliers are removed as ground point clouds, and non-ground point clouds are retained. Then, a statistical filtering algorithm is used to remove outlier noise points. The average distance from each non-ground point to its 20 nearest neighbors is calculated, and the mean and standard deviation of all average distances are calculated. Points with an average distance greater than the mean plus twice the standard deviation are marked as outliers and removed, resulting in the effective target point cloud.

[0042] Dynamic traffic participants are detected in the effective target point cloud using a Euclidean clustering algorithm. The clustering distance threshold is set to 0.5m, the minimum number of points in a cluster is 10, and the maximum number of points in a cluster is 10000. An unlabeled point is selected from the effective target point cloud as a seed point. All points with an Euclidean distance less than 0.5m from the seed point are grouped into the same cluster. This process is repeated until all points are labeled, with each cluster corresponding to an independent target. Dynamic traffic participants refer to objects capable of autonomous movement in the road environment, including moving motor vehicles, riding non-motorized vehicles, and walking pedestrians; any participant refers to each individual among all detected dynamic traffic participants. Then, the minimum bounding box of each cluster is calculated to obtain the spatial coordinates and size parameters of the three-dimensional bounding box. The clusters are classified according to their size and shape characteristics. For example, clusters with a length greater than 3m and a width greater than 1.5m are labeled as cars, clusters with a length less than 2m and a width less than 1m are labeled as pedestrians, and the remaining clusters are labeled as non-motorized vehicles, thus obtaining the category identifier of each participant.

[0043] Furthermore, the detected dynamic traffic participants are tracked using a Kalman filter algorithm, with a uniform motion model employed. A Kalman filter is initialized for each newly detected participant, and the state vector is set to... , representing the target's 3D position coordinates and 3D velocity components, respectively. The state transition matrix is ​​a 6×6 identity matrix, where the corresponding position elements of the position and velocity components are set to the sampling time interval Δt. The observation matrix is ​​a 3×6 matrix, retaining only the observation relationship of the position components, with the corresponding elements of the velocity components set to 0. In the prediction step, the state vector and covariance matrix of the current time step are predicted based on the state vector and state transition matrix of the previous time step. In the update step, the center coordinates of the 3D bounding box of the target detected at the current time step are used as observations, the Kalman gain is calculated, and the state vector and covariance matrix are updated. The Hungarian algorithm is used to associate the target detected at the current time step with the tracking trajectory of the previous time step. Trajectories that are successfully associated execute the update step, while unassociated targets initialize a new tracking trajectory. Trajectories that are not associated with the detected target for 5 consecutive frames are terminated. The center coordinates of the 3D bounding box of each tracking trajectory in multiple consecutive time frames are recorded to form a historical trajectory sequence of 5 to 20 frames in length. Finally, the category identifier, 3D bounding box, and historical trajectory sequence of each participant are integrated to obtain the second semantic layer.

[0044] Then, the road surface condition data is processed to estimate the adhesion coefficient, which can be achieved using a pre-defined mapping table method. Specifically: the road surface wetness / dampness level and road surface smoothness values ​​output from the road surface condition sensor are received. The road surface wetness / dampness level is divided into four levels: dry, wet, waterlogged, and icy. The road surface smoothness value ranges from 0 to 10, with a higher value indicating a more uneven road surface. The pre-calibrated mapping table between road surface conditions and adhesion coefficients, obtained through road surface experiments, is consulted to obtain the road surface adhesion coefficient value corresponding to the current road surface wetness / dampness level and smoothness. The estimated road surface adhesion coefficient is stored in a unified format, resulting in the third semantic layer.

[0045] Next, the target motion state data of the target autonomous vehicle is extracted, and the three parameters of vehicle speed, heading angle and position coordinates are directly standardized. The unit of vehicle speed is unified to m / s, the unit of heading angle is unified to degrees, and the unit of position coordinates is unified to UTM coordinate system. The precision of all parameters is retained to two decimal places, thus obtaining the fourth semantic layer.

[0046] Finally, timestamp alignment is performed on the four semantic layers. The example uses a time synchronization method based on the second pulse signal from the Global Positioning System (GPS). A corresponding timestamp is added to each data unit in the first, second, third, and fourth semantic layers, with a timestamp precision of 1ms. Using the timestamp of the first semantic layer as a reference, data units with a time difference of no more than 20ms are sequentially searched in the other three semantic layers, and the data from the four semantic layers corresponding to the same time point are matched and integrated. For data with time discrepancies, linear interpolation is used for data completion. The specific calculation logic is as follows: for two adjacent time points... and Corresponding known data values and The interpolation result corresponding to the time point t to be completed Ensure that all semantic layer data corresponds one-to-one in the time dimension to ultimately obtain the target parsing result.

[0047] By extracting semantic features from multimodal data in a hierarchical manner and aligning them with timestamps, heterogeneous raw data is transformed into structured, time-series unified semantic information, providing an accurate and reliable feature foundation for the subsequent construction of heterogeneous graph neural networks and traffic behavior analysis.

[0048] Furthermore, the method provided in this application embodiment includes: Using the dynamic traffic participants as nodes and the spatial distance and speed difference between them as edge attributes, a location graph, a speed graph, and a direction graph are constructed sequentially. The location graph, the speed graph, and the direction graph are then concatenated to obtain an initial graph structure. The initial graph structure is then input into a graph convolutional network for node feature extraction to obtain the heterogeneous graph neural network.

[0049] As a specific implementation, the information of all dynamic traffic participants is first extracted from the second semantic layer of the target parsing result, and each independent dynamic traffic participant is defined as a node in the heterogeneous graph neural network. The initial feature vector of each node contains 6 dimensions, namely the participant's category code, the x-coordinate of the center of the 3D bounding box, the y-coordinate of the center of the 3D bounding box, the z-coordinate of the center of the 3D bounding box, the driving speed, and the heading angle. The category code adopts a one-hot encoding method. For example, cars correspond to [1,0,0], non-motorized vehicles correspond to [0,1,0], and pedestrians correspond to [0,0,1]. All node features are normalized: the coordinate values ​​(x,y,z) are mapped to the interval of 0 to 1, specifically by dividing by the maximum value of the perception range, such as 100m; the speed values ​​are mapped to the interval of 0 to 1, specifically by dividing by the maximum expected speed, such as 40m / s; the heading angle ranges from 0° to 360°, and the normalized value = heading angle / 180-1, is mapped to the interval of -1 to 1, resulting in a standardized node feature matrix. When calculating the angle between the directions of movement of traffic participants, the difference between any two heading angles is taken as the smallest angle, so the range of this difference θ is 0° to 180°.

[0050] A location graph is constructed to represent the spatial relationships between dynamic traffic participants. All node pairs are traversed, and the Euclidean distance between the centers of the 3D bounding boxes of any two nodes is calculated. When the Euclidean distance between two nodes is less than 50m, an undirected edge is established between them. The edge attribute is the normalized spatial distance, calculated by dividing the actual distance by 50 to obtain a value between 0 and 1. All edge attributes form the adjacency matrix of the location graph, which, combined with the standardized node feature matrix, forms the complete location graph structure.

[0051] A velocity graph is constructed to represent the speed differences between dynamic traffic participants. All node pairs are traversed, and the absolute difference in speed between any two nodes is calculated. When the absolute speed difference between two nodes is less than the example threshold of 20 m / s, an undirected edge is created between these two nodes. It should be understood that this threshold can be preset and adjusted according to road types such as urban roads and highways; 20 m / s is only an example here. The edge attribute is the normalized speed difference, calculated by dividing the actual speed difference by 20 to obtain a value between 0 and 1. All edge attributes form the adjacency matrix of the velocity graph, which, combined with the standardized node feature matrix, forms the complete velocity graph structure.

[0052] A direction graph is constructed to represent the consistency of travel directions among dynamic traffic participants. All node pairs are traversed, and the cosine similarity of the heading angles between any two nodes is calculated. When the cosine similarity of the heading angles between two nodes is greater than 0, an undirected edge is established between these two nodes. The edge attribute is the cosine similarity of the heading angles, with a value ranging from 0 to 1; a larger value indicates a more consistent travel direction between the two participants. All edge attributes form the adjacency matrix of the direction graph, which, combined with the standardized node feature matrix, forms the complete direction graph structure.

[0053] Next, the location map, velocity map, and direction map are concatenated using a channel-dimensional concatenation method. The node feature matrices of the three maps are all N×6, where N is the number of dynamic traffic participants. The three node feature matrices are then concatenated along the channel dimension to obtain a concatenated node feature matrix of dimension N×18. Simultaneously, the adjacency matrices of the three maps are summed element-wise to obtain a concatenated adjacency matrix of dimension N×N. The concatenation process uses an equal-weighting method, with each of the three map features having a weight of 1, requiring no additional weighting. The concatenated node feature matrix and the concatenated adjacency matrix together form the initial graph structure.

[0054] Furthermore, the initial graph structure is input into a graph convolutional network for node feature extraction. This graph convolutional network employs a two-layer graph convolutional architecture. The first graph convolutional layer has an input dimension of 18 and an output dimension of 64, using the ReLU activation function. The second graph convolutional layer has an input dimension of 64 and an output dimension of 32, also using the ReLU activation function. The graph convolutional layers use mean aggregation, with the basic calculation formula as follows: ,in, and They represent the first Layer and first The node feature matrix of the layer, where A is the concatenated adjacency matrix and D is the degree matrix of A. For the first The trainable weight matrix of the layer, The activation function is ReLU. The model input is the spliced ​​node feature matrix and spliced ​​adjacency matrix of the initial graph structure, and the output is a high-dimensional node feature matrix of dimension N×32, where each row vector corresponds to the interaction feature vector of a dynamic traffic participant.

[0055] The aforementioned graph convolutional network is not trained independently, but rather jointly trained end-to-end with the graph attention network and trajectory decoder in subsequent steps. During joint training, the total loss function consists of three parts: trajectory prediction loss, behavior intent classification loss, and optional graph structure regularization loss. As a specific implementation, the trajectory prediction loss uses the negative log-likelihood loss function, i.e., the negative log-likelihood between the predicted trajectory distribution and the true trajectory; the behavior intent classification loss uses the cross-entropy loss function; the joint loss function is expressed as: ,in and To balance the weights, 1.0 and 0.5 are used for example. Training utilizes publicly available autonomous driving scenario datasets, such as nuScenes or Waymo Open Dataset. These datasets contain complete annotations of dynamic traffic participants, including cars, non-motorized vehicles, and pedestrians, as well as necessary information such as 3D bounding boxes, speeds, heading angles, and trajectories. During training, random occlusion and random translation data augmentation methods are employed to enhance the model's generalization ability. The model uses the Adam optimizer for parameter updates, with an initial learning rate of 0.001 and a batch size of 32. During training, a weight decay coefficient of 0.0001 is used, and the learning rate decays to 0.1 every 20 epochs. Training stops when the joint loss value on the validation set decreases by less than 0.001 for 10 consecutive epochs, or when the total number of training epochs reaches 100, resulting in the final end-to-end model. In the inference phase, the target resolution results are input into this end-to-end model, and the graph convolutional network outputs a high-dimensional node feature matrix, thus completing the construction of a heterogeneous graph neural network and obtaining multi-dimensional interaction node features of each dynamic traffic participant.

[0056] By constructing a heterogeneous graph neural network that includes three relationships: position, speed, and direction, and extracting node features, we can comprehensively explore the multi-dimensional interaction relationships between dynamic traffic participants, providing accurate feature support for subsequent trajectory prediction and behavioral intention analysis.

[0057] Furthermore, the method provided in this application embodiment includes: The first and second participants in the dynamic traffic participants are extracted. Based on the social influence theory, the intensity dimension, immediacy dimension, and quantity dimension are quantified by relative distance, meeting time, and angle of movement direction, respectively, and the interaction weights of the first and second participants are obtained through standardized analysis. The interaction weights are input into a graph attention network for feature aggregation and update to obtain an interaction feature vector. According to the interaction feature vector, the first historical trajectory sequence of the first participant and the second historical trajectory sequence of the second participant are jointly decoded to obtain the first probabilistic predicted trajectory and behavioral intention of the first participant and the second probabilistic predicted trajectory and behavioral intention of the second participant, respectively. The first probabilistic predicted trajectory and behavioral intention, and the second probabilistic predicted trajectory and behavioral intention constitute the multimodal probabilistic predicted trajectory and behavioral intention.

[0058] As a specific implementation, information on all dynamic traffic participants is extracted from the node features output by the heterogeneous graph neural network. The target autonomous vehicle is defined as the first participant, and all other dynamic traffic participants are sequentially defined as second participants. All combinations of first and second participants are iterated over, and the interaction weights between each group of participants are calculated.

[0059] The intensity dimension value is calculated based on social influence theory, corresponding to the relative distance between participants. The Euclidean distance *d* between the centers of the three-dimensional bounding boxes of the first and second participants is calculated, and the intensity dimension value is calculated using a negative exponential mapping formula. ,in This is the distance scale parameter, with a value of 20m. When d is greater than 100m, the intensity dimension value is directly set to 0, indicating that there is no significant interaction between the two.

[0060] The immediacy dimension value is calculated based on social influence theory. The immediacy dimension corresponds to the meeting time between participants. Based on the current positions and velocity vectors of the first and second participants, calculate the meeting time τ while they remain in their current motion states. The specific calculation formula is as follows: τ is the relative position vector of the two participants, and v is the relative velocity vector of the two participants. When |v| < 0.01 m / s, τ takes the value of infinity, i.e., the instantaneous dimension value I = 0. When the two participants move in opposite directions and there is a possibility of collision, It is a positive number; when the two are moving in the same direction or there is no possibility of collision, Take infinity. Calculate the instantaneous dimension value using the negative exponential mapping formula. ,in This is a time scale parameter, with a value of 5 seconds. When... When the time exceeds 20 seconds, the real-time dimension value is directly set to 0.

[0061] The quantitative dimension value is calculated based on social influence theory, corresponding to the angle between the movement directions of the participants. The difference θ between the heading angles of the first and second participants is calculated, with θ ranging from 0° to 180°. The quantitative dimension value is calculated using the direction consistency mapping formula. Where, when θ=0° This indicates that the two are moving in the same direction; when θ = 180° The angle between the two directions indicates that they are completely opposite. The larger the value, the more consistent their directions of movement are, and the lower the spatial distribution density of their influence, which aligns with the physical meaning of traffic scenarios. In traffic scenarios, a smaller directional angle means a relatively simple interaction pattern, equivalent to a smaller number of influence sources in social influence theory; a larger angle increases the complexity of the interaction, equivalent to the combined effect of influence sources from multiple directions.

[0062] Next, the three dimensions are standardized and fused to obtain the interaction weights. Since the values ​​of the intensity, immediacy, and quantity dimensions all range from 0 to 1, no additional normalization is required. The interaction weights are then fused using an equal-weighted product method. The interaction weights of all second participants corresponding to the same first participant are normalized so that the sum of all interaction weights is 1, thus obtaining the final standardized interaction weights.

[0063] Next, the standardized interaction weights are input into a graph attention network for feature aggregation and updating. The graph attention network adopts a single-layer graph attention architecture. The model input is an N×32 node feature matrix output by the heterogeneous graph neural network, where N is the number of dynamic traffic participants. The graph attention network removes the conventional self-attention calculation module and directly uses the standardized interaction weights calculated above as attention coefficients. During feature aggregation, the new feature of each node is the weighted sum of its own feature and the features of all neighboring nodes; the weights are the corresponding interaction weights. After aggregation, a fully connected layer is used for feature transformation. The fully connected layer has an input dimension of 32 and an output dimension of 64, and the ReLU activation function is used. The model output is an N×64 interaction feature vector matrix, where each row vector corresponds to the interaction feature vector of a dynamic traffic participant.

[0064] Furthermore, the graph attention network is not trained alone, but is jointly trained end-to-end with the graph convolutional network from the previous steps and the subsequent trajectory decoder. The joint loss function is: ,in The negative log-likelihood loss is used for trajectory prediction. For the cross-entropy loss of behavioral intent classification, the balanced weights are taken as an example. , Training utilizes publicly available autonomous driving scenario datasets, such as nuScenes or Waymo Open Dataset. These datasets contain complete dynamic annotations of traffic participants, including cars, non-motorized vehicles, and pedestrians, along with necessary information such as 3D bounding boxes, speeds, heading angles, and trajectories. The optimizer is Adam, with an initial learning rate of 0.001 and a batch size of 32. During training, a weight decay coefficient of 0.0001 is used, and the learning rate decreases to 0.1 every 20 epochs. Training stops when the joint loss on the validation set decreases to less than 0.001 for 10 consecutive epochs or when the total number of epochs reaches 100. During the inference phase, the target resolution result is input into the end-to-end model, and the graph attention network outputs an interaction feature vector.

[0065] Finally, joint decoding is performed based on the interaction feature vectors, specifically using a two-layer LSTM decoder with a hidden layer dimension of 128. The interaction feature vector of the first participant is concatenated with the first historical trajectory sequence, which is a 10-frame sequence of position coordinates. The decoder outputs three possible future trajectories, each containing the position coordinates of the next 20 frames, along with the probability value corresponding to each trajectory, with the sum of all probability values ​​being 1. The decoder also outputs a classification result, namely the behavioral intent label, including four categories: straight ahead, left turn, right turn, and decelerate / stop, using one-hot encoding. For example, straight ahead corresponds to [1,0,0,0], left turn to [0,1,0,0], right turn to [0,0,1,0], and decelerate / stop to [0,0,0,1]. The same method is used to jointly decode the interaction feature vector of the second participant and the second historical trajectory sequence to obtain the second participant's second probabilistic predicted trajectory and behavioral intent. Finally, the probabilistic predicted trajectories and behavioral intents of all participants together constitute a multimodal probabilistic predicted trajectory and behavioral intent.

[0066] By mapping the three dimensions of social influence theory into computable physical quantities and using them as attention coefficients in graph attention networks, the intensity of interactive influence among dynamic traffic participants can be accurately quantified, significantly improving the accuracy and robustness of trajectory prediction and behavioral intention recognition.

[0067] Furthermore, the method provided in this application embodiment includes: The relative distance, relative speed, and angle of motion between any participant and the target autonomous vehicle are collected in real time, and a weighted comprehensive threat coefficient for any participant is calculated. Any obstacle is extracted from the static obstacles, and an arbitrary threat coefficient is matched based on the arbitrary category attribute of the arbitrary obstacle. The arbitrary comprehensive threat coefficient and the arbitrary threat coefficient are used as weight coefficients of a predetermined safe distance constraint to dynamically adjust the predetermined autonomous driving route for obstacle avoidance, thereby obtaining the target planned path.

[0068] Specifically, firstly, relative distance, relative speed, and the angle between the directions of motion are used as threat assessment indicators. The objective weights of each indicator are determined by the entropy weight method. Then, the comprehensive score is completed by combining the approximation ideal solution ranking method. In this way, the comprehensive threat coefficient of any participant can be calculated. The specific implementation process of this step will be described in detail later.

[0069] After calculating the arbitrary comprehensive threat coefficient for any participant using the entropy weighting method and the approximation ideal solution ranking method, all static obstacle information is extracted from the first semantic layer of the target parsing results, and each independent static obstacle is defined as an arbitrary obstacle. Examples of static obstacle category attributes include six categories: road guardrails, traffic cones, parked vehicles, roadside buildings, trees, and traffic signs. A mapping table between static obstacle categories and threat coefficients is pre-established. For example, road guardrails correspond to a threat coefficient of 0.3, traffic cones to 0.4, parked vehicles to 0.8, roadside buildings to 0.2, trees to 0.1, and traffic signs to 0.15. These threat coefficients are calibrated through real-vehicle road tests and can be adaptively adjusted according to different driving scenarios. The mapping table is queried based on the category attributes of the arbitrary obstacle to obtain the corresponding arbitrary threat coefficient. The arbitrary threat coefficient ranges from 0 to 1; a higher value indicates a higher threat level for the static obstacle.

[0070] Next, the predetermined safe distance constraints between the target autonomous vehicle and any participant and any obstacle are calculated. First, a basic safe distance is determined, which is proportional to the target autonomous vehicle's current speed. The calculation formula is: Basic safe distance = Current speed × 1.5, where the current speed is in m / s, the basic safe distance is in meters, and the coefficient 1.5 represents a preset safe time interval (in seconds), meaning the vehicle needs 1.5 seconds to react at the current speed. Those skilled in the art can adjust this according to actual conditions. An arbitrary comprehensive threat coefficient is used as the dynamic safe distance weighting coefficient, and an arbitrary threat coefficient is used as the static safe distance weighting coefficient. For any participant, the actual required safe distance = Basic safe distance × (1 + Arbitrary comprehensive threat coefficient). For any obstacle, the actual required safe distance = Basic safe distance × (1 + Arbitrary threat coefficient). Simultaneously, combining multimodal probabilistic predicted trajectories and behavioral intentions, when the minimum distance between any participant and the target autonomous vehicle in the probabilistic predicted trajectory is less than 80% of the current actual distance, it is determined that they will approach in the future. Their corresponding safe distance is then multiplied by a magnification factor of 1.2 to ensure sufficient reaction time.

[0071] Next, a dynamic window method is used to dynamically adjust the predetermined autonomous driving route for obstacle avoidance. Starting from the current position of the target autonomous vehicle and taking the next waypoint of the predetermined autonomous driving route as the temporary endpoint, multiple feasible speed combinations are generated in the speed space. For example, the speed space range is 0 to 20 m / s, with 0 to 15 m / s recommended for urban roads and 0 to 30 m / s recommended for highways, and a maximum acceleration of 2 m / s². 2 The maximum deceleration is 4 m / s². 2The maximum angular velocity is 0.5 rad / s. For each speed combination, the simulated vehicle's trajectory is calculated for the next second. The minimum distance between the simulated trajectory and any participant or obstacle is calculated to determine if the corresponding safe distance constraint is met. All feasible trajectories that meet the safe distance constraint are filtered out, and the optimal trajectory is selected. The evaluation function for the optimal trajectory is: 0.7 × trajectory deviation coefficient + 0.3 × speed deviation coefficient, with smaller values ​​indicating a better trajectory. Here, 0.7 and 0.3 are merely exemplary weights; in practical applications, those skilled in the art can adaptively adjust them according to the needs of biased path following or speed maintenance. The trajectory deviation coefficient is the Euclidean distance between the simulated trajectory endpoint and the temporary endpoint divided by the initial distance; the speed deviation coefficient is the absolute difference between the simulated speed and the current vehicle speed divided by the current vehicle speed. The path point sequence of the optimal trajectory is concatenated with the remaining path points of the predetermined autonomous driving route to obtain the final target planned path. The above process is repeated every 100 milliseconds to achieve real-time dynamic obstacle avoidance adjustment.

[0072] By using the comprehensive threat coefficient of dynamic traffic participants and the category threat coefficient of static obstacles as weighting coefficients for safe distance constraints, and combining multimodal predicted trajectories for dynamic route adjustment, differentiated avoidance of obstacles with different threat levels can be achieved, improving the smoothness of path planning and traffic efficiency while ensuring driving safety.

[0073] Furthermore, the method provided in this application embodiment includes: The relative distance, relative speed, and angle between the directions of motion are used as threat assessment indicators; the entropy weight method is used to determine the objective weight allocation of the threat assessment indicators; the threat assessment indicators are comprehensively scored using the approximation ideal solution ranking method, and the arbitrary comprehensive threat coefficient is calculated in combination with the objective weight allocation.

[0074] In one embodiment, the relative distance, relative speed, and angle of motion between any participant and the target autonomous vehicle are collected in real time. The relative speed is the magnitude of the relative speed between the participant and the target autonomous vehicle, and the angle of motion is the difference between the heading angles of the participant and the target autonomous vehicle, ranging from 0° to 180°. These three parameters are used as threat assessment indicators. The relative distance, relative speed, and angle of motion can all be directly extracted from the relative position vector and velocity vector calculated in the aforementioned steps of quantifying the intensity, immediacy, and quantity dimensions: the relative distance is d, the relative speed is the magnitude of the relative speed vector v |v|, and the angle of motion is θ. It should be noted that the encounter time used in the aforementioned steps to calculate the immediacy dimension... The time quantity is derived further from the above parameters, while in this step, the relative speed is directly used as an independent threat assessment indicator.

[0075] A raw data matrix is ​​then constructed, where rows correspond to all detected dynamic traffic participants at the current moment, and columns correspond to three threat assessment indicators. The raw data matrix is ​​then standardized using range standardization. For positive indicators (indicators with higher values ​​indicating greater threat), the standardized value is calculated as (raw value - minimum value) / (maximum value - minimum value). For negative indicators (indicators with lower values ​​indicating greater threat), the standardized value is calculated as (maximum value - raw value) / (maximum value - minimum value). Relative distance is a negative indicator, while relative speed and the angle between the direction of movement are positive indicators. When the maximum value of an indicator equals its minimum value among all participants, all standardized values ​​for that indicator are set to 0.5. After standardization, all indicators range from 0 to 1, resulting in the standardized data matrix.

[0076] Next, the entropy weight method is used to determine the objective weight allocation of the three threat assessment indicators. Specifically: when the number of dynamic traffic participants is less than 2, the entropy weight method is not applicable, and the weights of the three indicators are evenly distributed, i.e., each takes 1 / 3. When the number of dynamic traffic participants is not less than 2, the calculation is performed as follows: Calculate the proportion of the j-th indicator for the i-th participant in the standardized data matrix. The proportion value = standardized value of the j-th indicator for the i-th participant / sum of standardized values ​​of the j-th indicator for all participants. Calculate the entropy value of the j-th indicator. The entropy value = -k multiplied by the sum of the proportion values ​​of the j-th indicator for all participants multiplied by the natural logarithm of the proportion values, where k is a constant, taken as 1 divided by the number of participants (the natural logarithm). When the proportion value is 0, the result of multiplying the proportion value by the natural logarithm of the proportion value is 0. When all standardized values ​​of an indicator are the same, the entropy value of that indicator is 1, and the weight is 0. Calculate the difference coefficient of the j-th indicator. The difference coefficient = 1 - entropy value. Calculate the weight of the j-th indicator: weight value = difference coefficient of the j-th indicator / sum of difference coefficients of all indicators. The sum of the weight values ​​of the three indicators is 1, resulting in the final objective weight allocation.

[0077] Finally, the threat assessment indicators are comprehensively scored using the approximation-ideal-solution ranking method. First, a weighted standardized decision matrix is ​​constructed, multiplying each element of the standardized data matrix by the weight value of the corresponding indicator. Positive and negative ideal solutions are determined: a positive ideal solution is a vector composed of the maximum values ​​of each indicator in the weighted standardized decision matrix, and a negative ideal solution is a vector composed of the minimum values ​​of each indicator in the weighted standardized decision matrix. The Euclidean distance from each participant to the positive ideal solution and the Euclidean distance from each participant to the negative ideal solution are calculated. The relative proximity of each participant is calculated as: relative proximity = distance to the negative ideal solution / (distance to the positive ideal solution + distance to the negative ideal solution). The relative proximity ranges from 0 to 1, with a higher value indicating a higher threat level. The relative proximity is used as the arbitrary comprehensive threat coefficient for any participant, with the comprehensive threat coefficient ranging from 0 to 1, where 0 represents no threat and 1 represents the highest threat.

[0078] By combining the entropy weight method with the approximation ideal solution ranking method to calculate the comprehensive threat coefficient of dynamic traffic participants, the threat level of different participants to the target autonomous vehicle can be objectively quantified, avoiding the bias caused by subjective weight allocation, and providing an accurate and reliable decision-making basis for subsequent dynamic obstacle avoidance adjustments.

[0079] Furthermore, the method provided in this application embodiment includes: The target control sequence of the target autonomous vehicle is generated based on the underlying control instructions; the target control sequence is sent to the target autonomous driving execution component via the vehicle controller local area network bus; the target autonomous driving execution component guides the target autonomous vehicle to perform autonomous driving according to the target planned path; wherein, the target control sequence includes steering angle instructions, acceleration instructions, and braking instructions; wherein, the target autonomous driving execution component includes a steering actuator, a drive motor, and a brake.

[0080] In this embodiment, the target control sequence is a set of continuous signals arranged in chronological order by the autonomous driving system, containing specific action commands such as steering angle, acceleration, and braking, used to directly drive the vehicle execution components to complete driving control.

[0081] Optionally, the determined target planned path is obtained, and the path trajectory information is converted into the low-level control commands required for vehicle operation. First, the coordinate deviation between the vehicle's real-time position and the target planned path is compared point-by-point. Simultaneously, the difference between the vehicle's current heading angle and the path direction, as well as the difference between the current vehicle speed and the preset speed for the path, are detected. A pure tracking algorithm is used to calculate the steering angle command, and the wheel deflection angle is determined based on the vehicle's wheelbase and the pre-aiming distance. The base value of the pre-aiming distance is set to 5 meters (corresponding to a vehicle speed of 0 m / s), and it is linearly adjusted with the current vehicle speed. For every 1 m / s increase in speed, the pre-aiming distance increases by 0.5 meters. ,in The current vehicle speed (m / s) is used. A proportional-integral-derivative (PID) controller is employed to adjust the acceleration and braking command outputs, with a proportional coefficient of 0.8, an integral coefficient of 0.1, and a derivative coefficient of 0.05. The command output intensity is adjusted in real time based on the difference between the current vehicle speed and the preset speed for the path. The steering angle value is adjusted synchronously with the magnitude of the heading deviation. The acceleration command output intensity corresponds to the vehicle speed increase requirement, and the braking command content matches the required deceleration magnitude. The three types of commands are sequentially arranged and integrated according to the vehicle's travel time to form a continuous and complete target control sequence. The target control sequence uniformly includes steering angle commands, acceleration commands, and braking commands. The target control sequence is also updated every 100 milliseconds to ensure real-time control.

[0082] The vehicle is equipped with an onboard controller local area network (ADC) bus as its internal signal transmission channel. The generated target control sequences undergo unified data encapsulation processing, and data frames are divided according to onboard communication specifications, with instruction types and receiving device identifiers clearly labeled. The bus communication rate is set to 500kbps, using a standard data frame format, with each data frame being 8 bytes long. After encapsulation, data signals are continuously pushed to the ADC bus. The bus, based on preset device addressing rules, transmits instructions for different functions to the corresponding target autonomous driving execution components. Steering angle instructions are sent to the steering actuator, acceleration instructions to the drive motor, and braking instructions to the brake. Each execution component maintains a real-time monitoring state, promptly receiving the corresponding instruction data transmitted from the bus.

[0083] Upon receiving their specific instructions, each target autonomous driving execution component immediately performs the corresponding action. Each component provides real-time feedback on its execution status. When the execution deviation exceeds a preset threshold (steering angle deviation exceeding 2 degrees, vehicle speed deviation exceeding 0.5 m / s), the control sequence regeneration process in this step is triggered. This involves recalculating the steering angle, acceleration, and braking commands. This process differs from the path replanning triggered by lateral following deviations or mileage in subsequent steps; it represents real-time correction at the execution layer. The steering actuator adjusts the wheel deflection angle according to the steering angle command, continuously correcting the vehicle's direction and reducing the deviation between the vehicle's position and the target planned path. The drive motor changes its power output torque according to the acceleration command, adjusting the vehicle speed to match the driving requirements of the target planned path. The brakes decelerate according to the braking command, smoothly reducing the vehicle speed to adapt to the path's driving conditions. These three types of execution components work together, continuously adjusting the vehicle's operating status to guide the target autonomous vehicle along the target planned path. The entire process operates cyclically according to a fixed period, achieving fully automated driving guidance.

[0084] By generating control sequences through path parsing and distributing instructions via the vehicle bus to drive the execution components, the vehicle's driving state can be precisely controlled, ensuring that the vehicle stably follows the target planned path and enabling the orderly implementation of autonomous driving actions.

[0085] Furthermore, the method provided in this application embodiment includes: The lateral following deviation of the target autonomous vehicle in the Frenet coordinate system is monitored in real time. When the lateral following deviation exceeds a preset allowable deviation threshold or the real-time driving mileage of the target autonomous vehicle exceeds a preset execution mileage threshold, path replanning is triggered. Based on the path replanning, the target planned path is dynamically planned and adjusted.

[0086] In one embodiment, during the driving of the target autonomous vehicle, real-time data from the vehicle's Global Positioning System (GPS) and Inertial Measurement Unit (INS) are collected to establish a Frenet coordinate system corresponding to the target planned path. The Frenet coordinate system uses the tangent direction of the target planned path as the vertical axis, the normal direction as the horizontal axis, and the origin as the point on the target planned path closest to the vehicle's center of mass. The vehicle's global coordinates are transformed to this Frenet coordinate system, and the lateral coordinate value of the vehicle's center of mass in the Frenet coordinate system is calculated. This value represents the lateral following deviation of the vehicle relative to the target planned path. Positive and negative values ​​of the lateral following deviation indicate whether the vehicle is on the left or right side of the target planned path, respectively. The lateral following deviation value is continuously updated at a frequency of 10 Hz to ensure real-time monitoring.

[0087] For example, the preset allowable deviation threshold can be obtained using a dynamic adaptive method. First, a basic allowable deviation value of 0.3 meters is determined, obtained through calibration via real-vehicle road testing. The basic allowable deviation value is adjusted based on the vehicle's current speed using the following formula: Allowable deviation threshold = Basic allowable deviation value × (1 + Current vehicle speed / 40), where the current vehicle speed is in meters per second, and 40 is a reference upper limit for vehicle speed, a fixed parameter obtained by combining the application scenario and real-vehicle test calibration. When the calculated allowable deviation threshold is less than 0.15 meters, 0.15 meters is taken as the final preset allowable deviation threshold; when the calculated allowable deviation threshold is greater than 0.5 meters, 0.5 meters is taken as the final preset allowable deviation threshold. Under this formula, the threshold is 0.3 meters when the vehicle speed is 0, and the threshold increases with higher speeds, consistent with the actual situation where greater lateral deviation is allowed at high speeds. Furthermore, the calculated value can be greater than 0.5 meters; for example, the threshold is 0.6 meters at a vehicle speed of 40 m / s, therefore the subsequent limiting logic uses 0.5 meters.

[0088] For example, the preset execution mileage threshold is also obtained using a dynamic adaptive method. First, the base execution mileage value is determined to be 50 meters. The base execution mileage value is adjusted based on the vehicle's current speed and road type. The speed adjustment coefficient is the current speed divided by 10. The road type is obtained from the road attribute information of the in-vehicle navigation map. For example, the road type adjustment coefficient is 0.6 for urban roads, 1.2 for highways, and 0.8 for rural roads. If there is no in-vehicle navigation map or road attribute information cannot be obtained, the road type adjustment coefficient can be set to 1.0 by default (i.e., no adjustment). The execution mileage threshold = base execution mileage value × speed adjustment coefficient × road type adjustment coefficient. When the calculated execution mileage threshold is less than 20 meters, 20 meters is taken as the final preset execution mileage threshold; when the calculated execution mileage threshold is greater than 200 meters, 200 meters is taken as the final preset execution mileage threshold.

[0089] Next, the absolute value of the current lateral following deviation is compared in real time with a preset allowable deviation threshold. Simultaneously, the vehicle's mileage since the last path planning is completed is accumulated. This accumulated mileage is calculated by integrating data from the vehicle's wheel speed sensors and compared with a preset execution mileage threshold. When the absolute value of the lateral following deviation exceeds the preset allowable deviation threshold, or the accumulated mileage exceeds the preset execution mileage threshold, the time interval since the last path replanning is checked. The minimum execution interval for path replanning is 500 milliseconds. This limitation is used to avoid frequent replanning due to noise or instantaneous fluctuations, ensuring driving stability. If the time interval is less than 500 milliseconds, replanning is not triggered until the minimum interval is met. When both trigger conditions are met simultaneously, the path replanning process is executed only once.

[0090] After path replanning is triggered, environmental perception data around the vehicle at the current moment is re-collected, including the position and speed information of dynamic traffic participants and the position and category information of static obstacles. Based on the latest environmental perception data, the complete process of obstacle detection, trajectory prediction, threat assessment, and path adjustment is repeated to generate a new target planned path. Path replanning only adjusts the road segment from the vehicle's current position to the end point of the original target planned path, preserving the overall direction of the untraveled portion of the original path to ensure the continuity of the driving route.

[0091] By dynamically and adaptively acquiring the lateral following deviation threshold and execution mileage threshold, setting dual trigger conditions and adding a minimum replanning interval limit to initiate path replanning, the vehicle driving deviation can be corrected in a timely manner, ensuring the accuracy of path tracking, while avoiding vehicle driving vibration problems caused by frequent replanning.

[0092] In summary, the autonomous driving path planning method based on large model analysis provided in this application has the following technical effects: This application extracts dynamic traffic participant information through large-scale model analysis combined with heterogeneous graph neural networks, calculates interaction weights and obstacle threat coefficients based on social impact theory, dynamically adjusts driving routes, monitors driving deviations in real time and triggers path replanning as needed, and generates control commands to guide vehicle operation. It can complete autonomous driving path planning and dynamic obstacle avoidance, effectively improving path tracking accuracy and driving safety. It achieves the technical effect of predicting driving trends by combining the interaction patterns of road participants, flexibly adjusting driving routes based on the evaluation results, and correcting driving status in real time.

[0093] Example 2, as Figure 2 As shown, based on the same inventive concept as in Embodiment 1 above, this application provides an autonomous driving path planning system based on large model analysis, the system comprising: Multimodal perception data acquisition module 1 is used to acquire target motion state data of the target autonomous vehicle and perceive the surrounding environment of the target autonomous vehicle to obtain target multimodal perception data.

[0094] Heterogeneous graph neural network construction module 2 is used to perform joint semantic parsing on the target motion state data and the target multimodal perception data to obtain the target parsing result, and construct a heterogeneous graph neural network based on the target parsing result.

[0095] The prediction trajectory and intent acquisition module 3 is used to introduce social influence theory to analyze the heterogeneous graph neural network to obtain multimodal probabilistic prediction trajectories and behavioral intentions.

[0096] The target planning path acquisition module 4 is used to dynamically adjust the predetermined autonomous driving route of the target autonomous driving vehicle based on the multimodal probabilistic predicted trajectory and behavioral intention as constraints, so as to obtain the target planning path.

[0097] The underlying control instruction execution module 5 is used to guide the target autonomous driving vehicle to perform autonomous driving execution using the target planned path as the underlying control instruction.

[0098] Furthermore, the multimodal sensing data acquisition module 1 is used to perform the following steps: The target motion state data includes vehicle speed, heading angle, and position coordinates, and the target multimodal perception data includes image data, point cloud data, and road surface state data.

[0099] Furthermore, the heterogeneous graph neural network construction module 2 is used to perform the following steps: The image data is semantically segmented to obtain a first semantic layer, consisting of road boundaries, lane lines, traffic signs, and static obstacles. Dynamic traffic participants in the point cloud data are detected and tracked to obtain arbitrary category identifiers, arbitrary 3D bounding boxes, and arbitrary historical trajectory sequences for any participant, forming a second semantic layer. The road surface state data is subjected to adhesion coefficient estimation to obtain a third semantic layer. The vehicle speed, heading angle, and position coordinates are used as a fourth semantic layer. The first, second, third, and fourth semantic layers are timestamped to obtain the target parsing result.

[0100] Furthermore, the heterogeneous graph neural network construction module 2 is used to perform the following steps: Using the dynamic traffic participants as nodes and the spatial distance and speed difference between them as edge attributes, a location graph, a speed graph, and a direction graph are constructed sequentially. The location graph, the speed graph, and the direction graph are then concatenated to obtain an initial graph structure. The initial graph structure is then input into a graph convolutional network for node feature extraction to obtain the heterogeneous graph neural network.

[0101] Furthermore, the predicted trajectory and intent acquisition module 3 is used to perform the following steps: The first and second participants in the dynamic traffic participants are extracted. Based on the social influence theory, the intensity dimension, immediacy dimension, and quantity dimension are quantified by relative distance, meeting time, and angle of movement direction, respectively, and the interaction weights of the first and second participants are obtained through standardized analysis. The interaction weights are input into a graph attention network for feature aggregation and update to obtain an interaction feature vector. According to the interaction feature vector, the first historical trajectory sequence of the first participant and the second historical trajectory sequence of the second participant are jointly decoded to obtain the first probabilistic predicted trajectory and behavioral intention of the first participant and the second probabilistic predicted trajectory and behavioral intention of the second participant, respectively. The first probabilistic predicted trajectory and behavioral intention, and the second probabilistic predicted trajectory and behavioral intention constitute the multimodal probabilistic predicted trajectory and behavioral intention.

[0102] Furthermore, the target planning path acquisition module 4 is used to perform the following steps: The relative distance, relative speed, and angle of motion between any participant and the target autonomous vehicle are collected in real time, and a weighted comprehensive threat coefficient for any participant is calculated. Any obstacle is extracted from the static obstacles, and an arbitrary threat coefficient is matched based on the arbitrary category attribute of the arbitrary obstacle. The arbitrary comprehensive threat coefficient and the arbitrary threat coefficient are used as weight coefficients of a predetermined safe distance constraint to dynamically adjust the predetermined autonomous driving route for obstacle avoidance, thereby obtaining the target planned path.

[0103] Furthermore, the target planning path acquisition module 4 is used to perform the following steps: The relative distance, relative speed, and angle between the directions of motion are used as threat assessment indicators; the entropy weight method is used to determine the objective weight allocation of the threat assessment indicators; the threat assessment indicators are comprehensively scored using the approximation ideal solution ranking method, and the arbitrary comprehensive threat coefficient is calculated in combination with the objective weight allocation.

[0104] Furthermore, the underlying control instruction execution module 5 is used to perform the following steps: The target control sequence of the target autonomous vehicle is generated based on the underlying control instructions; the target control sequence is sent to the target autonomous driving execution component via the vehicle controller local area network bus; the target autonomous driving execution component guides the target autonomous vehicle to perform autonomous driving according to the target planned path; wherein, the target control sequence includes steering angle instructions, acceleration instructions, and braking instructions; wherein, the target autonomous driving execution component includes a steering actuator, a drive motor, and a brake.

[0105] Furthermore, the underlying control instruction execution module 5 is used to perform the following steps: The lateral following deviation of the target autonomous vehicle in the Frenet coordinate system is monitored in real time. When the lateral following deviation exceeds a preset allowable deviation threshold or the real-time driving mileage of the target autonomous vehicle exceeds a preset execution mileage threshold, path replanning is triggered. Based on the path replanning, the target planned path is dynamically planned and adjusted.

[0106] The autonomous driving path planning system based on large model analysis provided in the embodiments of the present invention can execute the autonomous driving path planning method based on large model analysis provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0107] Although this application makes various references to certain modules in the system according to the embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy distinction between each other and are not used to limit the scope of protection of this invention.

[0108] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application. In some cases, the actions or steps described in this application can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An autonomous driving path planning method based on large model analysis, characterized in that, include: Acquire target motion state data of the target autonomous vehicle, and perceive the surrounding environment of the target autonomous vehicle to obtain target multimodal perception data. Joint semantic parsing is performed on the target motion state data and the target multimodal perception data to obtain the target parsing result, and a heterogeneous graph neural network is constructed based on the target parsing result; By introducing social influence theory to analyze the heterogeneous graph neural network, we can obtain multimodal probabilistic prediction trajectories and behavioral intentions. The predetermined autonomous driving route of the target autonomous vehicle is dynamically adjusted for obstacle avoidance based on the multimodal probabilistic predicted trajectory and behavioral intent, thereby obtaining the target planned path. The target autonomous driving vehicle is guided to perform autonomous driving execution using the target planned path as the underlying control command; Among them, social influence theory is introduced to analyze the heterogeneous graph neural network to obtain multimodal probabilistic prediction trajectories and behavioral intentions, including: Extract the first and second participants from the dynamic traffic participants; Based on the aforementioned social influence theory, the intensity dimension, immediacy dimension, and quantity dimension are quantified by relative distance, meeting time, and angle of movement direction, respectively, and the interaction weights between the first participant and the second participant are obtained through standardized analysis. The interaction weights are input into a graph attention network for feature aggregation and updating to obtain the interaction feature vector; Based on the interaction feature vector, the first historical trajectory sequence of the first participant and the second historical trajectory sequence of the second participant are jointly decoded in sequence to obtain the first probabilistic predicted trajectory and behavioral intention of the first participant and the second probabilistic predicted trajectory and behavioral intention of the second participant, respectively. The first probabilistic prediction trajectory and behavioral intention, and the second probabilistic prediction trajectory and behavioral intention, constitute the multimodal probabilistic prediction trajectory and behavioral intention.

2. The autonomous driving path planning method based on large model analysis as described in claim 1, characterized in that, The target motion state data includes vehicle speed, heading angle, and position coordinates, and the target multimodal perception data includes image data, point cloud data, and road surface state data.

3. The autonomous driving path planning method based on large model analysis as described in claim 2, characterized in that, The target parsing result is obtained by jointly semantically parsing the target motion state data and the target multimodal perception data, including: Semantic segmentation is performed on road boundaries, lane lines, traffic signs, and static obstacles in the image data to obtain the first semantic layer; The dynamic traffic participants in the point cloud data are detected and tracked to obtain any category identifier, any three-dimensional bounding box and any historical trajectory sequence of any participant in the dynamic traffic participants, and form a second semantic layer. The adhesion coefficient is estimated from the road surface condition data to obtain the third semantic layer; The vehicle speed, the heading angle, and the position coordinates are used as the fourth semantic layer; The first semantic layer, the second semantic layer, the third semantic layer, and the fourth semantic layer are timestamped to obtain the target parsing result.

4. The autonomous driving path planning method based on large model analysis as described in claim 3, characterized in that, Constructing a heterogeneous graph neural network based on the target parsing results includes: Using the dynamic traffic participants as nodes and the spatial distance and speed difference between the dynamic traffic participants as edge attributes, a location map, a speed map, and a direction map are constructed sequentially. The position map, the velocity map, and the orientation map are feature-stitched together to obtain an initial map structure; The initial graph structure is input into a graph convolutional network for node feature extraction to obtain the heterogeneous graph neural network.

5. The autonomous driving path planning method based on large model analysis as described in claim 3, characterized in that, Using the probabilistic predicted trajectory and behavioral intent of the multimodal autonomous driving system as constraints, the predetermined autonomous driving route of the target autonomous vehicle is dynamically adjusted for obstacle avoidance to obtain the target planned path, including: The relative distance, relative speed, and angle of motion between any participant and the target autonomous vehicle are collected in real time, and the weighted comprehensive threat coefficient of any participant is calculated. Extract any obstacle from the static obstacles and match any threat coefficient based on any category attribute of the random obstacle; The arbitrary comprehensive threat coefficient and the arbitrary threat coefficient are used as weighting coefficients for the predetermined safe distance constraint, and the predetermined autonomous driving route is dynamically adjusted for obstacle avoidance to obtain the target planned path.

6. The autonomous driving path planning method based on large model analysis as described in claim 5, characterized in that, The relative distance, relative speed, and angle of motion between any participant and the target autonomous vehicle are collected in real time, and a weighted comprehensive threat coefficient for any participant is calculated, including: The relative distance, the relative speed, and the angle between the directions of motion are used as threat assessment indicators. The objective weight allocation of the threat assessment indicators is determined using the entropy weight method; The threat assessment indicators are comprehensively scored using the approximation ideal solution ranking method, and the arbitrary comprehensive threat coefficient is calculated by combining the objective weight allocation.

7. The autonomous driving path planning method based on large model analysis as described in claim 5, characterized in that, Using the target planned path as the underlying control command, guide the target autonomous driving vehicle to perform autonomous driving execution, including: Generate the target control sequence for the target autonomous vehicle based on the underlying control instructions; The target control sequence is sent to the target autonomous driving execution component via the vehicle controller local area network bus; The target autonomous driving execution component guides the target autonomous driving vehicle to perform autonomous driving according to the target planned path; The target control sequence includes steering angle command, acceleration command, and braking command; The target autonomous driving execution components include a steering actuator, a drive motor, and a brake.

8. The autonomous driving path planning method based on large model analysis as described in claim 1, characterized in that, Using the target planned path as the underlying control command, guide the target autonomous driving vehicle to perform autonomous driving execution, including: The lateral following deviation of the target autonomous vehicle in the Frenet coordinate system is obtained through real-time monitoring. When the lateral following deviation exceeds a preset allowable deviation threshold or the real-time driving mileage of the target autonomous vehicle exceeds a preset execution mileage threshold, path replanning is triggered. Based on the path replanning, the target planned path is dynamically planned and adjusted.

9. An autonomous driving path planning system based on large model analysis, characterized in that, The system is used to implement the autonomous driving path planning method based on large model analysis as described in any one of claims 1-8, the system comprising: The multimodal perception data acquisition module is used to acquire target motion state data of the target autonomous vehicle and perceive the surrounding environment of the target autonomous vehicle to obtain target multimodal perception data. The heterogeneous graph neural network construction module is used to perform joint semantic parsing on the target motion state data and the target multimodal perception data to obtain the target parsing result, and to construct a heterogeneous graph neural network based on the target parsing result; The predicted trajectory and intent acquisition module is used to introduce social influence theory to analyze the heterogeneous graph neural network to obtain multimodal probabilistic predicted trajectories and behavioral intentions. The target planning path acquisition module is used to dynamically adjust the predetermined autonomous driving route of the target autonomous vehicle based on the multimodal probabilistic predicted trajectory and behavioral intention as constraints, so as to obtain the target planning path. The underlying control instruction execution module is used to guide the target autonomous driving vehicle to perform autonomous driving execution using the target planned path as the underlying control instruction.

Images