Space-time joint planning and decision-making method and system based on uncertainty perception

Abstract

The application provides a spatio-temporal joint planning and decision-making method and system based on uncertainty perception, which obtains a detection result of detecting a moving target in a preset range of an autonomous vehicle, and static map data in the preset range of the autonomous vehicle, generates a spatio-temporal probability occupation heat map of the moving target, fuses the static map data and the spatio-temporal probability occupation heat map, updates an occupation probability distribution at a current time based on a probability propagation mechanism, constructs a three-dimensional probability occupation grid, maps a value in the three-dimensional probability occupation grid and a collision time to a risk field, performs opportunity-risk analysis on a preset candidate strategy, determines an output decision, determines a planning space, and determines an optimal perception trajectory in the planning space; the application can avoid blind passing in a high-uncertainty condition, effectively transfer perception uncertainty to a planning layer, and significantly improve the robustness and safety of an autonomous driving system in a long-tail scenario.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a spatiotemporal joint planning and decision-making method and system based on uncertainty perception. Background Technology

[0002] Currently, most Level 4 autonomous driving systems adopt a pipelined processing architecture of "perception-prediction-planning-control". In this architecture, uncertainty is transmitted and accumulates between modules. For example, the perceived result (position, speed) of a distant obstacle itself contains noise and uncertainty. This uncertainty is input into the prediction module, resulting in a confidence interval for predicting its future trajectory. However, the planning module in related technologies often treats these input uncertainties as deterministic, planning based solely on a single value. This neglect of uncertainty is the root cause of the "blindness" and "unsafety" of autonomous driving systems in long-tail scenarios.

[0003] Although some work has begun to focus on the uncertainty of forecasting, how to effectively transmit this uncertainty to the planning level and make risk-aware decisions remains an unsolved problem.

[0004] Therefore, there is an urgent need for a method that not only enables planners to plan routes, but also enables them to make safer decisions based on the uncertainty of predictions. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, this application provides a spatiotemporal joint planning and decision-making method and system based on uncertainty perception to solve the above-mentioned technical problems.

[0006] According to one aspect of the embodiments of this application, a spatiotemporal joint planning and decision-making method based on uncertainty perception is provided. The spatiotemporal joint planning and decision-making method is implemented based on a spatiotemporal joint planning and decision-making model. The method includes: acquiring detection results obtained by detecting moving targets within a preset range of an autonomous vehicle, and static map data within the preset range of the autonomous vehicle; the detection results include: detection category, detection location, and covariance matrix of the detection location; generating a spatiotemporal probability occupancy heatmap of the moving targets based on a multimodal prediction model, using the detection results as input; the multimodal prediction model is obtained by performing a preset multimodal prediction model based on sample data; fusing the static map data and the spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment; and updating the occupancy probability distribution at the current moment based on a probability propagation mechanism. The following steps are performed: First, the occupancy probability distribution for future times is obtained. Second, a three-dimensional probability occupancy grid is constructed based on the current and future occupancy probability distributions. Third, the values ​​in the three-dimensional probability occupancy grid are mapped to collision times to form a risk field. The collision time is determined by the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target. Fourth, based on the risk field, a chance-risk analysis is performed on preset candidate strategies to obtain their chance and risk values. Fifth, an output decision is determined based on the chance and risk values. Sixth, a planning space is determined based on the comparison between the risk value in the risk field and a preset risk threshold. Seventh, using the output decision as boundary conditions and minimizing the cost function value of the risk perception trajectory as the objective, the optimal perception trajectory within the planning space is determined. The cost function is determined based on the risk cost, progress cost, and comfort cost of the risk perception trajectory.

[0007] According to another aspect of the embodiments of this application, a spatiotemporal joint planning and decision-making system based on uncertainty perception is also provided, comprising: a target detection module, configured to acquire detection results obtained by detecting moving targets within a preset range of an autonomous vehicle, and static map data within the preset range of the autonomous vehicle; the detection results include: detection category, detection location, and covariance matrix of the detection location; a heatmap generation module, configured to generate a spatiotemporal probability occupancy heatmap of the moving targets based on a multimodal prediction model and with the detection results as input; the multimodal prediction model is obtained by performing a preset multimodal prediction model based on sample data; a risk field generation module, configured to fuse the static map data and the spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment; and update the occupancy probability distribution at the current moment based on a probability propagation mechanism to obtain the occupancy probability distribution at future moments. Based on the occupancy probability distribution at the current moment and the occupancy probability distribution at the future moment, a three-dimensional probabilistic occupancy grid is constructed; the values ​​in the three-dimensional probabilistic occupancy grid are mapped to the collision time as a risk field; the collision time is determined by the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target; a decision output module is used to perform opportunity-risk analysis on preset candidate strategies based on the risk field to obtain the opportunity value and risk value of the preset candidate strategies; and determine the output decision based on the opportunity value and the risk value; a trajectory optimization module is used to determine the planning space based on the comparison result between the risk value in the risk field and the preset risk threshold; using the output decision as the boundary condition and minimizing the cost function value of the risk perception trajectory as the objective, the optimal perception trajectory within the planning space is determined; the cost function is determined based on the risk cost, progress cost, and comfort cost of the risk perception trajectory.

[0008] The beneficial effects of this application are as follows: This application obtains the detection results of moving targets within a preset range of an autonomous vehicle, as well as static map data within the preset range of the autonomous vehicle. Based on a multimodal prediction model and with the detection results as input, it generates a spatiotemporal probability occupancy heatmap of the moving targets. The static map data and the spatiotemporal probability occupancy heatmap are fused to obtain the occupancy probability distribution at the current moment. Based on a probability propagation mechanism, the occupancy probability distribution at the current moment is updated to obtain the occupancy probability distribution at future moments. Based on the occupancy probability distribution at the current moment and the occupancy probability distribution at future moments, a three-dimensional probability occupancy is constructed. The grid maps the values ​​and collision times within the 3D probability occupancy grid to a risk field. Based on this risk field, a chance-risk analysis is performed on preset candidate strategies to obtain their chance and risk values. The output decision is then determined based on these values. A planning space is defined by comparing the risk values ​​in the risk field with a preset risk threshold. Using the output decision as boundary condition and minimizing the cost function value of the risk perception trajectory as the objective, the optimal perception trajectory within the planning space is determined. This process, by introducing a covariance matrix, quantifies the uncertainty of the detection location and facilitates the generation of the spatiotemporal probability occupancy heatmap and the construction of the 3D probability occupancy grid. During this process, this uncertainty is explicitly transformed into occupancy probability, thus avoiding blind passage under high uncertainty conditions. This achieves effective transmission of perceived uncertainty at the planning layer, significantly improving the robustness and safety of the autonomous driving system in long-tail scenarios (such as distant obstacles, severe weather, occlusion, etc.). By combining the values ​​in the three-dimensional probability occupancy grid with the collision time, a risk field is jointly mapped. The collision time considers the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target. Therefore, the risk field reflects not only the spatial "congestion level" but also the temporal "urgency level," enabling... More accurate identification of potential collision risks enables the planner to react by slowing down or avoiding collisions earlier, improving the accuracy of risk assessment. Quantifying the trade-off between "opportunity value" (i.e., the probability and efficiency of passage) and "risk value" allows for more human-like and flexible driving decisions while ensuring safety, balancing passage efficiency and safety risks. By setting cost functions and boundary conditions, the planner ensures that it does not ignore minor changes in risk while pursuing progress and comfort. This ensures smooth and efficient driving while strictly adhering to the safety baseline, and reduces the generation of invalid trajectories, improving the planner's computational efficiency and success rate.

[0009] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0011] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0012] Figure 2 This is a flowchart illustrating an exemplary embodiment of the spatiotemporal joint planning and decision-making method based on uncertainty awareness, as shown in this application.

[0013] Figure 3 This is a schematic diagram illustrating a spatiotemporal joint planning and decision-making architecture based on uncertainty awareness, as shown in another exemplary embodiment of this application.

[0014] Figure 4 This is a schematic diagram of a spatiotemporal probability occupancy heatmap illustrated in an exemplary embodiment of this application;

[0015] Figure 5 This is a schematic diagram illustrating the planning of a risk perception trajectory, as shown in an exemplary embodiment of this application.

[0016] Figure 6 This is a schematic diagram illustrating the occupancy probability distribution at t=3s, as shown in an exemplary embodiment of this application.

[0017] Figure 7 This is a schematic diagram illustrating the risk integral of a risk perception trajectory, as shown in an exemplary embodiment of this application.

[0018] Figure 8 This is a block diagram illustrating an uncertainty-aware spatiotemporal joint planning and decision-making system, as shown in an exemplary embodiment of this application. Detailed Implementation

[0019] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0022] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0023] Reference Figure 1 As shown, the system architecture may include a detection device 101 and a processing device 102. The processing device 102 may be at least one of a desktop graphics processing unit (GPU) computer, a GPU computing cluster, a neural network computer, etc. Technical personnel can use the processing device 102 to acquire detection results of moving targets within a preset range of the autonomous vehicle, as well as static map data within the preset range of the autonomous vehicle. Based on a multimodal prediction model and with the detection results as input, a spatiotemporal probability occupancy heatmap of the moving targets is generated. The static map data and the spatiotemporal probability occupancy heatmap are fused to obtain the occupancy probability distribution at the current moment. Based on the probability propagation mechanism, the occupancy probability distribution at the current moment is updated to obtain the occupancy probability distribution at future moments. Based on the occupancy probability distribution at the current moment and the occupancy probability distribution at future moments, a three-dimensional probability occupancy grid is constructed. The values ​​in the three-dimensional probability occupancy grid and the collision time are mapped to a risk field. Based on the risk field, a chance-risk analysis of the preset candidate strategies is performed to obtain the chance value and risk value of the preset candidate strategies. Based on the chance value and risk value, the output decision is determined. Based on the comparison result of the risk value in the risk field and the preset risk threshold, the planning space is determined. With the output decision as the boundary condition and minimizing the cost function value of the risk perception trajectory as the objective, the optimal perception trajectory within the planning space is determined. The detection device 101 is used to detect moving targets within a preset range of the autonomous vehicle and to provide the detection results to the processing device 102 for processing.

[0024] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0025] Figure 2 This is a flowchart illustrating an exemplary embodiment of the spatiotemporal joint planning and decision-making method based on uncertainty awareness, as shown in this application. (Refer to...) Figure 2 As shown, the spatiotemporal joint planning and decision-making method based on uncertainty perception includes at least steps S210 to S250, which are described in detail below:

[0026] In step S210, detection results obtained from detecting moving targets within a preset range of the autonomous vehicle, and static map data within the preset range of the autonomous vehicle, are acquired. In one embodiment of this application, the preset range is set according to actual conditions. The detection results include: detection category, detection location, and covariance matrix of the detection location, etc. The detection category is represented by the category of the moving target bounding box, the detection location is used to represent the position of the moving target bounding box (e.g., center position), and the covariance matrix of the detection location is used to represent the uncertainty of the detection location. The static map data exists in the form of vector data and includes: lane center lines, lane boundary lines, intersections, pedestrian crossings, static obstacles, etc.

[0027] In step S220, a spatiotemporal probability occupancy heatmap of the moving target is generated based on the multimodal prediction model and with the detection results as input. In one embodiment of this application, the multimodal prediction model is obtained by applying a preset multimodal prediction model based on sample data. The spatiotemporal probability occupancy heatmap can reflect the probability of different locations being occupied by moving targets at different times. Combined with the uncertainty quantified by the detection position covariance matrix, the occupancy probability of each location will be adaptively adjusted according to the uncertainty, avoiding overestimation or underestimation of scene risks.

[0028] In step S230, static map data is fused with a spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment; based on a probability propagation mechanism, the occupancy probability distribution at the current moment is updated to obtain the occupancy probability distribution at future moments; a three-dimensional probability occupancy grid is constructed based on the occupancy probability distribution at the current moment and the occupancy probability distribution at future moments; the values ​​in the three-dimensional probability occupancy grid are mapped to collision times as a risk field. In one embodiment of this application, the process of constructing a three-dimensional probability occupancy grid based on the occupancy probability distribution at the current moment and the occupancy probability distribution at future moments includes: defining a three-dimensional coordinate system, wherein, The axis is parallel to the direction of travel of the autonomous vehicle; The axis is perpendicular to the direction of travel of the autonomous vehicle. The axis represents the future planning time domain, divided into several planning time domains; the occupancy probability distribution at the current moment is mapped to... On the time step plane; future moments ( , , ..., The occupancy probability distribution of ) is mapped sequentially to Mapped to ,..., On the time step plane, a three-dimensional data block showing the occupancy status from the current time to future time is obtained.

[0029] In one embodiment of this application, the formula for calculating the risk field includes:

[0030] Equation (1)

[0031] in, Represents grid At any moment The risk value, Represents grid At any moment The probability of occupancy, The collision time represents the time it takes for an autonomous vehicle to maintain its current relative motion and reach the grid cell occupied by the moving target. The time required This represents a constant, with the denominator set to 0.

[0032] In one embodiment of this application, the collision time is determined by the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target. In an accelerating approach scenario (i.e., the relative speed between the autonomous vehicle and the moving target is greater than 0, and the relative acceleration between the autonomous vehicle and the moving target is greater than 0), the formula for calculating the collision time is as follows:

[0033] Equation (2)

[0034] in, Indicates the collision time. This indicates the relative speed between the autonomous vehicle and the moving target. This represents the relative acceleration between the autonomous vehicle and the moving target. This indicates the relative distance between an autonomous vehicle and a moving target.

[0035] In deceleration approach scenarios (i.e., the relative speed between the autonomous vehicle and the moving target is greater than 0, and the relative acceleration between them is less than 0), firstly, it is determined whether the braking distance is less than the relative distance between the autonomous vehicle and the moving target. If the braking distance is less than this distance, a collision will not occur, and the collision time is set to infinity. If the braking distance is greater than or equal to the relative distance, the collision time is calculated using the following formula:

[0036] Equation (3)

[0037] In one embodiment of this application, in a uniform velocity scenario (where the acceleration between the autonomous vehicle and the moving target is greater than 0), the collision time is calculated using the following formula:

[0038] Equation (4)

[0039] in, Indicates the collision time. This indicates the relative speed between the autonomous vehicle and the moving target. This represents the relative distance between the autonomous vehicle and the moving target. If there are other conditions that prevent a collision (e.g., the relative speed between the autonomous vehicle and the moving target is less than or equal to 0), the collision time is set to infinity.

[0040] In step S240, based on the risk field, a chance-risk analysis is performed on the preset candidate strategies to obtain the chance value and risk value of the preset candidate strategies; and based on the chance value and risk value, the output decision is determined. In one embodiment of this application, the calculation formula for the output decision is as follows:

[0041] Equation (5)

[0042] in, Indicates the output decision. Indicates the preset candidate strategy Opportunity value, Indicates the preset candidate strategy The risk value, This represents the risk penalty coefficient.

[0043] In step S250, the planning space is determined based on the comparison between the risk value in the risk field and the preset risk threshold; the optimal perception trajectory within the planning space is determined using the output decision as the boundary condition and minimizing the cost function value of the risk perception trajectory as the objective. In one embodiment of this application, the cost function is determined based on the risk cost, schedule cost, and comfort cost of the risk perception trajectory.

[0044] This application introduces a covariance matrix to quantify the uncertainty of the detection location. During the generation of the spatiotemporal probability occupancy heatmap and the construction of the 3D probability occupancy grid, this uncertainty is explicitly transformed into occupancy probability, thus avoiding blind passage under high uncertainty conditions. This achieves effective transmission of perceptual uncertainty at the planning layer, significantly improving the robustness and safety of the autonomous driving system in long-tail scenarios (such as distant obstacles, inclement weather, occlusion, etc.). By combining the values ​​in the 3D probability occupancy grid with the collision time, a risk field is mapped. The collision time considers the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target, thus the risk field reflects not only spatial... The "congestion level" also reflects the "urgency" in terms of time, enabling more accurate identification of potential collision risks. This allows the planner to react by slowing down or avoiding collisions earlier, improving the accuracy of risk assessment. Quantifying the trade-off between "opportunity value" (i.e., the probability and efficiency of passage) and "risk value" allows for more human-like and flexible driving decisions while ensuring safety, balancing traffic efficiency and safety risks. By setting cost functions and boundary conditions, the planner ensures that it does not ignore minor changes in risk while pursuing progress and comfort. This ensures smooth and efficient driving while strictly adhering to safety standards, reducing the generation of invalid trajectories, and improving the planner's computational efficiency and success rate.

[0045] In one embodiment of this application, the process of determining the planning space based on the comparison between the risk value in the risk field and a preset risk threshold includes:

[0046] If the risk value of any grid in the risk field is greater than a preset risk threshold, the grid is determined to be impassable. In one embodiment of this application, the preset risk threshold is set according to actual conditions, for example, 0.5. If the risk value of any grid in the risk field is greater than the preset risk threshold, it indicates that the probability of passage is low.

[0047] If the risk value of each grid cell in the risk field is less than or equal to a preset risk threshold, the grid cell is determined to be passable. In one embodiment of this application, if the risk value of each grid cell in the risk field is less than or equal to the preset risk threshold, it indicates that the probability of passage is relatively high. A feasible region is formed by all passable grid cells, the boundary of the feasible region is extracted, and morphological processing is performed on the boundary of the feasible region to obtain the planning space. In one embodiment of this application, the boundary of the feasible region is extracted using an edge extraction algorithm (e.g., Canny operator, Sobel operator, etc.). Morphological processing includes opening operations, closing operations, dilation, erosion, etc. For example, closing operations are used to fill boundary holes, opening operations are used to remove boundary burrs, and dilation or erosion operations are used to reserve a safe distance for the boundary, thereby transforming the discrete, coarse grid boundary into a continuous, smooth trajectory boundary containing safety redundancy, and using the area within the trajectory boundary as the planning space.

[0048] In one embodiment of this application, the process of determining the optimal sensing trajectory within the planning space, using the output decision as the boundary condition and minimizing the cost function value of the risk perception trajectory as the objective, includes:

[0049] The current state and target location of the autonomous vehicle are obtained. In one embodiment of this application, the current state of the autonomous vehicle includes: the current position, speed, acceleration, orientation, and driving posture of the autonomous vehicle.

[0050] Based on the current state and the output decision, a reference line for the autonomous vehicle is determined. In one embodiment of this application, the reference line for the autonomous vehicle is obtained by using a QP (Quadratic Programming) smoothing algorithm or a residual neural network, given the current state and the output decision as inputs.

[0051] Based on the current state, reference lines, and target location, multiple risk perception trajectories for the autonomous vehicle within the planned space are generated. In one embodiment of this application, a sampling-based method is used to generate multiple risk perception trajectories for the autonomous vehicle within the planned space, based on the current state, reference lines, and target location.

[0052] The cost function value of each risk perception trajectory is calculated, and the risk perception trajectory with the minimum cost function value is selected as the optimal perception trajectory. In one embodiment of this application, the expression for the optimal perception trajectory is as follows:

[0053] Equation (6)

[0054] in, This represents the optimal sensing trajectory. Represents the cost function, Indicates the risk perception trajectory. This represents the set of risk perception trajectories. This indicates the starting state of the risk perception trajectory. Indicates the current state. This indicates the endpoint state of the risk perception trajectory. Indicates the target location. Indicates that autonomous vehicles are in acceleration at any moment Indicates the maximum permissible acceleration. Indicates that autonomous vehicles are in At the turning point of time, Indicates the maximum permissible turning angle. This indicates a preset risk threshold. This represents the risk value of the risk perception trajectory within the risk field;

[0055] The expression for the cost function includes:

[0056] Equation (7)

[0057] in, Represents the cost function, Indicates the risk perception trajectory. This represents the risk value of the risk perception trajectory within the risk field. Indicates the maximum risk threshold. Indicates the desired speed. Indicates that autonomous vehicles are in The speed of time Indicates that autonomous vehicles are in acceleration at any moment Indicates the maximum permissible acceleration. Indicates that autonomous vehicles are in The acceleration of time, Indicates the maximum permissible jerk. Indicates the weight of the acceleration term. This indicates the weight of the acceleration term, and the sum of the acceleration term weights equals 1.

[0058] In one embodiment of this application, the process of detecting moving targets within a preset range of an autonomous vehicle includes:

[0059] The system acquires multi-view images and radar point clouds within a preset range for autonomous vehicles. In one embodiment of this application, the multi-view images are obtained by capturing images using an onboard camera, and the radar point clouds are obtained by scanning using a lidar system or similar device.

[0060] The multi-view images and radar point clouds are preprocessed, and then spatiotemporally aligned to obtain spatiotemporally aligned multi-view images and radar point clouds. In one embodiment of this application, the preprocessing operations for the multi-view images include: geometric correction, viewpoint transformation, image enhancement, and image normalization. The preprocessing operations for the radar point clouds include: denoising, filtering, downsampling, and region cropping.

[0061] Semantic features are extracted from spatiotemporally aligned multi-view images, and geometric features are extracted from spatiotemporally aligned radar point clouds. Semantic and geometric features are fused to obtain fused features. Moving target detection is performed on the fused features to obtain the detection category, multiple predicted locations, and the confidence score for each predicted location. In one embodiment of this application, the process of extracting semantic features from spatiotemporally aligned multi-view images is implemented using a 2D base model or a multi-view Transformer model. The process of extracting geometric features from spatiotemporally aligned radar point clouds is implemented using a local feature extraction method based on covariance analysis, a PointNet / PointNet++ model, or a sparse convolutional network model. The process of fusing semantic and geometric features to obtain fused features can be implemented using a feature fusion method based on a unified space or a feature fusion method based on an attention mechanism. The process of performing moving target detection on the fused features to obtain the detection category, multiple predicted locations, and the confidence score for each predicted location is implemented using a 3D object detection head (e.g., an anchor-box-based detection head, a query-based detection head) model.

[0062] Multiple predicted locations are filtered according to the detection category and the confidence level of each predicted location. The filtered predicted locations are then matched with existing historical trajectories to obtain a matching trajectory. In one embodiment of this application, the process of filtering multiple predicted locations according to the detection category and the confidence level of each predicted location includes: comparing the confidence level of each predicted location with a preset confidence threshold for each detection category; and selecting predicted locations in each detection category whose confidence level is greater than or equal to the preset confidence threshold as the filtered predicted locations. The process of matching the filtered predicted locations with existing historical trajectories to obtain a matching trajectory is implemented using the Hungarian algorithm.

[0063] Based on Kalman filtering, the state of the matched trajectory is updated to obtain the updated position and its covariance matrix. The updated position is then used as the detection position, and its covariance matrix is ​​used as the covariance matrix of the detection position. In one embodiment of this application, the expression for the updated position is as follows:

[0064] Equation (8)

[0065] in, Indicates updating the position. This represents the prior state estimate, obtained by prediction based on the position information at time k-1. Indicates Kalman gain, This represents the position observation information at time k. Represents the observation matrix;

[0066] The formula for calculating Kalman gain is shown below:

[0067] Equation (9)

[0068] in, Indicates Kalman gain, Let the covariance matrix of the prior estimate be denoted as . Represents the transpose of the observation matrix. This represents the observation noise covariance.

[0069] The expression for updating the covariance matrix of the location is as follows:

[0070] Equation (10)

[0071] in, The covariance matrix representing the updated position. Represents the identity matrix. Indicates Kalman gain, Represents the observation matrix. This represents the covariance matrix of the prior estimate.

[0072] The formula for calculating the covariance matrix of the prior estimate is as follows:

[0073] Equation (11)

[0074] in, Let the covariance matrix of the prior estimate be denoted as . This represents the state transition matrix at the previous time step. Let represent the posterior covariance matrix of the previous time step. This represents the process noise covariance at the previous time step.

[0075] In one embodiment of this application, the process of obtaining a multimodal prediction model based on a preset multimodal prediction model using sample data includes:

[0076] The process involves extracting obstacle categories, obstacle locations, and historical trajectories of obstacles within a preset historical time period from sample data; and calculating the historical covariance matrix of obstacle locations based on the historical trajectories. In one embodiment of this application, the sample data includes historical multi-view images and historical radar point clouds, with the trigger acquisition time of the historical multi-view images corresponding to the trigger acquisition time of the historical radar point clouds. The process of extracting obstacle categories, obstacle locations, and historical trajectories of obstacles within a preset historical time period from the sample data includes: extracting features from historical multi-view images using an image feature extraction network model (e.g., a residual network model) to obtain image features; extracting features from historical radar point clouds using a point cloud feature extraction network model (e.g., a voxel network model) to obtain point cloud features; fusing point cloud features and image features to obtain fused features; inputting the fused features into a 3D detection head (e.g., a detection method based on a preset bounding box or a detection method based on a centerline point) to obtain obstacle categories and obstacle locations; and using the Hungarian algorithm, matching obstacle locations with existing historical trajectories according to obstacle categories to obtain the historical trajectories of obstacles within the preset historical time period. The process of calculating the historical covariance matrix of obstacle positions based on historical trajectories involves updating the state of the historical trajectory using Kalman filtering to obtain the historical covariance matrix of obstacle positions. This process is the same as updating the state of the matched trajectory based on Kalman filtering to obtain the covariance matrix of the updated position.

[0077] The obstacle category, obstacle location, and historical covariance matrix of the obstacle location are input into a preset multimodal prediction model to obtain the predicted spatiotemporal probability occupancy heatmap, predicted category, and predicted covariance matrix of the obstacle at a preset time. In one embodiment of this application, the preset multimodal prediction model can be selected from multimodal fusion occupancy networks, spatio-temporal probabilistic graphical models, etc.

[0078] Based on the differences between the predicted spatiotemporal probability occupancy heatmap and the true Gaussian heatmap, the differences between the predicted category and the true category, and the differences between the predicted covariance matrix and the true covariance matrix, a loss function is constructed. The parameters in the preset multimodal prediction model are adjusted with the goal of minimizing this loss function, resulting in the multimodal prediction model. In one embodiment of this application, the true Gaussian heatmap is calculated based on the historical trajectory of the obstacle within a preset historical time period. It is used to characterize the true position probability distribution of the obstacle at a preset time. The process of calculating the true Gaussian heatmap based on the historical trajectory of the obstacle within the preset historical time period includes: obtaining multiple historical position coordinates of the obstacle within the preset historical time period; inputting a pre-constructed two-dimensional Gaussian kernel function with multiple historical position coordinates as the center in a preset grid map to obtain the probability density value of each grid cell; and using the matrix composed of all probability density values ​​as the true Gaussian heatmap. The true covariance matrix is ​​calculated based on the historical trajectories of obstacles within a preset historical time period. This calculation is performed using a sliding window statistical method (or the sample covariance calculation formula), where the preset time point is after the preset historical time period and before the current time point. The loss function is calculated using the following formulas:

[0079] Equation (12)

[0080] in, Represents the loss function. This indicates the weight of the difference loss in the heatmap. This indicates the loss due to differences in the heatmap. Indicates the weight of the category difference loss. Indicates category difference loss. This represents the weights of the difference loss in the covariance matrix. This represents the difference loss in the covariance matrix. Indicates the weight of the regularization term. This represents the regularization term; the sum of the heatmap difference loss weight, the category difference loss weight, the covariance matrix difference loss weight, and the regularization term weight equals 1.

[0081] The expressions for heatmap difference loss include:

[0082] Equation (13)

[0083] in, This indicates the loss due to differences in the heatmap. This indicates the total number of samples. This indicates the weight of the mean square error term. Indicates the first The predicted spatiotemporal probability of each sample occupies a heatmap. Indicates the first The true Gaussian heatmap of each sample Indicates mean square error. Indicates the Kullback-Leibler divergence. This indicates the weight of the Kullback-Leibler divergence term; the sum of the weight of the mean squared error term and the weight of the Kullback-Leibler divergence term equals 1.

[0084] The expressions for class difference loss include:

[0085] Equation (14)

[0086] in, Indicates category difference loss. This indicates the total number of samples. Indicates the total number of categories. Indicates the first The true category of each sample is the category. This is in the form of one-hot encoding. Indicates the first Each sample belongs to category The predicted probability, This represents the weighting balance coefficient. This represents the sample focus loss for the predefined category;

[0087] Equation (15)

[0088] in, This represents the difference loss in the covariance matrix. This indicates the total number of samples. Represents the Frobenius norm weights. Indicates the Wasserstein distance weight. Indicates the first The prediction covariance matrix of each sample. Indicates the first The true covariance matrix of each sample Denotes the Frobenius norm. This represents the Wasserstein distance; the sum of the Frobenius norm weight and the Wasserstein distance weight equals 1.

[0089] The formulas for calculating the regularization term include:

[0090] Equation (16)

[0091] in, Represents the regularization term. Indicates the first The prediction covariance matrix of each sample. This indicates the calculation of eigenvalues. Denotes the least concave penalty function. This represents the weight of the least concave penalty function.

[0092] In one embodiment of this application, the process of fusing static map data with a spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment includes:

[0093] A coordinate system transformation is performed on the static map to obtain static map data in the autonomous vehicle coordinate system; then, the static map data in the autonomous vehicle coordinate system is rasterized to obtain a map data matrix. In one embodiment of this application, the map data matrix has the same dimension as the spatiotemporal probability occupancy heatmap; the process of transforming the static map to obtain static map data in the autonomous vehicle coordinate system is implemented through a rigid body transformation method. The process of rasterizing the static map data in the autonomous vehicle coordinate system is the process of dividing a rectangular region of interest in the autonomous vehicle coordinate system into a uniform grid.

[0094] The probability values ​​in the map data matrix are determined based on the element types. In one embodiment of this application, when the element type in the map data matrix is ​​an obstacle or building, the probability value of the corresponding location in the map data matrix can be set to 0.99 or other values; when the element type in the map data matrix is ​​a passable road, the probability value of the corresponding location in the map data matrix can be set to 0.001 or other values; when the element type in the map data matrix is ​​passable grassland, the probability value of the corresponding location in the map data matrix can be set to 0.5 or other values.

[0095] Based on the Bayesian update formula, the probability values ​​in the map data matrix and the probability values ​​in the spatiotemporal probability occupancy heatmap are spatially fused to obtain the spatial fused probability. Then, using the Kalman filter update equation, motion compensation is applied to the occupancy probability distribution of the previous time step to obtain an estimate of the occupancy probability of the previous time step. In one embodiment of this application, the process of spatially fusing the probability values ​​in the map data matrix and the probability values ​​in the spatiotemporal probability occupancy heatmap based on the Bayesian update formula to obtain the spatial fused probability includes: converting both the probability values ​​in the map data matrix and the probability values ​​in the spatiotemporal probability occupancy heatmap into logarithmic probability values; adding the logarithmic probability values ​​obtained from the conversion of the probability values ​​in the map data matrix to the logarithmic probability values ​​obtained from the conversion of the probability values ​​in the spatiotemporal probability occupancy heatmap to obtain the fused posterior logarithmic probability; and converting the fused posterior logarithmic probability into a probability value to obtain the spatial fused probability.

[0096] In one embodiment of this application, the expression for converting a probability value into a logarithmic odds value is as follows:

[0097] Equation (17)

[0098] in, Represents the logarithmic probability value. This represents the probability value. When... When the probability value is in the map data matrix, The logarithmic odds value is obtained by transforming the probability values ​​in the map data matrix. When the spatiotemporal probability occupies the probability value in the heatmap, The logarithmic probability value is obtained by converting the probability values ​​in the spatiotemporal probability heatmap.

[0099] The expression for the fused posterior log odds is as follows:

[0100] Equation (18)

[0101] in, This represents the posterior log-odds after fusion. This represents the logarithmic odds value obtained by transforming the probability values ​​in the map data matrix. This represents the logarithmic probability value obtained by converting the probability values ​​in the spatiotemporal probability heatmap. This represents the initial prior log odds (e.g., setting it to 0 corresponds to a probability of 0.5, i.e., an unknown state).

[0102] The expression for the spatial fusion probability is as follows:

[0103] Equation (19)

[0104] in, Represents the spatial fusion probability. This represents the posterior logarithmic probability after fusion.

[0105] In one embodiment of this application, the occupancy probability estimate of the previous time step includes: the prior position at the current time step and the prior covariance at the current time step. The formula for calculating the prior position at the current time step is as follows:

[0106] Equation (20)

[0107] in, This represents the prior position (i.e., the prior state estimate) at the current moment. Represents the state transition matrix. Indicates the posterior position of the previous time step. This represents the control input matrix from the previous time step. This represents the control vector at the previous time step. The formula for calculating the prior covariance (i.e., the prior estimated covariance matrix) at the current time step is shown in formula (11). The occupancy probability distribution at the previous time step is the result obtained by fusing the static map data with the spatiotemporal probability occupancy heatmap at the previous time step.

[0108] Based on the dynamic weight adjustment mechanism, the spatial fusion probability and the previous time-occupancy probability estimate are normalized to obtain the occupancy probability distribution. In one embodiment of this application, the process of normalizing the spatial fusion probability and the previous time-occupancy probability estimate based on the dynamic weight adjustment mechanism to obtain the occupancy probability distribution includes: calculating the Kalman gain as the dynamic weight; and calculating the position in the current time-occupancy probability distribution. The formula for calculating the Kalman gain is shown in formula (9), and the expression for calculating the position in the current time-occupancy probability distribution is shown in formula (8). In the calculation process using formula (8), the spatial fusion probability is used as the position observation information.

[0109] In one embodiment of this application, the process of updating the occupancy probability distribution at the current time to obtain the occupancy probability distribution at future time based on the probability propagation mechanism includes:

[0110] Based on the probability propagation mechanism, the occupancy probability distribution at the current time is predicted under multiple prediction modes to obtain the occupancy probability distribution under multiple prediction modes at future times. In one embodiment of this application, the occupancy probability distribution under multiple prediction modes at future times includes: the center position of the occupancy probability distribution under multiple prediction modes at future times and the covariance matrix of the center position of the occupancy probability distribution under multiple prediction modes at future times. The formula for calculating the center position of the occupancy probability distribution under multiple prediction modes at future times is as follows:

[0111] Equation (21)

[0112] in, This indicates the central position of the occupancy probability distribution at future time steps under the m-th prediction model. This represents the state transition matrix under the m-th prediction mode. Indicates the posterior position at the current moment. This represents the control input matrix under the m-th prediction mode. This represents the control vector at the current moment.

[0113] The formulas for calculating the covariance matrix of the occupancy probability distribution under various prediction models for future times are as follows:

[0114] Equation (22)

[0115] in, This represents the covariance matrix of the occupancy probability distribution at future times under the m-th prediction model. This represents the state transition matrix under the m-th prediction mode. This represents the posterior covariance matrix at the current time step under the m-th prediction model. This represents the process noise covariance at the current time step under the m-th prediction mode.

[0116] Based on the uncertainty measure of each prediction mode, the occupancy probability distributions of multiple prediction modes at future time are fused to obtain the occupancy probability distribution at future time. In one embodiment of this application, the process of fusing the occupancy probability distributions of multiple prediction modes at future time based on the uncertainty measure of each prediction mode includes: calculating the likelihood of each prediction mode; calculating the mode probability of each prediction mode based on the likelihood of each prediction mode; fusing the center positions of the occupancy probability distributions of multiple prediction modes at future time based on the mode probabilities of each prediction mode to obtain the fused position; and fusing the covariance matrices of the occupancy probability distributions of multiple prediction modes at future time based on the fused position and the mode probabilities of each prediction mode to obtain the fused covariance. The formula for calculating the likelihood of each prediction mode is as follows:

[0117] Equation (23)

[0118] in, This represents the likelihood at time k under the m-th prediction pattern. It represents the innovation covariance at time k under the m-th prediction mode (obtained by mapping the covariance matrix of the occupancy probability distribution at the current time under the m-th prediction mode to the observation space and adding it to the observation noise covariance). This represents the position observation information at time k. It represents the predicted observation value at time k under the m-th prediction mode (obtained by mapping the center position of the occupancy probability distribution at the current time under the m-th prediction mode, that is, mapping the center position of the occupancy probability distribution at the current time to the observation space under the m-th prediction mode).

[0119] The formula for calculating the probability of each prediction pattern is as follows:

[0120] Equation (24)

[0121] in, This represents the probability of the prediction at time k under the m-th prediction pattern. This represents the likelihood at time k under the m-th prediction pattern. This represents the probability of the prediction at time k-1 under the m-th prediction pattern. This represents the total number of prediction patterns. It represents the total likelihood of all predicted modes at time k.

[0122] The formula for calculating the fusion position is as follows:

[0123] Equation (25)

[0124] in, Indicates the fusion position. This represents the probability of the prediction at time k under the m-th prediction pattern. This indicates the central position of the occupancy probability distribution at future time steps under the m-th prediction model. This represents the total number of prediction patterns.

[0125] The formula for calculating the fusion covariance is as follows:

[0126] Equation (26)

[0127] in, Indicates the fusion covariance. Indicates the fusion position. This represents the probability of the prediction at time k under the m-th prediction pattern. This indicates the central position of the occupancy probability distribution at future time steps under the m-th prediction model. This represents the total number of prediction patterns. Let represent the covariance matrix of the occupancy probability distribution at future times under the m-th prediction model.

[0128] In one embodiment of this application, the process of obtaining the chance value and risk value of a preset candidate strategy based on a risk field through chance-risk analysis includes:

[0129] The current state and target position of the autonomous vehicle are obtained. In one embodiment of this application, the current state of the autonomous vehicle includes: the current position, driving speed, heading angle, and acceleration of the autonomous vehicle, and the target position of the autonomous vehicle is given by the navigation system.

[0130] Based on the current state and target location, a preset candidate strategy is mapped onto the risk field to obtain the candidate trajectory corresponding to the preset candidate strategy. In one embodiment of this application, the expression for the Y-axis coordinate point in the candidate trajectory is as follows:

[0131] Equation (27)

[0132] in, Indicates the first The Y-axis coordinate value at time t. Indicates the first The Y-axis coordinate value at time t. Indicates the first The velocity value at that moment, Indicates the first The heading angle at any moment, Indicates the time step.

[0133] The expression for the X-axis coordinates of the candidate trajectory is as follows:

[0134] Equation (28)

[0135] in, Indicates the first The X-axis coordinate value at time t. Indicates the first The X-axis coordinate value at time t. Indicates the first The velocity value at that moment, Indicates the first The heading angle at any moment, This indicates the time step. The velocity and heading angle of the candidate trajectories differ depending on the preset candidate strategy.

[0136] After obtaining the X-axis and Y-axis coordinates of the candidate trajectory, the X-axis and Y-axis coordinates of the candidate trajectory are mapped to the risk field to obtain the trajectory points in the candidate trajectory.

[0137] The mapping formula for the X-axis coordinates of the candidate trajectory is as follows:

[0138] Equation (29)

[0139] in, This represents the x-coordinate index of the risk field raster matrix. Indicates the first The X-axis coordinate value at time t. This represents the x-coordinate of the bottom left corner of the risk field raster matrix. This indicates the resolution of the risk field raster map.

[0140] The mapping formula for the Y-axis coordinates of the candidate trajectory is as follows:

[0141] Equation (30)

[0142] in, This represents the x-coordinate index of the risk field raster matrix. Indicates the first The Y-axis coordinate value at time t. The ordinate represents the bottom left corner of the risk field raster matrix. This indicates the resolution of the risk field raster map.

[0143] The sum of the probability values ​​of each trajectory point in the candidate trajectory is used as the risk value of the preset candidate strategy. In one embodiment of this application, the accumulation of risk values ​​helps to more comprehensively reflect the cumulative risk of the candidate trajectory encountering dynamic obstacles or unknown areas throughout the entire driving process, avoiding decision-making bias caused by only assessing the risk of a single point, and better meeting the needs of considering the safety of the entire process in actual autonomous driving.

[0144] Based on the efficiency and comfort of candidate trajectories, the chance value of a preset candidate strategy is determined. In one embodiment of this application, the formula for calculating the chance value of the preset candidate strategy includes:

[0145] Equation (31)

[0146] in, Indicates the preset candidate strategy Opportunity value, Indicates the comfort weight. This indicates the comfort level of the candidate trajectory. Indicates efficiency weight. This represents the efficiency of the candidate trajectory. The sum of the comfort weight and the efficiency weight equals 1.

[0147] The formula for calculating the comfort level of the candidate trajectory is as follows:

[0148] Equation (32)

[0149] in, This indicates the comfort level of the candidate trajectory. This represents the jerk, which is the maximum jerk of the autonomous vehicle at all times. This indicates the tolerance threshold.

[0150] Equation (33)

[0151] in, This indicates the efficiency of the candidate trajectory. This represents the longitudinal travel distance of the candidate trajectory. This indicates the theoretical maximum driving distance.

[0152] Figure 3 This is a schematic diagram illustrating a spatiotemporal joint planning and decision-making architecture based on uncertainty awareness, as shown in another exemplary embodiment of this application. Figure 3In this context, the spatiotemporal joint planning and decision-making architecture based on uncertainty perception includes: (1) a perception module (located in the perception layer): used to detect moving targets within a preset range of the autonomous vehicle and obtain the detection results; (2) a multimodal prediction module (located in the prediction layer): used to generate a spatiotemporal probability occupancy heatmap of moving targets with the detection results as input; (3) a spatiotemporal passable area builder (located in the risk construction layer): used to fuse static map data with the spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment; based on the probability propagation mechanism, update the occupancy probability distribution at the current moment to obtain the occupancy probability distribution at future moments; based on the current moment's... The three-dimensional probability occupation grid is constructed by occupancy probability distribution and future time occupancy probability distribution; the values ​​in the three-dimensional probability occupation grid and the collision time are mapped to the risk field; (4) Robust decision arbitrator (located in the decision layer): based on the risk field, the opportunity-risk analysis of the preset candidate strategy is performed to obtain the opportunity value and risk value of the preset candidate strategy; and the output decision is determined according to the opportunity value and risk value; (5) Spatiotemporal joint planner (located in the planning layer): based on the comparison result of the risk value in the risk field and the preset risk threshold, the planning space is determined; the optimal perception trajectory in the planning space is determined with the output decision as the boundary condition and the cost function value of minimizing the risk perception trajectory as the objective.

[0153] Figure 4 This is a schematic diagram of a spatiotemporal probability occupancy heatmap illustrated in an exemplary embodiment of this application. Figure 4 In this study, a three-dimensional probabilistic occupancy grid is constructed based on the spatiotemporal coordinate system and the occupancy probability distributions at t=1s, t=2s, t=3s, and t=4s. The occupancy probability distributions at different times intuitively show the possible activity areas of moving targets in different time domains. The higher the probability, the darker the color of the area, indicating that the position is more likely to be occupied by the moving target, thus providing an accurate probabilistic basis for the subsequent construction of the risk field.

[0154] Figure 5 This is a schematic diagram illustrating the planning of a risk perception trajectory, as shown in an exemplary embodiment of this application. Figure 5 In this process, the risk value of each trajectory point in the planned trajectory is integrated to obtain the trajectory risk integral, and the optimal sensing trajectory is determined with the goal of minimizing the trajectory risk integral.

[0155] Figure 6 This is a schematic diagram illustrating the occupancy probability distribution at t=3s, as shown in an exemplary embodiment of this application. Figure 6As shown in the figure, blue dots represent autonomous vehicles (autonomous vehicles), yellow dots represent pedestrians, yellow probability clouds represent the uncertainty of pedestrians' positions and their intention to cross the road, pink dots represent oncoming vehicles, and red probability clouds represent the uncertainty of oncoming vehicles' positions.

[0156] Figure 7 This is a schematic diagram illustrating the risk integral of a risk perception trajectory, as shown in an exemplary embodiment of this application. Figure 7 As shown in the figure, blue dots represent autonomous vehicles (autonomous vehicles), yellow dots represent pedestrians, and pink dots represent oncoming vehicles. Pink trajectories represent high-risk trajectories, yellow trajectories represent medium-risk trajectories, and green trajectories represent low-risk trajectories. High-risk trajectories are those to be avoided, while low-risk trajectories are recommended trajectories, i.e., the most perceptive trajectories.

[0157] The advantages of this application compared to existing technologies are shown in the table below:

[0158] Table 1

[0159]

[0160] As can be summarized from Table 1, this application, by explicitly modeling uncertainty, makes the spatiotemporal joint planning and decision-making system based on uncertainty awareness more robust in complex and rare scenarios; from the perspective of risk quantification in the decision-making process, it makes it easier for engineers to understand why the spatiotemporal joint planning and decision-making system based on uncertainty awareness makes specific decisions, facilitating debugging and optimization, and improving decision interpretability; the spatiotemporal joint planner makes the risk perception trajectory smoother and safer, especially in scenarios with dense dynamic obstacles, improving planning quality; end-to-end optimizability enables the entire spatiotemporal joint planning and decision-making system based on uncertainty awareness to be jointly optimized, reducing information loss between modules and improving the reliability of the spatiotemporal joint planning and decision-making system based on uncertainty awareness.

[0161] This application systematically introduces the uncertainties in the perception and prediction stages into the planning module (spatiotemporal joint planner) in the form of a probability occupancy heatmap, constructing a risk perception decision-making system to handle the uncertainty of moving targets. It elevates the traditional path planning problem to an optimization problem in a 3D spatiotemporal risk field, achieving true spatiotemporal joint planning and enabling autonomous vehicles to manage dynamic risks more precisely. The decision-making process is no longer based on heuristic rules, but on a quantitative comparison of "opportunities" (such as efficiency gains) and "risks" (trajectory risk integrals), making the decision results more interpretable and robust. By explicitly modeling uncertainty, autonomous vehicles can proactively choose low-risk coping strategies when facing long-tailed and complex scenarios, rather than "making mistakes" after uncertainty accumulates, greatly improving the safety and robustness of the uncertainty-perception-based spatiotemporal joint planning and decision-making system. The entire architecture is built using neural networks, enabling end-to-end optimization of the entire chain from perception and prediction to planning, directly reducing the long-term risk of the final trajectory.

[0162] The following describes an embodiment of the apparatus described in this application, which can be used to execute the spatiotemporal joint planning and decision-making system based on uncertainty perception described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the spatiotemporal joint planning and decision-making method based on uncertainty perception described in the above applications.

[0163] Figure 8 This is a block diagram illustrating an uncertainty-aware spatiotemporal joint planning and decision-making system, as shown in an exemplary embodiment of this application.

[0164] As shown in Figure 8, this exemplary spatiotemporal joint planning and decision-making system 800 based on uncertainty perception includes:

[0165] The target detection module 801 is used to acquire the detection results obtained by detecting moving targets within a preset range of the autonomous vehicle, as well as static map data within the preset range of the autonomous vehicle.

[0166] The heatmap generation module 802 is used to generate a spatiotemporal probability occupancy heatmap of moving targets based on a multimodal prediction model and with the detection results as input.

[0167] The risk field generation module 803 is used to fuse static map data with spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment; based on the probability propagation mechanism, the occupancy probability distribution at the current moment is updated to obtain the occupancy probability distribution at the future moment; based on the occupancy probability distribution at the current moment and the occupancy probability distribution at the future moment, a three-dimensional probability occupancy grid is constructed; and the values ​​in the three-dimensional probability occupancy grid and the collision time are mapped to the risk field.

[0168] The decision output module 804 is used to perform opportunity-risk analysis on preset candidate strategies based on the risk field, obtain the opportunity value and risk value of the preset candidate strategies, and determine the output decision based on the opportunity value and risk value.

[0169] The trajectory optimization module 805 is used to determine the planning space based on the comparison between the risk value in the risk field and the preset risk threshold; and to determine the optimal perception trajectory within the planning space with the output decision as the boundary condition and the cost function value of minimizing the risk perception trajectory as the objective.

[0170] In one embodiment of this application, the preset range is set according to actual conditions. The detection results include: detection category, detection location, and covariance matrix of the detection location. The detection category is used to characterize the category of the moving target bounding box, the detection location is used to characterize the location of the moving target bounding box (e.g., center location), and the covariance matrix of the detection location is used to characterize the uncertainty of the detection location. Static map data exists in the form of vector data, including: lane center lines, lane boundary lines, intersections, pedestrian crossings, etc.

[0171] In one embodiment of this application, the multimodal prediction model is obtained by performing a preset multimodal prediction model based on sample data. The spatiotemporal probability occupancy heatmap can reflect the probability of different locations being occupied by moving targets at different times. Combined with the uncertainty quantified by the detection location covariance matrix, the occupancy probability of each location will be adaptively adjusted according to the uncertainty, avoiding overestimation or underestimation of scene risks.

[0172] In one embodiment of this application, the process of constructing a three-dimensional probabilistic occupancy grid based on the occupancy probability distribution at the current time and the occupancy probability distribution at future times includes: defining a three-dimensional coordinate system, wherein, The axis is parallel to the direction of travel of the autonomous vehicle; The axis is perpendicular to the direction of travel of the autonomous vehicle. The axis represents the future planning time domain, divided into several planning time domains; the occupancy probability distribution at the current moment is mapped to... On the time step plane; future moments ( , , ..., The occupancy probability distribution of ) is mapped sequentially to Mapped to ,..., On the time step plane, a three-dimensional data block showing the occupancy status from the current time to future times is obtained. The calculation formula for the risk field is shown in formula (1). The calculation formula for the output decision is shown in formula (5).

[0173] In one embodiment of this application, the cost function is determined based on the risk cost, schedule cost, and comfort cost of the risk-aware trajectory.

[0174] This application introduces a covariance matrix to quantify the uncertainty of the detection location. During the generation of the spatiotemporal probability occupancy heatmap and the construction of the 3D probability occupancy grid, this uncertainty is explicitly transformed into occupancy probability, thus avoiding blind passage under high uncertainty conditions. This achieves effective transmission of perceptual uncertainty at the planning layer, significantly improving the robustness and safety of the autonomous driving system in long-tail scenarios (such as distant obstacles, inclement weather, occlusion, etc.). By combining the values ​​in the 3D probability occupancy grid with the collision time, a risk field is mapped. The collision time considers the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target, thus the risk field reflects not only spatial... The "congestion level" also reflects the "urgency" in terms of time, enabling more accurate identification of potential collision risks. This allows the planner to react by slowing down or avoiding collisions earlier, improving the accuracy of risk assessment. Quantifying the trade-off between "opportunity value" (i.e., the probability and efficiency of passage) and "risk value" allows for more human-like and flexible driving decisions while ensuring safety, balancing traffic efficiency and safety risks. By setting cost functions and boundary conditions, the planner ensures that it does not ignore minor changes in risk while pursuing progress and comfort. This ensures smooth and efficient driving while strictly adhering to safety standards, reducing the generation of invalid trajectories, and improving the planner's computational efficiency and success rate.

[0175] It should be noted that the spatiotemporal joint planning and decision-making system based on uncertainty perception provided in the above embodiments and the spatiotemporal joint planning and decision-making method based on uncertainty perception provided in the above embodiments belong to the same concept. The specific methods of execution of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the spatiotemporal joint planning and decision-making system based on uncertainty perception provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0176] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A spatiotemporal joint planning and decision-making method based on uncertainty perception, characterized in that, The spatiotemporal joint planning and decision-making method is implemented based on a spatiotemporal joint planning and decision-making model, and includes: The system acquires detection results obtained by detecting moving targets within a preset range of the autonomous vehicle, as well as static map data within the preset range of the autonomous vehicle; the detection results include: detection category, detection location, and covariance matrix of the detection location; Based on a multimodal prediction model and with the detection results as input, a spatiotemporal probability occupancy heatmap of the moving target is generated; the multimodal prediction model is obtained by training a preset multimodal prediction model based on sample data; The static map data is fused with the spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment. Based on the probability propagation mechanism, the occupancy probability distribution at the current moment is updated to obtain the occupancy probability distribution at future moments. Based on the occupancy probability distribution at the current moment and the occupancy probability distribution at future moments, a three-dimensional probability occupancy grid is constructed. The values ​​in the three-dimensional probability occupancy grid are mapped to the collision time as a risk field. The collision time is determined by the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target. Based on the risk field, a chance-risk analysis is performed on the preset candidate strategies to obtain the chance value and risk value of the preset candidate strategies; and the output decision is determined based on the chance value and the risk value. Based on the comparison between the risk value in the risk field and the preset risk threshold, the planning space is determined; using the output decision as the boundary condition and minimizing the cost function value of the risk perception trajectory as the objective, the optimal perception trajectory within the planning space is determined; the cost function is determined based on the risk cost, schedule cost, and comfort cost of the risk perception trajectory.

2. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 1, characterized in that, The process of determining the planning space based on the comparison between the risk value in the risk field and the preset risk threshold includes: If the risk value of each grid in the risk field is greater than the preset risk threshold, the grid is determined to be impassable. If the risk value of each grid in the risk field is less than or equal to the preset risk threshold, then the grid is determined to be passable; The feasible region is composed of all accessible grid cells. The boundary of the feasible region is extracted and morphological processing is performed on the boundary of the feasible region to obtain the planning space.

3. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 1 or 2, characterized in that, The process of determining the optimal perception trajectory within the planning space, using the output decision as the boundary condition and minimizing the cost function value of the risk perception trajectory as the objective, includes: Obtain the current state and target location of the autonomous vehicle; Based on the current state and the output decision, the reference line of the autonomous vehicle is determined; Based on the current state, reference line and target position, the autonomous vehicle generates multiple risk perception trajectories within the planned space; Calculate the cost function value for each risk perception trajectory, and select the risk perception trajectory with the minimum cost function value as the optimal perception trajectory.

4. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 1 or 2, characterized in that, The process of detecting moving targets within a preset range for autonomous vehicles includes: Acquire multi-view images and radar point clouds within a preset range of the autonomous vehicle; The multi-view image and the radar point cloud are preprocessed, and the preprocessed multi-view image and the preprocessed radar point cloud are spatiotemporally aligned to obtain the spatiotemporally aligned multi-view image and radar point cloud. Semantic features are extracted from the spatiotemporally aligned multi-view images, and geometric features are extracted from the spatiotemporally aligned radar point cloud; the semantic features and the geometric features are fused to obtain fused features; moving target detection is performed on the fused features to obtain the detection category, multiple predicted positions, and the confidence level of each predicted position; According to the detection category and the confidence level of each predicted location, multiple predicted locations are filtered, and the filtered predicted locations are matched with existing historical trajectories to obtain the matched trajectory. Based on Kalman filtering, the matching trajectory is updated to obtain the updated position and the covariance matrix of the updated position; the updated position is used as the detection position, and the covariance matrix of the updated position is used as the covariance matrix of the detection position.

5. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 1 or 2, characterized in that, The process of training a pre-defined multimodal prediction model based on sample data to obtain the multimodal prediction model includes: Extract obstacle categories, obstacle locations, and historical trajectories of obstacles within a preset historical time period from the sample data; and calculate the historical covariance matrix of the obstacle locations based on the historical trajectories. The obstacle category, the obstacle location, and the historical covariance matrix of the obstacle location are input into the preset multimodal prediction model to obtain the predicted spatiotemporal probability occupancy heatmap, predicted category, and predicted covariance matrix of the obstacle at a preset time. A loss function is constructed based on the differences between the predicted spatiotemporal probability occupancy heatmap and the actual Gaussian heatmap, the differences between the predicted category and the actual category, and the differences between the predicted covariance matrix and the actual covariance matrix. The parameters in the preset multimodal prediction model are adjusted with the objective of minimizing this loss function to obtain the multimodal prediction model. The actual Gaussian heatmap is calculated based on the historical trajectory of the obstacle within a preset time period. The actual covariance matrix is ​​calculated based on the historical trajectory of the obstacle within a preset time period.

6. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 5, characterized in that, The formula for calculating the loss function includes: ， in, Represents the loss function. This indicates the weight of the difference loss in the heatmap. This indicates the loss due to differences in the heatmap. Indicates the weight of the category difference loss. Indicates category difference loss. This represents the weights of the difference loss in the covariance matrix. This represents the difference loss in the covariance matrix. Indicates the weight of the regularization term. Represents the regularization term; The expression for the heatmap difference loss includes: ， in, This indicates the loss due to differences in the heatmap. This indicates the total number of samples. This indicates the weight of the mean square error term. Indicates the first The predicted spatiotemporal probability of each sample occupies a heatmap. Indicates the first The true Gaussian heatmap of each sample Indicates mean square error. Indicates the Kullback-Leibler divergence. Indicates the weight of the Kullback-Leibler divergence term; The expression for the category difference loss includes: ， in, Indicates category difference loss. This indicates the total number of samples. Indicates the total number of categories. Indicates the first The true category of each sample is the category. This is in the form of one-hot encoding. Indicates the first Each sample belongs to category The predicted probability, This represents the weighting balance coefficient. This represents the sample focus loss for the predefined category; ， in, This represents the difference loss in the covariance matrix. This indicates the total number of samples. Represents the Frobenius norm weights. Indicates the Wasserstein distance weight. Indicates the first The prediction covariance matrix of each sample. Indicates the first The true covariance matrix of each sample Denotes the Frobenius norm. Represents the Wasserstein distance; The formula for calculating the regularization term includes: ， in, Represents the regularization term. Indicates the first The prediction covariance matrix of each sample. This indicates the calculation of eigenvalues. Denotes the least concave penalty function. This represents the weight of the least concave penalty function.

7. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 1 or 2, characterized in that, The process of fusing the static map data with the spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment includes: The static map is transformed to obtain static map data in the coordinate system of the autonomous vehicle; and the static map data in the coordinate system of the autonomous vehicle is rasterized to obtain a map data matrix; the map data matrix has the same dimension as the spatiotemporal probability occupancy heatmap. Determine the probability value in the map data matrix based on the element type in the map data matrix; Based on the Bayesian update formula, the probability values ​​in the map data matrix and the probability values ​​in the spatiotemporal probability occupancy heatmap are spatially fused to obtain the spatial fused probability; and the occupancy probability distribution of the previous time step is motion compensated by the Kalman filter update equation to obtain the occupancy probability estimate of the previous time step. Based on the dynamic weight adjustment mechanism, the spatial fusion probability and the previous time step occupancy probability estimate are normalized to obtain the occupancy probability distribution.

8. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 1 or 2, characterized in that, The process of updating the occupancy probability distribution at the current moment to obtain the occupancy probability distribution at future moments based on the probability propagation mechanism includes: Based on the probability propagation mechanism, the occupancy probability distribution at the current moment is predicted under multiple prediction modes to obtain the occupancy probability distribution under multiple prediction modes at the future moment. Based on the uncertainty measure of each prediction mode, the occupancy probability distributions of multiple prediction modes for the future time are fused to obtain the occupancy probability distribution for the future time.

9. The spatiotemporal joint planning and decision-making method based on uncertainty perception according to claim 1 or 2, characterized in that, Based on the risk field, the process of obtaining the chance value and risk value of the preset candidate strategy through chance-risk analysis includes: Obtain the current state and target location of the autonomous vehicle; Based on the current state and the target location, the preset candidate strategy is mapped onto the risk field to obtain the candidate trajectory corresponding to the preset candidate strategy; The sum of the probability values ​​of each trajectory point in the candidate trajectory is used as the risk value of the preset candidate strategy; Based on the efficiency and comfort of the candidate trajectories, the chance value of the preset candidate strategy is determined.

10. A spatiotemporal joint planning and decision-making system based on uncertainty perception, characterized in that, include: The target detection module is used to acquire the detection results obtained by detecting moving targets within a preset range of the autonomous vehicle, as well as the static map data within the preset range of the autonomous vehicle. The detection results include: detection category, detection location, and the covariance matrix of the detection location; The heatmap generation module is used to generate a spatiotemporal probability occupancy heatmap of the moving target based on a multimodal prediction model and with the detection results as input; the multimodal prediction model is obtained by training a preset multimodal prediction model based on sample data; The risk field generation module is used to fuse the static map data with the spatiotemporal probability occupancy heatmap to obtain the occupancy probability distribution at the current moment; based on the probability propagation mechanism, the occupancy probability distribution at the current moment is updated to obtain the occupancy probability distribution at future moments; based on the occupancy probability distribution at the current moment and the occupancy probability distribution at future moments, a three-dimensional probability occupancy grid is constructed; the values ​​in the three-dimensional probability occupancy grid are mapped to the collision time as a risk field; the collision time is determined by the relative distance, relative speed, and relative acceleration between the autonomous vehicle and the moving target. The decision output module is used to perform opportunity-risk analysis on preset candidate strategies based on the risk field to obtain the opportunity value and risk value of the preset candidate strategies; and to determine the output decision based on the opportunity value and the risk value. The trajectory optimization module is used to determine the planning space based on the comparison result between the risk value in the risk field and the preset risk threshold; using the output decision as the boundary condition and minimizing the cost function value of the risk perception trajectory as the objective, it determines the optimal perception trajectory within the planning space; the cost function is determined based on the risk cost, schedule cost and comfort cost of the risk perception trajectory.

Images