Intelligent trajectory planning prediction method and device, equipment and storage medium

Abstract

The application discloses an intelligent trajectory planning and prediction method and device, equipment and a storage medium, relates to the technical field of automatic driving, and comprises the following steps: acquiring detection bird's-eye view features, occupancy network voxel features and scene semantic map features; generating a self-vehicle trajectory planning based on the detection bird's-eye view features, the occupancy network voxel features and the scene semantic map features, performing trajectory promotion and fusion of potential trajectories, and determining a self-vehicle optimized trajectory planning; determining a predicted target trajectory based on the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features, and completing intelligent trajectory planning and prediction based on the predicted target trajectory. The application generates a self-vehicle trajectory planning by comprehensively integrating different features of multiple sources of information, provides richer scene information, performs trajectory promotion and interactive fusion of potential trajectories, better processes data that has not been seen in the data set, calculates a relation matrix to optimize trajectory prediction, and significantly improves prediction accuracy and reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to an intelligent trajectory planning and prediction method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of automatic driving technology, intelligent planning algorithms are usually used to realize autonomous navigation and decision-making of vehicles to deal with various complex traffic scenes, but there are still some limitations and challenges, such as relying on two-dimensional or three-dimensional features for scene representation, lacking a comprehensive understanding of the depth and complexity of the scene, and existing algorithms often cannot make accurate predictions and decisions when facing rare or extreme traffic situations. Therefore, in a complex traffic environment, the intelligent trajectory of the vehicle needs to be planned and predicted to accurately predict the future driving trajectory of the vehicle.

[0003] At present, the existing method mainly depends on the fusion of various sensor data to obtain three-dimensional features, and then compresses the three-dimensional features into two-dimensional features through high compression, uses two-dimensional features to represent driving scene information, and uses a single trajectory for prediction when predicting the trajectory.

[0004] However, the existing method lacks depth information in three-dimensional space when representing driving scene information, limiting the richness and accuracy of the information, and a large amount of effective data is lost when data compression is performed, and a single prediction trajectory is used, limiting the optimization direction of the gradient during network training, affecting the diversity of trajectory prediction and the optimization effect of network training. Therefore, how to more efficiently and accurately plan and predict intelligent trajectories has become a problem to be solved.

[0005] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0006] The main purpose of the present application is to provide an intelligent trajectory planning and prediction method, device, equipment and storage medium, which aims to solve the technical problem of how to more efficiently and accurately plan and predict intelligent trajectories.

[0007] To achieve the above purpose, the present application provides an intelligent trajectory planning and prediction method, which comprises:

[0008] obtaining detection bird's eye view features, occupancy network voxel features and scene semantic map features;

[0009] generating a self-vehicle trajectory planning based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory promotion fusion potential trajectory to determine a self-vehicle optimized trajectory planning;

[0010] determine a predicted target trajectory based on the ego vehicle optimized trajectory planning, the occupancy network voxel feature, and the scene semantic map feature, and complete intelligent trajectory planning prediction based on the predicted target trajectory.

[0011] In an embodiment, the step of obtaining the detection bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature comprises:

[0012] obtaining a target detection algorithm, surround view camera data, lane information, and driving area information;

[0013] generating an object detection task based on the target detection algorithm and the surround view camera data, detecting a scene object position determination detection bird's eye view feature and an occupancy network voxel dataset based on the object detection task, the occupancy network voxel dataset comprising an occupancy network voxel length, an occupancy network voxel width, and an occupancy network voxel depth;

[0014] determining an occupancy network voxel feature based on the detection bird's eye view feature and the occupancy network voxel dataset;

[0015] establishing a semantic map based on the lane information and the driving area information, and determining a scene semantic map feature.

[0016] In an embodiment, the step of generating an ego vehicle trajectory planning based on the detection bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature, and performing trajectory promotion fusion of potential trajectories to determine an ego vehicle optimized trajectory planning comprises:

[0017] determining an ego vehicle copy trajectory and an ego vehicle update trajectory based on the detection bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature;

[0018] obtaining an ego vehicle trajectory planning based on the ego vehicle copy trajectory and the ego vehicle update trajectory;

[0019] performing trajectory promotion fusion of potential trajectories based on the ego vehicle copy trajectory and the ego vehicle update trajectory to obtain an ego vehicle optimized trajectory planning.

[0020] In an embodiment, the step of determining an ego vehicle copy trajectory and an ego vehicle update trajectory based on the detection bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature comprises:

[0021] determining an ego vehicle trajectory based on the detection bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature;

[0022] copying the ego vehicle trajectory to determine an ego vehicle copy trajectory;

[0023] determine a self vehicle updated trajectory based on the self vehicle trajectory, the detected aerial view feature, the occupancy network voxel feature, and the scene semantic map feature.

[0024] In an embodiment, the step of determining the self vehicle updated trajectory based on the self vehicle trajectory, the detected aerial view feature, the occupancy network voxel feature, and the scene semantic map feature comprises:

[0025] obtaining a scene representation cascaded interaction manner, the scene representation cascaded interaction manner comprising a cross-attention interaction manner and a feature concatenation interaction manner;

[0026] concatenating the self vehicle trajectory, the detected aerial view feature, the occupancy network voxel feature, and the scene semantic map feature based on the scene representation cascaded interaction manner to obtain the self vehicle updated trajectory.

[0027] In an embodiment, the step of performing trajectory promotion fusion potential trajectory based on the self vehicle replicated trajectory and the self vehicle updated trajectory to obtain the self vehicle optimized trajectory planning comprises:

[0028] obtaining a feature dimension;

[0029] performing trajectory promotion fusion potential trajectory based on the feature dimension, the self vehicle replicated trajectory, and the self vehicle updated trajectory to calculate a relationship matrix;

[0030] calculating the self vehicle optimized trajectory planning based on the relationship matrix, the self vehicle replicated trajectory, and the self vehicle updated trajectory.

[0031] In an embodiment, the step of determining the predicted target trajectory based on the self vehicle optimized trajectory planning, the occupancy network voxel feature, and the scene semantic map feature comprises:

[0032] obtaining a scene cascaded operation method;

[0033] concatenating the self vehicle optimized trajectory planning, the occupancy network voxel feature, and the scene semantic map feature based on the scene cascaded operation method to obtain the predicted target trajectory.

[0034] In addition, to achieve the above object, the present application further provides an intelligent trajectory planning and prediction device, which comprises:

[0035] an obtaining module, configured to obtain a detected aerial view feature, an occupancy network voxel feature, and a scene semantic map feature;

[0036] a processing module, configured to generate a self vehicle trajectory planning based on the detected aerial view feature, the occupancy network voxel feature, and the scene semantic map feature, and perform trajectory promotion fusion potential trajectory to determine a self vehicle optimized trajectory planning;

[0037] The execution module is configured to determine a predicted target trajectory based on the ego-vehicle optimized trajectory planning, the occupancy network voxel features, and the scene semantic map features, and to complete intelligent trajectory planning prediction based on the predicted target trajectory.

[0038] In addition, to achieve the above object, the present application further provides an intelligent trajectory planning prediction device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the intelligent trajectory planning prediction method as described above.

[0039] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the intelligent trajectory planning prediction method as described above.

[0040] The one or more technical solutions provided by the present application have at least the following technical effects:

[0041] The intelligent trajectory planning prediction method provided by the present embodiment comprises the following steps: obtaining detection bird's eye view features, occupancy network voxel features, and scene semantic map features; generating ego-vehicle trajectory planning based on the detection bird's eye view features, the occupancy network voxel features, and the scene semantic map features, and performing trajectory promotion to fuse potential trajectories to determine ego-vehicle optimized trajectory planning; determining a predicted target trajectory based on the ego-vehicle optimized trajectory planning, the occupancy network voxel features, and the scene semantic map features, and completing intelligent trajectory planning prediction based on the predicted target trajectory. The present application generates ego-vehicle trajectory planning by integrating detection bird's eye view features, occupancy network voxel features, and scene semantic map features, provides richer scene information, enhances the adaptability and robustness of the algorithm, makes trajectory prediction no longer rely on a single data source or feature, but integrates multi-source information, improves the accuracy and reliability of the prediction result, performs trajectory promotion, enhances feature interaction and fuses potential trajectories through a multi-head cross-attention mechanism and feature splicing operation, better handles data that has not been seen in the data set, improves the decision-making ability of the autonomous driving system when facing rare or extreme traffic situations, calculates a relationship matrix to optimize trajectory prediction, and determines optimized ego-vehicle trajectory planning, thereby realizing accurate prediction of the trajectory of an autonomous vehicle, and significantly improving the adaptability, accuracy, and reliability of the autonomous driving system in complex traffic scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings are only for the purpose of illustrating the embodiments of the present application, and for the person skilled in the art, other drawings can also be obtained without creative labor.

[0044] Figure 1 The flowchart provided by the intelligent trajectory planning prediction method embodiment one of the present application;

[0045] Figure 2 The network voxel representation diagram provided by the intelligent trajectory planning prediction method of the present application;

[0046] Figure 3 The flowchart provided by the intelligent trajectory planning prediction method embodiment two of the present application;

[0047] Figure 4 The brief flowchart of the intelligent trajectory planning prediction method provided by the embodiment of the present application;

[0048] Figure 5 The comparison diagram of the intelligent trajectory planning prediction method high compression method of the present application;

[0049] Figure 6 The structure diagram of the target prediction trajectory obtained by the traditional serial method of the intelligent trajectory planning prediction method of the present application;

[0050] Figure 7 The module structure diagram of the intelligent trajectory planning prediction device of the embodiment of the present application;

[0051] Figure 8 The device structure diagram of the hardware running environment involved in the intelligent trajectory planning prediction method in the embodiment of the present application.

[0052] The purpose of the present application, the function characteristics and the advantages will be further explained by combining with the embodiments and referring to the drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0054] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings in the specification and the specific embodiments.

[0055] The main solution of the embodiment of the present application is: obtaining detection bird's eye view features, occupancy network voxel features and scene semantic map features; generating a self-vehicle trajectory plan based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory promotion to fuse potential trajectories to determine a self-vehicle optimized trajectory plan; determining a predicted target trajectory based on the self-vehicle optimized trajectory plan, the occupancy network voxel features and the scene semantic map features, and completing intelligent trajectory planning prediction based on the predicted target trajectory.

[0056] In the embodiment, the following is described with the identification intelligent trajectory planning prediction device as the execution subject for convenience of description.

[0057] Since the prior art lacks depth information of a three-dimensional space when representing driving scene information, the richness and accuracy of information are limited, and a large amount of effective data is lost when data compression is performed, and a single predicted trajectory is adopted, which limits the optimization direction of a gradient during network training and affects the diversity of trajectory prediction and the optimization effect of network training.

[0058] The present application provides a solution, obtaining detection bird's eye view features, occupancy network voxel features and scene semantic map features; generating a self-vehicle trajectory plan based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory promotion to fuse potential trajectories to determine a self-vehicle optimized trajectory plan; determining a predicted target trajectory based on the self-vehicle optimized trajectory plan, the occupancy network voxel features and the scene semantic map features, and completing intelligent trajectory planning prediction based on the predicted target trajectory.

[0059] From the above embodiment, it can be known that the present application generates a self-vehicle trajectory plan by integrating detection bird's eye view features, occupancy network voxel features and scene semantic map features, provides richer scene information, enhances the adaptability and robustness of the algorithm, so that the trajectory prediction is no longer dependent on a single data source or feature, but integrates multi-source information, improves the accuracy and reliability of the prediction result, and performs trajectory promotion, enhances feature interaction and fusion potential trajectories through a multi-head cross attention mechanism and feature splicing operation, better processes data that has not been seen in the data set, improves the decision-making ability of the automatic driving system when facing rare or extreme traffic situations, calculates a relationship matrix to optimize trajectory prediction, and determines an optimized self-vehicle trajectory plan, thereby realizing accurate prediction of the trajectory of an autonomous vehicle, and significantly improving the adaptability, accuracy and reliability of the automatic driving system in a complex traffic scene.

[0060] Based on this, the embodiment of the present application provides an intelligent trajectory planning prediction method, which is described below with reference to Figure 1 , Figure 1 The flowchart of the first embodiment of the intelligent trajectory planning prediction method of the present application is shown in the figure.

[0061] In this embodiment, the intelligent trajectory planning prediction method comprises steps S10-S30:

[0062] Step S10, obtain the detection bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature;

[0063] It should be noted that the detection bird's eye view feature is to convert multiple perspective surround camera image data into a unified overhead perspective feature centered on the vehicle, the occupancy network voxel feature is to divide a three-dimensional space into many small cubic units, i.e. voxels, to represent the position and volume of objects in space, and the scene semantic map feature is a high-level map representation form describing environmental features and semantic information.

[0064] It can be understood that the detection bird's eye view feature relies on the target detection algorithm LSS-BEV and the surround camera data generation auxiliary task, i.e. using the CetnerPoint method to generate a 3D object detection task, completing the 3D object detection task through the bird's eye view BEV feature built by multiple 360° surround cameras, which can represent the position of objects in the scene, the occupancy network voxel feature retains the occupancy voxel representation method without height compression, and represents the three-dimensional environment with higher accuracy, thereby planning the trajectory, which can make the predicted trajectory more adaptive to the changes of the scene, rather than simply fitting the network training, and the scene semantic map feature includes geographic spatial position geometric information, environmental object information, road information, traffic rule semantic information, which is used to enable the autonomous driving system to more comprehensively understand and perceive the environment, to cover as much as possible the semantic map of the lane and the driving area, to better satisfy the intelligent driving model, to control the driving trajectory of the ego vehicle within the lane range, and to enable the autonomous driving vehicle to more safely and reliably navigate and make decisions.

[0065] For ease of understanding, the acquisition of the detection bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature is taken as an example for description, wherein the information acquisition device is an information acquisition module, and the storage device is a memory.

[0066] The information acquisition module acquires a target detection algorithm, such as the LSS-BEV algorithm, acquires surround camera data, generates an object detection task based on the target detection algorithm and the surround camera data, detects the scene object position based on the object detection task to determine the detection bird's eye view feature and the occupancy network voxel dataset, i.e. using the CetnerPoint method in the LSS-BEV algorithm to generate a 3D object detection task, completing the 3D object detection task through the bird's eye view BEV feature built by multiple 360° surround cameras to obtain the detection bird's eye view feature and the occupancy network voxel dataset, as shown in Figure 2 Figure 2 ​The occupancy network voxel representation diagram is used for the intelligent trajectory planning prediction method of the present application, and the scene representation transmitted in the network model is an occupancy voxel feature, which is:

[0067] B Occ ∈R C×D×H×W

[0068] wherein C represents a single voxel, W represents the length of an occupancy network voxel, H represents the width of an occupancy network voxel, and D represents the depth of an occupancy network voxel, the occupancy network voxel feature is determined based on the detected bird's eye view feature and the occupancy network voxel dataset, that is, a reserved occupancy voxel representation method, and the occupancy network voxel feature obtained by converting the occupancy voxel representation method can be:

[0069]

[0070] wherein R represents a real number field, i corresponds to the depth D of an occupancy network voxel, j corresponds to the width H of an occupancy network voxel, and k corresponds to the length W of an occupancy network voxel.

[0071] Lane information and driving area information are obtained, a semantic map is established based on the lane information and the driving area information, scene semantic map features are determined, that is, a semantic map covering as many lanes and driving areas as possible is obtained, which can better meet the intelligent driving model and control the driving trajectory of the ego vehicle within the lane range.

[0072] In a feasible implementation, step S10 can include steps A11-A14:

[0073] Step A11, obtaining a target detection algorithm, surround view camera data, lane information, and driving area information;

[0074] It should be noted that the target detection algorithm reflects the characteristics of processing image data from a surround view camera and identifying and locating objects in a scene therefrom, the surround view camera data reflects the characteristics of capturing details of the environment around the vehicle, the lane information reflects the characteristics of structured road information, and the driving area information reflects the characteristics of the area in which the vehicle can travel.

[0075] It can be understood that the target detection algorithm, such as the LSS-BEV algorithm, can analyze the surround view camera image content, extract key information, and generate detection bird's eye view features. The surround view camera data can provide 360-degree visual information around the vehicle, capturing environmental details around the vehicle, including road edges, obstacles, other vehicles, and pedestrians. The lane information can represent detailed information of lane boundaries, lane types, and lane directions, determine the exact position of the vehicle on the road, and predict the path it should follow. The driving area information is used to determine the area where the vehicle is driving, thereby handling complex traffic scenarios and providing structured road information to make path planning more reliable and reduce navigation errors caused by environmental understanding errors.

[0076] Step A12, generating an object detection task based on the target detection algorithm and the surround view camera data, detecting scene object positions based on the object detection task, determining detection bird's eye view features and occupancy network voxel data sets, the occupancy network voxel data set including occupancy network voxel length, occupancy network voxel width, and occupancy network voxel depth;

[0077] It can be understood that generating an object detection task to detect scene object positions can construct a comprehensive environmental perception map, including object position information and object three-dimensional space information, enhancing the understanding of scene depth and complexity, more accurately identifying the position of the vehicle on the road, and significantly improving the safety of autonomous vehicles.

[0078] Step A13, determining occupancy network voxel features based on the detection bird's eye view features and the occupancy network voxel data set;

[0079] It can be understood that the occupancy network voxel features can represent scene information through multiple voxels, including color information, transparency information, and object occupancy status, enabling perception to determine object position information, shape information, and speed information in three-dimensional space, effectively identifying and handling obstacles that are not explicitly labeled or have complex shapes, such as irregular vehicles, road stones, and scattered cartons, and more accurately understanding the surrounding environment.

[0080] Step A14, establishing a semantic map based on the lane information and the driving area information, and determining scene semantic map features.

[0081] It can be understood that by establishing a semantic map containing lane and driving area information, the system can more accurately identify the position of the vehicle on the road and plan a path that complies with traffic rules, more precisely control the vehicle's driving trajectory, and ensure its safe driving within the lane range, thereby improving the accuracy of path planning.

[0082] Step S20, based on the detection bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature, generating a self-vehicle trajectory planning, and performing trajectory promotion fusion potential trajectory, determining a self-vehicle optimized trajectory planning;

[0083] It should be noted that the self-vehicle trajectory planning reflects the characteristics of the preliminary trajectory and the potential trajectory generated by comprehensively reflecting the environmental information around the vehicle, and the self-vehicle optimized trajectory planning reflects the characteristics of the more stable trajectory generated by fusing the potential trajectory.

[0084] It can be understood that by integrating multi-source information and dynamic interaction, optimizing the trajectory planning, the safety of the trajectory planning is improved, the risk of potential accidents is reduced, the adaptability and flexibility of the system are improved, and the adaptability and flexibility of the system are improved. Adapt to changing traffic environment and different driving scenarios.

[0085] For the convenience of understanding, taking the determination of the self-vehicle optimized trajectory planning as an example for description, wherein the information acquisition device is an information acquisition module, the storage device is a memory, and the processing device is a processing module.

[0086] The information acquisition module acquires the detection bird's eye view feature B Obj , acquires the occupancy network voxel feature B Occ , acquires the scene semantic map feature B Map , determines the self-vehicle trajectory Q ego based on the detection bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature, copies the self-vehicle trajectory Q ego , determines the self-vehicle copy trajectory Query1, and splices the self-vehicle trajectory, the detection bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature based on the scene representation cascading interaction mode to obtain the self-vehicle updated trajectory Query2, that is, the scene representation cascading interaction mode such as cross attention interaction mode and feature splicing interaction mode is acquired, and after Q ego and B Obj interaction, an updated Q ego is obtained, which is completed by using the multi-head cross attention of transformer, wherein MHCA represents multi-head cross attention, and is represented as:

[0087] Q ego = MHCA(Q ego , B Obj )

[0088] Then B Obj and B Occ are spliced, and then Q ego is interacted, which is completed by using the multi-head cross attention of transformer, and is represented as:

[0089] Q ego= MHCA(Q ego , Concat(B Obj , B Occ ))

[0090] And continue to interact with the concatenation of B ego and B Occ and B Map , using the multi-head cross attention of the transformer, denoted as:

[0091] Q ego = MHCA(Q ego , Concat(B Occ , B Map ))

[0092] Concatenate multiple voxels, that is, concatenate multiple feature dimensions, where MHCA represents multi-head cross attention, Concat represents the concatenation operation of features, R represents the real number field, and the specific calculation is represented as:

[0093]

[0094] The self-vehicle copy trajectory and the self-vehicle update trajectory are subjected to multi-trajectory Query enhancement, potential trajectories are fused based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle update trajectory, a relationship matrix is calculated, and a self-vehicle optimization trajectory planning is calculated based on the relationship matrix, the self-vehicle copy trajectory and the self-vehicle update trajectory, that is, the feature dimension The self-vehicle copy trajectory Query1 is taken as The self-vehicle update trajectory Query2 is taken as Q ego , where Q ego uses the way of cascading scene feature interaction to obtain a group of more stable trajectory prediction results, Then only interact with the 3D object detection BEV feature, and more attention is paid to the drivable path of the predicted trajectory, through more refined Q ego and with more randomness are combined, that is, multi-trajectory Query enhancement is performed, and better results are obtained through network training.

[0095] With the relationship between Q ego and , the relationship matrix between the two can be obtained, defined as M QQ , and can be calculated by relying on Softmax and M QQ can be represented as:

[0096]

[0097] i.e. calculating is represented as:

[0098]

[0099] i.e. calculating is represented as:

[0100]

[0101] wherein, representing Q ego and The feature dimension of K represents 256 because it is a common practice to encode the feature dimension of Query as 256 in the network coding process.

[0102] After obtaining the relationship matrix, reconstruct Q ego and , and calculate

[0103]

[0104] wherein, using Q new representing the enhanced trajectory Query, i.e. obtaining the ego-vehicle optimized trajectory planning.

[0105] Step S30, determining a predicted target trajectory based on the ego-vehicle optimized trajectory planning, the occupancy network voxel feature and the scene semantic map feature, and completing intelligent trajectory planning prediction based on the predicted target trajectory.

[0106] It should be noted that the predicted target trajectory reflects the characteristics of trajectory prediction of the vehicle in a complex and dynamic road environment.

[0107] It can be understood that by accurately predicting the target trajectory of the vehicle, potential collision risks can be effectively avoided, the safety of autonomous driving can be improved, the best driving strategy can be selected in complex traffic conditions, or avoidance decisions can be made quickly in emergency situations, adapt to changing traffic environments and different driving scenarios, improve the adaptability and flexibility of the system, and in the face of rare or extreme traffic conditions, the prediction accuracy can be significantly improved.

[0108] For ease of understanding, the determination of the predicted target trajectory is taken as an example for description, wherein the information collection device is an information collection module, the storage device is a memory, and the execution device is an execution module.

[0109] The information collection module obtains the ego-vehicle optimized trajectory planning Q new , obtains the occupancy network voxel feature B Occ , and obtains the scene semantic map feature B Map, obtain a scene cascading operation method, splice the ego-vehicle optimized trajectory planning, the occupancy network voxel feature and the scene semantic map feature based on the scene cascading operation method to obtain a predicted target trajectory, that is, use Q new Again, the scene cascading operation is performed to complete Q new The interaction with the scene semantic map feature B Map and the occupancy network voxel feature B Occ is represented as:

[0110] Q end = MHCA(Q new , Concat(B Map , B Occ ))

[0111] Finally, the predicted trajectory Q end is obtained, that is, the predicted target trajectory is obtained, and intelligent trajectory planning prediction is completed based on the predicted target trajectory.

[0112] In a feasible implementation, the step S30 can include steps B11-B12.

[0113] Step B11, obtain a scene cascading operation method;

[0114] It should be noted that the scene cascading operation method is to use feature fusion technology to process and integrate feature information from different levels and dimensions.

[0115] Step B12, splice the ego-vehicle optimized trajectory planning, the occupancy network voxel feature and the scene semantic map feature based on the scene cascading operation method to obtain a predicted target trajectory.

[0116] It can be understood that the scene cascading operation method can include feature splicing and feature addition, combine different feature vectors into a complex vector, enhance the processing capacity of the model to information, improve the effect of image processing, and improve the accuracy and reliability of the prediction result.

[0117] The intelligent trajectory planning and prediction method provided in the embodiment comprises the following steps: acquiring detection bird's-eye view features, occupancy network voxel features and scene semantic map features; generating a self-vehicle trajectory planning based on the detection bird's-eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory promotion and fusion of potential trajectories to determine a self-vehicle optimized trajectory planning; determining a predicted target trajectory based on the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features, and completing intelligent trajectory planning and prediction based on the predicted target trajectory. The technical problem of how to more efficiently perform intelligent trajectory planning and prediction is solved. Compared with the prior art, the present application provides richer scene information by integrating detection bird's-eye view features, occupancy network voxel features and scene semantic map features to generate a self-vehicle trajectory planning, enhances the adaptability and robustness of the algorithm, so that trajectory prediction is no longer dependent on a single data source or feature, but integrates multi-source information, improves the accuracy and reliability of the prediction result, and performs trajectory promotion, enhances feature interaction and fusion of potential trajectories through a multi-head cross-attention mechanism and feature splicing operation, better handles data that has not been seen in the data set, improves the decision-making ability of the autonomous driving system when facing rare or extreme traffic situations, calculates a relation matrix to optimize trajectory prediction, determines an optimized self-vehicle trajectory planning, thereby realizing accurate prediction of the trajectory of an autonomous vehicle, and significantly improving the adaptability, accuracy and reliability of the autonomous driving system in complex traffic scenarios.

[0118] Based on the first embodiment of the present application, the same or similar contents as the above-mentioned embodiment one can be referred to the above introduction, and the subsequent will not be described in detail.

[0119] In the embodiment, with reference to Figure 3 , Figure 3 The flowchart provided in the second embodiment of the intelligent trajectory planning and prediction method of the present application comprises steps S21-S23.

[0120] In step S21, a self-vehicle copy trajectory and a self-vehicle update trajectory are determined based on the detection bird's-eye view features, the occupancy network voxel features and the scene semantic map features.

[0121] It should be noted that the self-vehicle copy trajectory reflects the preliminary mapping features of the self-vehicle trajectory, and the self-vehicle update trajectory reflects the dynamic adjustment and optimization features of the self-vehicle trajectory.

[0122] For the convenience of understanding, the determination of the self-vehicle copy trajectory and the self-vehicle update trajectory is taken as an example for description, wherein the information acquisition device is an information acquisition module, the storage device is a memory, and the processing device is a processing module.

[0123] The information acquisition module acquires detection bird's-eye view features B Obj , and acquires occupancy network voxel features BOcc obtaining the scene semantic map feature B Map determining a self vehicle trajectory Q based on the detected bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature ego copying the self vehicle trajectory Q ego determining a self vehicle copy trajectory Query1, and performing subsequent processing based on the self vehicle copy trajectory and a self vehicle update trajectory.

[0124] In an embodiment, step S21 can include steps C11-C13:

[0125] Step C11, determining a self vehicle trajectory based on the detected bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature.

[0126] It should be noted that the self vehicle trajectory reflects the predicted trajectory change of the self vehicle.

[0127] It can be understood that by considering the state of the self vehicle, the future trajectory of the vehicle can be more accurately predicted, potential collisions and conflicts can be avoided, and the safety of the autonomous driving system can be improved.

[0128] Step C12, copying the self vehicle trajectory to determine a self vehicle copy trajectory.

[0129] It can be understood that different trajectory interactions can result in different predicted trajectories, and different trajectory data can produce different optimization directions for the gradient of the network during network training. In order to ensure that the network can more easily find the global optimal optimization direction during training, and to improve the final trajectory prediction result, the self vehicle copy trajectory can be determined by copying the self vehicle trajectory based on the current self vehicle trajectory information, which is similar to the current state. Only interact with the 3D object detection BEV feature, and pay more attention to the drivable path of the predicted trajectory.

[0130] Step C13, determining a self vehicle update trajectory based on the self vehicle trajectory, the detected bird's eye view feature, the occupancy network voxel feature, and the scene semantic map feature.

[0131] It can be understood that the self vehicle update trajectory takes into account more environmental factors and the output of the prediction model to generate a more accurate and adaptive trajectory to future changes, better representing the dynamic intention and predicted path of the self vehicle, thereby improving the accuracy and reliability of trajectory prediction.

[0132] In an embodiment, step C13 can include steps D11-D12:

[0133] Step D11, obtain a scene representation cascaded interaction mode, the scene representation cascaded interaction mode comprising a cross-attention interaction mode and a feature concatenation interaction mode;

[0134] It should be noted that the scene representation cascaded interaction mode reflects the features of the feature concatenation fusion realized by the cascaded structure.

[0135] It can be understood that through the scene representation cascaded interaction mode, the model can more accurately capture the feature relationship between different semantic levels, thereby improving the accuracy of the ego vehicle trajectory prediction, better understanding and generalizing the unseen scenes, improving the adaptability of the model in complex and variable environments, while reducing the consumption of computing resources and improving the efficiency of the model.

[0136] Step D12, based on the scene representation cascaded interaction mode, concatenating the ego vehicle trajectory, detecting the bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature to obtain an ego vehicle updated trajectory.

[0137] It can be understood that the ego vehicle trajectory prediction relies on the accurate understanding of the environment around the vehicle, and the cascaded interaction mode provides more rich and detailed environmental information by integrating features of different scales, thereby more accurately capturing the feature relationship between different semantic levels, thereby improving the accuracy of the ego vehicle trajectory prediction.

[0138] Step S22, based on the ego vehicle copy trajectory and the ego vehicle updated trajectory, obtaining an ego vehicle trajectory planning;

[0139] It can be understood that the ego vehicle trajectory planning integrates the detected bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature, can predict the motion trajectory of the vehicle, and deeply interacts with the surrounding environment and other traffic participants to realize dynamic adaptive trajectory planning. The ego vehicle optimization trajectory planning can process data in the data set that has not been seen before, improve the decision-making ability of the automatic driving system when facing rare or extreme traffic situations, and optimize the trajectory prediction by calculating the relationship matrix to determine the optimized ego vehicle trajectory planning, to realize accurate prediction of the trajectory of the autonomous vehicle.

[0140] For ease of understanding, taking obtaining an ego vehicle trajectory planning as an example, the information acquisition device is an information acquisition module, the storage device is a memory, and the processing device is a processing module.

[0141] The information acquisition module acquires a detected bird's eye view feature B Obj , acquires an occupancy network voxel feature B Occ , acquires a scene semantic map feature B Map , based on the detected bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature, determines an ego vehicle trajectory Q egocopy the ego vehicle trajectory Q ego The ego vehicle copy trajectory Query1 is determined, the ego vehicle trajectory is spliced based on the scene representation cascading interaction mode, the bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature are detected to obtain the ego vehicle update trajectory Query2, that is, the scene representation cascading interaction mode such as cross attention interaction mode and feature splicing interaction mode is obtained, and the ego vehicle trajectory is copied Q ego After interacting with B Obj , the updated Q ego is obtained

[0142] Q ego = MHCA (Q ego , B Obj )

[0143] Then, B Obj is spliced with B Occ , and then Q ego is interacted with, which is completed by using the multi-head cross attention of the transformer, and is represented as:

[0144] Q ego = MHCA (Q ego , Concat (B Obj , B Occ ))

[0145] Q ego is further interacted with the splicing of B Occ and B Map , which is completed by using the multi-head cross attention of the transformer, and is represented as:

[0146] Q ego = MHCA (Q ego , Concat (B Occ , B Map ))

[0147] Splicing multiple voxels, that is, splicing multiple feature dimensions, wherein MHCA represents multi-head cross attention, Concat represents splicing operation of features, R represents real number field, and the specific calculation is represented as:

[0148]

[0149] The ego vehicle trajectory planning is obtained based on the ego vehicle copy trajectory and the ego vehicle update trajectory, and subsequent processing is performed based on the ego vehicle trajectory planning.

[0150] Step S23, based on the self-vehicle copy trajectory and the self-vehicle updated trajectory, trajectory promotion fusion potential trajectory is carried out to obtain self-vehicle optimized trajectory planning.

[0151] It can be understood that by carrying out trajectory promotion fusion potential trajectory to obtain self-vehicle optimized trajectory planning, the safety of trajectory planning is improved, the risk of potential accidents is reduced, the changing traffic environment and different driving scenes are adapted, and the accuracy of self-vehicle trajectory prediction is significantly improved.

[0152] For the convenience of understanding, taking obtaining self-vehicle optimized trajectory planning as an example for description, wherein the information collection device is an information collection module, the storage device is a memory, and the processing device is a processing module.

[0153] The information collection module acquires the self-vehicle copy trajectory Query1, acquires the self-vehicle updated trajectory Query2, carries out multi-trajectory Query enhancement on the self-vehicle copy trajectory and the self-vehicle updated trajectory, carries out trajectory promotion fusion potential trajectory based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle updated trajectory, calculates the relationship matrix, and calculates the self-vehicle optimized trajectory planning based on the relationship matrix, the self-vehicle copy trajectory and the self-vehicle updated trajectory, that is, acquires the feature dimension The self-vehicle copy trajectory Query1 is taken as The self-vehicle updated trajectory Query2 is taken as Q ego , wherein Q ego A set of more stable trajectory prediction results are obtained by using the cascaded scene feature interaction mode, Only interact with the 3D object detection BEV feature, and more attention is paid to the drivable path of the predicted trajectory, and a more refined Q ego and has more randomness are combined, that is, multi-trajectory Query enhancement is carried out, and better results are obtained through network training.

[0154] With the help of Q ego and the relationship between The relationship matrix between the two can be obtained, which is defined as M QQ , and can be calculated by relying on Softmax and M QQ can be represented as:

[0155]

[0156] That is, calculate is represented as:

[0157]

[0158] That is, calculate is represented as:

[0159]

[0160] wherein, representing Q ego and The feature dimension is represented by K, because in the network coding process, the common practice is to encode the feature dimension of Query as 256 dimensions, so K represents 256.

[0161] After obtaining the relationship matrix, reconstruct Q ego and , and calculate the expression as:

[0162]

[0163] wherein, using Q new representing the enhanced trajectory Query, that is, obtaining the optimized trajectory planning of the ego vehicle.

[0164] In a feasible implementation, step S23 can include steps E11-E13:

[0165] Step E11, obtaining a feature dimension;

[0166] It should be noted that the feature dimension reflects the feature of the dimension of the feature space.

[0167] It can be understood that the feature dimension can represent the number of elements in the feature vector, and in image processing, the feature dimension can correspond to the number of feature vectors in the image, affecting the learning and generalization ability of the model. Higher feature dimension can represent more features, thereby capturing more instance information, but the calculation will be more complex and prone to overfitting. Selecting a reasonable feature dimension can help the model learn and generalize better, improve the prediction accuracy of the model, reduce the computational burden of the model, improve the operation efficiency, and optimize the performance of the model.

[0168] Step E12, based on the feature dimension, the ego vehicle copy trajectory and the ego vehicle update trajectory, performing trajectory promotion fusion on the potential trajectory, and calculating a relationship matrix;

[0169] It should be noted that the relationship matrix reflects the correlation and interaction information between features.

[0170] It can be understood that the relationship matrix is used to represent the association strength between different features or data points. The elements of the matrix can represent the existence state of a specific relationship or interaction, thereby quantifying and analyzing the complex relationship between features. In the relationship matrix, 0 and 1 can represent no relationship or relationship between features, thereby converting the formal expression of the relationship into a structure that can be mathematically operated.

[0171] Step E13, calculating the ego-vehicle optimization trajectory planning based on the relationship matrix, the ego-vehicle copy trajectory and the ego-vehicle update trajectory.

[0172] It can be understood that by calculating the ego-vehicle optimization trajectory planning, the trajectory prediction is no longer dependent on a single data source or feature, but integrates multi-source information, improves the accuracy and reliability of the prediction result, significantly improves the safety of trajectory planning, reduces the risk of potential accidents, and adapts to changing traffic environments and different driving scenarios.

[0173] The intelligent trajectory planning prediction method proposed in this embodiment determines the ego-vehicle copy trajectory and the ego-vehicle update trajectory based on the detected bird's eye view features, the occupancy network voxel features and the scene semantic map features; obtains the ego-vehicle trajectory planning based on the ego-vehicle copy trajectory and the ego-vehicle update trajectory; and performs trajectory promotion to fuse potential trajectories based on the ego-vehicle copy trajectory and the ego-vehicle update trajectory to obtain the ego-vehicle optimization trajectory planning. The technical problem of how to more accurately perform intelligent trajectory planning prediction is solved. Compared with the prior art, the present application provides richer scene information by integrating detected bird's eye view features, occupancy network voxel features and scene semantic map features, enhances the adaptability and robustness of the algorithm, accurately generates and fuses potential trajectories to optimize the trajectory planning of the autonomous vehicle, better handles data that has never been seen in the data set, and integrates multi-source information to improve the accuracy and reliability of the prediction result, significantly improving the adaptability, accuracy and reliability of the autonomous driving system in complex scenarios.

[0174] For the sake of understanding the implementation process of the intelligent trajectory planning prediction method obtained after the above-mentioned embodiment one, an example is provided as follows. Figure 4 , Figure 4 A brief flowchart of an intelligent trajectory planning prediction method is provided, specifically:

[0175] Referring to embodiment one, the detected bird's eye view features, the occupancy network voxel features and the scene semantic map features are obtained; the ego-vehicle trajectory planning is generated based on the detected bird's eye view features, the occupancy network voxel features and the scene semantic map features, and potential trajectories are fused to determine the ego-vehicle optimization trajectory planning; the prediction target trajectory is determined based on the ego-vehicle optimization trajectory planning, the occupancy network voxel features and the scene semantic map features, and the intelligent trajectory planning prediction is completed based on the prediction target trajectory. Referring to embodiment two, the ego-vehicle copy trajectory and the ego-vehicle update trajectory are determined based on the detected bird's eye view features, the occupancy network voxel features and the scene semantic map features; the ego-vehicle trajectory planning is obtained based on the ego-vehicle copy trajectory and the ego-vehicle update trajectory; and potential trajectories are fused based on the ego-vehicle copy trajectory and the ego-vehicle update trajectory to obtain the ego-vehicle optimization trajectory planning.

[0176] The existing technical method is mostly composed of 3D detection BEV features and map features in the scene representation of end-to-end intelligent driving. However, with the continuous change of the scene, more data in the dataset that has never been seen before comes along, which is called a corner case. In the face of such problems, the use of the dataset as a training standard cannot meet the requirements of driving, therefore, the 3D occupancy voxel is introduced into the end-to-end autonomous driving task, as shown in Figure 5 . Figure 5 For the intelligent trajectory planning and prediction method of the present application, the existing technical method obtains BEV features by compressing LSS-BEV features, which is a compression method that obtains by compressing the height, simply referred to as height compression, and loses a large amount of effective data, such as converting the original feature dimension from DxHxW to HxW in the compression process, wherein the compression method adopts an accumulation method, that is, converting the D-layer features into 1 layer by layer addition, so that the features change from DxHxW to 1xHxW, at this time, the scene represented by the different values of the D layer is now represented by adding 1 layer, which will lose a lot of representation information, therefore, a more advanced scene representation method is designed, compared with BEV features, 3D occupancy voxel can represent the driving scene information in three-dimensional space, ensuring more information intervention.

[0177] Different scene representations can perceive different details of the scene, the existing technical method will be through a direct way or a serial way, as shown in Figure 6 . Figure 6 For the structure diagram of the target prediction trajectory obtained by the traditional serial way of the intelligent trajectory planning and prediction method of the present application, according to the self-defined order, different scene representation features are interacted with the trajectory planning of the ego vehicle through the serial way, and then the final prediction result is directly obtained, only one feature can be interacted at a time, and in the next interaction, it will be affected by the new feature, therefore, the scene cascade representation interaction method is redesigned, and the multi-head cross attention is used to complete the interaction, which significantly improves the efficiency of the interaction.

[0178] Different trajectory interactions will get different predicted trajectories, and different trajectory data can produce different optimization directions for the gradient of the network during network training. In order to ensure that the network can more easily find the global optimal optimization direction during training, and improve the final trajectory prediction result, a multi-trajectory improvement strategy is designed. The existing method is usually to copy the trajectory branch and calculate the enhancement of the two groups of trajectories respectively. Obviously, it will increase the training difficulty of the network, and if the network structure is completely copied, it will increase the parameter amount of the model and reduce the calculation speed of the network, which is not conducive to the deployment and application of the model. Therefore, the application additionally initializes a set of trajectory predictions, forms a multi-trajectory with the original trajectory, obtains a set of more stable trajectory prediction results by using the cascading scene feature interaction mode of the application, and jointly through more refined trajectories and potential trajectories with more randomness, a better result is obtained through network training.

[0179] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the intelligent trajectory planning and prediction method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0180] The present application also provides an intelligent trajectory planning and prediction device, please refer to Figure 7 , the intelligent trajectory planning and prediction device comprises:

[0181] The acquisition module 10 is used for acquiring the detection bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature;

[0182] The processing module 20 is used for generating the ego vehicle trajectory planning based on the detection bird's eye view feature, the occupancy network voxel feature and the scene semantic map feature, and fusing the potential trajectory for trajectory improvement to determine the ego vehicle optimized trajectory planning;

[0183] The execution module 30 is used for determining the predicted target trajectory based on the ego vehicle optimized trajectory planning, the occupancy network voxel feature and the scene semantic map feature, and completing the intelligent trajectory planning and prediction based on the predicted target trajectory.

[0184] The acquisition module 10 is also used for acquiring the target detection algorithm, the surround view camera data, the lane information and the driving area information;

[0185] The object detection task is generated based on the target detection algorithm and the surround view camera data, the detection bird's eye view feature and the occupancy network voxel dataset are determined based on the object detection task to detect the scene object position, and the occupancy network voxel dataset comprises the occupancy network voxel length, the occupancy network voxel width and the occupancy network voxel depth;

[0186] The occupancy network voxel feature is determined based on the detection bird's eye view feature and the occupancy network voxel dataset;

[0187] establish a semantic map based on the lane information and the travel area information, and determine a scene semantic map feature.

[0188] The processing module 20 is further configured to determine a self-vehicle replication trajectory and a self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0189] The processing module 20 is further configured to determine a self-vehicle replication trajectory and a self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0190] The processing module 20 is further configured to determine a self-vehicle replication trajectory and a self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0191] The processing module 20 is further configured to determine a self-vehicle replication trajectory and a self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0192] The processing module 20 is further configured to determine a self-vehicle replication trajectory and a self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0193] The processing module 20 is further configured to determine a self-vehicle replication trajectory and a self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0194] The processing module 20 is further configured to obtain a scene representation cascading interaction method, the scene representation cascading interaction method including a cross-attention interaction method and a feature splicing interaction method.

[0195] The processing module 20 is further configured to obtain a scene representation cascading interaction method, the scene representation cascading interaction method including a cross-attention interaction method and a feature splicing interaction method.

[0196] The processing module 20 is further configured to obtain a feature dimension.

[0197] The processing module 20 is further configured to obtain a feature dimension.

[0198] The processing module 20 is further configured to obtain a feature dimension.

[0199] The execution module 30 is further configured to obtain a scene cascading operation method.

[0200] The execution module 30 is further configured to obtain a scene cascading operation method.

[0201] The intelligent trajectory planning prediction device provided in the present application adopts the intelligent trajectory planning prediction method in the above embodiment, and can solve the technical problem of how to more efficiently and accurately perform intelligent trajectory planning prediction. Compared with the prior art, the intelligent trajectory planning prediction device provided in the present application has the same beneficial effects as the intelligent trajectory planning prediction method provided in the above embodiment, and other technical features in the intelligent trajectory planning prediction device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0202] The present application provides an intelligent trajectory planning prediction device, which comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the intelligent trajectory planning prediction method in the above embodiment one.

[0203] Reference will now be made to the drawings, in which Figure 8 which shows a structural schematic diagram of an intelligent trajectory planning prediction device suitable for implementing the embodiments of the present application. The intelligent trajectory planning prediction device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 8 The intelligent trajectory planning prediction device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0204] As Figure 8As shown, the intelligent trajectory planning and prediction device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 1002 or loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the intelligent trajectory planning and prediction device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the intelligent trajectory planning and prediction device to communicate wirelessly or wired with other devices to exchange data. Although the intelligent trajectory planning and prediction device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or less systems can be alternatively implemented or possessed.

[0205] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0206] The intelligent trajectory planning and prediction device provided by the present disclosure adopts the intelligent trajectory planning and prediction method in the above embodiments, and can solve the technical problem of how to more efficiently and accurately perform intelligent trajectory planning and prediction. Compared with the prior art, the intelligent trajectory planning and prediction device provided by the present disclosure has the same beneficial effects as the intelligent trajectory planning and prediction method provided by the above embodiments, and other technical features in the intelligent trajectory planning and prediction device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0207] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0208] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any variations and modifications that can be made by any person skilled in the art within the spirit and scope of the application are intended to be encompassed by the application. The scope of the application is defined by the appended claims.

[0209] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the intelligent trajectory planning and prediction method in the above embodiments.

[0210] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any appropriate medium, including but not limited to an electrical wire, an optical cable, an RF (Radio Frequency) cable, etc., or any appropriate combination thereof.

[0211] The above computer readable storage medium can be included in the intelligent trajectory planning and prediction device; or can exist separately and not be assembled into the intelligent trajectory planning and prediction device.

[0212] The computer readable storage medium described above carries one or more programs, when the one or more programs are executed by the intelligent trajectory planning and prediction device, the intelligent trajectory planning and prediction device is caused to: acquire detection bird's eye view features, occupancy network voxel features and scene semantic map features; generate a self-vehicle trajectory planning based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features, and perform trajectory promotion fusion on potential trajectories to determine a self-vehicle optimized trajectory planning; determine a predicted target trajectory based on the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features, and complete intelligent trajectory planning and prediction based on the predicted target trajectory.

[0213] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0214] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified functions. It should also be noted that, in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0215] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0216] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., computer programs) for executing the intelligent trajectory planning and prediction method described above, and can solve the technical problem of how to more efficiently and accurately perform intelligent trajectory planning and prediction. Compared with the prior art, the beneficial effects of the computer readable storage medium provided by the present application are the same as those of the intelligent trajectory planning and prediction method provided by the above-mentioned embodiments, and are not described here.

[0217] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. An intelligent trajectory planning prediction method, characterized in that, The method comprises: acquiring detection bird's eye view features, occupancy network voxel features and scene semantic map features; generating a self-vehicle trajectory plan based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory promotion fusion of potential trajectories to determine a self-vehicle optimized trajectory plan; determining a predicted target trajectory based on the self-vehicle optimized trajectory plan, the occupancy network voxel features and the scene semantic map features, and completing intelligent trajectory planning prediction based on the predicted target trajectory; The step of generating a self-vehicle trajectory plan based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory promotion fusion of potential trajectories to determine a self-vehicle optimized trajectory plan comprises: determining a self-vehicle copy trajectory and a self-vehicle update trajectory based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features; obtaining a self-vehicle trajectory plan based on the self-vehicle copy trajectory and the self-vehicle update trajectory; performing trajectory promotion fusion of potential trajectories based on the self-vehicle copy trajectory and the self-vehicle update trajectory to obtain a self-vehicle optimized trajectory plan.

2. The method of claim 1, wherein, The step of acquiring detection bird's eye view features, occupancy network voxel features and scene semantic map features comprises: acquiring a target detection algorithm, surround camera data, lane information and driving area information; generating an object detection task based on the target detection algorithm and the surround camera data, detecting scene object positions based on the object detection task to determine detection bird's eye view features and occupancy network voxel data sets, the occupancy network voxel data sets comprising occupancy network voxel length, occupancy network voxel width and occupancy network voxel depth; determining occupancy network voxel features based on the detection bird's eye view features and the occupancy network voxel data sets; establishing a semantic map based on the lane information and the driving area information to determine scene semantic map features.

3. The method of claim 1, wherein, The step of determining a self-vehicle copy trajectory and a self-vehicle update trajectory based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features comprises: determining a self-vehicle trajectory based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features; copying the self-vehicle trajectory to determine a self-vehicle copy trajectory; determining a self-vehicle update trajectory based on the self-vehicle trajectory, detection bird's eye view features, the occupancy network voxel features and the scene semantic map features.

4. The method of claim 3, wherein, The step of determining a self-vehicle update trajectory based on the self-vehicle trajectory, detection bird's eye view features, the occupancy network voxel features and the scene semantic map features comprises: acquiring a scene representation cascading interaction mode, the scene representation cascading interaction mode comprising a cross-attention interaction mode and a feature splicing interaction mode; splicing the self-vehicle trajectory, detection bird's eye view features, the occupancy network voxel features and the scene semantic map features based on the scene representation cascading interaction mode to obtain a self-vehicle update trajectory.

5. The method of claim 1, wherein, The step of performing trajectory promotion fusion of potential trajectories based on the self-vehicle copy trajectory and the self-vehicle update trajectory to obtain a self-vehicle optimized trajectory plan comprises: acquiring feature dimensions; The feature dimension, the ego vehicle copy trajectory and the ego vehicle update trajectory are used to perform trajectory promotion fusion on potential trajectories, and a relationship matrix is calculated; The ego vehicle optimization trajectory planning is calculated based on the relationship matrix, the ego vehicle copy trajectory and the ego vehicle update trajectory.

6. The method of claim 1, wherein, The step of determining the predicted target trajectory based on the ego vehicle optimization trajectory planning, the occupancy network voxel feature and the scene semantic map feature comprises: An acquisition method of a scene cascade operation is acquired; The ego vehicle optimization trajectory planning, the occupancy network voxel feature and the scene semantic map feature are spliced based on the acquisition method of the scene cascade operation to obtain the predicted target trajectory.

7. An intelligent trajectory planning prediction device, characterized by, The device comprises: An acquisition module is configured to acquire detection bird's eye view features, occupancy network voxel features and scene semantic map features; A processing module is configured to generate an ego vehicle trajectory planning based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features, perform trajectory promotion fusion on potential trajectories, and determine an ego vehicle optimization trajectory planning; An execution module is configured to determine a predicted target trajectory based on the ego vehicle optimization trajectory planning, the occupancy network voxel feature and the scene semantic map feature, and complete intelligent trajectory planning prediction based on the predicted target trajectory. The processing module is further configured to determine an ego vehicle copy trajectory and an ego vehicle update trajectory based on the detection bird's eye view features, the occupancy network voxel features and the scene semantic map features; The ego vehicle trajectory planning is obtained based on the ego vehicle copy trajectory and the ego vehicle update trajectory; The ego vehicle optimization trajectory planning is obtained based on the ego vehicle copy trajectory and the ego vehicle update trajectory.

8. An intelligent trajectory planning and prediction device, characterized by, The device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the intelligent trajectory planning prediction method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the intelligent trajectory planning prediction method according to any one of claims 1 to 6.

Images