Intelligent trajectory planning prediction method and device, equipment and storage medium

Abstract

The invention discloses an intelligent trajectory planning prediction method and device, equipment and a storage medium, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining and detecting aerial view features, occupied network voxel features and scene semantic map features; generating a self-vehicle trajectory plan based on the detected aerial view features, the occupied network voxel features and the scene semantic map features, performing trajectory lifting and fusing a potential trajectory, and determining a self-vehicle optimized trajectory plan; and determining a prediction target trajectory based on the self-vehicle optimization trajectory planning, the occupied network voxel features and the scene semantic map features, and completing intelligent trajectory planning prediction based on the prediction target trajectory. According to the method and the device, the self-vehicle trajectory planning is generated by integrating different features through integrating multi-source information, richer scene information is provided, trajectory improvement is performed, potential trajectories are interactively fused, data which are not seen in a data set are better processed, a relation matrix is calculated to optimize trajectory prediction, and prediction accuracy and reliability are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to intelligent trajectory planning and prediction methods, devices, equipment and storage media. Background Art

[0002] With the development of autonomous driving technology, intelligent planning algorithms are usually used to achieve autonomous vehicle navigation and decision-making to handle a variety of complex traffic scenarios. However, there are still some limitations and challenges. For example, the scene representation mostly relies on two-dimensional or three-dimensional features and lacks a comprehensive understanding of the depth and complexity of the scene. In addition, existing algorithms are often unable to make accurate predictions and decisions when faced with rare or extreme traffic conditions. Therefore, in complex traffic environments, it is necessary to plan and predict the intelligent trajectory of the vehicle so that the vehicle can accurately predict its future driving trajectory.

[0003] At present, the existing practices mainly rely on the fusion of multiple sensor data to obtain three-dimensional features, and then compress the three-dimensional features into two-dimensional features in a highly compressed manner. The two-dimensional features are used to represent the driving scene information, and when predicting the trajectory, a single trajectory is used for prediction.

[0004] However, the existing approach lacks three-dimensional depth information when representing driving scene information, which limits the richness and accuracy of information, and leads to a large amount of effective data loss during data compression. In addition, the use of a single predicted trajectory limits the optimization direction of the gradient during network training, affecting the diversity of trajectory prediction and the optimization effect of network training. Therefore, how to perform intelligent trajectory planning prediction more efficiently and accurately has become an urgent problem to be solved.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide an intelligent trajectory planning prediction method, device, equipment and storage medium, aiming to solve the technical problem of how to perform intelligent trajectory planning prediction more efficiently and accurately.

[0007] To achieve the above objectives, the present application proposes an intelligent trajectory planning prediction method, the method comprising:

[0008] Obtain detection bird's-eye view features, occupancy network voxel features, and scene semantic map features;

[0009] Generate the self-vehicle trajectory planning based on the detected bird's-eye view features, the occupied network voxel features and the scene semantic map features, and perform trajectory enhancement and fusion of potential trajectories to determine the self-vehicle optimized trajectory planning;

[0010] A predicted target trajectory is determined based on the ego-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features, and intelligent trajectory planning prediction is completed based on the predicted target trajectory.

[0011] In one embodiment, the step of acquiring and detecting bird's-eye view features, occupancy network voxel features, and scene semantic map features comprises:

[0012] Obtain object detection algorithms, surround camera data, lane information, and driving area information;

[0013] Generate an object detection task based on the target detection algorithm and the surround camera data, detect the scene object position based on the object detection task to determine the detection bird's-eye view features and an occupied network voxel dataset, wherein the occupied network voxel dataset includes an occupied network voxel length, an occupied network voxel width, and an occupied network voxel depth;

[0014] determining an occupancy network voxel feature based on the detected bird's eye view feature and the occupancy network voxel dataset;

[0015] A semantic map is established based on the lane information and the driving area information, and scene semantic map features are determined.

[0016] In one embodiment, the step of generating the self-vehicle trajectory planning based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory enhancement to fuse potential trajectories to determine the self-vehicle optimized trajectory planning includes:

[0017] Determine the ego-vehicle copy trajectory and the ego-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features;

[0018] Obtaining a self-vehicle trajectory planning based on the self-vehicle copy trajectory and the self-vehicle update trajectory;

[0019] Based on the self-vehicle copied trajectory and the self-vehicle updated trajectory, trajectory enhancement and fusion of potential trajectories are performed to obtain the self-vehicle optimized trajectory planning.

[0020] In one embodiment, the step of determining the self-vehicle copy trajectory and the self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features comprises:

[0021] Determining an ego-vehicle trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features;

[0022] Copying the ego vehicle trajectory to determine an ego vehicle copy trajectory;

[0023] An updated trajectory of the ego vehicle is determined based on the ego vehicle trajectory, the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0024] In one embodiment, the step of determining the updated trajectory of the ego vehicle based on the ego vehicle trajectory, the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features comprises:

[0025] Acquire a scene representation cascade interaction mode, wherein the scene representation cascade interaction mode includes a cross-attention interaction mode and a feature splicing interaction mode;

[0026] Based on the scene representation cascade interaction mode, the ego vehicle trajectory is spliced, the bird's-eye view features, the occupancy network voxel features and the scene semantic map features are detected to obtain an updated trajectory of the ego vehicle.

[0027] In one embodiment, the step of performing trajectory enhancement and fusion of potential trajectories based on the self-vehicle copied trajectory and the self-vehicle updated trajectory to obtain the self-vehicle optimized trajectory planning includes:

[0028] Get feature dimensions;

[0029] Based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle update trajectory, trajectory enhancement and fusion of potential trajectories are performed to calculate a relationship matrix;

[0030] The ego-vehicle optimized trajectory planning is calculated based on the relationship matrix, the ego-vehicle copied trajectory and the ego-vehicle updated trajectory.

[0031] In one embodiment, the step of determining the predicted target trajectory based on the ego-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features comprises:

[0032] Get the scene cascade operation method;

[0033] Based on the scene cascade operation method, the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features are spliced ​​to obtain a predicted target trajectory.

[0034] In addition, to achieve the above purpose, the present application also proposes an intelligent trajectory planning prediction device, which includes:

[0035] An acquisition module is used to acquire detection bird's-eye view features, occupancy network voxel features, and scene semantic map features;

[0036] A processing module, configured to generate a trajectory plan for the ego vehicle based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features, and to perform trajectory enhancement and fusion of potential trajectories to determine an optimized trajectory plan for the ego vehicle;

[0037] An execution module is used to determine a predicted target trajectory based on the optimized trajectory planning of the ego-vehicle, the occupancy network voxel features and the scene semantic map features, and complete intelligent trajectory planning prediction based on the predicted target trajectory.

[0038] In addition, to achieve the above-mentioned purpose, the present application also proposes an intelligent trajectory planning prediction device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the intelligent trajectory planning prediction method as described above.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the intelligent trajectory planning prediction method described above are implemented.

[0040] One or more technical solutions proposed in this application have at least the following technical effects:

[0041] The present embodiment proposes an intelligent trajectory planning prediction method, which obtains detection bird's-eye view features, occupation network voxel features and scene semantic map features; generates an ego vehicle trajectory planning based on the detection bird's-eye view features, the occupation network voxel features and the scene semantic map features, and performs trajectory lifting and fusion of potential trajectories to determine the ego vehicle optimized trajectory planning; determines a predicted target trajectory based on the ego vehicle optimized trajectory planning, the occupation network voxel features and the scene semantic map features, and completes intelligent trajectory planning prediction based on the predicted target trajectory. This application generates ego vehicle trajectory planning by integrating detection bird's-eye view features, occupancy network voxel features and scene semantic map features, providing richer scene information and enhancing the adaptability and robustness of the algorithm, so that trajectory prediction no longer relies on a single data source or feature, but integrates multi-source information, improving the accuracy and reliability of the prediction results, and performing trajectory enhancement. It enhances the potential trajectory of feature interaction fusion through multi-head cross-attention mechanism and feature splicing operation, better handles unseen data in the data set, improves the decision-making ability of the autonomous driving system in the face of rare or extreme traffic conditions, calculates the relationship matrix to optimize trajectory prediction, and determines the optimized ego vehicle trajectory planning, thereby achieving accurate prediction of the trajectory of the autonomous driving vehicle, significantly improving the adaptability, accuracy and reliability of the autonomous driving system in complex traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 A flowchart of the first embodiment of the intelligent trajectory planning prediction method of the present application is provided;

[0045] Figure 2 A schematic diagram of the voxel representation of the network occupied by the intelligent trajectory planning prediction method of this application;

[0046] Figure 3 A flowchart of the second embodiment of the intelligent trajectory planning prediction method of the present application is provided;

[0047] Figure 4 A brief flowchart of the intelligent trajectory planning prediction method provided in the embodiment of the present application;

[0048] Figure 5 This is a comparison chart of highly compressed methods for intelligent trajectory planning prediction methods in this application;

[0049] Figure 6 A schematic diagram of the target prediction trajectory structure obtained by the traditional serial method of the intelligent trajectory planning prediction method of this application;

[0050] Figure 7 This is a schematic diagram of the module structure of the intelligent trajectory planning and prediction device according to an embodiment of the present application;

[0051] Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the intelligent trajectory planning prediction method in the embodiment of the present application.

[0052] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0054] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0055] The main solution of the embodiment of the present application is: obtaining detection bird's-eye view features, occupation network voxel features and scene semantic map features; generating self-vehicle trajectory planning based on the detection bird's-eye view features, the occupation network voxel features and the scene semantic map features, and performing trajectory enhancement and fusion of potential trajectories to determine the self-vehicle optimized trajectory planning; determining the predicted target trajectory based on the self-vehicle optimized trajectory planning, the occupation network voxel features and the scene semantic map features, and completing intelligent trajectory planning prediction based on the predicted target trajectory.

[0056] In this embodiment, for ease of description, the following description is made by taking the identification of the intelligent trajectory planning prediction device as the execution subject.

[0057] Because the existing technology lacks three-dimensional depth information when representing driving scene information, it limits the richness and accuracy of the information, and will cause a large amount of effective data loss during data compression. In addition, the use of a single prediction trajectory limits the optimization direction of the gradient during network training, affecting the diversity of trajectory prediction and the optimization effect of network training.

[0058] The present application provides a solution, which includes acquiring detection bird's-eye view features, occupation network voxel features and scene semantic map features; generating a self-driving vehicle trajectory planning based on the detection bird's-eye view features, the occupation network voxel features and the scene semantic map features, and performing trajectory enhancement and fusion of potential trajectories to determine the self-driving vehicle optimized trajectory planning; determining a predicted target trajectory based on the self-driving vehicle optimized trajectory planning, the occupation network voxel features and the scene semantic map features, and completing intelligent trajectory planning prediction based on the predicted target trajectory.

[0059] It can be seen from the above embodiments that the present application generates the self-vehicle trajectory planning by integrating the detection of bird's-eye view features, occupancy network voxel features and scene semantic map features, thereby providing richer scene information and enhancing the adaptability and robustness of the algorithm, so that trajectory prediction no longer relies on a single data source or feature, but integrates multi-source information, thereby improving the accuracy and reliability of the prediction results, and performing trajectory enhancement. Through the multi-head cross-attention mechanism and feature splicing operation, the feature interaction fusion potential trajectory is enhanced, and the data that has not been seen in the data set is better processed, and the decision-making ability of the autonomous driving system in the face of rare or extreme traffic conditions is improved. The relationship matrix is ​​calculated to optimize the trajectory prediction, and the optimized self-vehicle trajectory planning is determined, thereby achieving accurate prediction of the trajectory of the autonomous driving vehicle, significantly improving the adaptability, accuracy and reliability of the autonomous driving system in complex traffic scenarios.

[0060] Based on this, the present application embodiment provides an intelligent trajectory planning prediction method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the intelligent trajectory planning and prediction method of the present application.

[0061] In this embodiment, the intelligent trajectory planning prediction method includes steps S10 to S30:

[0062] Step S10, obtaining detection bird's-eye view features, occupancy network voxel features and scene semantic map features;

[0063] It should be noted that the detection bird's-eye view feature converts the surround camera image data of multiple perspectives into a feature map of a unified bird's-eye view centered on the vehicle. The occupancy network voxel feature represents the position and volume of objects in the space by dividing the three-dimensional space into many small cubic units, namely voxels. The scene semantic map feature is a high-level map representation that describes environmental features and semantic information.

[0064] It can be understood that the detection of the bird's-eye view feature relies on the target detection algorithm LSS-BEV and the surround camera data to generate auxiliary tasks, that is, the CetnerPoint method is used to generate a 3D object detection task. The bird's-eye view BEV feature constructed by multiple 360° surround cameras completes the 3D object detection task, which can characterize the location of objects in the scene. The occupancy network voxel feature retains the occupancy voxel representation method without high compression, and represents the three-dimensional environment with higher accuracy, thereby planning the trajectory, which can make the predicted trajectory more adaptable to the changes in the scene, rather than a simple network training fitting. The scene semantic map feature includes geographic spatial position geometry information, environmental object information, road information, and traffic rules semantic information, which is used to enable the autonomous driving system to understand and perceive the environment more comprehensively, and can cover as many semantic maps of lanes and driving areas as possible, better meet the intelligent driving model, and control the driving trajectory of the vehicle within the lane range, so that the autonomous driving vehicle can navigate and make decisions more safely and reliably.

[0065] For ease of understanding, the following is explained by taking the acquisition and detection of bird's-eye view features, occupancy network voxel features and scene semantic map features as an example, wherein the information acquisition device is an information acquisition module and the storage device is a memory.

[0066] The information acquisition module obtains a target detection algorithm, such as the LSS-BEV algorithm, obtains surround camera data, generates an object detection task based on the target detection algorithm and the surround camera data, detects the scene object position based on the object detection task, determines the detection bird's-eye view features and the occupancy network voxel data set, that is, uses the CetnerPoint method in the LSS-BEV algorithm to generate a 3D object detection task, and completes the 3D object detection task through the bird's-eye view BEV features constructed by multiple 360° surround cameras, and obtains the detection bird's-eye view features and the occupancy network voxel data set, such as Figure 2 As shown, Figure 2This is a schematic diagram of the network voxel representation of the intelligent trajectory planning prediction method of this application. The scene transmitted in the network model is represented by the occupied voxel features:

[0067] B Occ ∈R C×D×H×W

[0068] Wherein, C represents a single voxel, W represents the length of the occupied network voxel, H represents the width of the occupied network voxel, and D represents the depth of the occupied network voxel. Based on the detected bird's-eye view features and the occupied network voxel data set, the occupied network voxel features are determined, that is, the occupied voxel representation method is retained, and the occupied network voxel features obtained by converting the occupied voxel representation method can be:

[0069]

[0070] Among them, R represents the real number field, i corresponds to the occupied network voxel depth D, j corresponds to the occupied network voxel width H, and k corresponds to the occupied network voxel length W.

[0071] The lane information and the driving area information are obtained, a semantic map is established based on the lane information and the driving area information, and the scene semantic map features are determined, that is, a semantic map covering as many lanes and driving areas as possible is obtained, which can better meet the intelligent driving model and control the driving trajectory of the vehicle within the lane range.

[0072] In a feasible implementation, step S10 may include steps A11 to A14:

[0073] Step A11, obtaining target detection algorithm, surround camera data, lane information and driving area information;

[0074] It should be noted that the target detection algorithm reflects the characteristics of processing image data from the surround view camera and identifying and locating objects in the scene therefrom, the surround view camera data reflects the characteristics of capturing the environmental details around the vehicle, the lane information reflects the characteristics of structured road information, and the driving area information reflects the characteristics of the area where the vehicle can drive.

[0075] It can be understood that the target detection algorithm, such as the LSS-BEV algorithm, can analyze the surround camera image content and extract key information to generate detection bird's-eye view features. The surround camera data can provide 360-degree viewing information around the vehicle and capture environmental details around the vehicle, including road edges, obstacles, other vehicles and pedestrians. The lane information can characterize detailed information on lane boundaries, lane types and lane directions, determine the exact position of the vehicle on the road, and predict the path it should follow. The driving area information is used to determine the area where the vehicle is driving, thereby handling complex traffic scenes and providing structured road information, making path planning more reliable and reducing navigation errors caused by misunderstanding of the environment.

[0076] Step A12, generating an object detection task based on the target detection algorithm and the surround camera data, detecting the scene object position based on the object detection task to determine the detection bird's-eye view features and an occupied network voxel dataset, wherein the occupied network voxel dataset includes an occupied network voxel length, an occupied network voxel width, and an occupied network voxel depth;

[0077] It is understandable that generating object detection tasks to detect the location of scene objects can build a comprehensive environmental perception map, including object location information and object three-dimensional spatial information, enhance the understanding of scene depth and complexity, more accurately identify the vehicle's position on the road, and significantly improve the safety of autonomous vehicles.

[0078] Step A13, determining an occupancy network voxel feature based on the detected bird's-eye view feature and the occupancy network voxel dataset;

[0079] It can be understood that the occupancy network voxel feature can enrich the scene representation information through multiple voxels, and the voxels include color information, transparency information and object occupancy status, so that perception can determine the position information, shape information and speed information of the object in three-dimensional space, effectively identify and process obstacles that are not clearly marked or have complex shapes, such as special-shaped cars, stones on the road and scattered cardboard boxes, and more accurately understand the surrounding environment.

[0080] Step A14, establishing a semantic map based on the lane information and the driving area information, and determining scene semantic map features.

[0081] It can be understood that by establishing a semantic map containing lane and driving area information, the system can more accurately identify the vehicle's position on the road and plan a path that complies with traffic regulations, more accurately control the vehicle's driving trajectory, and ensure its safe driving within the lane range, thereby improving the accuracy of path planning.

[0082] Step S20, generating a trajectory plan for the ego vehicle based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features, and performing trajectory enhancement and fusion of potential trajectories to determine an optimized trajectory plan for the ego vehicle;

[0083] It should be noted that the ego vehicle trajectory planning reflects the characteristics of the preliminary trajectory and potential trajectory generated by integrating the environmental information around the vehicle, and the ego vehicle optimized trajectory planning reflects the characteristics of a more stable trajectory generated by integrating the potential trajectory.

[0084] It can be understood that by integrating multi-source information and dynamic interaction, trajectory planning is optimized, the safety of trajectory planning is improved, the risk of potential accidents is reduced, the system can adapt to changing traffic environments and different driving scenarios, and its adaptability and flexibility are improved.

[0085] For ease of understanding, the optimization trajectory planning of the self-vehicle is determined as an example for explanation, wherein the information collection device is the information collection module, the storage device is the memory, and the processing device is the processing module.

[0086] The information acquisition module obtains the detection bird's-eye view feature B Obj , obtain the occupancy network voxel feature B Occ , obtain the scene semantic map feature B Map , determine the vehicle trajectory Q based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features ego , copy the vehicle trajectory Q ego Determine the ego vehicle copy trajectory Query1, splice the ego vehicle trajectory based on the scene representation cascade interaction mode, detect the bird's-eye view features, the occupancy network voxel features and the scene semantic map features to obtain the ego vehicle update trajectory Query2, that is, obtain the scene representation cascade interaction mode, such as the cross-attention interaction mode and the feature splicing interaction mode, in Q ego With B Obj After interaction, we get the updated Q ego , which is done using transformer’s multi-head cross attention, where MHCA stands for multi-head cross attention, expressed as:

[0087] Q ego =MHCA(Q ego ,B Obj )

[0088] Then B Obj With B Occ Perform feature splicing and then combine with Q ego To interact, we use the multi-head cross attention of transformer, which can be expressed as:

[0089] Q ego=MHCA(Q ego ,Concat(B Obj ,B Occ ))

[0090] And Q ego Continue with B Occ and B Map The splicing is interactive, which is completed by using the multi-head cross attention of transformer, expressed as:

[0091] Q ego =MHCA(Q ego ,Concat(B Occ ,B Map ))

[0092] Concatenating multiple voxels means concatenating multiple feature dimensions, where MHCA stands for multi-head cross attention, Concat stands for the concatenation operation of features, and R stands for the real number domain. The specific calculation is expressed as:

[0093]

[0094] The self-vehicle copy trajectory and the self-vehicle update trajectory are subjected to multi-trajectory query enhancement, and the trajectory is enhanced and fused with potential trajectories based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle update trajectory, and the relationship matrix is ​​calculated. The self-vehicle optimized trajectory planning is calculated based on the relationship matrix, the self-vehicle copy trajectory and the self-vehicle update trajectory, that is, the feature dimension is obtained. The self-car copy trajectory Query1 is used as The updated trajectory Query2 of the ego vehicle is used as Q ego , where Q ego A more stable set of trajectory prediction results is obtained by using the cascade scene feature interaction method. It only interacts with the 3D object detection BEV features and focuses more on predicting the drivable path of the trajectory. ego and more random Perform a combination, that is, perform multi-trajectory query enhancement, and obtain better results through network training.

[0095] With the help of Q ego and The relationship between the two can be obtained by defining the relationship matrix between them as M QQ , and can be calculated by relying on Softmax and M QQ It can be expressed as:

[0096]

[0097] Calculate It is expressed as:

[0098]

[0099] Calculate It is expressed as:

[0100]

[0101] in, Representative Q ego and In the process of network coding, the common practice is to encode the feature dimension of Query into 256 dimensions, so K represents 256.

[0102] After obtaining the relationship matrix, ego and After reconstruction, the calculation is expressed as:

[0103]

[0104] Among them, using Q new Represents the enhanced trajectory query, that is, the optimized trajectory planning of the vehicle is obtained.

[0105] Step S30, determining a predicted target trajectory based on the ego-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features, and completing intelligent trajectory planning prediction based on the predicted target trajectory.

[0106] It should be noted that the predicted target trajectory reflects the characteristics of vehicle trajectory prediction in a complex and dynamic road environment.

[0107] It is understandable that by accurately predicting the target trajectory of the vehicle, potential collision risks can be effectively avoided, the safety of autonomous driving can be improved, and the best driving strategy can be selected in complex traffic conditions, or avoidance decisions can be made quickly in emergency situations, adapting to changing traffic environments and different driving scenarios. The adaptability and flexibility of the system are improved, and accurate predictions can be made even in the face of rare or extreme traffic conditions, significantly improving the accuracy of the predictions.

[0108] For ease of understanding, the following is explained by taking the determination of the predicted target trajectory as an example, wherein the information collection device is an information collection module, the storage device is a memory, and the execution device is an execution module.

[0109] The information collection module obtains the vehicle's optimized trajectory planning Q new , obtain the occupancy network voxel feature B Occ , obtain the scene semantic map feature B Map, obtain a scene cascade operation method, and splice the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features based on the scene cascade operation method to obtain a predicted target trajectory, that is, use Q new Perform the scene cascade operation again to complete Q new and scene semantic map feature B Map and occupancy network voxel feature B Occ The interaction is expressed as:

[0110] Q end =MHCA(Q new ,Concat(B Map ,B Occ ))

[0111] Get the final predicted trajectory Q end , that is, the predicted target trajectory is obtained, and the intelligent trajectory planning prediction is completed based on the predicted target trajectory.

[0112] In a feasible implementation, step S30 may include steps B11 to B12:

[0113] Step B11, obtaining the scene cascade operation method;

[0114] It should be noted that the scene cascade operation method uses feature fusion technology to process and integrate feature information from different levels and dimensions.

[0115] Step B12, based on the scene cascade operation method, the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features are spliced ​​to obtain a predicted target trajectory.

[0116] It can be understood that the scene cascade operation method may include feature concatenation and feature addition, combining different feature vectors into complex vectors, enhancing the model's ability to process information, improving the effect of image processing, and improving the accuracy and reliability of prediction results.

[0117] The present embodiment proposes an intelligent trajectory planning prediction method, which obtains detection bird's-eye view features, occupation network voxel features and scene semantic map features; generates an ego vehicle trajectory planning based on the detection bird's-eye view features, the occupation network voxel features and the scene semantic map features, and performs trajectory lifting and fusion of potential trajectories to determine the ego vehicle optimized trajectory planning; determines a predicted target trajectory based on the ego vehicle optimized trajectory planning, the occupation network voxel features and the scene semantic map features, and completes intelligent trajectory planning prediction based on the predicted target trajectory. The technical problem of how to perform intelligent trajectory planning and prediction more efficiently is solved. Compared with the existing technology, the present application generates the self-vehicle trajectory planning by integrating the detection of bird's-eye view features, occupancy network voxel features and scene semantic map features, providing richer scene information and enhancing the adaptability and robustness of the algorithm, so that trajectory prediction no longer relies on a single data source or feature, but integrates multi-source information, improves the accuracy and reliability of the prediction results, and performs trajectory enhancement. Through the multi-head cross-attention mechanism and feature splicing operation, the feature interaction fusion potential trajectory is enhanced, and the data that has not been seen in the data set is better processed, and the decision-making ability of the autonomous driving system in the face of rare or extreme traffic conditions is improved. The relationship matrix is ​​calculated to optimize the trajectory prediction, and the optimized self-vehicle trajectory planning is determined, thereby realizing accurate prediction of the trajectory of the autonomous driving vehicle, significantly improving the adaptability, accuracy and reliability of the autonomous driving system in complex traffic scenarios.

[0118] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction and will not be repeated later.

[0119] In this embodiment, refer to Figure 3 , Figure 3 The flowchart of the second embodiment of the intelligent trajectory planning prediction method of this application is provided, and step S20 specifically includes steps S21 to S23:

[0120] Step S21, determining the self-vehicle copy trajectory and the self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features;

[0121] It should be noted that the ego vehicle copy trajectory reflects the characteristics of the preliminary mapping of the ego vehicle trajectory, and the ego vehicle update trajectory reflects the characteristics of the dynamic adjustment and optimization of the ego vehicle trajectory.

[0122] For ease of understanding, the determination of the self-vehicle copy trajectory and the self-vehicle update trajectory is taken as an example for explanation, wherein the information collection device is the information collection module, the storage device is the memory, and the processing device is the processing module.

[0123] The information acquisition module obtains the detection bird's-eye view feature B Obj , obtain the occupancy network voxel feature BOcc , obtain the scene semantic map feature B Map , determine the vehicle trajectory Q based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features ego , copy the vehicle trajectory Q ego Determine the self-vehicle duplicate trajectory Query1, and perform subsequent processing based on the self-vehicle duplicate trajectory and the self-vehicle updated trajectory.

[0124] In a feasible implementation, step S21 may include steps C11 to C13:

[0125] Step C11, determining the vehicle trajectory based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features;

[0126] It should be noted that the vehicle trajectory reflects the characteristics of the predicted trajectory change of the vehicle.

[0127] It can be understood that by considering the state of the ego vehicle, the future trajectory of the vehicle can be predicted more accurately, avoiding potential collisions and conflicts, thereby improving the safety of the autonomous driving system.

[0128] Step C12, copying the vehicle trajectory to determine the vehicle copy trajectory;

[0129] It is understandable that different trajectory interactions can result in different predicted trajectories, and different trajectory data can produce different optimization directions for the network gradient during network training. In order to ensure that the network can more easily find the global optimal optimization direction during training and improve the final trajectory prediction result, the ego vehicle trajectory can be copied to determine the ego vehicle copy trajectory, which is based on the current ego vehicle trajectory information, copies a trajectory similar to the current state, and only interacts with the 3D object detection BEV feature, focusing more on the drivable path of the predicted trajectory.

[0130] Step C13, determining an updated trajectory of the ego vehicle based on the ego vehicle trajectory, the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0131] It can be understood that the updated trajectory of the ego vehicle takes into account more environmental factors and the output of the prediction model to generate a trajectory that is more accurate and adaptable to future changes, better characterizes the dynamic intention and predicted path of the ego vehicle, and thus improves the accuracy and reliability of trajectory prediction.

[0132] In a feasible implementation, step C13 may include steps D11 to D12:

[0133] Step D11, obtaining a scene representation cascade interaction mode, wherein the scene representation cascade interaction mode includes a cross-attention interaction mode and a feature splicing interaction mode;

[0134] It should be noted that the scene representation cascade interaction method reflects the feature of realizing feature splicing and fusion through a cascade structure.

[0135] It is understandable that through the cascading interaction of scene representation, the model can more accurately capture the feature relationship between different semantic levels, thereby improving the accuracy of vehicle trajectory prediction, and better understand and generalize unseen scenes, improving the adaptability of the model in complex and changing environments, while reducing the consumption of computing resources and improving the efficiency of the model.

[0136] Step D12, based on the scene representation cascade interaction mode, the ego vehicle trajectory is spliced, the bird's-eye view features are detected, the occupancy network voxel features and the scene semantic map features are detected to obtain an updated trajectory of the ego vehicle.

[0137] Understandably, ego-vehicle trajectory prediction relies on accurate understanding of the vehicle’s surroundings, and the cascaded interaction approach provides richer and more detailed environmental information by integrating features at different scales, thereby more accurately capturing the feature relationships between different semantic levels, thereby improving the accuracy of ego-vehicle trajectory prediction.

[0138] Step S22, obtaining a trajectory planning of the ego vehicle based on the ego vehicle copy trajectory and the ego vehicle update trajectory;

[0139] It can be understood that the self-driving vehicle trajectory planning integrates the detection of bird's-eye view features, occupancy network voxel features and scene semantic map features, can predict the vehicle's movement trajectory, and deeply interact with the surrounding environment and other traffic participants to achieve dynamic adaptive trajectory planning. The self-driving vehicle optimized trajectory planning can process data that has not been seen in the data set, improve the decision-making ability of the autonomous driving system when facing rare or extreme traffic conditions, and optimize the trajectory prediction by calculating the relationship matrix to determine the optimized self-driving vehicle trajectory planning, so as to achieve accurate prediction of the trajectory of the autonomous driving vehicle.

[0140] For ease of understanding, the following is explained by taking the vehicle trajectory planning as an example, wherein the information collection device is the information collection module, the storage device is the memory, and the processing device is the processing module.

[0141] The information acquisition module obtains the detection bird's-eye view feature B Obj , obtain the occupancy network voxel feature B Occ , obtain the scene semantic map feature B Map , determine the vehicle trajectory Q based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features ego, copy the vehicle trajectory Q ego Determine the ego vehicle copy trajectory Query1, splice the ego vehicle trajectory based on the scene representation cascade interaction mode, detect the bird's-eye view features, the occupancy network voxel features and the scene semantic map features to obtain the ego vehicle update trajectory Query2, that is, obtain the scene representation cascade interaction mode, such as the cross-attention interaction mode and the feature splicing interaction mode, in Q ego With B Obj After interaction, we get the updated Q ego , which is done using transformer’s multi-head cross attention, where MHCA stands for multi-head cross attention, expressed as:

[0142] Q ego =MHCA(Q ego ,B Obj )

[0143] Then B Obj With B Occ Perform feature splicing and then combine with Q ego To interact, we use the multi-head cross attention of transformer, which can be expressed as:

[0144] Q ego =MHCA(Q ego ,Concat(B Obj ,B Occ ))

[0145] And Q ego Continue with B Occ and B Map The splicing is interactive, which is completed by using the multi-head cross attention of transformer, expressed as:

[0146] Q ego =MHCA(Q ego ,Concat(B Occ ,B Map ))

[0147] Concatenating multiple voxels means concatenating multiple feature dimensions, where MHCA stands for multi-head cross attention, Concat stands for the concatenation operation of features, and R stands for the real number domain. The specific calculation is expressed as:

[0148]

[0149] A self-vehicle trajectory planning is obtained based on the self-vehicle copy trajectory and the self-vehicle update trajectory, and subsequent processing is performed based on the self-vehicle trajectory planning.

[0150] Step S23, performing trajectory enhancement and fusion of potential trajectories based on the ego vehicle copied trajectory and the ego vehicle updated trajectory to obtain an ego vehicle optimized trajectory planning.

[0151] It can be understood that by performing trajectory improvement and fusing potential trajectories to obtain the optimized trajectory planning of the ego vehicle, the safety of trajectory planning is improved, the risk of potential accidents is reduced, it adapts to changing traffic environments and different driving scenarios, and significantly improves the accuracy of ego vehicle trajectory prediction.

[0152] For ease of understanding, the optimization trajectory planning of the self-vehicle is taken as an example for explanation, wherein the information collection device is the information collection module, the storage device is the memory, and the processing device is the processing module.

[0153] The information acquisition module obtains the self-vehicle copy trajectory Query1, obtains the self-vehicle update trajectory Query2, performs multi-trajectory Query enhancement on the self-vehicle copy trajectory and the self-vehicle update trajectory, performs trajectory enhancement and fusion of potential trajectories based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle update trajectory, calculates the relationship matrix, and calculates the self-vehicle optimized trajectory planning based on the relationship matrix, the self-vehicle copy trajectory and the self-vehicle update trajectory, that is, obtains the feature dimension The self-car copy trajectory Query1 is used as The updated trajectory Query2 of the ego vehicle is used as Q ego , where Q ego A more stable set of trajectory prediction results is obtained by using the cascade scene feature interaction method. It only interacts with the 3D object detection BEV features and focuses more on predicting the drivable path of the trajectory. ego and more random Perform a combination, that is, perform multi-trajectory query enhancement, and obtain better results through network training.

[0154] With the help of Q ego and The relationship between the two can be obtained by defining the relationship matrix between them as M QQ , and can be calculated by relying on Softmax and M QQ It can be expressed as:

[0155]

[0156] Calculate It is expressed as:

[0157]

[0158] Calculate It is expressed as:

[0159]

[0160] in, Representative Q ego and In the process of network coding, the common practice is to encode the feature dimension of Query into 256 dimensions, so K represents 256.

[0161] After obtaining the relationship matrix, ego and After reconstruction, the calculation is expressed as:

[0162]

[0163] Among them, using Q new Represents the enhanced trajectory query, that is, the optimized trajectory planning of the vehicle is obtained.

[0164] In a feasible implementation, step S23 may include steps E11 to E13:

[0165] Step E11, obtaining feature dimensions;

[0166] It should be noted that the feature dimension reflects the characteristics of the dimensionality of the feature space.

[0167] It can be understood that the feature dimension can represent the number of elements in the feature vector. In image processing, the feature dimension can correspond to the number of feature vectors in the image, which affects the learning and generalization ability of the model. Higher feature dimensions can represent more features, thereby capturing more instance information, but the calculation will be more complicated and prone to overfitting. Choosing a reasonable feature dimension can help the model learn and generalize better, improve the model's prediction accuracy, reduce the model's computational burden, improve operational efficiency, and optimize the model's performance.

[0168] Step E12, performing trajectory enhancement and fusion of potential trajectories based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle update trajectory, and calculating a relationship matrix;

[0169] It should be noted that the relationship matrix reflects the correlation between features and the characteristics of the interactive information.

[0170] It can be understood that the relationship matrix is ​​used to represent the strength of association between different features or data points, and the elements of the matrix can characterize the existence state of a specific relationship or interaction, thereby quantifying and analyzing the complex relationship between features, wherein 0 and 1 in the relationship matrix can indicate that there is no relationship or a relationship between the features, thereby converting the formal expression of the relationship into a structure that can be mathematically operated.

[0171] Step E13, calculating the ego vehicle optimized trajectory planning based on the relationship matrix, the ego vehicle copied trajectory and the ego vehicle updated trajectory.

[0172] It is understandable that by calculating the optimized trajectory planning of the self-driving car, the trajectory prediction no longer relies on a single data source or feature, but integrates multi-source information, which improves the accuracy and reliability of the prediction results, significantly improves the safety of trajectory planning, reduces the risk of potential accidents, and adapts to changing traffic environments and different driving scenarios.

[0173] This embodiment proposes an intelligent trajectory planning prediction method, which determines the self-vehicle copy trajectory and the self-vehicle update trajectory based on the detected bird's-eye view features, the occupied network voxel features and the scene semantic map features; obtains the self-vehicle trajectory planning based on the self-vehicle copy trajectory and the self-vehicle update trajectory; and performs trajectory enhancement and fusion of potential trajectories based on the self-vehicle copy trajectory and the self-vehicle update trajectory to obtain the self-vehicle optimized trajectory planning. The technical problem of how to more accurately perform intelligent trajectory planning prediction is solved. Compared with the prior art, this application provides richer scene information by comprehensively detecting bird's-eye view features, occupied network voxel features and scene semantic map features, enhances the adaptability and robustness of the algorithm, accurately generates and fuses potential trajectories to optimize the trajectory planning of the autonomous driving vehicle, better handles data that has never been seen in the data set, and integrates multi-source information to improve the accuracy and reliability of the prediction results, significantly improving the adaptability, accuracy and reliability of the autonomous driving system in complex scenes.

[0174] For example, in order to help understand the implementation process of the intelligent trajectory planning prediction method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 4 , Figure 4 A brief flowchart of an intelligent trajectory planning prediction method is provided, specifically:

[0175] Referring to Example 1, the detection bird's-eye view features, the occupancy network voxel features, and the scene semantic map features are obtained; based on the detection bird's-eye view features, the occupancy network voxel features, and the scene semantic map features, the self-vehicle trajectory planning is generated, and the trajectory is enhanced and fused with potential trajectories to determine the self-vehicle optimized trajectory planning; based on the self-vehicle optimized trajectory planning, the occupancy network voxel features, and the scene semantic map features, the predicted target trajectory is determined, and the intelligent trajectory planning prediction is completed based on the predicted target trajectory. Referring to Example 2, the self-vehicle copy trajectory and the self-vehicle update trajectory are determined based on the detection bird's-eye view features, the occupancy network voxel features, and the scene semantic map features; the self-vehicle trajectory planning is obtained based on the self-vehicle copy trajectory and the self-vehicle update trajectory; based on the self-vehicle copy trajectory and the self-vehicle update trajectory, the trajectory is enhanced and fused with potential trajectories to obtain the self-vehicle optimized trajectory planning.

[0176] In the scene representation of end-to-end intelligent driving, the existing technical methods mostly use 3D detection BEV features and map features. However, as the scene continues to change, more data that has never been seen in the data set comes along, which is called conrner case. Faced with such a problem, the method of using the data set as the training standard can no longer meet the requirements of driving. Therefore, this application introduces 3D occupied voxels into the end-to-end autonomous driving task, such as Figure 5 As shown, Figure 5 This is a comparison chart of the height compression method of the intelligent trajectory planning prediction method of this application. The existing technical method compresses the LSS-BEV feature to obtain the BEV feature. This compression method is obtained by compressing the height, which is referred to as height compression. A large amount of effective data is lost. For example, the original feature dimension is converted from DxHxW to HxW during the compression process, wherein the compression method is obtained in an accumulation manner, that is, the feature of the D layer is converted into 1 layer by adding layer by layer, so that the feature changes from DxHxW to 1xHxW. At this time, the scene originally represented by different numerical values ​​of the D layer is now represented by adding them into 1 layer, which will lose a lot of representation information. Therefore, a more advanced scene representation method is designed. Compared with the BEV feature, the 3D occupied voxel can represent the driving scene information in three-dimensional space, ensuring the intervention of more information.

[0177] Different scene representations can perceive different details of the scene. Existing technical methods use direct or serial methods, such as Figure 6 As shown, Figure 6 The intelligent trajectory planning prediction method of this application obtains the target prediction trajectory structure schematic diagram in a traditional serial manner. Different scene representation features are gradually interacted with the trajectory planning of the vehicle in a series manner in a customized order, and then the final prediction result is directly obtained. Only one feature can be interacted at a time, and it will be affected by the new feature in the next interaction. Therefore, this application redesigns the scene cascade representation interaction method and uses multi-head cross-attention to complete the interaction, which significantly improves the efficiency of the interaction.

[0178] Different trajectory interactions will result in different predicted trajectories, and different trajectory data can produce different optimization directions for the network's gradient during network training. In order to ensure that the network can more easily find the global optimal optimization direction during training and improve the final trajectory prediction results, a multi-trajectory improvement strategy is designed. The existing practice is usually to copy the trajectory branch and perform computational enhancement on the two sets of trajectories respectively. This will obviously increase the difficulty of network training. Moreover, if a complete copy of the network structure is made, the number of model parameters will increase, the network's calculation speed will be reduced, and it will be detrimental to the deployment and application of the model. Therefore, the present application initializes an additional set of trajectory predictions to form multiple trajectories with the original trajectories, and uses the cascaded scene feature interaction method of the present application to obtain a set of more stable trajectory prediction results. By combining more refined trajectories with potential trajectories with more randomness, better results are obtained through network training.

[0179] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the intelligent trajectory planning prediction method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0180] This application also provides an intelligent trajectory planning prediction device, please refer to Figure 7 , the intelligent trajectory planning prediction device comprises:

[0181] An acquisition module 10 is used to acquire detection bird's-eye view features, occupancy network voxel features and scene semantic map features;

[0182] A processing module 20 is used to generate a trajectory plan of the ego vehicle based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features, and to perform trajectory enhancement and fusion of potential trajectories to determine an optimized trajectory plan of the ego vehicle;

[0183] The execution module 30 is used to determine the predicted target trajectory based on the optimized trajectory planning of the ego-vehicle, the occupancy network voxel features and the scene semantic map features, and complete the intelligent trajectory planning prediction based on the predicted target trajectory.

[0184] The acquisition module 10 is also used to acquire target detection algorithms, surround camera data, lane information and driving area information;

[0185] Generate an object detection task based on the target detection algorithm and the surround camera data, detect the scene object position based on the object detection task to determine the detection bird's-eye view features and an occupied network voxel dataset, wherein the occupied network voxel dataset includes an occupied network voxel length, an occupied network voxel width, and an occupied network voxel depth;

[0186] determining an occupancy network voxel feature based on the detected bird's eye view feature and the occupancy network voxel dataset;

[0187] A semantic map is established based on the lane information and the driving area information, and scene semantic map features are determined.

[0188] The processing module 20 is further used to determine the ego-vehicle copy trajectory and the ego-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features;

[0189] Obtaining a self-vehicle trajectory planning based on the self-vehicle copy trajectory and the self-vehicle update trajectory;

[0190] Based on the self-vehicle copied trajectory and the self-vehicle updated trajectory, trajectory enhancement and fusion of potential trajectories are performed to obtain the self-vehicle optimized trajectory planning.

[0191] The processing module 20 is further used to determine the trajectory of the vehicle based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features;

[0192] Copying the ego vehicle trajectory to determine an ego vehicle copy trajectory;

[0193] An updated trajectory of the ego vehicle is determined based on the ego vehicle trajectory, the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

[0194] The processing module 20 is further used to obtain a scene representation cascade interaction mode, wherein the scene representation cascade interaction mode includes a cross-attention interaction mode and a feature splicing interaction mode;

[0195] Based on the scene representation cascade interaction mode, the ego vehicle trajectory is spliced, the bird's-eye view features, the occupancy network voxel features and the scene semantic map features are detected to obtain an updated trajectory of the ego vehicle.

[0196] The processing module 20 is also used to obtain feature dimensions;

[0197] Based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle update trajectory, trajectory enhancement and fusion of potential trajectories are performed to calculate a relationship matrix;

[0198] The ego-vehicle optimized trajectory planning is calculated based on the relationship matrix, the ego-vehicle copied trajectory and the ego-vehicle updated trajectory.

[0199] The execution module 30 is also used to obtain a scene cascade operation method;

[0200] Based on the scene cascade operation method, the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features are spliced ​​to obtain a predicted target trajectory.

[0201] The intelligent trajectory planning prediction device provided by the present application adopts the intelligent trajectory planning prediction method in the above embodiment, which can solve the technical problem of how to perform intelligent trajectory planning prediction more efficiently and accurately. Compared with the prior art, the beneficial effects of the intelligent trajectory planning prediction device provided by the present application are the same as the beneficial effects of the intelligent trajectory planning prediction method provided by the above embodiment, and the other technical features in the intelligent trajectory planning prediction device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0202] The present application provides an intelligent trajectory planning prediction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the intelligent trajectory planning prediction method in the above-mentioned embodiment 1.

[0203] Reference below Figure 8 , which shows a schematic diagram of the structure of an intelligent trajectory planning prediction device suitable for implementing the embodiment of the present application. The intelligent trajectory planning prediction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The intelligent trajectory planning prediction device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0204] like Figure 8As shown, the intelligent trajectory planning prediction device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the intelligent trajectory planning prediction device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the intelligent trajectory planning prediction device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an intelligent trajectory planning prediction device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0205] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0206] The intelligent trajectory planning prediction device provided by this application adopts the intelligent trajectory planning prediction method in the above embodiment, which can solve the technical problem of how to perform intelligent trajectory planning prediction more efficiently and accurately. Compared with the prior art, the beneficial effects of the intelligent trajectory planning prediction device provided by this application are the same as the beneficial effects of the intelligent trajectory planning prediction method provided by the above embodiment, and the other technical features in the intelligent trajectory planning prediction device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0207] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0208] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0209] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the intelligent trajectory planning prediction method in the above-mentioned embodiment.

[0210] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0211] The computer-readable storage medium may be included in the intelligent trajectory planning prediction device; or may exist independently without being assembled into the intelligent trajectory planning prediction device.

[0212] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the intelligent trajectory planning prediction device, the intelligent trajectory planning prediction device: obtains detection bird's-eye view features, occupation network voxel features and scene semantic map features; generates self-vehicle trajectory planning based on the detection bird's-eye view features, the occupation network voxel features and the scene semantic map features, and performs trajectory enhancement and fusion of potential trajectories to determine the optimized trajectory planning of the self-vehicle; determines the predicted target trajectory based on the optimized trajectory planning of the self-vehicle, the occupation network voxel features and the scene semantic map features, and completes the intelligent trajectory planning prediction based on the predicted target trajectory.

[0213] The computer program code for performing the operation of the present application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0214] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0215] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0216] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned intelligent trajectory planning prediction method, and can solve the technical problem of how to perform intelligent trajectory planning prediction more efficiently and accurately. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the intelligent trajectory planning prediction method provided in the above-mentioned embodiment, and will not be repeated here.

[0217] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An intelligent trajectory planning prediction method, characterized in that: The method includes: Obtain detection bird's-eye view features, occupancy network voxel features, and scene semantic map features; Generate the self-vehicle trajectory planning based on the detected bird's-eye view features, the occupied network voxel features and the scene semantic map features, and perform trajectory enhancement and fusion of potential trajectories to determine the self-vehicle optimized trajectory planning; A predicted target trajectory is determined based on the ego-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features, and intelligent trajectory planning prediction is completed based on the predicted target trajectory.

2. The method according to claim 1, characterized in that The step of acquiring and detecting bird's-eye view features, occupancy network voxel features and scene semantic map features comprises: Obtain object detection algorithms, surround camera data, lane information, and driving area information; Generate an object detection task based on the target detection algorithm and the surround camera data, detect the scene object position based on the object detection task to determine the detection bird's-eye view features and an occupied network voxel dataset, wherein the occupied network voxel dataset includes an occupied network voxel length, an occupied network voxel width, and an occupied network voxel depth; determining an occupancy network voxel feature based on the detected bird's eye view feature and the occupancy network voxel dataset; A semantic map is established based on the lane information and the driving area information, and scene semantic map features are determined.

3. The method according to claim 1, characterized in that The steps of generating the self-vehicle trajectory planning based on the detected bird's-eye view features, the occupied network voxel features and the scene semantic map features, and performing trajectory enhancement and fusion of potential trajectories to determine the self-vehicle optimized trajectory planning include: Determine the ego-vehicle copy trajectory and the ego-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features; Obtaining a self-vehicle trajectory planning based on the self-vehicle copy trajectory and the self-vehicle update trajectory; Based on the self-vehicle copied trajectory and the self-vehicle updated trajectory, trajectory enhancement and fusion of potential trajectories are performed to obtain the self-vehicle optimized trajectory planning.

4. The method according to claim 3, characterized in that The step of determining the self-vehicle copy trajectory and the self-vehicle update trajectory based on the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features comprises: Determining an ego-vehicle trajectory based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features; Copying the ego vehicle trajectory to determine an ego vehicle copy trajectory; An updated trajectory of the ego vehicle is determined based on the ego vehicle trajectory, the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features.

5. The method according to claim 4, characterized in that The step of determining the updated trajectory of the ego vehicle based on the ego vehicle trajectory, the detected bird's-eye view features, the occupancy network voxel features and the scene semantic map features comprises: Acquire a scene representation cascade interaction mode, wherein the scene representation cascade interaction mode includes a cross-attention interaction mode and a feature splicing interaction mode; Based on the scene representation cascade interaction mode, the ego vehicle trajectory is spliced, the bird's-eye view features, the occupancy network voxel features and the scene semantic map features are detected to obtain an updated trajectory of the ego vehicle.

6. The method according to claim 3, characterized in that The step of performing trajectory enhancement and fusion of potential trajectories based on the self-vehicle copied trajectory and the self-vehicle updated trajectory to obtain the self-vehicle optimized trajectory planning comprises: Get feature dimensions; Based on the feature dimension, the self-vehicle copy trajectory and the self-vehicle update trajectory, trajectory enhancement and fusion of potential trajectories are performed to calculate a relationship matrix; The ego-vehicle optimized trajectory planning is calculated based on the relationship matrix, the ego-vehicle copied trajectory and the ego-vehicle updated trajectory.

7. The method according to claim 1, characterized in that The step of determining the predicted target trajectory based on the ego-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features comprises: Get the scene cascade operation method; Based on the scene cascade operation method, the self-vehicle optimized trajectory planning, the occupancy network voxel features and the scene semantic map features are spliced ​​to obtain a predicted target trajectory.

8. An intelligent trajectory planning and prediction device, characterized in that: The device comprises: An acquisition module is used to acquire detection bird's-eye view features, occupancy network voxel features, and scene semantic map features; A processing module, configured to generate a trajectory plan for the ego vehicle based on the detected bird's-eye view features, the occupancy network voxel features, and the scene semantic map features, and to perform trajectory enhancement and fusion of potential trajectories to determine an optimized trajectory plan for the ego vehicle; An execution module is used to determine a predicted target trajectory based on the optimized trajectory planning of the ego-vehicle, the occupancy network voxel features and the scene semantic map features, and complete intelligent trajectory planning prediction based on the predicted target trajectory.

9. An intelligent trajectory planning and prediction device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the intelligent trajectory planning prediction method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the intelligent trajectory planning prediction method according to any one of claims 1 to 7 are implemented.

Images