Driving track planning method and related equipment

Abstract

The invention discloses a driving track planning method and related equipment, and relates to the technical field of auxiliary driving, and the method comprises the steps: obtaining environment perception data and driving track data of a target vehicle; performing fusion reasoning on the environment perception data and the driving track data through a target large language model to generate a reasoning result text; matching traffic rules and driving experience related to the reasoning result text from a preset cognitive knowledge base through the target large language model; according to the traffic rule and the driving experience, generating a task planning instruction for the target vehicle through the target large language model; and determining a planned trajectory of the target vehicle based on the task planning instruction and the environmental perception data. According to the method, perception data and knowledge information are fused through a large language model, an intelligent decision closed loop from environment understanding to task planning to track generation is realized, and the accuracy, compliance and safety of automatic driving track planning can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of assisted driving, and more particularly, to a driving trajectory planning method and related equipment. BACKGROUND

[0002] With the rapid development of assisted driving technology, trajectory planning, as a key link for vehicles to achieve safe, stable and intelligent driving, directly determines the decision quality and driving safety of the whole vehicle. In complex traffic environments, the assisted driving system needs to comprehensively perceive the external environment, identify dynamic targets, predict motion trends, and generate reasonable driving decisions and planning trajectories accordingly. However, due to the diversity and dynamics of traffic scenarios, the behavior patterns of vehicles under different weather, road, traffic density and unexpected events are extremely complex, which poses extremely high computational and cognitive challenges to trajectory planning algorithms.

[0003] In related technologies, traditional trajectory planning methods usually rely on rule-based decision logic or deep learning models for path prediction and planning. The rule-based method relies too much on the rule system set by humans and is difficult to adapt to variable and unstructured road scenes. The end-to-end method based on deep learning has a difficult-to-explain internal decision-making process, and the model is often trained only for specific scenarios, with limited generalization ability. That is, the scene adaptability of trajectory planning information is not strong in related technologies, leading to technical problems such as inaccurate trajectory prediction, decision errors and reduced driving safety of the assisted driving system in complex dynamic traffic environments. SUMMARY

[0004] A series of simplified concepts are introduced in the summary part of the present application, which will be further described in detail in the specific embodiment part. The summary part of the present application does not mean to try to limit the key features and necessary technical features of the claimed technical solutions, and even less means to determine the protection scope of the claimed technical solutions.

[0005] The driving trajectory planning method and related equipment provided by the present application can fuse perception data and knowledge information through a large language model, realize an intelligent decision-making closed loop from environment understanding to task planning to trajectory generation, and improve the accuracy, compliance and safety of automatic driving trajectory planning.

[0006] In a first aspect, a driving trajectory planning method is provided, including: obtaining environment perception data and driving trajectory data of a target vehicle; performing fusion reasoning on the environment perception data and the driving trajectory data by a target large language model to generate a reasoning result text; matching traffic rules and driving experience related to the reasoning result text from a preset cognitive knowledge base by the target large language model; generating task planning instructions for the target vehicle by the target large language model according to the traffic rules and the driving experience; and determining a planning trajectory of the target vehicle based on the task planning instructions and the environment perception data.

[0007] In some embodiments, the process of obtaining the environment perception data includes: determining a target perception algorithm from a preset perception algorithm library by the target large language model based on a current scene of the target vehicle, wherein the current scene includes at least one of weather, traffic density, and road type; and obtaining the environment perception data by analyzing vehicle external sensor data of the target vehicle by the target perception algorithm.

[0008] In some embodiments, the process of performing fusion reasoning on the environment perception data and the driving trajectory data by the target large language model to generate a reasoning result text includes: embedding and replacing the environment perception data and the driving trajectory data based on a predefined reasoning template to obtain a preset prompt word, wherein the predefined reasoning template includes object output constraints and influence output constraints; and inputting the preset prompt word into the target large language model to obtain the reasoning result text output by the target large language model, wherein the reasoning result text includes a road participating object and a possibility of the road participating object affecting the target vehicle.

[0009] In some embodiments, the process of matching traffic rules and driving experience related to the reasoning result text from a preset cognitive knowledge base by the target large language model includes: encoding the reasoning result text to obtain text vector data; retrieving a plurality of candidate knowledge items related to the first vector data from the preset cognitive knowledge base by a preset similarity search algorithm; and filtering common sense rules and driving experience matching the current scene of the target vehicle by performing semantic analysis on the plurality of candidate knowledge items by the target large language model.

[0010] In some embodiments, the generating, by the target large language model, the task planning instruction for the target vehicle according to the traffic rules and the driving experience comprises: generating, by the target large language model, an initial task planning instruction based on the reasoning result text; performing, by the target large language model, matching evaluation on the initial task planning instruction and the traffic rules and the driving experience to obtain an evaluation result; and performing, by the target large language model, correction and optimization on the initial task planning instruction based on the evaluation result to generate the task planning instruction.

[0011] In some embodiments, the determining, based on the task planning instruction and the environment perception data, the planning trajectory of the target vehicle comprises: generating an initial planning trajectory of the target vehicle based on the task planning instruction and the environment perception data; performing safety verification on the initial planning trajectory; if the safety verification fails, performing optimization processing on the initial planning trajectory to generate an optimized trajectory as the planning trajectory; and if the safety verification passes, determining the initial planning trajectory as the planning trajectory.

[0012] In some embodiments, the generating, based on the task planning instruction and the environment perception data, the initial planning trajectory of the target vehicle comprises: if the target vehicle has a navigation instruction or an external driving instruction, determining a target reward function from a preset reward function library based on the task planning instruction, and generating the initial planning trajectory by a preset diffusion model based on the target reward function and a historical planning trajectory distribution; and if the target vehicle does not have the navigation instruction and the external driving instruction, performing analysis and processing on the environment perception data, the driving trajectory data, the traffic rules and the driving experience by the target large language model to output the initial planning trajectory.

[0013] In a second aspect, the present application further provides a driving trajectory planning device, comprising: a data acquisition unit configured to acquire environment perception data and driving trajectory data of a target vehicle; a text generation unit configured to perform fusion reasoning on the environment perception data and the driving trajectory data by a target large language model to generate a reasoning result text; a knowledge matching unit configured to match, by the target large language model, traffic rules and driving experience related to the reasoning result text from a preset cognitive knowledge base; an instruction generation unit configured to generate, by the target large language model, a task planning instruction for the target vehicle according to the traffic rules and the driving experience; and a trajectory determination unit configured to determine a planning trajectory of the target vehicle based on the task planning instruction and the environment perception data.

[0014] In a third aspect, the present application also provides an electronic device, comprising a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory to implement the steps of the driving trajectory planning method according to the first aspect.

[0015] In a fourth aspect, the present application also provides a computer readable storage medium storing a computer program, wherein the computer program is configured to be executed by a processor to implement the steps of the driving trajectory planning method according to the first aspect.

[0016] In a fifth aspect, the present application also provides a computer program product comprising a computer program or computer executable instructions, wherein the computer program or computer executable instructions are configured to be executed by a processor to implement the steps of the driving trajectory planning method according to the embodiments of the present application.

[0017] To sum up, the present application fuses and reasons the environmental perception data and the driving trajectory data through the target large language model, can understand the external environment status and the dynamic information of the vehicle itself at the same time, compared with the scheme of relying on only a single sensor or fixed model, this fusion mode can more comprehensively reflect the actual driving scene of the vehicle, thereby improving the accuracy and scene adaptability of the trajectory planning; using the target large language model to match the related traffic rules and driving experience from the preset cognitive knowledge base, makes the trajectory planning process have knowledge support, can abide by the traffic regulations and learn the reasonable driving habits when making decisions, thereby improving the compliance and practicality of the planning result; generating task planning instructions through the large language model, can automatically make high-level driving decisions based on the reasoning result, so that the trajectory planning not only depends on algorithm calculation, but also integrates logical reasoning based on language understanding, making the task generation process more flexible, interpretable and traceable; according to the generated task planning instructions and the environmental perception data, determining the final planning trajectory of the target vehicle, the trajectory planning result can dynamically adapt to the current environmental changes and decision requirements, ensuring the rationality and safety of the vehicle motion path, thereby effectively reducing the potential risks. To sum up, the driving trajectory planning method provided by the present application fuses the perception data and knowledge information through the large language model, realizes the intelligent decision-making closed loop from environment understanding to task planning and then to trajectory generation, and can improve the accuracy, compliance and safety of the automatic driving trajectory planning. BRIEF DESCRIPTION OF DRAWINGS

[0018] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments, and are not meant to limit the present application. Moreover, the same reference numerals in the attached drawings refer to the same or similar components. In the drawings: Figure 1 A flowchart of a driving trajectory planning method provided by an embodiment of the present application; Figure 2 FIG. 1 is a schematic diagram of a driving trajectory planning device according to an embodiment of the present application; Figure 3 FIG. 2 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] The terms in the specification, claims and drawings of the present application, such as "first", "second", "third", "fourth" and the like (if any), are used to distinguish similar objects, and are not intended to describe a specific order or sequence. Therefore, it is understood that these terms can be used interchangeably under appropriate circumstances, so that the described embodiments can be implemented in different orders, unless the drawings or descriptions specifically require otherwise. In addition, the terms "is" and "has" and any variants thereof in the present application are intended to cover non-exclusive inclusion of all possible constituent elements. For example, a process, method, system, product or device including several steps or units does not necessarily limit to only the explicitly listed steps or units, but can also include other steps or units not explicitly listed, or steps or units inherent to the process, method, product or device.

[0020] In the present application, "module" or "unit" refers to a computer program or a part of a computer program with a specific function, and cooperates with other related parts to achieve a predetermined target. These modules or units can be implemented by software, hardware (such as processing circuitry or memory) or a combination of both. One or more processors or memories can implement one or more modules or units. At the same time, each module or unit can also be part of a larger module or unit.

[0021] The technical solutions in the present application will be described in detail below in conjunction with the drawings in the embodiments. It should be noted that the described embodiments are only a part of the present application, not all embodiments. In the following description, "some embodiments" mentioned is only a subset of all possible embodiments, which can be the same or different subset, and different embodiments can be combined with each other without conflict.

[0022] Figure 1 FIG. 1 is a schematic diagram of a driving trajectory planning device according to an embodiment of the present application; Figure 1 The driving trajectory planning method provided by the embodiments of the present application can include the following steps 101 to 105: Step 101, obtaining environment perception data and driving trajectory data of a target vehicle; In some examples, the target vehicle is a motor vehicle equipped with an advanced driver assistance system (ADAS) or an autonomous driving system (ADS) to which the driving trajectory planning method of the embodiments of the present application is applied. The core feature of the target vehicle is that it has hardware and software foundations for data acquisition, transmission, processing, and execution of trajectory planning instructions, including but not limited to passenger cars, commercial vehicles, special operation vehicles, etc. For example, the target vehicle can be a family sedan equipped with an L2+ level advanced driver assistance system or an urban bus with autonomous driving function, etc. The environmental perception data is a multi-dimensional data set used to describe the surrounding traffic environment, road conditions, and external environmental parameters of the target vehicle. Its core role is to provide environmental input for subsequent trajectory planning to ensure that the planning result adapts to the real-time surrounding scene. The acquisition of environmental perception data depends on the multi-type sensors and external data interfaces equipped on the target vehicle, which can specifically include: collecting three-dimensional position, size, and motion trend data of surrounding obstacles (such as vehicles, pedestrians, and non-motor vehicles) through a laser radar (LiDAR); collecting image information of road markings, traffic signal lights, and traffic signs through a camera (Camera), and analyzing them into structured data (such as "red light", "speed limit 60 km / h", etc.) through image recognition algorithms; collecting speed, distance, and azimuth angle data of long-distance targets through a millimeter wave radar (MMR) to supplement the sensing shortcomings of the laser radar in bad weather; obtaining static road information such as road topology (such as the number of lanes, turning restrictions), road curvature, and slope through a high-definition map (HD Map) interface; for example, in a city intersection scenario, the laser radar obtains a pedestrian crossing horizontally in front of 30 m, with a three-dimensional coordinate of (X: 125.3 m, Y: 48.6 m, Z: 1.2 m) and a size of (length 0.6 m x width 0.5 m x height 1.7 m); the camera recognizes that the current intersection traffic signal light is "green", and the remaining green light duration is 8 s; the millimeter wave radar detects a small car 15 m away from the side and rear at a speed of 40 km / h; the high-definition map feedbacks that the current road is a two-way four-lane road, the current lane is a straight lane, and left turns are prohibited.The driving track data is a data set for recording the historical driving state and real-time motion parameters of the target vehicle. The driving track data is mainly obtained by the vehicle positioning system and vehicle bus of the target vehicle. The real-time position coordinates (such as latitude, longitude and altitude) and heading angle (the angle between the driving direction of the vehicle and the north direction) of the vehicle can be obtained by combining the global navigation satellite system (GNSS) with the inertial measurement unit (IMU). The real-time driving parameters of the vehicle, including the driving speed, acceleration (longitudinal acceleration and lateral acceleration), steering wheel angle, brake pedal opening, throttle pedal opening and the like, can be collected through the controller area network bus (CAN). The continuous driving parameters of the target vehicle within the last 30 seconds are extracted through the historical data storage module of the auxiliary driving system to form a historical driving track sequence.

[0023] Through the implementation of step 101, the environment perception data and driving track data of the target vehicle are obtained, the current external environment information and the motion state of the vehicle can be comprehensively mastered, a complete data basis is provided for subsequent intelligent decision-making, the timeliness and accuracy of the input information in the trajectory planning process can be ensured, the actual driving scene in which the vehicle is located can be better reflected, and thus the reliability of the overall decision-making is improved.

[0024] In step 102, the environment perception data and driving track data are fused and reasoned by the target large language model to generate a reasoning result text. In some examples, the target large language model is a large language model (LLM) that is specifically optimized for driving scenarios and has the ability to understand multi-modal data and logical reasoning. Its core function is to receive environmental perception data and trajectory data, output structured reasoning results through built-in driving scenario reasoning logic, and provide decision-making basis for subsequent trajectory planning. A general large language model that supports multi-modal input (such as GPT-4V, LLaMA 3) can be selected as the base model, fine-tuned using driving scenario-specific datasets, including annotated texts of vehicle sensor data (such as "a truck is driving on the road 50 meters ahead"), structured texts of traffic rules (such as "the minimum speed on urban expressways cannot be lower than 60 km / h"), and analysis texts of historical driving cases (such as "the braking distance at a road intersection increases by 20% in rainy weather"). Finally, the model is adapted to the vehicle hardware environment through model compression (such as quantization and pruning) to reduce the computational resource occupation, obtaining the target large language model to ensure that it can meet the millisecond-level reasoning delay requirement when deployed to the vehicle edge computing unit (VECU). The fusion reasoning process refers to the process of the target large language model correlating and analyzing environmental perception data (describing the surrounding environment) and trajectory data (describing the vehicle's own state), identifying road participants, judging the impact of objects on the vehicle, and extracting key scene features. The core purpose is to break the isolation of the two types of data and form a reasoning conclusion that links the environment and the vehicle. The reasoning result text is a structured natural language text output by the target large language model through fusion reasoning. It contains three key pieces of information: road participants, the likelihood of the object's impact on the vehicle, and key scene factors, providing clear evidence for matching cognitive knowledge bases and generating task planning instructions. For example, the reasoning result text generated for a school surrounding road segment scene is: "

Road participants

Object impact likelihood

Key scene factors

[0025] Through the implementation of step 102, the target large language model is used to fuse and reason the environmental perception data and the driving trajectory data, which can understand the external environment and the vehicle dynamic behavior at the same time, help the model to extract key semantics from multi-dimensional information, form a comprehensive understanding of the driving scene, and improve the logical consistency and scene adaptability of the reasoning result.

[0026] In step 103, the target large language model is used to match the traffic rules and driving experience related to the reasoning result text from the preset cognitive knowledge base. In some examples, the preset cognitive knowledge base is a structured knowledge set that is pre-constructed and dynamically updated to support driving trajectory planning decisions, stores traffic rules and driving experience, and its core function is to provide scenario-based knowledge support for the target large language model to ensure that the trajectory planning meets the regulatory requirements and actual driving rules. The data collection tool can crawl the traffic regulation texts published by the state and local governments (such as the “Road Traffic Safety Law of the People’s Republic of China”), the real car driving case data accumulated by automobile manufacturers, and the extreme scenario processing solutions generated by simulation platforms, and convert them into structured items (including scene tags, knowledge content, and applicable conditions) through natural language processing (NLP) technology, and store them in a vector database (such as Milvus) to support fast retrieval. Then, through the feedback loop mechanism of the target vehicle, manually intervened cases, newly released traffic control information, and user-defined driving preferences (such as “cautious acceleration in rainy weather”) are entered into the knowledge base in real time to ensure the timeliness of the knowledge. Traffic rules are mandatory provisions in the preset cognitive knowledge base that are formulated by legal institutions or recognized by the industry to regulate vehicle driving behavior, and their core role is to ensure that the trajectory planning meets legal requirements and road traffic order, and to avoid decision-making errors caused by illegal operations. Driving experience is a non-mandatory operation guide in the preset cognitive knowledge base that is summarized based on historical driving cases and has scene adaptability, and its core role is to make up for the scene gaps of traffic rules and improve the rationality and safety of trajectory planning in complex dynamic environments.

[0027] Through the implementation of step 103, the target large language model is used to match related traffic rules and driving experience from the preset cognitive knowledge base, so that the trajectory planning process has knowledge support, and the perception reasoning result is combined with existing traffic common sense and historical experience, which helps to comply with traffic regulations and refer to reasonable driving behavior patterns during planning, thereby improving the compliance and practicality of the decision.

[0028] In step 104, the target large language model is used to generate task planning instructions for the target vehicle according to the traffic rules and driving experience. In some examples, task planning instructions are high-level instructions generated by the target large language model based on traffic rules, driving experience, and inference results from the current scene. These instructions guide the target vehicle to perform specific driving behaviors. Their core features are scene adaptability, clear operation, and safety constraints, allowing them to be directly used as the basis for subsequent trajectory planning. Task planning instructions can cover action types (such as braking, acceleration, and steering), action parameter ranges (such as braking deceleration and steering angle), and auxiliary operations (such as prompting the driver to take over and activating hazard warning lights), ensuring that the instructions can be parsed into specific control signals by the execution layer. For example, the task planning instruction for a scenario where children are crossing near a school is: "[Action Type] Emergency Braking + Warning Prompt; [Action Parameter] Braking Deceleration 3.5 m / s²". 2 (Continue until vehicle speed drops to 0km / h), hazard warning lights activate (continue until vehicle comes to a complete stop); [Assisted Operation] The driver is prompted via the in-vehicle voice system: 'Children are crossing ahead, emergency braking has been triggered, please prepare to take over'; [Constraints] Maintain straight-line driving during braking, with a steering angle not exceeding ±5°. Based on traffic rules and driving experience, the process of generating task planning instructions for the target vehicle through the target large language model is a decision-making stage where the target large language model integrates the legal constraints of traffic rules, practical guidance from driving experience, and real-time needs of the current scenario into specific driving instructions. The core purpose is to ensure that the instructions comply with regulations while adapting to complex dynamic environments, avoiding contradictions of compliance without safety or safety without violation.

[0029] By implementing step 104, task planning instructions are generated by the target large language model based on traffic rules and driving experience. This enables the automatic formulation of high-level driving behaviors based on the reasoning results, achieving a natural transformation from knowledge understanding to action decision-making. This makes the task planning process more logical, interpretable, and flexible, and helps to achieve more intelligent driving behavior decisions.

[0030] Step 105: Determine the planned trajectory of the target vehicle based on the task planning instructions and environmental perception data; In some examples, the planning trajectory is a set of spatiotemporal parameters describing a driving path of the target vehicle in a future period of time, which is generated based on the task planning instruction and the environment perception data, can include a sequence of spatial coordinates, motion state parameters, and safety constraint boundaries, and can be directly converted into vehicle bottom layer control signals (such as steering angle, brake pressure); the planning trajectory needs to meet the behavior requirements of the task instruction (such as emergency braking, steering avoidance) and the spatial restrictions of the environment perception data (such as obstacle position, lane boundary), to ensure the safe driving of the vehicle along the preset path in a dynamic traffic environment. Based on the task planning instruction and the environment perception data, the process of determining the planning trajectory of the target vehicle is the execution link of converting high-level task planning instruction into specific driving path, aiming to ensure that the generated trajectory meets the behavior requirements of the task instruction and can avoid potential risks (such as obstacles, road boundary restrictions) in the environment perception data.

[0031] Through the implementation of step 105, the planning trajectory of the target vehicle is determined based on the task planning instruction and the environment perception data, which can make the trajectory generation process take into account real-time environmental changes and task execution targets, ensure the rationality and safety of the vehicle driving path, thereby reducing potential risks and improving the stability and safety and reliability of the assisted driving system in complex dynamic scenarios.

[0032] In summary, the embodiments of the present application fuse and reason the environment perception data and the driving trajectory data through the target large language model, which can understand the external environment conditions and the dynamic information of the vehicle itself at the same time. Compared with the scheme of relying only on a single sensor or a fixed model, this fusion method can more comprehensively reflect the actual driving scene in which the vehicle is located, thereby improving the accuracy and scene adaptability of trajectory planning; the target large language model is used to match relevant traffic rules and driving experience from a preset cognitive knowledge base, so that the trajectory planning process has knowledge support and can comply with traffic regulations and learn from reasonable driving habits when making decisions, thereby improving the compliance and practicality of the planning result; the task planning instruction is generated through the large language model, which can automatically make high-level driving decisions based on the reasoning result, so that the trajectory planning not only relies on algorithm calculation, but also incorporates logical reasoning based on language understanding, making the task generation process more flexible, interpretable and traceable; according to the generated task planning instruction and the environment perception data, the final planning trajectory of the target vehicle is determined, and the trajectory planning result can dynamically adapt to the current environmental changes and decision requirements, ensuring the rationality and safety of the vehicle motion path, thereby effectively reducing potential risks. In summary, the driving trajectory planning method provided by the embodiments of the present application fuses perception data and knowledge information through a large language model, realizes an intelligent decision-making closed loop from environment understanding to task planning and then to trajectory generation, and can improve the accuracy, compliance and safety of automatic driving trajectory planning.

[0033] In some embodiments, the process of obtaining the aforementioned environment perception data can include: determining, by the target large language model, a target perception algorithm from a preset perception algorithm library based on a current scene of the target vehicle, wherein the current scene can include at least one of weather, traffic density, and road type; and performing analysis and processing on vehicle external sensor data of the target vehicle by the target perception algorithm to obtain the environment perception data.

[0034] In some examples, the current scene is a set of real-time environmental state affecting the selection of the perception algorithm of the target vehicle at the current time, the core features of which include at least one element containing weather, traffic density and road type, which is used to provide the target large language model with algorithm matching scene basis, to ensure that the subsequent perception data analysis adapts to the actual environment. Weather is an atmospheric environmental condition that affects visibility, road surface state and sensor perception effect in the current scene, which can include rainfall, visibility, temperature, wind power and other parameters, and is a key basis for selecting anti-interference type perception algorithms; The acquisition method of weather is to collect real-time data through vehicle-mounted meteorological sensors (such as rain sensors, visibility sensors), and the data is transmitted to the domain controller through the controller area network bus; It can also receive shared data from roadside weather stations through vehicle-to-everything (V2X). Traffic density is the number and distribution of traffic participants (motor vehicles, non-motor vehicles, pedestrians) per unit road length in the current scene, which core reflects the degree of road congestion, and is a key basis for selecting high-density target detection algorithms; The acquisition method of traffic density can be that the camera counts the number of traffic participants in the field of view through image recognition technology, and the laser radar analyzes the target distribution density through point cloud clustering, and after the data of the two is fused by the target large language model, the density level is divided into "low (≤10 vehicles / km), medium (10-30 vehicles / km), high (≥30 vehicles / km)". Road type is the road attribute classification of the target vehicle driving in the current scene, which core includes road grade, lane number, speed limit standard, traffic rules, etc., which is a key basis for selecting perception algorithms that adapt to road features; The acquisition method of road type can determine the real-time position through the global navigation satellite system of the target vehicle combined with the inertial measurement unit, and the domain controller retrieves the corresponding road attribute label from the high-precision map according to the position; It can also be assisted by camera recognition of road markings (such as continuous solid lines on expressways). The preset perception algorithm library is a structured database that stores various perception algorithms, which is pre-constructed and dynamically updated to adapt to the perception needs of different scenes, and its core function is to provide the target large language model with an algorithm selection pool to ensure that there is corresponding algorithm support for environmental perception data analysis in different scenes.The target perception algorithm is selected from the preset perception algorithm library and is adapted to the current scene. The core features are scene adaptability and data analysis effectiveness, which can ensure accurate analysis of vehicle external sensor data in the current scene and avoid perception deviation caused by mismatch between the algorithm and the scene. The target large language model can first receive structured current scene description (such as "heavy rain + high traffic density + urban expressway"); then, the target large language model traverses the scene labels of the preset perception algorithm library to preliminarily select candidate algorithms matching the labels (such as MMR anti-interference algorithm and BEV multi-target detection algorithm); finally, the model combines the current sensor state (such as LiDAR with heavy rain point cloud noise) to finally determine 1-2 core algorithms as the target perception algorithm; for example, the current scene is "heavy rain + high traffic density + urban expressway", and the LiDAR is excluded from the candidate algorithms due to excessive rain noise, and the target large language model finally determines "MMR anti-interference target tracking algorithm (detection distance 150 m, refresh frequency 20 Hz)" and "front camera image enhancement algorithm in rainy weather (contrast ratio 30%, fog intensity 50%)" as the target perception algorithm. The vehicle external sensor data is the original environmental data collected by the external perception sensor of the target vehicle without analysis, which can include LiDAR point cloud data, camera image data, MMR echo data, ultrasonic radar data, etc., and is the original material for subsequent environmental perception data analysis. The process of analyzing and processing the vehicle external sensor data of the target vehicle through the target perception algorithm to obtain the environmental perception data is the core link of converting the unanalyzed sensor raw data into structured environmental perception data, so as to ensure that the output data contains key information such as target type, position, motion state and road boundary, which can be directly used for subsequent trajectory planning; for example, for the original data of the "heavy rain + high traffic density + urban expressway" scene, first, the MMR anti-interference algorithm is used to filter echo noise (remove invalid data with a reflection intensity lower than 5), and the camera image enhancement algorithm is used to improve the picture contrast (from 40 to 70); then, the MMR algorithm analyzes 10 effective targets (excluding 2 noise targets) and outputs target 1 (ID: V001, type: passenger car, distance 85 m, speed 60 km / h) and target 2 (ID: P001, type: pedestrian, distance 30 m, speed 1.2 m / s); the camera algorithm analyzes the traffic signal light state (currently green, remaining time 10 s); finally, the target large language model integrates the data to generate the environmental perception data: "target list (V001: passenger car, 85 m / 60 km / h; P001: pedestrian, 30 m / 1.2 m / s), traffic signal light: green (10 s), road boundary: current lane width 3.75 m, left lane line distance 0.5 m".

[0035] Through implementation of the foregoing embodiments, the most suitable target perception algorithm is automatically selected from the perception algorithm library according to the current driving scene (such as weather, traffic density, and road type) by using the target large language model, so that the perception module can be dynamically and adaptively optimized according to environmental characteristics, the problem that a single algorithm fails in a specific scene is avoided, the accuracy and real-time performance of environmental information extraction are improved, higher-quality input data is provided for subsequent trajectory planning, and the adaptability of the method to complex scenes is enhanced.

[0036] In some embodiments, the foregoing step 102 can include: embedding and replacing the environment perception data and the driving trajectory data based on a predefined reasoning template to obtain a preset prompt word, wherein the predefined reasoning template can include object output constraints and influence output constraints; and inputting the preset prompt word into the target large language model to obtain a reasoning result text output by the target large language model, wherein the reasoning result text includes a road participant object and a possibility of an influence of the road participant object on the target vehicle.

[0037] In some examples, the predefined inference template is a pre-designed text framework containing fixed field constraints and variable data placeholders for the fusion inference logic of the target large language model. Its core role is to avoid information omission or logical deviation in the inference of the target large language model, and to ensure that the inference result text focuses on the two key information of road participant objects and object impact possibility. The process of embedding replacement is to match and fill the environment perception data and driving trajectory data obtained in step 101 into the reserved placeholders of the predefined inference template according to the field requirements of the predefined inference template. Its core purpose is to convert unstructured raw data into structured text that meets the input format of the target large language model, providing clear data basis for subsequent inference. The preset prompt word is a structured text that meets the format requirements of the predefined inference template after embedding replacement. Its core features are data integrity and logical specification, and it can be directly used as the input of the target large language model to guide the model to output the inference result text according to the template constraints. When the environment perception data and driving trajectory data are filled into the template placeholders, the complete text generated is the preset prompt word, which does not need additional processing, but needs to be checked by a format verification tool to check whether there are placeholders that have not been replaced, data format errors, etc. The object output constraint is a mandatory output requirement set for the "road participant object" field in the predefined inference template. Its core content can include the "ID identification", "type classification", "location coordinates" and "motion parameters" of the object, etc., to ensure that the target large language model does not miss the key object information during inference, and to provide complete object data basis for subsequent impact possibility analysis. The impact output constraint is a mandatory output requirement set for the "object impact possibility" field in the predefined inference template. Its core content can include "risk type", "risk level" and "impact time", etc., to ensure that the target large language model can clearly analyze the potential impact of each road participant object on the target vehicle during inference, and avoid decision-making errors caused by ambiguous impact description. The process of inputting the preset prompt word into the target large language model to obtain the inference result text output by the target large language model is the core link of the target large language model based on the structured data and constraint requirements of the preset prompt word to complete data association, logical inference and text generation. Its core purpose is to convert the input environment and trajectory data into natural language inference results containing road participant objects and impact possibilities, providing basis for subsequent knowledge matching. Among them, the road participant object is all traffic participants and moving objects around the target vehicle that affect its driving trajectory, including motor vehicles (passenger cars, trucks, buses, etc.), non-motor vehicles (electric bicycles, bicycles, etc.), pedestrians, and temporary obstacles (such as fallen goods, construction barriers), etc., which are the objects that need to be avoided or cooperated in trajectory planning.The influence possibility of the road participant object on the target vehicle is the interaction between the motion state of each road participant object and the driving trajectory, speed, and other parameters of the target vehicle, and the degree of interference that can be generated on the driving safety or trajectory execution of the target vehicle. The core includes collision risk, lane occupation interference, speed limit influence, and other types, which need to be determined in combination with distance, time trajectory, and other parameters.

[0038] For example, in the scenario of the target vehicle driving to an urban intersection, first, the vehicle-mounted VECU calls the pre-defined "urban intersection reasoning template", which includes fields such as "scene description", "road participant object", "object influence possibility", and "key conclusion" and corresponding constraints; then, the "road type: urban intersection, weather: sunny, traffic density: medium, object list: P001 - pedestrian (X: 120.5m, Y: 50.3m, speed 1.2m / s), V002 - passenger car (X: 125.3m, Y: 48.6m, speed 30km / h)" are extracted from the environment perception data obtained in step 101, and the "target vehicle speed 35km / h, heading angle 90°" are extracted from the driving trajectory data, which are embedded and replaced into the template placeholders to generate preset prompt words; subsequently, the VECU inputs the preset prompt words into the target large language model, the target large language model checks the completeness of the object information according to the object output constraint, and calculates the collision risk (P001 intersection time 2.5s, distance 18m, high risk) according to the influence output constraint; finally, the model outputs the reasoning result text in the template format, which includes the complete road participant object list and influence possibility analysis, and is stored in the vehicle-mounted cache module to provide accurate basis for the knowledge matching of step 103.

[0039] Through the implementation of the above embodiments, the environment perception data and driving trajectory data are structured and embedded using the pre-defined reasoning template, so that the large language model can clearly distinguish the key objects and their potential influences when generating the reasoning result. This not only improves the logical consistency and output controllability of the model, but also avoids the problem of model hallucination or missing key factors, so that the potential risks and interaction relationships can be more accurately identified, providing a clear and interpretable semantic basis for decision-making reasoning for assisted driving.

[0040] In some embodiments, the aforementioned step 103 can include: encoding the reasoning result text to obtain text vector data; retrieving a plurality of candidate knowledge items related to the first vector data from the pre-set cognitive knowledge base through a pre-set similarity search algorithm; and performing semantic analysis on the plurality of candidate knowledge items by the target large language model to filter out common sense rules and driving experience that match the current scene of the target vehicle.

[0041] In some examples, the text vector data is a high-dimensional numerical vector that is computer-recognizable and is converted from the inference result text by a text encoding technology. The core feature is semantic mapping, that is, the dimensions and numerical distribution of the text vector data correspond to the semantic information of the text, so that texts with similar semantics correspond to vectors that are closer in high-dimensional space, providing a quantitative basis for subsequent similarity search. The preset similarity search algorithm is an algorithm that is integrated in advance and is used to calculate the similarity between the text vector data and the knowledge item vector data in the preset cognitive knowledge base, and quickly retrieve high-similarity items. The core function is to solve the problem of fast matching of high-dimensional vectors and ensure efficient screening of potential related items from massive knowledge. The preset similarity search algorithm can be integrated by combining multiple algorithms, such as the Hierarchical Navigable Small World Graphs (HNSW) algorithm (responsible for fast approximate search of high-dimensional vectors) and the K-Nearest Neighbors (KNN) algorithm (responsible for accurate calculation of neighbor similarity). Algorithm parameters are optimized through offline testing (such as setting the number of neighbors of HNSW to 32 and the K value of KNN to 10) to balance the retrieval speed (≤50 ms) and accuracy (≥95%). The algorithm is called through the algorithm interface of the vehicle-mounted edge computing unit. For example, for 1024-dimensional text vector data, first, the HNSW algorithm is used to perform coarse screening in the vector index of the preset cognitive knowledge base, quickly narrowing the search range to 100 candidate vectors. Then, the KNN algorithm is used to calculate the cosine similarity of the 100 vectors and the target vector, and the top 10 vectors with a similarity of ≥0.8 are selected as the preliminary retrieval results. The candidate knowledge items are a set of knowledge items that are similar in semantics to the inference result text and are retrieved from the preset cognitive knowledge base by the preset similarity search algorithm. They may contain traffic rules or driving experience related to the current scene, but need to be further screened to exclude items that are similar in semantics but do not match the scene. The candidate knowledge items come from the preset cognitive knowledge base, and their acquisition process is the output result of the preset similarity search algorithm. They can be sorted according to the similarity between the text vector data and the knowledge item vector data in the knowledge base, and the top K (such as K=10) are selected as candidate items. Each candidate item contains "knowledge ID", "knowledge type (rule / experience)", "knowledge content", "applicable scene label", and "vector data" fields, ensuring complete analysis basis for subsequent screening.The process of semantic analysis is a process in which the target large language model performs deep semantic understanding on the candidate knowledge items, combines the current scene of the target vehicle (such as weather, road type, and traffic density), judges the adaptability of the items to the current scene, the relevance of the knowledge content, and the potential conflicts, and the core purpose is to eliminate the items that are semantically similar but not suitable for the scene from the candidate items, and retain the truly matched knowledge; the process of semantic analysis is realized by the scene understanding ability of the target large language model, the target large language model first extracts the "applicable scene tags" (such as "highway, pedestrian") of the candidate knowledge items and the "current scene features" (such as "rainy day, urban intersection, pedestrian") in the reasoning result text; then judge the scene consistency (such as "highway" and "urban intersection" are scene conflicts); finally, combine the relevance of the knowledge content and the current reasoning conclusion (such as the relevance of "braking distance" and "pedestrian crossing needs to slow down"), and output the analysis result; for example, for the candidate item R078 (applicable scene "highway, pedestrian"), the target large language model extracts the current scene features "urban intersection, rainy day", and finds that "highway" and "urban intersection" conflict through scene element alignment, and determines that the item is not matched with the current scene; for the item E056 (applicable scene "intersection, rainy day, braking"), the target large language model finds that its scene tag is completely matched with the current scene, and the content "rainy day braking distance increases" is highly relevant to the reasoning result "pedestrian crossing needs to slow down", and determines it as a matched item. The common sense rules and driving experience that match the current scene of the target vehicle are the knowledge items that are completely matched with the current scene (weather, road type, traffic participants, etc.) of the target vehicle and can support trajectory planning decision after semantic analysis, and the core features are scene consistency and decision support, that is, both conform to the current environmental characteristics and can provide clear rule constraints or experience guidance for task planning instruction generation.

[0042] Through the implementation of the above embodiments, the reasoning result text is vectorized and combined with a similarity search algorithm to match the most relevant traffic rules and driving experience from the cognitive knowledge base to the current scene, so that the trajectory planning has knowledge-level constraints and guidance, the method can automatically call the corresponding driving common sense and safety standards in different countries, roads, and traffic situations, realize knowledge-driven dynamic decision-making, effectively improve the compliance and behavior rationality of the planned trajectory, and reduce the risk of misjudgment and violation.

[0043] In some embodiments, the foregoing step 104 can include: based on the reasoning result text, generating an initial task planning instruction by the target large language model; matching and evaluating the initial task planning instruction with the traffic rules and driving experience by the target large language model to obtain an evaluation result; based on the evaluation result, modifying and optimizing the initial task planning instruction by the target large language model to generate the task planning instruction.

[0044] In some examples, the initial task planning instruction is a driving instruction generated by the target large language model based on the scene features and risk judgments in the inference result text. The initial task planning instruction prioritizes the emergency handling of the current risk, but may not fully comply with traffic rules or driving experience and needs to be subsequently verified and optimized. The target large language model first extracts the core risks (such as "pedestrian crossing, high collision risk") and the target vehicle state (such as "current speed 35 km / h") from the inference result text. Then, based on the pre-set risk-action mapping rules (such as "high collision risk-braking deceleration"), the initial instruction containing the action type and basic parameters is generated. The instruction format must include the action type, core parameters, and execution timing to ensure that subsequent evaluation has a clear object. For example, the initial task planning instruction for the rain road intersection pedestrian crossing scene is: "

Action type

Core parameters

Execution timing

Auxiliary operation

Compliance Evaluation

Experience Adaptation Evaluation

Risk Residue Evaluation

[0045] Through the implementation of the above embodiments, the generation, evaluation and correction process of task planning instructions is introduced into the large language model, which can continuously optimize task decisions under the constraints of traffic rules and driving experience, make the generated planning instructions have self-examination and correction ability, and automatically adjust the driving behavior instructions according to the scene feedback, thereby improving the accuracy and robustness of task planning and reducing the decision deviation in trajectory planning.

[0046] In some embodiments, the foregoing step 105 can include: generating an initial planning trajectory of the target vehicle based on the task planning instruction and the environment perception data; performing safety verification on the initial planning trajectory; if the safety verification fails, performing optimization processing on the initial planning trajectory to generate an optimized trajectory as the planning trajectory; if the safety verification passes, determining the initial planning trajectory as the planning trajectory.

[0047] In some examples, the initial planning trajectory is a spatiotemporal parameter sequence describing the future driving path of the target vehicle, which is preliminarily generated by a trajectory generation algorithm based on the task planning instruction and the environment perception data, strictly following the action requirements (such as braking, steering) of the task planning instruction, but may not completely avoid potential safety risks in the environment, and needs to be confirmed whether it is available after safety verification; the corresponding algorithm can be selected according to the type of the task planning instruction (such as emergency braking, constant speed cruising), for example, the polynomial trajectory generator (Polynomial Trajectory Generator) is used for braking instructions to generate a continuous path by fitting the time-displacement curve; the generation process needs to input the road boundary (such as lane width, shoulder position) and obstacle position in the environment perception data to ensure that the trajectory does not exceed the physical feasible region; the final output includes the structured initial trajectory data containing "time sequence", "spatial coordinates", "velocity / acceleration curve". The safety verification is a process of detecting the risks of the initial planning trajectory by a preset algorithm, judging whether the trajectory has collision risks, whether it exceeds the road feasible region, and whether it meets the vehicle dynamics constraints, the core purpose of which is to identify the safety blind area in the initial trajectory to avoid driving risks caused by trajectory planning omissions; the safety verification process can be realized by multi-dimensional verification functions, one is collision risk verification, calling the collision detection algorithm (Collision Detection Algorithm) to calculate the minimum distance between the target vehicle at each time on the trajectory and the obstacles (such as pedestrians, vehicles) in the environment perception data, to judge whether it meets the safety threshold (such as the minimum distance to pedestrians ≥2m); two is the road boundary verification, checking whether the trajectory exceeds the physical boundaries such as lane lines and shoulders (such as lateral offset ≤±0.5m) by the boundary judgment function (Boundary Check Function); three is the dynamics constraint verification, verifying whether the acceleration and steering angle of the trajectory are within the hardware limit range of the vehicle (such as the maximum braking deceleration ≤-4.5m / s 2 ); the verification result can be output as "pass / fail" and specific risk point description; for example, the collision detection algorithm calculates that the distance between the target vehicle and the front pedestrian at 1.8s is only 1.5m (lower than the 2m safety threshold), and the collision risk is not eliminated; the road boundary verification shows that the lateral offset is 0.2m (compliant); the dynamics constraint verification shows that the acceleration is -3.0m / s 2(in hardware limits); the final verification result is "failed", and the risk point is marked as "1.8s distance from pedestrian is insufficient". When the initial planning trajectory fails the safety verification, the trajectory optimization algorithm adjusts the trajectory parameters (such as speed, acceleration, path coordinates) to eliminate the identified safety risks, and finally generates a trajectory that meets the safety requirements, ensuring that the optimized trajectory not only meets the task planning instructions, but also completely avoids environmental risks; specifically, first, risk positioning can be performed to clearly identify the specific risk points indicated by the safety verification (such as "1.8s distance from pedestrian is 1.5m"); then parameter adjustment is performed, the optimization direction is selected based on the risk point type (such as collision risk requires shortening the braking time, then increasing the braking deceleration), and the time-velocity curve or spatial coordinates are adjusted through the Gradient Descent Algorithm; finally, secondary verification is performed, and the safety verification is performed again on the adjusted trajectory until all risk points are eliminated, and the optimized trajectory is generated. When the initial planning trajectory passes the safety verification (i.e., no collision risk, no exceeding of road boundaries, and compliance with dynamic constraints), it is directly determined as the final planning trajectory executed by the target vehicle. The core role of this process is to simplify the process, improve efficiency, avoid unnecessary optimization calculations, and ensure the real-time nature of trajectory planning.

[0048] Through the implementation of the above embodiments, safety verification and optimization are set in the trajectory planning stage, so that the planning result is screened for risks before execution. If potential collisions or abnormal trajectories are detected, a safe path can be regenerated through optimization strategies, forming a safety closed-loop planning process that can improve the safety protection capability of the assisted driving system in complex dynamic traffic environments, ensuring that the vehicle motion trajectory is always within a safe and controllable range.

[0049] In some embodiments, the aforementioned generation of an initial planning trajectory for the target vehicle based on the task planning instructions and the environmental perception data can include: if the target vehicle has navigation instructions or external driving instructions, determining a target reward function from a pre-set reward function library based on the task planning instructions, and generating an initial planning trajectory based on the target reward function and historical planning trajectory distribution through a pre-set diffusion model; if the target vehicle does not have navigation instructions and external driving instructions, analyzing and processing the environmental perception data, driving trajectory data, traffic rules and driving experience through a target large language model to output an initial planning trajectory.

[0050] In some examples, the navigation instruction is structured guidance information generated by a vehicle navigation system (VNS) of the target vehicle for guiding the driving path of the target vehicle, the core content of which includes turning direction (left turn / right turn / straight), lane change prompt, destination path planning and key node distance (such as "turn left 500 m ahead"), and is the path reference for long-distance driving of the target vehicle. The external driving instruction is a control instruction from outside the target vehicle for temporarily adjusting its driving state, which can include traffic control information sent by a road side unit (RSU) (such as "reduce speed to 20 km / h due to construction ahead"), intervention instructions from a remote control center (such as "emergency stop"), and cooperative information from vehicle-to-vehicle (V2V) (such as "suggest reducing speed due to sudden braking of vehicle ahead"). The preset reward function library is a pre-constructed set of reward functions for evaluating the pros and cons of trajectories, and its core role is to provide an optimization target for trajectory generation, guide the generation of trajectories that meet the scene requirements through quantitative indicators such as safety distance, lane centering degree, and driving efficiency, and avoid random generation without a target. The preset reward function library can be constructed based on scene classification, divided into scenes according to task type (braking / turning / cruising), road type (highway / urban / rural), and risk level (high / medium / low), and designed with exclusive reward functions for each scene (such as "maximum safety distance reward function" for emergency braking scene and "comfort reward function" for cruising scene). The functions are stored in mathematical formula form and are associated with scene labels for retrieval. The target reward function is a reward function selected from the preset reward function library that matches the current task planning instruction and scene characteristics, which can quantitatively define the standard of a good trajectory in the current scene (such as prioritizing safety in emergency scenes and considering efficiency and comfort in cruising scenes), and provide a clear optimization direction for trajectory generation. Specifically, the core action of the task planning instruction can be analyzed first (such as "emergency braking" and "navigation left turn"); secondly, the current scene characteristics are extracted (such as "high risk", "urban intersection", etc.); finally, the scene labels of the preset reward function library are traversed to match the most suitable reward function, and if necessary, the weight parameters are adjusted through interpolation (such as increasing the safety distance weight in rainy weather); for example, the task planning instruction is "emergency braking to avoid pedestrians", and the scene characteristics are "rainy weather + urban intersection + high risk", the reward function for the "emergency braking + high risk" scene is matched from the preset reward function library, and the safety distance weight is adjusted from 0.8 to 0.9 (to emphasize the safety requirement in rainy weather), and the target reward function is obtained.The preset diffusion model is a pre-trained generation model based on a diffusion process to generate continuous trajectories conforming to a historical trajectory distribution. The core principle is to generate new trajectories consistent with the distribution of training data (historical planning trajectories) from random noise through a gradual denoising process. Specifically, historical planning trajectory data (including time, location, speed, etc.) of target vehicles and vehicles of the same type can be collected, and a training set can be constructed according to scene classification (such as braking, turning, and cruising). Then, a model can be trained based on an improved denoising diffusion probabilistic model (DDPM) architecture, with input being time-series parameters of trajectories and output being smoothed trajectories after denoising. Finally, model parameters can be fine-tuned for different scenarios (such as high-speed and urban) to ensure that the generated trajectories conform to the characteristics of the scenarios. For example, for a diffusion model trained for the "urban intersection emergency braking" scenario, input is a random noise vector (with dimensions matching trajectory parameters), and through a 100-step denoising process, a trajectory containing "speed decreasing from 35 km / h to 0 km / h within 0-3s and continuous change of position coordinates" is generated, and the distribution of the generated trajectory is consistent with the distribution of 100,000 historical emergency braking trajectories with a coincidence degree of ≥90%. Based on the target reward function and the distribution of historical planning trajectories, the process of generating an initial planning trajectory through the preset diffusion model is a method of generating an initial planning trajectory through a preset diffusion model when the target vehicle has a navigation instruction or an external driving instruction, combining the optimization target of the target reward function and the distribution characteristics of the historical trajectory, to ensure that the generated trajectory not only conforms to the regular driving rules but also meets the specific needs of the current task.In the case where the target vehicle does not have navigation instructions and external driving instructions, the method for the target large language model to autonomously generate an initial planning trajectory based on multi-source data is to comprehensively understand the global semantics and logically deduce the decision-making logic, which is suitable for dynamic scenarios without preset path guidance (such as straight-line cruising and sudden obstacle avoidance). Specifically, first, data integration can be performed. The target large language model receives environmental perception data (such as "no obstacles 50m ahead, lane width 3.75m"), driving trajectory data (such as "current speed 40km / h, straight-line driving"), traffic rules (such as "current speed limit 50km / h"), and driving experience (such as "straight-line cruising suggests keeping the lane centered and maintaining a stable speed"). Then, reasoning and decision-making are performed. The target large language model determines the core parameters of the trajectory (such as maintaining a speed of 40km / h and a lateral offset of ≤0.3m) based on the logic chain analysis of safety, compliance, and comfort. Finally, trajectory generation is performed. The target large language model outputs a structured initial planning trajectory containing position, speed, and acceleration in time sequence. For example, in the case of a target vehicle driving on a straight line in a city without navigation and external instructions, the environmental perception data shows "no obstacles 100m ahead, two-way four-lane road", the driving trajectory data is "current speed 40km / h, heading angle 0°", the traffic rules are "speed limit 50km / h", and the driving experience is "straight-line cruising keeps the lane centered". After analysis by the target large language model, the initial planning trajectory is generated as follows: "0-5s, maintain a speed of 40km / h, position coordinates linearly extend from (X: 200.5m, Y: 50.0m) to (X: 222.8m, Y: 50.0m), lateral offset ≤0.2m, longitudinal acceleration 0m / s. 2

[0051] Through the implementation of the above embodiments, the diffusion model or the language model is used to generate an initial planning trajectory according to whether there are navigation instructions or external driving instructions, so that the trajectory planning process can respond to the driving task intention and independently reason to generate a reasonable path when there is a lack of clear instructions. The introduction of the diffusion model improves the smoothness and diversity of trajectory generation, while the language model reasoning based on knowledge and perception enhances the intelligence and explainability of decision-making, thereby realizing the trajectory planning capability of task orientation and environmental adaptation.

[0052] Furthermore, as an implementation of the foregoing method embodiments, the present application also provides a driving trajectory planning device for implementing the foregoing method embodiments. The device embodiment corresponds to the foregoing method embodiments, and for the sake of readability, the details of the foregoing method embodiments will not be described one by one. However, it should be clear that the device in the present embodiment can correspondingly implement all the contents in the foregoing method embodiments. For example, Figure 2 ​As shown, the driving trajectory planning apparatus 20 comprises a data acquisition unit 201, a text generation unit 202, a knowledge matching unit 203, an instruction generation unit 204, and a trajectory determination unit 205, wherein the data acquisition unit 201 is configured to acquire environmental perception data and driving trajectory data of a target vehicle; the text generation unit 202 is configured to generate an inference result text by fusing and inferring the environmental perception data and the driving trajectory data through a target large language model; the knowledge matching unit 203 is configured to match traffic rules and driving experiences related to the inference result text from a preset cognitive knowledge base through the target large language model; the instruction generation unit 204 is configured to generate a task planning instruction for the target vehicle through the target large language model according to the traffic rules and the driving experiences; and the trajectory determination unit 205 is configured to determine a planning trajectory of the target vehicle based on the task planning instruction and the environmental perception data.

[0053] In some embodiments, the data acquisition unit 201 is further configured to determine a target perception algorithm from a preset perception algorithm library through the target large language model based on a current scene of the target vehicle, wherein the current scene comprises at least one of weather, traffic density, and road type; and perform analysis and processing on vehicle external sensor data of the target vehicle through the target perception algorithm to obtain the environmental perception data.

[0054] In some embodiments, the text generation unit 202 is further configured to embed and replace the environmental perception data and the driving trajectory data based on a predefined inference template to obtain a preset prompt word, wherein the predefined inference template comprises object output constraints and influence output constraints; and input the preset prompt word into the target large language model to obtain an inference result text output by the target large language model, wherein the inference result text comprises a road participating object and an influence possibility of the road participating object on the target vehicle.

[0055] In some embodiments, the knowledge matching unit 203 is further configured to encode the inference result text to obtain text vector data; search a plurality of candidate knowledge items related to the first vector data from the preset cognitive knowledge base through a preset similarity search algorithm; and perform semantic analysis on the plurality of candidate knowledge items through the target large language model to filter out common sense rules and driving experiences that match the current scene of the target vehicle.

[0056] In some embodiments, the instruction generation unit 204 is further configured to generate an initial task planning instruction through the target large language model based on the inference result text; perform matching and evaluation of the initial task planning instruction with the traffic rules and the driving experiences through the target large language model to obtain an evaluation result; and perform correction and optimization on the initial task planning instruction through the target large language model based on the evaluation result to generate the task planning instruction.

[0057] In some embodiments, the trajectory determination unit 205 is further configured to generate an initial planning trajectory of the target vehicle based on the task planning instruction and the environment perception data, perform safety verification on the initial planning trajectory, and if the safety verification fails, perform optimization processing on the initial planning trajectory to generate an optimized trajectory as the planning trajectory, or if the safety verification passes, determine the initial planning trajectory as the planning trajectory.

[0058] In some embodiments, the trajectory determination unit 205 is further configured to, if the target vehicle has a navigation instruction or an external driving instruction, determine a target reward function from a preset reward function library based on the task planning instruction, and generate an initial planning trajectory based on the target reward function and a historical planning trajectory distribution through a preset diffusion model, or if the target vehicle does not have a navigation instruction or an external driving instruction, analyze and process the environment perception data, the driving trajectory data, the traffic rules and the driving experience through the target large language model to output the initial planning trajectory.

[0059] The present application also provides a computer-readable storage medium having computer-executable instructions or computer programs stored therein, which, when executed by a processor, cause the processor to perform any step of the driving trajectory planning method provided by the present application.

[0060] In some embodiments, the computer-readable storage medium can be a random access memory (RAM), a read-only memory (ROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc. storage; or can be various devices including one or any combination of the above storage.

[0061] In some embodiments, the computer-executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or as modules, components, subroutines or other units suitable for use in a computing environment.

[0062] In some embodiments, the computer-executable instructions can but not necessarily correspond to files in a file system, can be stored in part of a file storing other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple cooperating files (for example, files storing one or more modules, subroutines or code portions).

[0063] In some embodiments, the computer executable instructions can be deployed to execute on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected through a communication network.

[0064] As shown in Figure 3 The present application also provides an electronic device 30, comprising a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor, wherein the processor 320 implements any step of the driving trajectory planning method described above when executing the computer program 311.

[0065] The present application also provides a computer program product, comprising a computer program or computer executable instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer program or computer executable instructions from the computer readable storage medium, and the processor executes the computer program or computer executable instructions, so that the electronic device executes any step of the driving trajectory planning method described above.

[0066] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; even though the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A driving trajectory planning method, characterized in that, include: Acquire environmental perception data and driving trajectory data of the target vehicle; The environmental perception data and the driving trajectory data are fused and reasoned using a target large language model to generate reasoning result text. Using the target large language model, traffic rules and driving experiences related to the reasoning result text are matched from a preset cognitive knowledge base; Based on the traffic rules and the driving experience, task planning instructions for the target vehicle are generated using the target large language model. Based on the task planning instructions and the environmental perception data, the planned trajectory of the target vehicle is determined.

2. The driving trajectory planning method according to claim 1, characterized in that, The process of acquiring the environmental perception data includes: Based on the current scene of the target vehicle, a target perception algorithm is determined from a preset perception algorithm library through the target large language model, wherein the current scene includes at least one of weather, traffic density and road type; The target perception algorithm is used to analyze and process the external sensor data of the target vehicle to obtain the environmental perception data.

3. The driving trajectory planning method according to claim 1, characterized in that, The step involves fusing and reasoning the environmental perception data and the driving trajectory data using a target large language model to generate reasoning result text, including: The environmental perception data and the driving trajectory data are embedded and replaced based on a predefined reasoning template to obtain preset prompt words. The predefined reasoning template includes object output constraints and influence output constraints. The preset prompt words are input into the target large language model to obtain the inference result text output by the target large language model, wherein the inference result text includes the road participants and the probability of the road participants' influence on the target vehicle.

4. The driving trajectory planning method according to claim 1, characterized in that, The step of matching traffic rules and driving experiences related to the inference result text from a preset cognitive knowledge base using the target large language model includes: The inference result text is encoded to obtain text vector data; A preset similarity search algorithm is used to retrieve multiple candidate knowledge entries related to the first vector data from the preset cognitive knowledge base; The target large language model is used to perform semantic analysis on the multiple candidate knowledge items to filter out common sense rules and driving experience that match the current scenario of the target vehicle.

5. The driving trajectory planning method according to claim 1, characterized in that, The step of generating task planning instructions for the target vehicle based on the traffic rules and driving experience using the target large language model includes: Based on the inference result text, an initial task planning instruction is generated through the target large language model; The initial task planning instructions are matched and evaluated with the traffic rules and driving experience using the target large language model to obtain the evaluation results. Based on the evaluation results, the initial task planning instructions are modified and optimized using the target large language model to generate the task planning instructions.

6. The driving trajectory planning method according to claim 1, characterized in that, Determining the planned trajectory of the target vehicle based on the task planning instructions and the environmental perception data includes: Based on the task planning instructions and the environmental perception data, an initial planned trajectory for the target vehicle is generated; The initial planned trajectory is subjected to a safety check. If the security check fails, the initial planned trajectory is optimized to generate an optimized trajectory as the planned trajectory. If the security verification passes, the initial planned trajectory will be determined as the planned trajectory.

7. The driving trajectory planning method according to claim 6, characterized in that, The step of generating the initial planned trajectory of the target vehicle based on the task planning instructions and the environmental perception data includes: If the target vehicle has navigation instructions or external driving instructions, then the target reward function is determined from the preset reward function library based on the task planning instructions, and the initial planning trajectory is generated through the preset diffusion model based on the target reward function and the historical planning trajectory distribution. If the target vehicle does not have the navigation instructions and the external driving instructions, the environmental perception data, the driving trajectory data, the traffic rules and the driving experience are analyzed and processed by the target large language model to output the initial planned trajectory.

8. A driving trajectory planning device, characterized in that, include: The data acquisition unit is used to acquire environmental perception data and driving trajectory data of the target vehicle. The text generation unit is used to perform fusion reasoning on the environmental perception data and the driving trajectory data through the target large language model to generate reasoning result text. The knowledge matching unit is used to match traffic rules and driving experiences related to the reasoning result text from a preset cognitive knowledge base using the target large language model. The instruction generation unit is used to generate task planning instructions for the target vehicle based on the traffic rules and the driving experience, using the target large language model. The trajectory determination unit is used to determine the planned trajectory of the target vehicle based on the task planning instructions and the environmental perception data.

9. An electronic device, comprising: A memory and a processor, characterized in that the processor, when executing a computer program stored in the memory, implements the steps of the driving trajectory planning method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the driving trajectory planning method as described in any one of claims 1 to 7.

Images