A method and related device for predicting vehicle intent

Abstract

This application provides a vehicle intent prediction method applied to the field of autonomous driving. The method includes: acquiring the location of a target vehicle; determining multiple candidate lane groups within a preset distance that the target vehicle can drive on based on the location of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than the target threshold; acquiring road condition information for each candidate lane group; and determining a target candidate lane group from the multiple candidate lane groups based on the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intent of the target vehicle. This application defines the driving intent of the target vehicle as the lane direction of the most likely lane to be driven on. For complex road conditions, lane direction can more accurately represent the vehicle's driving intent compared to directional intents such as left turn or right turn.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method for predicting vehicle intentions and related apparatus. Background Technology

[0002] Driving intent refers to the vehicle's intended driving strategy in the future, which can be estimated based on road conditions and driving status. Vehicle trajectory prediction refers to predicting the vehicle's location at each point in time within a certain period of the future.

[0003] In the field of autonomous driving, by estimating the driving intentions of surrounding vehicles in real time, accurately and reliably, and predicting the future driving trajectory of vehicles, it can help autonomous vehicles anticipate traffic conditions ahead, establish traffic patterns around the vehicle, help judge the importance of other vehicles and targets, screen key targets for interaction, facilitate the vehicle to plan its path in advance, and safely pass through complex scenarios.

[0004] In the existing technology, driving intention is defined as directional intentions such as going straight, turning left, and turning right. However, the above definition of driving intention has limited ability to represent complex road scenarios and cannot cover all driving intentions. Summary of the Invention

[0005] Firstly, this application provides a method for predicting vehicle intent, the method comprising:

[0006] Obtain the location of the target vehicle;

[0007] In order to accurately predict whether other vehicles around the vehicle will affect its driving safety and driving decisions, and to control its driving strategy based on the surrounding vehicles, it is necessary to determine the driving intentions of at least one associated vehicle located around the vehicle. In this embodiment, the target vehicle is any one of the at least one associated vehicle located around the vehicle.

[0008] Based on the location of the target vehicle, multiple candidate lane groups that the target vehicle can drive within a preset distance are determined, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold.

[0009] The so-called lanes that the target vehicle can drive in can be understood as the lanes that the target vehicle can drive in within a preset distance, provided that it does not violate traffic regulations or other constraints.

[0010] In order to more accurately define the driving intention of the target vehicle, the lane direction of the lane in which the target vehicle is most likely to travel in the future can be taken as the driving intention of the target vehicle. Therefore, it is first necessary to determine which lanes the target vehicle may travel in the future within a preset distance. The lanes in which the target vehicle may travel within the preset distance can be used as candidate lanes.

[0011] The aforementioned preset distance can be pre-set, depending on the distance at which the target vehicle's driving intention is to be predicted; or, the preset distance can be determined based on the target vehicle's speed and the prediction time required for that driving intention.

[0012] Obtain traffic information for each of the multiple candidate lane groups; based on the traffic information, determine a target candidate lane group from the multiple candidate lane groups, and use the lane direction of the target candidate lane group as the driving intention of the target vehicle.

[0013] In this application embodiment, the driving intention of the target vehicle is defined as the lane direction of the most likely lane to be traveled. For complex road conditions, the lane direction can more accurately represent the driving intention of the vehicle compared with directional intentions such as left turn and right turn.

[0014] In one possible implementation, each candidate lane group includes at least one lane, and the lane direction difference of the lanes included in each candidate lane group within the preset distance is less than the target threshold.

[0015] Since the lane direction of the lane most likely to be traveled by the target vehicle in the future needs to be taken as the driving intention in this embodiment of the application, and since some lanes in the candidate lanes usually have the same or similar lane direction, multiple candidate lanes can be clustered based on the direction of the lane lines to obtain multiple candidate lane groups. The difference in lane direction between different candidate lane groups within the preset distance is greater than the target threshold. Each candidate lane group includes at least one lane, and the difference in lane direction between the lanes included in each candidate lane group is less than the target threshold.

[0016] In one possible implementation, the lane direction is the direction of the lane line within a preset distance, starting from the position of the target vehicle. That is, the lane direction indicates the direction in which the lane line extends within the preset distance, rather than simply the direction of the lane line at a single point. Specifically, the lane direction can refer to the direction indicated by the trajectory of the lane line; for example, the lane direction can refer to the trajectory of a vehicle traveling along the lane line.

[0017] In one possible implementation, the preset distance is a fixed value of less than 200 meters; or, the preset distance is obtained by calculating the driving speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the driving speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 7 seconds.

[0018] In one possible implementation, the method further includes: obtaining the historical driving route of the target vehicle; determining a target candidate lane group from the multiple candidate lane groups based on the road condition information, including: determining the target candidate lane group from the multiple candidate lane groups based on the road condition information and the historical driving route. Furthermore, in addition to obtaining the road condition information of each candidate lane group in the multiple candidate lane groups, the historical driving route of the target vehicle or other driving status information can also be obtained. The driving status information may include, but is not limited to, speed, acceleration, and steering angle relative to the lane centerline, etc., which are not limited here.

[0019] In one possible implementation, the traffic information includes at least one of the following:

[0020] Lane centerline information, drivable area information, obstacle information, and speed limit area information.

[0021] Among them, lane centerline information can indicate the position of the lane's centerline and the direction in which the centerline extends.

[0022] The lane can include drivable areas and non-drivable areas. Drivable area information can refer to the areas on the lane that are allowed to be driven in as stipulated by traffic regulations.

[0023] Among them, obstacle information can also be called roadblock information. For example, obstacles can refer to objects that obstruct the movement of the target vehicle, such as motor vehicles, non-motor vehicles, roadblocks, pedestrians, and animals. They can also refer to the end of a road, a collapsed section of road, a sunken road surface, or an intersection.

[0024] Among them, speed limit area information can refer to blind spots of vehicles, speed-limited sections of roads such as school campuses, and traffic congestion information.

[0025] In one possible implementation, the traffic information is represented as a rasterized image comprising multiple image channels, each image channel representing at least one piece of information in the traffic information.

[0026] This application embodiment uses a rasterized image encoding method, where different image channels correspond to different road condition information, thereby making the expression of road condition information more accurate and easier for neural networks to learn.

[0027] In one possible implementation, the method further includes: determining the target predicted trajectory of the target vehicle using a neural network model based on the road condition information of the target candidate lane group.

[0028] To more accurately predict the intention of a target vehicle, the target vehicle's predicted trajectory can also be calculated. In one possible implementation, the method further includes: obtaining the historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle; determining the degree of influence of the at least one associated vehicle on the target vehicle's driving based on the historical driving routes using an attention mechanism; and determining the target vehicle's predicted trajectory based on the road condition information of the target candidate lane group using a neural network model, including: determining the target vehicle's predicted trajectory based on the road condition information of the target candidate lane group and the degree of influence.

[0029] In one possible implementation, the target vehicle's predicted trajectory is determined using a neural network model based on the road condition information of the target candidate lane group, including:

[0030] Based on the road condition information of the target candidate lane group, the neural network model is used to determine multiple candidate trajectories of the target vehicle and the confidence level of each candidate trajectory.

[0031] Based on this confidence level, the target predicted trajectory of the target vehicle is determined from the multiple candidate trajectories.

[0032] The neural network model can include multiple prediction branches, each capable of predicting different trajectory types. Each of these branches can determine a candidate trajectory and its confidence level for the target vehicle based on the road condition information of the target candidate lane group. The candidate trajectory with the highest confidence level can then be selected as the target predicted trajectory, which can then be used as the predicted trajectory for the target vehicle. In subsequent embodiments, these prediction branches can also be referred to as neural network sub-models.

[0033] Secondly, this application provides a model training method, including:

[0034] Obtain the first neural network model, the position of the target vehicle, and its actual driving trajectory, wherein the actual driving trajectory of the target vehicle is located in the target lane;

[0035] Based on the location of the target vehicle, multiple candidate lane groups that the target vehicle can drive within a preset distance are determined, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold.

[0036] Obtain traffic information for each of the multiple candidate lane groups;

[0037] Based on the road condition information, the first neural network model is used to determine the target candidate lane group from the multiple candidate lane groups. Based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, the first neural network model is trained to obtain the second neural network model.

[0038] In one possible implementation, based on the road condition information, a target candidate lane group is determined from the multiple candidate lane groups using the first neural network model, and the first neural network model is trained based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, including:

[0039] Based on the road condition information of each candidate lane group, the probability of each candidate lane being selected is determined by the first neural network model;

[0040] Obtain the true probability of each candidate lane, where the true probability of the target lane being in a candidate lane group is 1, and the true probability of the target lane not being in a candidate lane group is 0;

[0041] The first neural network model is trained based on the difference between the probability of each candidate lane being selected and the true probability of each candidate lane.

[0042] In one possible implementation, the method further includes:

[0043] Obtain the third neural network model;

[0044] Based on the road condition information of the candidate lane group in which the target lane is located, the target predicted trajectory of the target vehicle is determined by the third neural network model;

[0045] Based on the difference between the predicted trajectory of the target and the actual driving trajectory, the third neural network model is trained to obtain the fourth neural network model.

[0046] In one possible implementation, the third neural network model includes multiple neural network sub-models. Based on road condition information of the candidate lane group in which the target lane is located, the third neural network model determines the target vehicle's predicted trajectory, including:

[0047] Based on the road condition information of the candidate lane group in which the target lane is located, multiple predicted trajectories of the target vehicle are determined through multiple neural network sub-models, wherein each neural network sub-model is used to determine one predicted trajectory.

[0048] Based on the difference between each of the multiple predicted trajectories and the actual driving trajectory, the trajectory with a difference less than a threshold is determined as the target predicted trajectory.

[0049] In one possible implementation, the multiple predicted trajectories of the target vehicle are determined through the multiple neural network sub-models, including:

[0050] The multiple neural network sub-models are used to determine multiple predicted trajectories of the target vehicle and the confidence level of each predicted trajectory.

[0051] Based on the difference between the predicted trajectory of the target and the actual driving trajectory, the third neural network model is trained to obtain a fourth neural network model, including:

[0052] Based on the difference between the predicted trajectory of the target and the actual driving trajectory, and the difference between the confidence level of the predicted trajectory of the target and the value 1, the third neural network model is trained to obtain the fourth neural network model.

[0053] Thirdly, this application provides a vehicle intention prediction device, the device comprising:

[0054] The acquisition module is used to acquire the location of the target vehicle; the acquisition module is also used to acquire the road condition information of each candidate lane group among the multiple candidate lane groups determined by the candidate determination module.

[0055] The lane determination module is used to determine multiple candidate lane groups that the target vehicle can drive within a preset distance based on the position of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold; the lane determination module is also used to determine a target candidate lane group from the multiple candidate lane groups based on the road condition information, and the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

[0056] In one possible implementation, each candidate lane group includes at least one lane, and the lane direction difference of the lanes included in each candidate lane group within the preset distance is less than the target threshold.

[0057] In one possible implementation, the lane direction is the direction of the lane line within a preset distance, starting from the position of the target vehicle.

[0058] In one possible implementation, the preset distance is a fixed value less than 200 meters; or,

[0059] The preset distance is calculated by taking the target vehicle’s speed and the intention prediction time. The preset distance is positively correlated with the target vehicle’s speed and the intention prediction time, and the intention prediction time is less than or equal to 7 seconds.

[0060] In one possible implementation, the acquisition module is also used to acquire the historical driving routes of the target vehicle;

[0061] The lane determination module is specifically used to determine the target candidate lane group from the multiple candidate lane groups based on the road condition information and the historical driving route.

[0062] In one possible implementation, the traffic information includes at least one of the following:

[0063] Lane centerline information, drivable area information, obstacle information, and speed limit area information.

[0064] In one possible implementation, the traffic information is represented as a rasterized image comprising multiple image channels, each image channel representing at least one piece of information in the traffic information.

[0065] In one possible implementation, the device further includes:

[0066] The trajectory prediction module is used to determine the target vehicle's predicted trajectory using a neural network model based on the road condition information of the target candidate lane group.

[0067] In one possible implementation, the acquisition module is also used to acquire the historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle;

[0068] The trajectory prediction module is used to determine the degree of influence of at least one associated vehicle on the target vehicle's driving based on the historical driving route and through an attention mechanism.

[0069] Based on the road condition information of the target candidate lane group and the degree of driving impact, the target vehicle's predicted trajectory is determined through a neural network model.

[0070] In one possible implementation, the trajectory prediction module is specifically used to determine multiple candidate trajectories of the target vehicle and the confidence level of each candidate trajectory based on the road condition information of the target candidate lane group and through the neural network model.

[0071] Based on this confidence level, the target predicted trajectory of the target vehicle is determined from the multiple candidate trajectories.

[0072] Fourthly, this application provides a model training apparatus, comprising:

[0073] The acquisition module is used to acquire the first neural network model, the position of the target vehicle, and the actual driving trajectory, wherein the actual driving trajectory of the target vehicle is located on the target lane; the acquisition module is also used to acquire the road condition information of each candidate lane group among the multiple candidate lane groups determined by the lane determination module.

[0074] The lane determination module is used to determine multiple candidate lane groups that the target vehicle can drive within a preset distance based on the position of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold.

[0075] The model training module is used to determine the target candidate lane group from the multiple candidate lane groups based on the road condition information obtained by the acquisition module, and to train the first neural network model based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, so as to obtain the second neural network model.

[0076] In one possible implementation, the model training module is specifically used to determine the probability of each candidate lane being selected by the first neural network model based on the road condition information of each candidate lane group.

[0077] Obtain the true probability of each candidate lane, where the true probability of the target lane being in a candidate lane group is 1, and the true probability of the target lane not being in a candidate lane group is 0;

[0078] The first neural network model is trained based on the difference between the probability of each candidate lane being selected and the true probability of each candidate lane.

[0079] In one possible implementation, the acquisition module is also used to acquire a third neural network model;

[0080] The device also includes:

[0081] The trajectory prediction module is used to determine the target predicted trajectory of the target vehicle through the third neural network model based on the road condition information of the candidate lane group in which the target lane is located.

[0082] The model training module is also used to train the third neural network model based on the difference between the predicted trajectory of the target and the actual driving trajectory, so as to obtain the fourth neural network model.

[0083] In one possible implementation, the third neural network model includes multiple neural network sub-models. The trajectory prediction module is specifically used to determine multiple predicted trajectories of the target vehicle through the multiple neural network sub-models based on the road condition information of the candidate lane group in which the target lane is located. Each neural network sub-model is used to determine one predicted trajectory.

[0084] Based on the difference between each of the multiple predicted trajectories and the actual driving trajectory, the trajectory with a difference less than a threshold is determined as the target predicted trajectory.

[0085] In one possible implementation, the trajectory prediction module is specifically used to determine multiple predicted trajectories of the target vehicle and the confidence level of each predicted trajectory through the multiple neural network sub-models.

[0086] Based on the difference between the predicted trajectory of the target and the actual driving trajectory, the third neural network model is trained to obtain a fourth neural network model, including:

[0087] Based on the difference between the predicted trajectory of the target and the actual driving trajectory, and the difference between the confidence level of the predicted trajectory of the target and the value 1, the third neural network model is trained to obtain the fourth neural network model.

[0088] Fifthly, this application provides a server including a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the server performs the method as described in the first aspect or any possible implementation thereof.

[0089] In a sixth aspect, this application provides a server including a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the server performs the method as described in the second aspect or any possible implementation thereof.

[0090] In a seventh aspect, this application provides a vehicle that includes a vehicle intention prediction device as described in the third aspect.

[0091] Eighthly, this application provides a computer storage medium storing a computer program that, when executed by a computer, causes the computer to perform the method as described in the first aspect or any possible implementation thereof.

[0092] Ninthly, this application provides a computer storage medium storing a computer program that, when executed by a computer, causes the computer to perform the method as described in the second aspect or any possible implementation thereof.

[0093] In a tenth aspect, this application provides a computer program product storing instructions that, when executed by a computer, cause the computer to perform the method as described in the first aspect or any possible implementation thereof.

[0094] In the eleventh aspect, this application provides a computer program product that stores instructions that, when executed by a computer, cause the computer to perform the method as described in the second aspect or any possible implementation thereof.

[0095] In a twelfth aspect, this application provides a chip system including a processor for supporting an execution device or training device in implementing the functions involved in the foregoing aspects, such as transmitting or processing data involved in the foregoing methods; or, information. In one possible design, the chip system further includes a memory for storing program instructions and data necessary for the execution device or training device. The chip system may be composed of chips or may include chips and other discrete devices.

[0096] This application provides a vehicle intention prediction method, which includes: obtaining the location of a target vehicle; determining multiple candidate lane groups within a preset distance that the target vehicle can travel on based on the location of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold; obtaining road condition information for each candidate lane group; and determining a target candidate lane group from the multiple candidate lane groups based on the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle. This application defines the driving intention of the target vehicle as the lane direction of the most likely lane to be traveled on. For complex road conditions, lane direction can more accurately represent the vehicle's driving intention compared to directional intentions such as left turn or right turn. Attached Figure Description

[0097] Figure 1a A structural diagram illustrating the main framework of artificial intelligence;

[0098] Figure 1b This is an illustration of a road condition.

[0099] Figure 1c A functional block diagram of an autonomous driving device with autonomous driving function provided in the embodiments of this application;

[0100] Figure 2 A flowchart illustrating a vehicle intent prediction method provided in an embodiment of this application;

[0101] Figure 3 This is an illustration of traffic information;

[0102] Figure 4 This is a diagram illustrating the direction of a lane.

[0103] Figure 5 This is a diagram illustrating the data processing for the attention head;

[0104] Figure 6 A flowchart illustrating a model training method provided in an embodiment of this application;

[0105] Figure 7 The overall framework of the neural network provided in the embodiments of this application;

[0106] Figure 8 A schematic diagram of the structure of a vehicle intent prediction device provided in an embodiment of this application;

[0107] Figure 9 This application provides a schematic diagram of the structure of a model training device according to an embodiment of the present application.

[0108] Figure 10 A schematic diagram of the structure of the execution device provided in the embodiments of this application;

[0109] Figure 11 A schematic diagram of the structure of the training device provided in the embodiments of this application;

[0110] Figure 12 This is a schematic diagram of a chip structure provided in an embodiment of this application. Detailed Implementation

[0111] The embodiments of the present invention will now be described with reference to the accompanying drawings. The terminology used in the embodiments section is for illustrative purposes only and is not intended to limit the scope of the invention.

[0112] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0113] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0114] First, the overall workflow of the artificial intelligence system is described; please refer to [link / reference]. Figure 1a , Figure 1aThe diagram illustrates a structural framework for artificial intelligence (AI). The framework is further elaborated below along two dimensions: the "Intelligent Information Chain" (horizontal axis) and the "IT Value Chain" (vertical axis). The "Intelligent Information Chain" reflects a series of processes from data acquisition to processing. For example, this process could be a general sequence of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. In this process, data undergoes a condensation process of "data—information—knowledge—wisdom." The "IT Value Chain" reflects the value that AI brings to the information technology industry, from the underlying infrastructure of human intelligence and information (provided and processed through technological means) to the industrial ecosystem of the system.

[0115] (1) Infrastructure

[0116] Infrastructure provides computing power to support artificial intelligence systems, enabling communication with the external world and providing support through a basic platform. This communication occurs through sensors; computing power is provided by intelligent chips (hardware acceleration chips such as CPUs, NPUs, GPUs, ASICs, and FPGAs); and the basic platform includes distributed computing frameworks and related platform guarantees and support, which may include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to acquire data, and this data is provided to intelligent chips in the distributed computing system provided by the basic platform for computation.

[0117] (2) Data

[0118] The data at the next layer of infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, text, and IoT data from traditional devices, including business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0119] (3) Data processing

[0120] Data processing typically includes methods such as data training, machine learning, deep learning, search, reasoning, and decision-making.

[0121] Among them, machine learning and deep learning can perform intelligent information modeling, extraction, preprocessing, and training of data by symbolizing and formalizing it.

[0122] Reasoning refers to the process in which, in a computer or intelligent system, the machine thinks and solves problems by simulating human intelligent reasoning, based on reasoning control strategies and using formalized information. Typical functions include search and matching.

[0123] Decision-making refers to the process of making decisions based on intelligent information after reasoning, and it typically provides functions such as classification, sorting, and prediction.

[0124] (4) General ability

[0125] After the data processing mentioned above, the results of the data processing can be used to form some general capabilities, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0126] (5) Smart Products and Industry Applications

[0127] Intelligent products and industry applications refer to products and applications of artificial intelligence systems in various fields. They are the encapsulation of overall artificial intelligence solutions, productizing intelligent information decision-making and realizing practical applications. Their application areas mainly include: intelligent terminals, intelligent transportation, intelligent healthcare, autonomous driving, smart cities, etc.

[0128] This application can be applied to the field of autonomous driving, specifically enabling the prediction of vehicle driving intentions and driving trajectories in the field of autonomous driving.

[0129] Next, we will introduce the concepts of driving intent and trajectory prediction:

[0130] Driving intent refers to the vehicle's intended driving strategy in the future, which can be estimated based on road conditions and driving status. Vehicle trajectory prediction refers to predicting the vehicle's location at each point in time within a certain period of the future.

[0131] In the field of autonomous driving, by estimating the driving intentions of surrounding vehicles in real time, accurately, and reliably, and predicting the future driving trajectories of vehicles, it can help the autonomous vehicle anticipate traffic conditions ahead, establish a traffic situation around the vehicle, facilitate the assessment of the importance of other vehicles and targets, filter key targets for interaction, and enable the autonomous vehicle to plan its path in advance and safely navigate complex scenarios. It should be understood that, in this embodiment, the aforementioned surrounding vehicles can also be referred to as associated vehicles located around the autonomous vehicle.

[0132] In existing technologies, driving intention is defined as directional intentions such as going straight, turning left, and turning right. For example, in an intersection scenario, a vehicle's driving intention may include going straight, turning left, and turning right. However, the above definition of driving intention has limited capacity to represent complex scenarios, and directional intentions cannot cover all driving intentions in some complex intersections or other complex lane scenarios. For example, refer to... Figure 1b , Figure 1b This is a diagram of a road condition, in which lanes 1 and 2 are left-turn lanes, lanes 3 and 4 are straight-ahead lanes, and lane 5 is an S-shaped lane. When a vehicle is traveling in lane 5, the vehicle's driving intention cannot be accurately expressed by simply indicating a left or right turn. In order to solve the above problem, this application defines the vehicle's driving intention by the lane direction.

[0133] For example, in some complex road conditions, there is a lane whose direction extends to the left with an arc of 30 degrees curvature and then to the right with an arc of 50 degrees curvature. If it is predicted that a vehicle will travel in this lane, the above lane direction can be used as the vehicle's driving intention. Compared with the driving intention of turning left first and then right, the driving intention defined by the lane direction as the smallest scale is more accurate and has a wider coverage.

[0134] This application provides a vehicle intent prediction method that can be applied to autonomous driving prediction systems. The prediction system can predict the vehicle's driving intent and trajectory based on road condition information, the vehicle's historical driving routes, and other information.

[0135] In this embodiment of the application, the prediction system may include hardware circuits (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.) or combinations of these hardware circuits. For example, the prediction system may be a hardware system with instruction execution capabilities, such as a CPU or DSP, or a hardware system without instruction execution capabilities, such as an ASIC or FPGA, or a combination of the aforementioned hardware systems without instruction execution capabilities and hardware systems with instruction execution capabilities.

[0136] Specifically, the prediction system can be a hardware system with the function of executing instructions. The vehicle intention prediction method provided in this application embodiment can be software code stored in memory. The prediction system can obtain the software code from the memory and execute the obtained software code to implement the vehicle intention prediction method provided in this application embodiment.

[0137] It should be understood that the prediction system can be a combination of a hardware system without the function of executing instructions and a hardware system with the function of executing instructions. Some steps of the vehicle intention prediction method provided in the embodiments of this application can also be implemented by a hardware system without the function of executing instructions in the prediction system, which is not limited here.

[0138] In this embodiment of the application, the prediction system can be deployed on a vehicle or a cloud-based server. The following describes the prediction process of the prediction system to realize the vehicle's driving intention and predict the trajectory, taking the prediction system deployed on a vehicle as an example, in conjunction with the software and hardware modules on the vehicle.

[0139] The vehicles in this application embodiment, such as the target vehicle and related vehicles around the target vehicle, can refer to internal combustion engine vehicles that use an engine as a power source, hybrid vehicles that use an engine and an electric motor as power sources, electric vehicles that use an electric motor as a power source, and so on.

[0140] In this embodiment of the application, the vehicle may include an autonomous driving device 100 with autonomous driving function.

[0141] Reference Figure 1c , Figure 1c This is a functional block diagram of an autonomous driving device 100 with autonomous driving capabilities provided in an embodiment of this application. In one embodiment, the autonomous driving device 100 may include various subsystems, such as a driving system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, and a user interface 116. Optionally, the autonomous driving device 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of the autonomous driving device 100 may be interconnected via wired or wireless means.

[0142] The mobility system 102 may include components that provide powered motion for the autonomous driving device 100. In one embodiment, the mobility system 102 may include an engine 118, an energy source 119, a transmission 120, and wheels / tires 121. The engine 118 may be an internal combustion engine, an electric motor, an air-compressed engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine 118 converts the energy source 119 into mechanical energy.

[0143] Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 119 may also provide power to other systems of the autonomous driving device 100.

[0144] The transmission 120 can transmit mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other components, such as a clutch. The drive shaft may include one or more axles that can be coupled to one or more wheels 121.

[0145] Sensor system 104 may include several sensors that sense information about the environment surrounding autonomous driving device 100. For example, sensor system 104 may include a positioning system 122 (which may be a Global Positioning System (GPS) system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU) 124, radar 126, a laser rangefinder 128, and a camera 130. Sensor system 104 may also include sensors monitoring the internal systems of autonomous driving device 100 (e.g., in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). This detection and identification is a key function for the safe operation of autonomous driving device 100.

[0146] The positioning system 122 can be used to estimate the geographic location of the autonomous driving device 100. The IMU 124 is used to sense changes in the position and orientation of the autonomous driving device 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope.

[0147] Radar 126 can use radio signals to sense objects in the surrounding environment of the autonomous driving device 100. In some embodiments, in addition to sensing objects, radar 126 can also be used to sense the speed and / or direction of travel of objects.

[0148] Radar 126 may include an electromagnetic wave transmitting unit and a receiving unit. In terms of electromagnetic wave transmission principle, radar 126 can be implemented as either pulse radar or continuous wave radar. In the continuous wave radar mode, radar 126 can be implemented as either frequency-modulated continuous wave (FMCW) or frequency shift keying (FSK) depending on the signal waveform.

[0149] Radar 126 uses electromagnetic waves as a medium to detect objects based on time-of-flight (TOF) or phase-shift methods, and detects the position of the detected object, the distance to the detected object, and the relative velocity. To detect objects located in front of, behind, or to the side of the vehicle, radar 126 can be configured at an appropriate location on the exterior of the vehicle. LiDAR 126 uses laser light as a medium to detect objects based on TOF or phase-shift methods, and detects the position of the detected object, the distance to the detected object, and the relative velocity.

[0150] Alternatively, in order to detect objects located in front of, behind or to the side of the vehicle, the lidar 126 may be configured at an appropriate location on the exterior of the vehicle.

[0151] The laser rangefinder 128 can use lasers to sense objects in the environment where the autonomous driving device 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0152] Camera 130 can be used to capture multiple images of the surrounding environment of the autonomous driving device 100. Camera 130 can be a still camera or a video camera.

[0153] Optionally, to acquire images of the vehicle's exterior, the camera 130 can be located at an appropriate position outside the vehicle. For example, to acquire images of the front of the vehicle, the camera 130 can be positioned inside the vehicle's interior, close to the windshield. Alternatively, the camera 130 can be positioned around the front bumper or radiator grille. For example, to acquire images of the rear of the vehicle, the camera 130 can be positioned inside the vehicle's interior, close to the rear window. Alternatively, the camera 130 can be positioned around the rear bumper, trunk, or tailgate. For example, to acquire images of the sides of the vehicle, the camera 130 can be positioned inside the vehicle's interior, close to at least one of the side windows. Alternatively, the camera 130 can be positioned around a side mirror, fender, or door.

[0154] In this embodiment of the application, road condition information, historical driving routes, and historical driving routes of related vehicles around the target vehicle can be obtained based on one or more sensors in the sensor system 104.

[0155] The control system 106 controls the operation of the autonomous driving device 100 and its components. The control system 106 may include various elements, including a steering system 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a route control system 142, and an obstacle avoidance system 144.

[0156] The steering system 132 is operable to adjust the forward direction of the automatic driving device 100. For example, in one embodiment, it can be a steering wheel system.

[0157] Throttle 134 is used to control the operating speed of engine 118 and thus the speed of automatic driving device 100.

[0158] Braking unit 136 is used to control the deceleration of the automatic driving device 100. Braking unit 136 can use friction to slow down the wheel 121. In other embodiments, braking unit 136 can convert the kinetic energy of the wheel 121 into electrical current. Braking unit 136 may also take other forms to slow down the rotational speed of the wheel 121 to control the speed of the automatic driving device 100.

[0159] The computer vision system 140 is operable to process and analyze images captured by the camera 130 to identify objects and / or features in the surrounding environment of the autonomous driving device 100. These objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 may use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 140 may be used to map the environment, track objects, estimate object velocities, and so on.

[0160] The route control system 142 is used to determine the driving route of the autonomous driving device 100. In some embodiments, the route control system 142 may combine data from the sensor 138, the positioning system 122, and one or more predetermined maps to determine the driving route for the autonomous driving device 100.

[0161] The obstacle avoidance system 144 is used to identify, assess and avoid or otherwise traverse potential obstacles in the environment of the autonomous driving device 100.

[0162] Of course, in one instance, the control system 106 may include additional or alternative components besides those shown and described. Alternatively, some of the components shown above may be reduced.

[0163] The autonomous driving device 100 interacts with external sensors, other autonomous driving devices, other computer systems, or users via peripheral devices 108. Peripheral devices 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152.

[0164] In some embodiments, peripheral device 108 provides a means for the user of autonomous driving device 100 to interact with user interface 116. For example, on-board computer 148 may provide information to the user of autonomous driving device 100. User interface 116 may also operate on-board computer 148 to receive user input. On-board computer 148 may be operated via touchscreen. In other cases, peripheral device 108 may provide a means for autonomous driving device 100 to communicate with other devices located within the vehicle. For example, microphone 150 may receive audio (e.g., voice commands or other audio input) from the user of autonomous driving device 100. Similarly, speaker 152 may output audio to the user of autonomous driving device 100.

[0165] The wireless communication system 146 can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as code division multiple access (CDMA), EVDO, Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or 4G cellular communication, such as long term evolution (LTE), or 5G cellular communication. The wireless communication system 146 can communicate using WiFi and a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 can communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various autonomous driving device communication systems, may also be used. For example, the wireless communication system 146 may include one or more dedicated short-range communications (DSRC) devices, which may include public and / or private data communications between autonomous driving devices and / or roadside stations.

[0166] In one implementation, the road condition information, historical driving trajectory, and other information in this application embodiment can be received by the vehicle from other vehicles or a cloud server through the wireless communication system 146.

[0167] When the prediction system is located on a server in the cloud, the vehicle can receive driving intention information for the target vehicle from the server via the wireless communication system 146.

[0168] Power source 110 can provide power to various components of autonomous driving device 100. In one embodiment, power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more such battery packs can be configured to provide power to various components of autonomous driving device 100. In some embodiments, power source 110 and energy source 119 can be implemented together, as is the case in some fully electric vehicles.

[0169] Some or all of the functions of the autonomous driving device 100 are controlled by a computer system 112. The computer system 112 may include at least one processor 113, which executes instructions 115 stored in a non-transitory computer-readable medium such as memory 114. The computer system 112 may also be multiple computing devices that control individual components or subsystems of the autonomous driving device 100 in a distributed manner.

[0170] Processor 113 can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a special-purpose device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1c The processor, memory, and other components of computer 110 within the same block are functionally illustrated; however, those skilled in the art will understand that the processor, computer, or memory may actually include multiple processors, computers, or memories that may or may not be stored in the same physical housing. For example, memory may be a hard disk drive or other storage media located in a housing different from that of computer 110. Therefore, references to processors or computers will be understood to include references to a collection of processors or computers or memories that may or may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor that performs calculations only relevant to the component's specific function.

[0171] In the various aspects described herein, the processor may be located remotely from the autonomous driving device and may communicate wirelessly with the autonomous driving device. In other aspects, some of the processes described herein are executed on a processor located within the autonomous driving device, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0172] In some embodiments, memory 114 may contain instructions 115 (e.g., program logic) that can be executed by processor 113 to perform various functions of the autonomous driving device 100, including those described above. Memory 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 102, sensor system 104, control system 106, and peripheral devices 108.

[0173] In addition to instruction 115, memory 114 may also store data such as road maps, route information, the location, direction, speed, and other data of the autonomous driving device, as well as other information. This information can be used by the autonomous driving device 100 and computer system 112 during operation of the autonomous, semi-autonomous, and / or manual modes.

[0174] The vehicle intent prediction method provided in this application embodiment can be software code stored in memory 114. Processor 113 can retrieve the software code from memory and execute the retrieved software code to implement the vehicle intent prediction method provided in this application embodiment. After obtaining the driving intent of the target vehicle, the driving intent can be transmitted to control system 106, and control system 106 can determine the vehicle driving strategy based on the driving intent.

[0175] User interface 116 is used to provide information to or receive information from the user of autonomous driving device 100. Optionally, user interface 116 may include one or more input / output devices within a set of peripheral devices 108, such as wireless communication system 146, vehicle computer 148, microphone 150, and speaker 152.

[0176] Computer system 112 can control the functions of autonomous driving device 100 based on input received from various subsystems (e.g., driving system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 can utilize input from control system 106 to control steering unit 132 to avoid obstacles detected by sensor system 104 and obstacle avoidance system 144. In some embodiments, computer system 112 is operable to provide control over many aspects of autonomous driving device 100 and its subsystems.

[0177] Alternatively, one or more of these components may be installed separately from or associated with the autonomous driving device 100. For example, the memory 114 may exist partially or completely separately from the autonomous driving device 100. The components may be communicatively coupled together in a wired and / or wireless manner.

[0178] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 1c This should not be construed as a limitation on the embodiments of this application.

[0179] The application architecture of the embodiments of this application has been described above. The intent prediction method provided by the embodiments of this application will be described in detail below.

[0180] Since the embodiments of this application involve neural networks, the relevant terms involved in the embodiments of this application will be introduced below for ease of understanding.

[0181] (1) Neural Network

[0182] Neural networks can be composed of neural units, which can refer to units represented by x. s The arithmetic unit that takes input data and an intercept of 1 as input can output the following:

[0183]

[0184] Where s = 1, 2, ..., n, n is a natural number greater than 1, W s For x s The weights are denoted by b, where b is the bias of the neural unit. f is the activation function of the neural unit, used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into an output signal. The output signal of this activation function can be used as the input of the next convolutional layer; the activation function can be the sigmoid function. A neural network is a network formed by connecting multiple of the above-mentioned individual neural units together; that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field, which can be a region composed of several neural units.

[0185] (2) A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers, which can be viewed as a filter. A convolutional layer refers to the layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron can be connected to only some of the neurons in its neighboring layers. A convolutional layer typically contains several feature planes, each composed of rectangularly arranged neural units. Neural units on the same feature plane share weights, which are the convolutional kernel. Shared weights can be understood as the way features are extracted being independent of their location. The convolutional kernel can be formalized as a matrix of random size, and during the training process of the CNN, the kernel can learn reasonable weights. Furthermore, the direct benefit of shared weights is reducing the connections between layers in the CNN, while also reducing the risk of overfitting.

[0186] (3) Deep Neural Networks

[0187] Deep Neural Networks (DNNs), also known as multilayer neural networks, can be understood as neural networks with many hidden layers, though there's no specific metric for "many." DNNs can be categorized into three layers based on their position: input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. All layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer. Although DNNs appear complex, the operation of each layer is actually quite simple, resembling a linear relationship as follows: in, It is the input vector. It is the output vector. α is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained through such a simple operation. Because DNNs have many layers, the coefficients W and the offset vector... The number of these parameters is therefore quite large. The definitions of these parameters in a DNN are as follows: Taking the coefficient W as an example: Assuming a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as... The superscript 3 represents the layer number where coefficient W is located, while the subscript corresponds to the third layer index 2 of the output and the second layer index 4 of the input.

[0188] In summary: the coefficients from the k-th neuron in layer L-1 to the j-th neuron in layer L are defined as follows:

[0189] It's important to note that the input layer does not have a W parameter. In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrix of all layers in the trained deep neural network (a weight matrix formed by the vectors W from many layers).

[0190] (4) Loss Function

[0191] In training a deep neural network, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the deep neural network predicts the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, and training the deep neural network becomes a process of minimizing this loss.

[0192] (5) Backpropagation algorithm

[0193] Convolutional neural networks can employ backpropagation (BP) to correct the parameters in the initial super-resolution model during training, thereby reducing the reconstruction error loss. Specifically, forward propagation of the input signal to the output generates an error loss; this error loss information is then propagated back to update the parameters in the initial super-resolution model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining the optimal parameters of the super-resolution model, such as the weight matrix.

[0194] (6) Attention mechanism

[0195] Attention mechanisms mimic the internal processes of biological observation—aligning internal experience with external senses to increase the precision of observation in specific areas. They enable the rapid sifting of high-value information from a large volume of data using limited attentional resources. Attention mechanisms can quickly extract important features from sparse data and are therefore widely used in natural language processing tasks, particularly machine translation. Self-attention mechanisms, an improvement on attention mechanisms, reduce reliance on external information and are better at capturing the internal correlations of data or features. The core idea of ​​attention mechanisms can be rewritten as follows:

[0196] In this formula, Lx = ||Source|| represents the length of the Source. The meaning is that the elements in the Source are imagined as a series of data pairs. Given a Query element in the Target, the similarity or relevance between the Query and each Key is calculated to obtain the weight coefficient of the Value corresponding to each Key. Then, the Values ​​are weighted and summed to obtain the final Attention value. Therefore, the Attention mechanism essentially performs a weighted sum of the Values ​​of the elements in the Source, while the Query and Key are used to calculate the weight coefficients of their corresponding Values. Conceptually, Attention can be understood as selectively filtering a small amount of important information from a large amount of information and focusing on this important information, ignoring most of the unimportant information. The focusing process is reflected in the calculation of the weight coefficients; the larger the weight, the more focused it is on its corresponding Value. That is, the weight represents the importance of the information, and the Value is the corresponding information. Self-attention can be understood as intra attention. The attention mechanism occurs between the elements of the Target (Query) and all elements of the Source. Self-attention refers to the attention mechanism that occurs between elements within the Source or between elements within the Target. It can also be understood as the attention calculation mechanism in the special case where Target = Source. The specific calculation process is the same, only the calculation object changes.

[0197] (7) Lane center line

[0198] The lane centerline is located between the two boundaries of the lane; it may be virtual or it may actually exist.

[0199] Reference Figure 2 , Figure 2 This is a flowchart illustrating a vehicle intent prediction method provided in an embodiment of this application, as follows: Figure 2As shown, the vehicle intent prediction method provided in this application includes:

[0200] 201. Obtain the location of the target vehicle.

[0201] In this embodiment of the application, the target vehicle can be a vehicle for which driving intention prediction is required.

[0202] In one implementation, in order to accurately predict whether other vehicles in the vicinity will affect the vehicle's driving safety, whether they will influence the vehicle's driving decisions, and how to control the vehicle's driving strategy based on the surrounding vehicles, it is necessary to determine the driving intentions of at least one associated vehicle located around the vehicle. In this embodiment of the application, the target vehicle is any one of the at least one associated vehicle located around the vehicle.

[0203] It should be understood that the aforementioned "associated vehicles" can be understood as vehicles that are within a certain preset range from the vehicle itself. That is, vehicles that are associated with the vehicle are determined based on their proximity, and these associated vehicles are then considered as associated vehicles of the vehicle itself. In addition, "associated vehicles" can also be understood as vehicles that will affect the vehicle's driving state decisions in the future. That is, vehicles that are associated with the vehicle are determined based on whether they will affect the vehicle's driving strategy in the future, and these associated vehicles are then considered as associated vehicles of the vehicle itself.

[0204] In one implementation, the vehicle's processor can control the relevant sensors on the vehicle to acquire the driving status information of surrounding vehicles based on the software code related to step 201 in memory 114, and determine which vehicles are associated vehicles based on the acquired driving status information, that is, determine which vehicles need to be predicted for intent.

[0205] Alternatively, the process of identifying the target vehicle described above can be determined by other vehicles or cloud-side servers; this is not a limitation.

[0206] To accurately predict the future driving intentions of a target vehicle, it is necessary to obtain information such as the vehicle's driving status and road conditions. This information can serve as the basis for predicting the vehicle's driving intentions. Driving status information can include the target vehicle's location.

[0207] The target vehicle's position can be its absolute position on the map or its relative position to the user vehicle. The absolute position of the target vehicle can be determined based on the user vehicle's absolute position and the relative position between the target vehicle and the user vehicle.

[0208] Taking the associated vehicle as the target vehicle as an example, in this embodiment of the application, the driving status information of the target vehicle can be obtained. The driving status information may include the location of the target vehicle. Specifically, the location of the target vehicle can be sensed by the sensors carried by the vehicle, or the location of the target vehicle can be obtained by interacting with other vehicles or cloud-side servers.

[0209] Taking the associated vehicle as the target vehicle as an example, in one implementation, the location of the target vehicle can be obtained in real time, or the location of the target vehicle can be obtained once at a certain time interval.

[0210] 202. Based on the location of the target vehicle, determine multiple candidate lane groups that the target vehicle can travel within a preset distance, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than the target threshold.

[0211] In this embodiment of the application, in order to more accurately define the driving intention of the target vehicle, the lane direction of the lane that the target vehicle is most likely to travel in the future can be taken as the driving intention of the target vehicle. Therefore, it is first necessary to determine which lanes the target vehicle may travel in the future within a preset distance. The lanes that the target vehicle may travel in the preset distance can be taken as candidate lanes.

[0212] It should be understood that the aforementioned preset distance can be pre-set, depending on how far the target vehicle's driving intention is to be predicted; or, the preset distance can be determined based on the target vehicle's speed and the prediction time required for that driving intention.

[0213] In one implementation, the preset distance is a fixed value of less than 200 meters; or, the preset distance is obtained by calculating the driving speed of the target vehicle and the prediction time of the intention, wherein the preset distance is positively correlated with the driving speed of the target vehicle and the prediction time required for the driving intention, and the prediction time required for the intention is less than or equal to 7 seconds.

[0214] It should be understood that the so-called lanes that the target vehicle can travel in can be understood as the lanes that the target vehicle can travel in within a preset distance, provided that it does not violate traffic regulations or other constraints. For example, refer to... Figure 3 , Figure 3This is a diagram illustrating traffic information, in which the target vehicle is located between lanes 3 and 4. At this time, lanes 3, 4, and 5 are lanes that the target vehicle can travel within a preset distance. Lanes 1 and 2 are lanes that are opposite to the target vehicle's current direction of travel. Since there is a solid line between lanes 2 and 3 within the preset distance from the target vehicle's location, the target vehicle cannot change lanes to lanes 1 or 2 or make a U-turn to lanes 1 or 2 within the preset distance. Therefore, lanes 1 and 2 are not lanes that the target vehicle can travel within the preset distance.

[0215] The following describes how to determine the lanes within a preset distance that the target vehicle can travel in, based on the target vehicle's location:

[0216] After obtaining the location of the target vehicle, the lane information around the target vehicle can be determined from the map information. The map information can include the location of lane lines or virtual lane lines, drivable areas, pedestrian crossings, lane boundaries, etc., and is stored and queried in vector form. Lane information can include the width of each lane, the direction of lane lines, etc.

[0217] After obtaining the target vehicle's location, the next lane the target vehicle can travel in can be determined from the map based on that location. In one possible implementation, the similarity between the target vehicle's location and direction of travel and various lanes over a historical period can be used to determine which lanes are associated with the target vehicle. Lanes with associated lanes can be considered as the target vehicle's next lanes. Specifically, the target vehicle's locations at the first and last historical moments can be selected. and It also calculates the target vehicle's direction of travel at the first and last moments, and filters the first moment based on map information. h All lanes whose distance and angle difference from the target vehicle are within a certain threshold range. This lane This refers to the lanes that are associated with the target vehicle at the first moment, and all lanes L whose distance and angle difference with the target vehicle at the last moment t is within a certain threshold range. t Lane L t This represents the lanes that are associated with the target vehicle at the last moment. The intersection of these two lanes can represent the lanes that are associated with the target vehicle during the time period from the first moment to the historical moment. These associated lanes can be used as candidate lanes that the target vehicle can travel within a preset distance in the future.

[0218] It should be understood that since the lane information stored in the map is a continuous sequence, where each sequence can indicate a short lane segment, a continuous sequence can indicate a complete lane. Therefore, the candidate lanes determined in this application embodiment can be stored in the form of lane sequences.

[0219] Since the lane direction of the lane most likely to be traveled by the target vehicle in the future needs to be taken as the driving intention of the target vehicle in this embodiment, and since some lanes in the candidate lanes usually have the same or similar lane directions, multiple candidate lanes can be clustered based on the direction of the lane lines to obtain multiple candidate lane groups. The lane direction difference between different candidate lane groups within the preset distance is greater than the target threshold. Each candidate lane group includes at least one lane, and the lane direction difference between the lanes included in each candidate lane group is less than the target threshold.

[0220] It should be understood that the lane direction refers to the direction of the lane line within the preset distance, starting from the position of the target vehicle. In other words, the lane direction indicates the direction in which the lane line extends within the preset distance, not just the direction of the lane line at a single point. Specifically, the lane direction can refer to the direction indicated by the trajectory of the lane line; for example, the lane direction can refer to the trajectory of a vehicle traveling along the lane line.

[0221] Reference Figure 4 , Figure 4 This is a schematic diagram of lane directions, where lane 5 can be straight, an S-shaped turn, or straight. In this embodiment, lane direction can be understood as the direction of the lane line or the direction of the lane centerline.

[0222] In this application embodiment, the lane direction difference can be the difference between the extension directions of lane lines or lane center lines. For example, lane A extends to the left, lane B extends to the right, lane C extends straight, and lane D extends to the left. The lane direction difference between lane A and lane D is small, while the lane direction difference between lane A, lane B, and lane C is large.

[0223] Reference Figure 5 , Figure 5 This is a schematic diagram of a candidate lane group in an embodiment of this application. Lane 1 and lane 2 are assigned to the same candidate lane group because their lane directions are similar. Lane 3 and lane 4 are assigned to the same candidate lane group because their lane directions are similar. Lane 5 is assigned to an independent candidate lane group because its lane direction is significantly different from the other lanes.

[0224] 203. Obtain the traffic information for each candidate lane group among the multiple candidate lane groups.

[0225] In this embodiment of the application, after determining multiple candidate lane groups, it is necessary to select the target lane group from the multiple candidate lane groups as the target lane group, and the lane direction of the target lane group is taken as the driving intention of the target vehicle. In order to select the target lane group from multiple candidate lane groups, it is necessary to obtain the road condition information of each candidate lane group. The following describes how to obtain the road condition information of each candidate lane group.

[0226] It should be understood that the traffic information for each candidate lane group can be understood as the traffic information for each lane included in each candidate lane group.

[0227] In this embodiment of the application, the road condition information may include at least one of the following: lane centerline information, drivable area information, obstacle information, and speed limit area information.

[0228] Among them, lane centerline information can indicate the position of the lane's centerline and the direction in which the centerline extends.

[0229] The lane can include drivable areas and non-drivable areas. Drivable area information can refer to the areas on the lane that are allowed to be driven in as stipulated by traffic regulations.

[0230] Among them, obstacle information can also be called roadblock information. For example, obstacles can refer to objects that obstruct the movement of the target vehicle, such as motor vehicles, non-motor vehicles, roadblocks, pedestrians, and animals. They can also refer to the end of a road, a collapsed section of road, a sunken road surface, or an intersection.

[0231] Among them, speed limit area information can refer to blind spots of vehicles, speed-limited sections of roads such as school campuses, and traffic congestion information.

[0232] In addition to obtaining traffic information for each candidate lane group among multiple candidate lane groups, it can also obtain the target vehicle's historical driving route or other driving status information. Driving status information may include, but is not limited to, speed, acceleration, and steering angle relative to the lane centerline, etc., which are not limited here.

[0233] In one possible implementation, the information obtained above for predicting driving intentions can be represented as a rasterized image, wherein the rasterized image includes multiple image channels, each image channel representing at least one type of information.

[0234] Specifically, road condition information can be rasterized to obtain rasterized images of map elements. The target vehicle's historical driving route is also encoded at the same resolution and image size. The resulting rasterized image is merged with the rasterized image of the map elements and used as input data for determining the target vehicle's driving intentions.

[0235] The following describes how to perform rasterization encoding on road condition information and the historical driving routes of the target vehicle:

[0236] In one possible implementation, the size of the rasterized image, such as 720*720 pixels, and the scaling ratio of the image to the actual location can be determined first, such as 4 pixels per meter, so the actual range that the image can represent is 180m*180m. A coordinate system is established with the target vehicle's last observed position as the origin and the target's orientation as the coordinate axis direction. All map elements and obstacles are transformed to this coordinate system and then mapped onto the raster image. Before mapping, they are map elements and obstacles in the actual physical space; after mapping, they are pixels on the raster image. Based on this mapping, map elements and obstacles in the physical space can be represented by pixels on the raster image. For example, if the coordinate center is selected at the horizontal center and 2 / 3 of the vertical direction of the image, the rasterized image can represent a range of 120m in front of the target vehicle, 60m behind it, and 90m to the left and right. Information encoding can use independent channel encoding, that is, each type of information is encoded into different image channels, or some information can be encoded into the same image channel. The total number of image channels is variable and can be added or removed as needed.

[0237] Specifically, they can be divided into three categories:

[0238] 1. Intent Path Related Channels

[0239] For candidate lane groups requiring encoding, partial road condition information of these lane groups is used as intent path-related channels, including but not limited to lane centerline channels and drivable area channels. Specifically, taking the encoding process of lane centerline information as an example, the lane centerline information of the candidate lane group can be plotted onto the image channel after coordinate transformation and scaling. The drivable area channel plots the drivable area corresponding to the intent path onto the image channel. This plotted image channel can be a binary image, meaning the filled area value is 1, otherwise it is 0.

[0240] 2. Lanes not related to the intended path

[0241] For candidate lane groups that need to be coded, road condition information that is not in the candidate lane group is taken as non-intended path related channels. Non-intended path related channels are selected and coded, including but not limited to lane centerline channels and drivable area channels.

[0242] 3. Shared information channels

[0243] Partial road condition information for candidate lane groups can be included, but is not limited to, historical driving route channels, historical information channels for surrounding obstacles, and speed-limited area channels. Taking the historical driving route channel as an example, the historical driving route channel can be plotted onto the image channel as a curve connecting the historical positions of the target vehicle. The plotted values ​​are filled with linear values, with larger values ​​closer to the current time, used to distinguish the historical driving direction of the target vehicle. The historical information channel for surrounding dynamic obstacles is similar, and all surrounding targets are encoded into the same image channel. The speed-limited area channel can be plotted onto the corresponding image channel as a binary map.

[0244] This application embodiment uses a rasterized image encoding method, where different image channels correspond to different road condition information, thereby making the expression of road condition information more accurate and easier for neural networks to learn.

[0245] 204. Based on the road condition information, determine the target candidate lane group from the multiple candidate lane groups. The lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

[0246] In this embodiment of the application, after obtaining the road condition information of each candidate lane group, a target candidate lane group can be determined from the multiple candidate lane groups based on the road condition information. The lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

[0247] It should be understood that a target candidate lane group can also be determined from these multiple candidate lane groups based on the road condition information of each candidate lane group and the historical driving route of the target vehicle.

[0248] In one implementation, the information used for predicting driving intentions can be represented as a rasterized image, and a target candidate lane group can be determined from the plurality of candidate lane groups based on the rasterized image.

[0249] The following describes how to determine the target candidate lane group from the multiple candidate lane groups based on the road condition information.

[0250] In one implementation, a neural network model can be obtained, which is pre-trained and has the ability to determine the probability of selection for each candidate lane group based on road condition information.

[0251] In one implementation, a neural network model can be used to extract features from traffic information to obtain feature vectors representing the traffic information of each candidate lane group. These feature vectors can be expanded into one-dimensional vectors, such as a one-dimensional vector of length 128. The neural network can then process the feature vectors corresponding to each candidate lane group to obtain the selection probability of each candidate lane group. The candidate lane group with the highest selection probability is selected as the target candidate lane group, and the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

[0252] For example, a CNN can be used to process traffic information to obtain a feature vector representing the traffic information of each candidate lane group. The feature vector can be expanded into a one-dimensional vector, such as a one-dimensional vector of length 128. This one-dimensional vector is then input into two fully connected layers, and the final output is a vector of length 1. After passing through the sigmoid function, the selection probability of each candidate lane group is obtained. The CNN network structure can be, but is not limited to, ResNet16.

[0253] How to train the aforementioned neural network model, which has the ability to determine the selection probability of each candidate lane group based on road condition information, will be described in subsequent embodiments and will not be repeated here.

[0254] In this embodiment, to more accurately predict the intention of the target vehicle, the target vehicle's predicted trajectory can also be calculated. Specifically, the road condition information of the aforementioned target candidate lane group can be input into a neural network model, which has the ability to output the target vehicle's predicted trajectory based on the road condition information. Therefore, the target vehicle's predicted trajectory can be determined through the neural network model based on the road condition information of the target candidate lane group.

[0255] In one implementation, the feature vector corresponding to the road condition information of the aforementioned target candidate lane group can be input into a neural network model. Then, based on the feature vector corresponding to the road condition information of the target candidate lane group, the target predicted trajectory of the target vehicle can be determined through the neural network model.

[0256] It should be understood that, in addition to road condition information, the degree of influence of surrounding vehicles on the target vehicle can also be used as input to the neural network. Specifically, the historical driving routes of the target vehicle and at least one surrounding vehicle can be obtained, and based on the historical driving routes, the degree of influence of the at least one surrounding vehicle on the target vehicle can be determined through an attention mechanism. Then, based on the road condition information of the target candidate lane group and the degree of influence, the target predicted trajectory of the target vehicle can be determined through a neural network model.

[0257] The attention mechanism will be described next:

[0258] The attention mechanism can be implemented by a multi-head attention layer, which can obtain N input vectors X. l Each input vector corresponds to a historical driving route. Specifically, the historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle can be feature extracted using, but not limited to, recurrent neural networks (RNNs) or long short-term memory (LSTM) networks to obtain N input vectors X. l .

[0259] N input vectors X l This can also be represented as matrix X. Employing a self-attention mechanism, it transforms each vector based on the correlation between them, resulting in N output vectors, which can also be represented as matrix Y. A multi-head attention layer can include multiple attention heads, specifically using a first transformation matrix Q, a second transformation matrix K, and a third transformation matrix V to transform the N input vectors respectively. <X1,X2,…,X N Each input vector X in > i The transformation yields the first intermediate vector (q vector), second intermediate vector (k vector), and third intermediate vector (v vector) corresponding to each input vector. Operationally, the input matrix X, composed of N input vectors, can be linearly transformed using the first transformation matrix Q, the second transformation matrix K, and the third transformation matrix V, respectively, to obtain the Q matrix, K matrix, and V matrix of the input matrix. These matrices are then split to obtain the q vector, k vector, and v vector corresponding to each input vector. For any i-th input vector Xi among the N input vectors, the first intermediate vector (q vector, qi) corresponding to the i-th input vector is used in conjunction with the other input vectors X. j The corresponding second intermediate vectors (k vectors, k) j The dot product operation is used to determine the relationship between the i-th input vector Xi and each of the input vectors X. j The various correlations. Although it is also possible to directly use q i With k j The dot product result is used to determine the correlation, but more classically, the dot product result is first divided by a constant, then a softmax operation is performed, and the result is used as the input vector X. i With X j The degree of correlation, that is:

[0260]

[0261] Therefore, the i-th input vector X can be used as... i With each input vector X j The various correlation degrees α i,jAs a weighting factor, for each input vector X j The corresponding third intermediate vector (v vector, v) j We perform a weighted combination to obtain the i-th combined vector C corresponding to the i-th input vector Xi. i :

[0262]

[0263] Therefore, we can obtain a vector sequence of N combined vectors corresponding to N input vectors. <C1,C2,…,C N >, or matrix C. Based on this combined vector sequence, N output vectors can be obtained. Specifically, in one embodiment, the vector sequence of N combined vectors can be directly used as N output vectors, i.e., Y. i =C i At this point, the output matrix Y is the combined vector matrix C, which can also be written as:

[0264]

[0265] The above describes the processing flow of an attention head. In the MHA architecture, the MHA layer maintains m sets of transformation matrices. Each set of transformation matrices includes the aforementioned first transformation matrix Q, second transformation matrix K, and third transformation matrix V, allowing the above operations to be performed in parallel to obtain m combined vector sequences (i.e., m matrices C). Each vector sequence includes N combined vectors obtained based on a set of transformation matrices. In this case, the MHA layer concatenates the m combined vector sequences to obtain a concatenated matrix; then, it transforms this concatenated matrix using the fourth transformation matrix W to obtain the final output matrix Y. This output matrix Y can be split into N output vectors.<Y1,Y2,…,YN> Through the above operations, the MHA layer performs transformation operations based on the correlation between the N input vectors to obtain N output vectors.

[0266] The N output vectors are merged and passed through a fully connected network to output the final interaction feature, which is the degree of influence of at least one of the associated vehicles on the target vehicle's driving.

[0267] In this embodiment of the application, the neural network model may include multiple prediction branches, each prediction branch having the ability to predict different prediction trajectory types. Each of the multiple prediction branches can determine a candidate trajectory of the target vehicle and the confidence level of the candidate trajectory based on the road condition information of the target candidate lane group. The confidence level can represent the probability that the corresponding candidate trajectory is the actual driving trajectory of the target vehicle.

[0268] The candidate trajectory with the highest confidence level can then be selected as the target prediction trajectory, which can be used as the predicted trajectory of the target vehicle. This prediction branch can also be referred to as a neural network sub-model in subsequent embodiments.

[0269] It should be understood that how to train a neural network model to have the ability to predict different types of predicted trajectories will be described in subsequent embodiments and will not be repeated here.

[0270] This application provides a vehicle intention prediction method, which includes: obtaining the location of a target vehicle; determining multiple candidate lane groups within a preset distance that the target vehicle can travel on based on the location of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold; obtaining road condition information for each candidate lane group; and determining a target candidate lane group from the multiple candidate lane groups based on the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle. This application defines the driving intention of the target vehicle as the lane direction of the most likely lane to be traveled on. For complex road conditions, lane direction can more accurately represent the vehicle's driving intention compared to directional intentions such as left turn or right turn.

[0271] Furthermore, since the input for generating the predicted trajectory is road condition information related to driving intent, the features used in trajectory prediction have a clear attribute of driving intent, making the input information for trajectory prediction more interpretable and thus making the trajectory prediction results more accurate.

[0272] The above describes the reasoning process for predicting driving intentions. Next, we will describe the training process of the neural network.

[0273] Reference Figure 6 , Figure 6 This is a flowchart illustrating a model training method provided in an embodiment of this application, as shown below. Figure 6 As shown, the model training method provided in this application includes:

[0274] 601. Obtain the first neural network model, the position of the target vehicle, and the actual driving trajectory, wherein the actual driving trajectory of the target vehicle is located in the target lane.

[0275] In this embodiment of the application, a first neural network model can be obtained, wherein the first neural network model is a neural network model to be trained, and the neural network model may include feature extraction networks such as CNN and fully connected networks, etc.

[0276] In this embodiment of the application, the training sample data may include the actual driving trajectory of the target vehicle.

[0277] 602. Based on the location of the target vehicle, determine multiple candidate lane groups that the target vehicle can travel within a preset distance, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than the target threshold.

[0278] In this embodiment of the application, multiple candidate lane groups that the target vehicle can drive within a preset distance can be determined based on the location of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold.

[0279] The specific description of step 602 can be found in the description of step 202 in the above embodiments, and will not be repeated here.

[0280] 603. Obtain the traffic information for each candidate lane group among the multiple candidate lane groups.

[0281] The specific description of step 603 can be found in the description of step 203 in the above embodiments, and will not be repeated here.

[0282] 604. Based on the road condition information, the first neural network model is used to determine the target candidate lane group from the multiple candidate lane groups, and the first neural network model is trained based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, to obtain the second neural network model.

[0283] It should be understood that a target candidate lane group can also be determined from these multiple candidate lane groups based on the road condition information of each candidate lane group and the historical driving route of the target vehicle.

[0284] In this embodiment of the application, the candidate lane group can be labeled based on whether the target lane is in the candidate lane group. The candidate lane group in which the target lane is located can be labeled as a positive sample. The lane direction of the candidate lane group labeled as a positive sample can be considered as the correct driving intention of the target vehicle. The candidate lane group in which the target lane is not located can be labeled as a negative sample. The lane direction of the candidate lane group labeled as a negative sample can be considered as the incorrect driving intention of the target vehicle.

[0285] In one implementation, the distance and orientation difference between the endpoint of the target vehicle's actual driving trajectory and all candidate lane groups can be calculated. If all differences are less than a certain threshold, the candidate lane group is considered a positive sample; otherwise, it is considered a negative sample. It should be understood that the number of positive samples in a candidate lane group may be greater than one. For example, when the target vehicle is about to enter an intersection, its actual driving trajectory may not have reached the fork in the path, thus not showing clear turning information. In this case, it is impossible to determine which candidate lane group's lane direction represents the target vehicle's true driving intention. Therefore, the lane directions of multiple possible candidate lane groups can be matched, and in this situation, the labels of multiple candidate lane groups should all be positive samples.

[0286] In this embodiment, the first neural network model can process road condition information to obtain the selection probability of each candidate lane group. Since the actual driving trajectory of the target vehicle lies in some candidate lane groups, the true probability corresponding to these candidate lane groups should be 1, while the true probability of candidate lane groups not in which the actual driving trajectory of the target vehicle lies is 0. Therefore, the first neural network model can be trained based on the difference between the selection probability of each candidate lane group and the true probability of each candidate lane group.

[0287] Specifically, a loss function can be constructed based on the difference between the selected probability of each candidate lane group and the true probability of each candidate lane group, and the first neural network model can be trained based on the constructed loss function. For example, the loss function can be, but is not limited to, the following formula:

[0288]

[0289] Where M is the number of candidate lane groups, C i Let X be the traffic information for the i-th candidate lane group, C be the traffic information for all candidate lane groups, and X be the traffic information for all candidate lane groups. a For the historical driving route of the vehicle, P(C) i |X a C) represents the predicted long-term intention probability obtained from the first neural network. Labels for candidate lane groups, This is the cross-entropy loss function.

[0290] In this embodiment of the application, a neural network model capable of outputting the target predicted trajectory of a target vehicle based on road condition information can also be trained.

[0291] Specifically, a third neural network model can be obtained, and the target predicted trajectory of the target vehicle can be determined through the third neural network model based on the road condition information of the candidate lane group in which the target lane is located. Then, the third neural network model can be trained based on the difference between the target predicted trajectory and the actual driving trajectory to obtain a fourth neural network model.

[0292] It should be understood that road condition information of candidate lane groups that the target lane is not in may not be included in the training process of the third neural network model.

[0293] It should be understood that, in addition to road condition information, the degree of influence of surrounding vehicles on the target vehicle can also be used as input to the third neural network. Specifically, the historical driving routes of the target vehicle and at least one surrounding vehicle can be obtained, and based on the historical driving routes, the degree of influence of the at least one surrounding vehicle on the target vehicle can be determined through an attention mechanism. Then, based on the road condition information of the target candidate lane group and the degree of influence, the target predicted trajectory of the target vehicle can be determined through the third neural network model.

[0294] In one possible implementation, the third neural network model includes multiple neural network sub-models. Based on the road condition information of the candidate lane group in which the target lane is located, multiple predicted trajectories of the target vehicle are determined through these multiple neural network sub-models. Each neural network sub-model is used to determine one predicted trajectory. Then, based on the difference between each predicted trajectory and the actual driving trajectory, the trajectory with a difference less than a threshold is determined as the target predicted trajectory. The neural network sub-model corresponding to the target predicted trajectory is trained based on the difference between the target predicted trajectory and the actual driving trajectory.

[0295] In one implementation, the predicted trajectory with the smallest difference from the actual driving trajectory among multiple predicted trajectories can be used as the target predicted trajectory.

[0296] Since the lane driving intentions defined in this application embodiment are difficult to define and label, this application embodiment does not predict explicit behavioral intentions. Instead, it uses multiple prediction branches, each branch being a predictor that identifies a driving intention. When training the third neural network based on multiple predicted trajectories output by the third neural network, only the predicted trajectory that is closest to the actual driving trajectory can be selected for supervised training. That is, only the network parameters of that branch are updated, ultimately enabling different branches to learn the trajectories of different samples, i.e., predicting trajectories of different behaviors.

[0297] Furthermore, since each neural network sub-model needs to process road condition information during the model's inference process to obtain a predicted trajectory, each sub-model also needs to output the confidence level corresponding to the predicted trajectory in order to indicate which predicted trajectory should be used as the target predicted trajectory. Correspondingly, during the training of the third neural network model, the loss function used for training the third neural network model needs to be constructed based on the difference between the confidence levels output by each sub-model and the actual confidence level. Specifically, multiple predicted trajectories of the target vehicle and the confidence level of each predicted trajectory can be determined through these multiple neural network sub-models. Based on the difference between the target predicted trajectory and the actual driving trajectory, and the difference between the confidence level of the target predicted trajectory and the value 1, the neural network sub-model corresponding to the target predicted trajectory is trained.

[0298] The model training method provided in this application embodiment will be described below with an example of training:

[0299] To prevent training samples from clustering in a single neural network sub-model, a two-stage training approach can be used. The first stage involves supervised training of all neural network sub-models, updating the parameters of each sub-model. The loss function can be:

[0300]

[0301] in, Let M' be the set of all candidate lane groups with positive labels, K be the number of candidate lane groups in this set, and Y be the number of neural network sub-models. t The actual driving trajectory of the target vehicle. Let m be the predicted trajectory of the m-th predicted trajectory and k be the predicted trajectory of the k-th predicted branch. This is the MSE loss function. Since all neural network sub-models are trained in the first stage, the loss is not constructed based on the confidence level of the output of each neural network sub-model.

[0302] In the second stage, independent neural network sub-models are trained, and the loss function can be:

[0303]

[0304] If the confidence label of the neural network sub-model closest to the actual driving trajectory is 1, and the rest are 0, then the loss function can be:

[0305]

[0306] in, The confidence level of the neural network predicting the trajectory. For confidence level labels, This represents the cross-entropy loss.

[0307] like Figure 7 As shown, Figure 7 The overall framework of the neural network provided in the embodiments of this application. Figure 7 The left side encodes the vehicle's historical driving route and the road condition information of each candidate lane group into a rasterized form and inputs it into the CNN to obtain the selection probability and environmental features of the candidate lane group. Figure 7 The system encodes the obstacles around the vehicle and the historical driving route of the target vehicle, and then passes them through a multi-head attention layer to obtain multi-target interaction features. These features are combined with environmental features and then input into a neural network that includes multiple neural network sub-models. Each neural network sub-model predicts the predicted trajectory and corresponding confidence level for the current candidate lane group. Figure 7 The road condition information of the M candidate lane groups is rasterized and encoded, and the environmental features and selection probability are calculated. K neural network sub-models predict different trajectories respectively. Finally, M×K predicted trajectories and corresponding confidence scores are predicted.

[0308] The above describes the intent prediction method and model training method provided in the embodiments of this application from the perspective of methodology. Next, the intent prediction device and model training device provided in the embodiments of this application will be described from the perspective of apparatus.

[0309] Reference Figure 8 , Figure 8 This is a schematic diagram of the structure of a vehicle intention prediction device provided in an embodiment of this application, such as... Figure 8 As shown, the device 800 includes:

[0310] The acquisition module 801 is used to acquire the location of the target vehicle; the acquisition module is also used to acquire the road condition information of each candidate lane group among the multiple candidate lane groups determined by the candidate determination module.

[0311] For a detailed description of the acquisition module 801, please refer to the descriptions of steps 201 and 203 in the above embodiments, which will not be repeated here.

[0312] The lane determination module 802 is used to determine multiple candidate lane groups that the target vehicle can drive within a preset distance based on the position of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold; the lane determination module is also used to determine a target candidate lane group from the multiple candidate lane groups based on the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle.

[0313] For a detailed description of the lane determination module 802, please refer to the descriptions of steps 202 and 204 in the above embodiments, which will not be repeated here.

[0314] In one possible implementation, each candidate lane group includes at least one lane, and the lane direction difference between the lanes included in each candidate lane group is less than the target threshold.

[0315] In one possible implementation, the lane direction is the direction of the lane line within a preset distance, starting from the position of the target vehicle.

[0316] In one possible implementation, the preset distance is a fixed value less than 200 meters; or,

[0317] The preset distance is calculated by taking the target vehicle’s speed and the intention prediction time. The preset distance is positively correlated with the target vehicle’s speed and the intention prediction time, and the intention prediction time is less than or equal to 7 seconds.

[0318] In one possible implementation, the acquisition module is also used to acquire the historical driving routes of the target vehicle;

[0319] The lane determination module is specifically used to determine the target candidate lane group from the multiple candidate lane groups based on the road condition information and the historical driving route.

[0320] In one possible implementation, the traffic information includes at least one of the following:

[0321] Lane centerline information, drivable area information, obstacle information, and speed limit area information.

[0322] In one possible implementation, the traffic information is represented as a rasterized image comprising multiple image channels, each image channel representing at least one piece of information in the traffic information.

[0323] In one possible implementation, the device further includes:

[0324] The trajectory prediction module is used to determine the target vehicle's predicted trajectory using a neural network model based on the road condition information of the target candidate lane group.

[0325] In one possible implementation, the acquisition module is also used to acquire the historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle;

[0326] The trajectory prediction module is used to determine the degree of influence of at least one associated vehicle on the target vehicle's driving based on the historical driving route and through an attention mechanism.

[0327] Based on the road condition information of the target candidate lane group and the degree of driving impact, the target vehicle's predicted trajectory is determined through a neural network model.

[0328] In one possible implementation, the trajectory prediction module is specifically used to determine multiple candidate trajectories of the target vehicle and the confidence level of each candidate trajectory based on the road condition information of the target candidate lane group and through the neural network model.

[0329] Based on this confidence level, the target predicted trajectory of the target vehicle is determined from the multiple candidate trajectories.

[0330] Reference Figure 9 , Figure 9 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application, as shown below. Figure 9 As shown, the device 900 includes:

[0331] The acquisition module 901 is used to acquire the first neural network model, the position of the target vehicle, and the actual driving trajectory, wherein the actual driving trajectory of the target vehicle is located on the target lane; the acquisition module is also used to acquire the road condition information of each candidate lane group among the multiple candidate lane groups determined by the lane determination module.

[0332] For a detailed description of the acquisition module 901, please refer to the descriptions of steps 601 and 603 in the above embodiments, which will not be repeated here.

[0333] The lane determination module 902 is used to determine multiple candidate lane groups that the target vehicle can drive within a preset distance based on the position of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold.

[0334] For a detailed description of the lane determination module 902, please refer to the description of step 602 in the above embodiment, which will not be repeated here.

[0335] The model training module 903 is used to determine the target candidate lane group from the multiple candidate lane groups through the first neural network model based on the road condition information obtained by the acquisition module, and to train the first neural network model based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, so as to obtain the second neural network model.

[0336] For a detailed description of the model training module 903, please refer to the description of step 604 in the above embodiment, which will not be repeated here.

[0337] In one possible implementation, the model training module is specifically used to determine the probability of each candidate lane being selected by the first neural network model based on the road condition information of each candidate lane group.

[0338] Obtain the true probability of each candidate lane, where the true probability of the target lane being in a candidate lane group is 1, and the true probability of the target lane not being in a candidate lane group is 0;

[0339] The first neural network model is trained based on the difference between the probability of each candidate lane being selected and the true probability of each candidate lane.

[0340] In one possible implementation, the acquisition module is also used to acquire a third neural network model;

[0341] The device also includes:

[0342] The trajectory prediction module is used to determine the target predicted trajectory of the target vehicle through the third neural network model based on the road condition information of the candidate lane group in which the target lane is located.

[0343] The model training module is also used to train the third neural network model based on the difference between the predicted trajectory of the target and the actual driving trajectory, so as to obtain the fourth neural network model.

[0344] In one possible implementation, the third neural network model includes multiple neural network sub-models. The trajectory prediction module is specifically used to determine multiple predicted trajectories of the target vehicle through the multiple neural network sub-models based on the road condition information of the candidate lane group in which the target lane is located. Each neural network sub-model is used to determine one predicted trajectory.

[0345] Based on the difference between each of the multiple predicted trajectories and the actual driving trajectory, the trajectory with a difference less than a threshold is determined as the target predicted trajectory.

[0346] In one possible implementation, the trajectory prediction module is specifically used to determine multiple predicted trajectories of the target vehicle and the confidence level of each predicted trajectory through the multiple neural network sub-models.

[0347] Based on the difference between the predicted trajectory of the target and the actual driving trajectory, the third neural network model is trained to obtain a fourth neural network model, including:

[0348] Based on the difference between the predicted trajectory of the target and the actual driving trajectory, and the difference between the confidence level of the predicted trajectory of the target and the value 1, the third neural network model is trained to obtain the fourth neural network model.

[0349] It should be noted that the information interaction and execution process between the modules / units of the above-mentioned device are based on the same concept as the method embodiment of this application, and the resulting technical effects are the same as those of the method embodiment of this application. For details, please refer to the description in the method embodiment shown above in the embodiment of this application, and it will not be repeated here.

[0350] The following describes an execution device provided in an embodiment of this application. Please refer to [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of an execution device provided in an embodiment of this application. The execution device 1000 can specifically be a virtual reality (VR) device, a mobile phone, a tablet, a laptop, a smart wearable device, a monitoring data processing device, or a server, etc., and is not limited thereto. Specifically, the execution device 1000 includes: a receiver 1001, a transmitter 1002, a processor 1003, and a memory 1004 (wherein the execution device 1000 may have one or more processors 1003). Figure 10 (Taking a processor as an example), processor 1003 may include application processor 10031 and communication processor 10032. In some embodiments of this application, receiver 1001, transmitter 1002, processor 1003 and memory 1004 may be connected via bus or other means.

[0351] Memory 1004 may include read-only memory and random access memory, and provides instructions and data to processor 1003. A portion of memory 1004 may also include non-volatile random access memory (NVRAM). Memory 1004 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0352] Processor 1003 controls the operation of the execution device. In specific applications, the various components of the execution device are coupled together through a bus system, which may include not only the data bus, but also power buses, control buses, and status signal buses. However, for clarity, all buses are referred to as the bus system in the diagram.

[0353] The methods disclosed in the embodiments of this application can be applied to or implemented by the processor 1003. The processor 1003 can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the integrated logic circuits in the hardware of the processor 1003 or by instructions in software form. The processor 1003 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor, or a microcontroller, and may further include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The processor 1003 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 1004. Processor 1003 reads the information in memory 1004 and, in conjunction with its hardware, completes the steps of the above method.

[0354] Receiver 1001 can be used to receive input digital or character information, and to generate signal inputs related to the settings and function control of the execution device. Transmitter 1002 can be used to output digital or character information through the first interface; transmitter 1002 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; transmitter 1002 may also include a display device such as a display screen.

[0355] In one embodiment of this application, the processor 1003 is configured to execute... Figure 2 The driving intention prediction method described in the corresponding embodiment.

[0356] This application also provides a training device; please refer to [link / reference]. Figure 11 , Figure 11 This is a schematic diagram of a training device provided in an embodiment of this application. The training device 1100 can be equipped with... Figure 9 The model training apparatus described in the corresponding embodiment is used to implement Figure 9In accordance with the functionality of the model training device in the corresponding embodiment, specifically, the training device 1100 is implemented by one or more servers. The training device 1100 can vary significantly due to differences in configuration or performance, and may include one or more central processing units (CPUs) 1119 (e.g., one or more processors) and memory 1132, and one or more storage media 1130 (e.g., one or more mass storage devices) for storing application programs 1142 or data 1144. The memory 1132 and storage media 1130 can be temporary or persistent storage. The program stored in the storage media 1130 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the training device. Furthermore, the CPU 1119 may be configured to communicate with the storage media 1130 and execute the series of instruction operations in the storage media 1130 on the training device 1100.

[0357] The training device 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input / output interfaces 1158; or, one or more operating systems 1141, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0358] In this embodiment, the central processing unit 1119 is used to execute... Figure 6 The model training method provided in the corresponding embodiment.

[0359] This application also provides a computer program product that, when run on a computer, causes the computer to perform steps as performed by the aforementioned execution device, or causes the computer to perform steps as performed by the aforementioned training device.

[0360] This application also provides a computer-readable storage medium storing a program for signal processing, which, when run on a computer, causes the computer to perform steps as performed by the aforementioned execution device, or causes the computer to perform steps as performed by the aforementioned training device.

[0361] The execution device, training device, or terminal device provided in this application embodiment can specifically be a chip. The chip includes a processing unit and a communication unit. The processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer execution instructions stored in the storage unit to cause the chip within the execution device to execute the data processing method described in the above embodiments, or to cause the chip within the training device to execute the data processing method described in the above embodiments. Optionally, the storage unit can be a storage unit within the chip, such as a register or cache. Alternatively, the storage unit can be a storage unit located outside the chip within the wireless access device, such as a read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0362] For details, please refer to Figure 12 , Figure 12 This is a schematic diagram of a chip provided in an embodiment of this application. The chip can be represented as a neural network processor (NPU) 1200. The NPU 1200 is mounted as a coprocessor on the host CPU, and tasks are assigned by the host CPU. The core part of the NPU is the arithmetic circuit 1203, which is controlled by the controller 1204 to extract matrix data from the memory and perform multiplication operations.

[0363] In some implementations, the arithmetic circuit 1203 internally includes multiple processing engines (PEs). In some implementations, the arithmetic circuit 1203 is a two-dimensional pulsating array. The arithmetic circuit 1203 can also be a one-dimensional pulsating array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1203 is a general-purpose matrix processor.

[0364] For example, suppose we have an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit retrieves the corresponding data of matrix B from the weight memory 1202 and caches it in each PE of the arithmetic circuit. The arithmetic circuit retrieves the data of matrix A from the input memory 1201 and performs matrix operations with matrix B. The partial result or the final result of the obtained matrix is ​​stored in the accumulator 1208.

[0365] Unified memory 1206 is used to store input and output data. Weight data is directly transferred to weight memory 1202 via Direct Memory Access Controller (DMAC) 1205. Input data is also transferred to unified memory 1206 via DMAC.

[0366] BIU stands for Bus Interface Unit, which is used for interaction between the AXI bus and the DMAC and the Instruction Fetch Buffer (IFB) 1209.

[0367] The Bus Interface Unit (BIU) 1210 is used by the instruction fetch memory 1209 to fetch instructions from external memory, and also by the memory access controller 1205 to fetch the original data of the input matrix A or the weight matrix B from external memory.

[0368] The DMAC is mainly used to move input data from external memory DDR to unified memory 1206, or to weight data to weight memory 1202, or to input data to input memory 1201.

[0369] The vector computation unit 1207 includes multiple arithmetic processing units that, when needed, further process the output of the computation circuit, such as vector multiplication, vector addition, exponential operations, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / fully connected layers of neural networks, such as Batch Normalization, pixel-level summation, and upsampling of feature planes.

[0370] In some implementations, the vector computation unit 1207 can store the processed output vector in the unified memory 1206. For example, the vector computation unit 1207 can apply a linear function, or a nonlinear function, to the output of the computation circuit 1203, such as linear interpolation of feature planes extracted by a convolutional layer, or, for example, a vector of accumulated values, to generate activation values. In some implementations, the vector computation unit 1207 generates normalized values, pixel-level summed values, or both. In some implementations, the processed output vector can be used as an activation input to the computation circuit 1203, for example, for use in subsequent layers of the neural network.

[0371] The instruction fetch buffer 1209 connected to the controller 1204 is used to store the instructions used by the controller 1204;

[0372] Unified memory 1206, input memory 1201, weight memory 1202, and instruction fetch memory 1209 are all on-chip memories. External memory is proprietary to this NPU hardware architecture.

[0373] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits used to control the execution of the above program.

[0374] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the accompanying drawings of the device embodiments provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0375] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods of the various embodiments of this application.

[0376] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0377] This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for predicting vehicle intent, characterized in that, The method includes: Obtain the location of the target vehicle, wherein the target vehicle is any one of at least one associated vehicle located around the resident vehicle; Based on the location of the target vehicle, multiple candidate lane groups that the target vehicle can drive within a preset distance are determined, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold. Obtain traffic information for each of the multiple candidate lane groups; Obtain the historical driving routes of the target vehicle; Based on the road condition information, a target candidate lane group is determined from the plurality of candidate lane groups. The lane direction of the target candidate lane group is used as the driving intention of the target vehicle. The target candidate lane group is determined based on a neural network model. The neural network model is pre-trained and has the ability to determine the selection probability of each candidate lane group based on the road condition information. The training sample data of the neural network model includes the actual driving trajectory of the target vehicle, and the actual driving trajectory of the target vehicle is located on the target lane. The step of determining the target candidate lane group from the plurality of candidate lane groups based on the road condition information includes: Based on the road condition information and the historical driving route, a target candidate lane group is determined from the plurality of candidate lane groups. The road condition information includes at least one of the following: lane centerline information, drivable area information, obstacle information, and speed limit area information.

2. The method according to claim 1, characterized in that, Each candidate lane group includes at least one lane, and the lane direction difference of the lanes included in each candidate lane group within the preset distance is less than the target threshold.

3. The method according to claim 1 or 2, characterized in that, The lane direction is the direction of the lane line within the preset distance, starting from the position of the target vehicle.

4. The method according to claim 1 or 2, characterized in that, The preset distance is a fixed value less than 200 meters; or, The preset distance is obtained by calculating the driving speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the driving speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 2 seconds.

5. The method according to claim 1 or 2, characterized in that, The road condition information is represented as a rasterized image, which includes multiple image channels, each image channel representing at least one piece of information in the road condition information.

6. The method according to claim 1 or 2, characterized in that, The method further includes: Based on the road condition information of the target candidate lane group, the target predicted trajectory of the target vehicle is determined by a neural network model.

7. The method according to claim 6, characterized in that, The method further includes: Obtain the historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle; Based on the historical driving route, the degree of influence of the at least one associated vehicle on the target vehicle's driving is determined through an attention mechanism. The step of determining the target predicted trajectory of the target vehicle using a neural network model based on the road condition information of the target candidate lane group includes: Based on the road condition information of the target candidate lane group and the degree of driving impact, the target predicted trajectory of the target vehicle is determined through a neural network model.

8. The method according to claim 7, characterized in that, The step of determining the target predicted trajectory of the target vehicle using a neural network model based on the road condition information of the target candidate lane group includes: Based on the road condition information of the target candidate lane group, the neural network model is used to determine multiple candidate trajectories of the target vehicle and the confidence level of each candidate trajectory. Based on the confidence level, the target predicted trajectory of the target vehicle is determined from the plurality of candidate trajectories.

9. A model training method, characterized in that, include: Acquire the first neural network model, the position of the target vehicle, and the actual driving trajectory, wherein the actual driving trajectory of the target vehicle is located on the target lane, and the target vehicle is any one of at least one associated vehicle located around the resident vehicle; Based on the location of the target vehicle, multiple candidate lane groups that the target vehicle can drive within a preset distance are determined, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold. Obtain road condition information for each of the plurality of candidate lane groups, wherein the road condition information includes at least one of the following: lane centerline information, drivable area information, obstacle information, and speed limit area information; Based on the road condition information, a target candidate lane group is determined from the plurality of candidate lane groups using the first neural network model. The first neural network model is then trained based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, to obtain a second neural network model. The target candidate lane group is also determined based on the historical driving route of the target vehicle. The first neural network model is pre-trained and has the ability to determine the selection probability of each candidate lane group based on road condition information. The training sample data of the first neural network model includes the actual driving trajectory of the target vehicle, which is located on the target lane.

10. The method according to claim 9, characterized in that, The step of determining a target candidate lane group from the plurality of candidate lane groups using the first neural network model based on the road condition information, and training the first neural network model based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, includes: Based on the road condition information of each candidate lane group, the probability of each candidate lane being selected is determined by the first neural network model; Obtain the true probability of each candidate lane, wherein the true probability of the target lane being in a candidate lane group is 1, and the true probability of the target lane not being in a candidate lane group is 0; The first neural network model is trained based on the difference between the probability of each candidate lane being selected and the true probability of each candidate lane.

11. The method according to claim 9 or 10, characterized in that, The method further includes: Obtain the third neural network model; Based on the road condition information of the candidate lane group in which the target lane is located, the target predicted trajectory of the target vehicle is determined by the third neural network model; Based on the difference between the predicted trajectory of the target and the actual driving trajectory, the third neural network model is trained to obtain the fourth neural network model.

12. The method according to claim 11, characterized in that, The third neural network model includes multiple neural network sub-models. The step of determining the target vehicle's predicted trajectory using the third neural network model based on road condition information of the candidate lane group to which the target lane is located includes: Based on the road condition information of the candidate lane group in which the target lane is located, multiple predicted trajectories of the target vehicle are determined through the multiple neural network sub-models, wherein each neural network sub-model is used to determine one predicted trajectory. Based on the difference between each of the multiple predicted trajectories and the actual driving trajectory, the trajectory with a difference less than a threshold is determined as the target predicted trajectory from the multiple predicted trajectories; The step of training the third neural network model based on the difference between the predicted target trajectory and the actual driving trajectory includes: Based on the difference between the predicted target trajectory and the actual driving trajectory, a neural network sub-model corresponding to the predicted target trajectory is trained.

13. The method according to claim 12, characterized in that, The process of determining multiple predicted trajectories of the target vehicle through the multiple neural network sub-models includes: Multiple predicted trajectories of the target vehicle and the confidence level of each predicted trajectory are determined through the multiple neural network sub-models. The step of training a neural network sub-model corresponding to the target predicted trajectory based on the difference between the target predicted trajectory and the actual driving trajectory includes: Based on the difference between the predicted target trajectory and the actual driving trajectory, and the difference between the confidence level of the predicted target trajectory and the value 1, the neural network sub-model corresponding to the predicted target trajectory is trained.

14. A vehicle intent prediction device, characterized in that, The device includes: The acquisition module is used to acquire the location of the target vehicle, which is any one of at least one associated vehicle located around the vehicle; the acquisition module is also used to acquire the road condition information of each candidate lane group among the multiple candidate lane groups determined by the candidate determination module. The lane determination module is used to determine multiple candidate lane groups within a preset distance for the target vehicle based on its position, wherein the lane direction difference between different candidate lane groups within the preset distance is greater than a target threshold; the lane determination module is also used to determine a target candidate lane group from the multiple candidate lane groups based on the road condition information, wherein the lane direction of the target candidate lane group is used as the driving intention of the target vehicle, and the target candidate lane group is determined based on a neural network model, wherein the neural network model is pre-trained and has the ability to determine the selection probability of each candidate lane group based on road condition information, and the training sample data of the neural network model includes the actual driving trajectory of the target vehicle, wherein the actual driving trajectory of the target vehicle is located on the target lane; The acquisition module is also used to acquire the historical driving routes of the target vehicle; The lane determination module is specifically used to determine a target candidate lane group from the plurality of candidate lane groups based on the road condition information and the historical driving route. The road condition information includes at least one of the following: lane centerline information, drivable area information, obstacle information, and speed limit area information.

15. The apparatus according to claim 14, characterized in that, Each candidate lane group includes at least one lane, and the lane direction difference of the lanes included in each candidate lane group within the preset distance is less than the target threshold.

16. The apparatus according to claim 14 or 15, characterized in that, The lane direction is the direction of the lane line within the preset distance, starting from the position of the target vehicle.

17. The apparatus according to claim 14 or 15, characterized in that, The preset distance is a fixed value less than 200 meters; or, The preset distance is obtained by calculating the driving speed of the target vehicle and the intention prediction time, wherein the preset distance is positively correlated with the driving speed of the target vehicle and the intention prediction time, and the intention prediction time is less than or equal to 2 seconds.

18. The apparatus according to claim 14 or 15, characterized in that, The road condition information is represented as a rasterized image, which includes multiple image channels, each image channel representing at least one piece of information in the road condition information.

19. The apparatus according to claim 14 or 15, characterized in that, The device further includes: The trajectory prediction module is used to determine the target predicted trajectory of the target vehicle through a neural network model based on the road condition information of the target candidate lane group.

20. The apparatus according to claim 19, characterized in that, The acquisition module is also used to acquire the historical driving routes of the target vehicle and at least one associated vehicle located around the target vehicle; The trajectory prediction module is used to determine the degree of influence of the at least one associated vehicle on the target vehicle's driving based on the historical driving route and through an attention mechanism. Based on the road condition information of the target candidate lane group and the degree of driving impact, the target predicted trajectory of the target vehicle is determined through a neural network model.

21. The apparatus according to claim 20, characterized in that, The trajectory prediction module is specifically used to determine multiple candidate trajectories of the target vehicle and the confidence level of each candidate trajectory based on the road condition information of the target candidate lane group and through the neural network model. Based on the confidence level, the target predicted trajectory of the target vehicle is determined from the plurality of candidate trajectories.

22. A model training device, characterized in that, include: The acquisition module is used to acquire the first neural network model, the position of the target vehicle, and the actual driving trajectory, wherein the actual driving trajectory of the target vehicle is located on the target lane, and the target vehicle is any one of at least one associated vehicle located around the vehicle; the acquisition module is also used to acquire the road condition information of each candidate lane group in the multiple candidate lane groups determined by the lane determination module, wherein the road condition information includes at least one of the following: lane centerline information, drivable area information, obstacle information, and speed limit area information; The lane determination module is used to determine multiple candidate lane groups that the target vehicle can drive within a preset distance based on the position of the target vehicle, wherein the difference in lane direction between different candidate lane groups within the preset distance is greater than a target threshold. The model training module is used to determine a target candidate lane group from the plurality of candidate lane groups based on the road condition information acquired by the acquisition module, and to train the first neural network model based on the difference between the target candidate lane group and the candidate lane group in which the target lane is located, to obtain a second neural network model. The target candidate lane group is also determined based on the historical driving route of the target vehicle. The first neural network model is pre-trained and has the ability to determine the selection probability of each candidate lane group based on road condition information. The training sample data of the first neural network model includes the actual driving trajectory of the target vehicle, and the actual driving trajectory of the target vehicle is located on the target lane.

23. The apparatus according to claim 22, characterized in that, The model training module is specifically used to determine the selection probability of each candidate lane based on the road condition information of each candidate lane group through the first neural network model; Obtain the true probability of each candidate lane, wherein the true probability of the target lane being in a candidate lane group is 1, and the true probability of the target lane not being in a candidate lane group is 0; The first neural network model is trained based on the difference between the probability of each candidate lane being selected and the true probability of each candidate lane.

24. The apparatus according to claim 22 or 23, characterized in that, The acquisition module is also used to acquire a third neural network model; The device further includes: The trajectory prediction module is used to determine the target predicted trajectory of the target vehicle through the third neural network model based on the road condition information of the candidate lane group in which the target lane is located. The model training module is also used to train the third neural network model based on the difference between the target predicted trajectory and the actual driving trajectory to obtain the fourth neural network model.

25. The apparatus according to claim 24, characterized in that, The third neural network model includes multiple neural network sub-models. The trajectory prediction module is specifically used to determine multiple predicted trajectories of the target vehicle through the multiple neural network sub-models based on the road condition information of the candidate lane group in which the target lane is located. Each neural network sub-model is used to determine one predicted trajectory. Based on the difference between each of the multiple predicted trajectories and the actual driving trajectory, the trajectory with a difference less than a threshold is determined as the target predicted trajectory from the multiple predicted trajectories; The model training module is specifically used to train the neural network sub-model corresponding to the target predicted trajectory based on the difference between the target predicted trajectory and the actual driving trajectory.

26. The apparatus according to claim 25, characterized in that, The trajectory prediction module is specifically used to determine multiple predicted trajectories of the target vehicle and the confidence level of each predicted trajectory through the multiple neural network sub-models. The model training module is specifically used to train the neural network sub-model corresponding to the target predicted trajectory based on the difference between the target predicted trajectory and the actual driving trajectory, and the difference between the confidence level of the target predicted trajectory and the value 1.

27. A server, characterized in that, The system includes a memory and a processor; the memory stores code, and the processor is configured to execute the code, wherein when the code is executed, the server performs the method as described in any one of claims 1 to 13.

28. A computer storage medium, characterized in that, The computer storage medium stores a computer program that, when executed by a computer, causes the computer to perform the method described in any one of claims 1 to 13.

29. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1 to 13.

Images