Vehicle trajectory prediction method and device and terminal equipment

Abstract

The invention is suitable for the technical field of neural networks, and provides a vehicle trajectory prediction method and device, and a terminal device, and the method comprises the steps: obtaining the vehicle information of each vehicle in a detection region, selecting a target vehicle according to the vehicle information, determining the intention recognition result of the target vehicle according to the vehicle information and the environment information corresponding to the target vehicle, and outputting the intention recognition result. And inputting the intention recognition result and the environment information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle, and sending the trajectory prediction result to the target terminal. According to the invention, the intention recognition result of the target vehicle is recognized in advance through the interaction relationship among the multiple vehicles, the intention recognition result of the target vehicle and the corresponding environment information are processed according to the pre-trained trajectory prediction neural network model, and the driving trajectory of the target vehicle in a certain period of time is predicted. And the precision and efficiency of comprehensive trajectory prediction of multiple vehicles under complex road conditions are improved.

Description

Technical Field

[0001] The present application belongs to the field of neural network technology, and in particular, relates to a vehicle trajectory prediction method, device and terminal equipment. Background Art

[0002] With the increase in the number of cars, the urban traffic environment is becoming more and more complex. Especially in tunnels, intersections and other complex roads where there are many types of traffic participants and the movement states of traffic participants are changeable, traffic accidents are very likely to occur. In view of the above situation, it is necessary to predict the vehicle's driving trajectory in order to grasp the traffic situation of complex road conditions and ensure the safety of vehicle traffic.

[0003] There are usually two types of vehicle trajectory prediction methods: trajectory prediction methods based on traditional physical models and trajectory prediction methods based on machine learning models. Among them, trajectory prediction methods based on traditional physical models mainly predict trajectories based on the vehicle's motion state and physical characteristics to obtain short-term trajectory prediction results. Trajectory prediction methods based on machine learning models can only predict trajectories for a single vehicle and cannot adapt to complex traffic environments.

[0004] In summary, the relevant vehicle trajectory prediction methods have the problems of short prediction results and only being able to predict a single vehicle trajectory. Summary of the invention

[0005] The embodiments of the present application provide a vehicle trajectory prediction method, apparatus, and terminal device, which can solve the problems that related vehicle trajectory prediction methods have short prediction results and can only predict a single vehicle trajectory.

[0006] In a first aspect, an embodiment of the present application provides a vehicle trajectory prediction method, comprising:

[0007] Obtain vehicle information of each vehicle in the detection area;

[0008] Selecting a target vehicle according to the vehicle information;

[0009] Determining an intention recognition result of the target vehicle according to the vehicle information and environmental information corresponding to the target vehicle;

[0010] Inputting the intention recognition result and the environmental information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle;

[0011] The trajectory prediction result is sent to the target terminal.

[0012] In a possible implementation manner of the first aspect, the vehicle information includes location information, and selecting a target vehicle according to the vehicle information includes:

[0013] Acquire the signal light information within the detection area;

[0014] Determining a target lane that is in a passable state according to the signal light information;

[0015] A vehicle located in the target lane is screened according to the position information and is taken as a target vehicle.

[0016] In a possible implementation manner of the first aspect, determining the intention recognition result of the target vehicle according to the vehicle information and the environmental information corresponding to the target vehicle includes:

[0017] Determine an adjacent vehicle connected to the target vehicle according to the vehicle information, and use the first vehicle information of the adjacent vehicle as the environmental information of the target vehicle;

[0018] constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environmental information;

[0019] The adjacency matrix is ​​input into a pre-trained intention recognition neural network model for processing to obtain the intention recognition result of the target vehicle; wherein the intention recognition result is used to characterize the driving direction of the target vehicle; the intention recognition neural network model is composed of at least one graph neural network layer, a fully connected layer, a long short-term memory network layer and a function layer connected in sequence, and the graph neural network layer, the fully connected layer and the long short-term memory network layer correspond to each other one by one.

[0020] In a possible implementation manner of the first aspect, determining, according to the vehicle information, an adjacent vehicle having a connection relationship with the target vehicle, and using first vehicle information of the adjacent vehicle as the environmental information of the target vehicle includes:

[0021] Determine a neighborhood area centered on the target vehicle according to the location information of the target vehicle;

[0022] All other vehicles in the neighborhood area are regarded as adjacent vehicles having a connection relationship with the target vehicle; wherein the other vehicles are vehicles other than the target vehicle;

[0023] The first vehicle information of the adjacent vehicle is used as the environmental information of the target vehicle.

[0024] In a possible implementation manner of the first aspect, constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environment information includes:

[0025] Normalizing the second vehicle information of the target vehicle and the environmental information respectively to obtain a first feature vector of the target vehicle and a second feature vector of an adjacent vehicle;

[0026] Calculating the similarity between the target vehicle and the adjacent vehicles according to the first feature vector and the second feature vector;

[0027] An adjacency matrix corresponding to the target vehicle is constructed according to the similarity.

[0028] In a possible implementation manner of the first aspect, determining a neighborhood area centered on the target vehicle according to the position information of the target vehicle includes:

[0029] Convert the target vehicle's position information into a coordinate system to obtain converted position coordinates;

[0030] Get preset area restriction data;

[0031] A neighborhood area centered on the target vehicle is constructed according to the preset area restriction data and the position coordinates.

[0032] In a possible implementation of the first aspect, the inputting the intention recognition result and the environmental information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle includes:

[0033] The second vehicle information of the target vehicle, the intention recognition result and the environmental information are input into the pre-trained trajectory prediction neural network model for processing to obtain the trajectory prediction result of the target vehicle; wherein the trajectory prediction neural network model includes an encoder and a decoder, the encoder is composed of at least one long short-term memory network, a context connection layer, and a dimensionality reduction layer connected in sequence, and the decoder is composed of at least one long short-term memory network and a fully connected layer connected in sequence.

[0034] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the intention recognition result of the target vehicle is pre-identified through the interactive relationship between multiple vehicles, and the intention recognition result of the target vehicle and the corresponding environmental information are processed according to the pre-trained trajectory prediction neural network model, so as to predict the driving trajectory of the target vehicle in a certain time period, thereby improving the accuracy and efficiency of comprehensive trajectory prediction of multiple vehicles under complex road conditions.

[0035] In a second aspect, an embodiment of the present application provides a vehicle trajectory prediction device, comprising:

[0036] A data acquisition module, used to acquire vehicle information of each vehicle in the detection area;

[0037] A vehicle screening module, used for selecting a target vehicle according to the vehicle information;

[0038] An intention recognition module, used to determine an intention recognition result of the target vehicle according to the vehicle information and environmental information corresponding to the target vehicle;

[0039] A trajectory prediction module, used for inputting the intention recognition result and the environmental information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle;

[0040] The sending module is used to send the trajectory prediction result to the target terminal.

[0041] In a possible implementation manner of the second aspect, the vehicle information includes location information, and the vehicle screening module includes:

[0042] A signal light information acquisition unit, used to acquire signal light information within the detection area;

[0043] A lane determination unit, configured to determine a target lane in a passable state according to the signal light information;

[0044] The vehicle screening unit is used to screen the vehicle located on the target lane according to the position information as the target vehicle.

[0045] In a possible implementation manner of the second aspect, the intention recognition module includes:

[0046] An environmental information determination submodule, used to determine an adjacent vehicle having a connection relationship with the target vehicle according to the vehicle information, and use the first vehicle information of the adjacent vehicle as the environmental information of the target vehicle;

[0047] A matrix construction submodule, used for constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environmental information;

[0048] An intention recognition submodule is used to input the adjacency matrix into a pre-trained intention recognition neural network model for processing to obtain the intention recognition result of the target vehicle; wherein the intention recognition result is used to characterize the driving direction of the target vehicle; the intention recognition neural network model is composed of at least one graph neural network layer, a fully connected layer, a long short-term memory network layer and a function layer connected in sequence, and the graph neural network layer, the fully connected layer and the long short-term memory network layer correspond to each other one by one.

[0049] In a possible implementation manner of the second aspect, the environment information determination submodule includes:

[0050] A neighborhood determination unit, configured to determine a neighborhood area centered on the target vehicle according to the location information of the target vehicle;

[0051] An adjacent vehicle determination unit, configured to determine all other vehicles in the neighborhood area as adjacent vehicles having a connection relationship with the target vehicle; wherein the other vehicles are vehicles other than the target vehicle;

[0052] The environmental information determining unit is used to use the first vehicle information of the adjacent vehicle as the environmental information of the target vehicle.

[0053] In a possible implementation manner of the second aspect, the matrix construction submodule includes:

[0054] A feature vector determination unit, used for normalizing the second vehicle information of the target vehicle and the environmental information respectively, to obtain a first feature vector of the target vehicle and a second feature vector of an adjacent vehicle;

[0055] A similarity calculation unit, configured to calculate the similarity between the target vehicle and the adjacent vehicle based on the first feature vector and the second feature vector;

[0056] A matrix construction unit is used to construct an adjacency matrix corresponding to the target vehicle according to the similarity.

[0057] In a possible implementation manner of the second aspect, the neighborhood determining unit includes:

[0058] A coordinate conversion subunit, used for converting the position information of the target vehicle into a coordinate system to obtain the converted position coordinates;

[0059] A data acquisition subunit, used to acquire preset area restriction data;

[0060] The neighborhood determination subunit is used to construct a neighborhood area centered on the target vehicle according to the preset area restriction data and the position coordinates.

[0061] In a possible implementation of the second aspect, the trajectory prediction module is specifically used to input the second vehicle information of the target vehicle, the intention recognition result and the environmental information into the pre-trained trajectory prediction neural network model for processing to obtain the trajectory prediction result of the target vehicle; wherein the trajectory prediction neural network model includes an encoder and a decoder, the encoder is composed of at least one long short-term memory network, a context connection layer, and a dimensionality reduction layer connected in sequence, and the decoder is composed of at least one long short-term memory network and a fully connected layer connected in sequence.

[0062] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the vehicle trajectory prediction method as described in any one of the first aspects above is implemented.

[0063] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the vehicle trajectory prediction method as described in any one of the first aspects above is implemented.

[0064] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the vehicle trajectory prediction method described in any one of the above-mentioned first aspects.

[0065] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0067] Figure 1 is a schematic diagram of the structure of a vehicle trajectory prediction system provided in an embodiment of the present application;

[0068] Figure 2 It is a flow chart of a vehicle trajectory prediction method provided in an embodiment of the present application;

[0069] Figure 3 It is a flow chart of step S103 of the vehicle trajectory prediction method provided in an embodiment of the present application;

[0070] Figure 4 is a schematic diagram of the structure of the intention recognition neural network model provided in the embodiment of the present application;

[0071] Figure 5 is a scene graph of a neighborhood area provided in an embodiment of the present application;

[0072] Figure 6 It is a schematic diagram of the structure of the trajectory prediction neural network model provided in the embodiment of the present application;

[0073] Figure 7 is a schematic diagram of a historical driving trajectory provided by an embodiment of the present application;

[0074] Figure 8 is a schematic diagram of the structure of a vehicle trajectory prediction device provided in an embodiment of the present application;

[0075] Fig. 9 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0077] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0078] It should also be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. As used in the present specification and the appended claims, the term "if" can be interpreted as "when..." or "once" or "in response to determination" or "in response to detection" depending on the context. Similarly, the phrase "if it is determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once it is determined" or "in response to determination" or "once [the described condition or event] is detected" or "in response to detection [the described condition or event]" depending on the context.

[0079] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0080] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0081] The vehicle trajectory prediction method provided in the embodiment of the present application can be applied to terminal devices such as servers, vehicle-mounted devices, and roadside computing devices. The embodiment of the present application does not impose any restrictions on the specific type of terminal devices.

[0082] In recent years, urban intelligent traffic management technology has achieved certain development, which usually relies on vehicle trajectory prediction technology to control traffic conditions. However, the relevant vehicle trajectory prediction methods have the problems of short prediction results and can only predict the trajectory of a single vehicle. In order to solve this problem, the present application proposes a vehicle trajectory prediction method, a vehicle trajectory prediction device, a terminal device and a computer-readable storage medium, which can obtain vehicle information of multiple vehicles in a complex traffic detection area, identify and determine the intention recognition result of the target vehicle, and process the intention recognition result and corresponding environmental information of the target vehicle according to the pre-trained trajectory prediction neural network model, predict the driving trajectory of the target vehicle in a certain time period, and improve the accuracy and efficiency of comprehensive trajectory prediction of multiple vehicles under complex road conditions.

[0083] To implement the technical solution proposed in this application, a vehicle trajectory prediction system can be constructed first. Figure 1 The vehicle trajectory prediction system consists of at least one sensing device ( Figure 1 Only one is shown), at least one roadside unit ( Figure 1 Only one is shown), more than one vehicle ( Figure 1Only two are shown, such as vehicle a, vehicle b) and a terminal device, and the sensing device, roadside unit, and vehicle are all connected to the terminal device in communication, and the terminal device is connected to the external target terminal in communication. Among them, the vehicle may have a trajectory prediction demand for other vehicles in the detection area to determine a safe driving route, and the terminal device is a computing device that can provide vehicle trajectory prediction services. The terminal device obtains the vehicle information of each vehicle in the detection area through the roadside sensing device, roadside unit, etc., selects the target vehicle and performs intent recognition processing on the target vehicle, and uses the pre-trained trajectory prediction neural network model to predict the driving trajectory of the target vehicle in a certain period of time based on the intention recognition result and the corresponding environmental information.

[0084] In order to illustrate the technical solution proposed in this application, a specific embodiment is provided below for illustration.

[0085] Figure 2 A schematic flowchart of the vehicle trajectory prediction method provided in the present application is shown. As an example but not a limitation, the method can be applied to the above-mentioned terminal device.

[0086] S101, obtaining vehicle information of each vehicle in the detection area.

[0087] Specifically, the detection area is a traffic road that needs to be processed for vehicle trajectory prediction, such as a tunnel, a long straight road section or an intersection section. Among them, the vehicle trajectory prediction results of the tunnel and the long straight road section are relatively controllable due to their fixed driving directions. However, due to the diversity of driving directions, the complexity and uncertainty of traffic behavior in the intersection section are higher. When performing vehicle trajectory prediction processing, it is necessary to consider various factors such as information interaction between multiple vehicles, lane attributes, signal light status, road conditions, etc. for different detection areas to perform trajectory prediction. In order to improve the accuracy and stability of vehicle trajectory prediction, a variety of sensing devices are pre-set at the detection area, for example, a roadside sensing device, a roadside unit (RSU, Road Side Unit), etc. are set at the detection area; wherein the roadside sensing device may include but is not limited to at least one of a camera, a millimeter wave radar, a laser radar, a pressure sensor, a distance sensor, etc. The vehicle includes but is not limited to a networked vehicle equipped with an on-board unit (OBU) and communicating with an RSU and a terminal device, and an ordinary vehicle that does not communicate with a roadside unit or a terminal device.

[0088] Specifically, the vehicle information of each vehicle in the detection area is collected by various sensing devices set up in the detection area, wherein the vehicle information includes but is not limited to location information, vehicle feature information (such as vehicle ID, license plate number, owner identity information, vehicle model, driving speed, heading angle, etc.).

[0089] As an example and not a limitation, the vehicle information of the networked vehicle connected to the terminal device can be obtained through communication between the roadside unit and the vehicle-mounted unit; and the image data, point cloud data, distance sensor data, etc. of ordinary vehicles can be collected through roadside sensing equipment, and the vehicle information corresponding to the above-mentioned ordinary vehicles can be obtained after preprocessing.

[0090] By setting up various types of sensing devices and communication devices, vehicle information of all vehicles (including connected vehicles and ordinary vehicles) in the detection area is collected, and the environmental information of the vehicle is constructed more accurately, thereby improving the accuracy of the intention recognition results and trajectory prediction results for the target vehicle.

[0091] S102: Select a target vehicle according to the vehicle information.

[0092] Specifically, traffic conditions and installed road facilities are different on different traffic roads. For example, on some roads, traffic lights are used to indicate the traffic status (including the prohibited traffic status indicated by a red light, the fast traffic status indicated by a yellow light, and the passable status indicated by a green light) as well as the direction of travel. For example, left-traffic lights, straight-ahead lights, or right-traffic lights. On some roads, traffic lights are only used to indicate the traffic status but not the direction of travel. And there are some roads without any traffic lights. The method of selecting target vehicles is different for different traffic roads.

[0093] Specifically, in a detection area where no traffic lights are set (for example, traffic roads or commercial parks in residential areas, etc.), each vehicle in the detection area is selected as a target vehicle, and a trajectory prediction process is performed on each vehicle in the detection area to obtain a trajectory prediction result for each vehicle in the detection area.

[0094] Specifically, in a detection area where a signal light for indicating the traffic driving state and the passing direction is set, each vehicle on the lane corresponding to the passable state in the detection area is selected as a target vehicle. For example, the detection area is a one-way lane, and the traffic light is only used to indicate the traffic driving state. When the signal light is detected to be green, each vehicle on the one-way lane is a target vehicle, and a trajectory prediction process is performed on each vehicle that can pass on the one-way lane to obtain a trajectory prediction result for each vehicle on the one-way lane. Alternatively, the detection area is an intersection, and the traffic light is used to indicate the passing direction in addition to indicating the traffic driving state. The lane where the green traffic light is located is selected as the target lane in the passable state, and the lane where each vehicle is located is determined according to the position information of each vehicle in the intersection, and each vehicle located on the target lane is screened and obtained as the target vehicle, so that a trajectory prediction process is performed on each vehicle in the passable lane in the intersection, and a trajectory prediction result for each vehicle in the passable lane in the intersection is obtained.

[0095] S103: Determine an intention recognition result of the target vehicle according to the vehicle information and environmental information corresponding to the target vehicle.

[0096] Specifically, usually, the vehicle's driving trajectory at a future moment is mainly determined by the vehicle's position, driving direction, driving speed, and interaction with surrounding vehicles. Therefore, when the vehicle information of the target vehicle is obtained, it is necessary to identify the adjacent vehicles that are connected to the target vehicle to determine the target vehicle's environmental information, and then input the target vehicle's vehicle information and environmental information into the intention recognition neural network model for processing to obtain the target vehicle's steering intention recognition result.

[0097] S104: input the intention recognition result and the environmental information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle.

[0098] Specifically, the intention recognition result of the target vehicle and the environmental information corresponding to the target vehicle are input into the pre-trained trajectory prediction neural network model for processing, and the future t f The turning intention of the target vehicle within a time period (or prediction time domain) and the driving trajectory sequence corresponding to the turning intention.

[0099] S105: Send the trajectory prediction result to the target terminal.

[0100] Specifically, after obtaining the vehicle trajectory prediction result of the target vehicle, the trajectory prediction result is sent to the target terminal. The target terminal includes but is not limited to the management terminal (such as a cloud platform or a terminal device of a management user), other networked vehicles other than the target vehicle, etc. In the intelligent traffic management system, the management terminal needs to obtain the trajectory prediction results of the vehicle to grasp the traffic situation in the detection area, and implement the corresponding traffic management strategy according to the predicted situation, and send warning information; the networked vehicle connected to the terminal device needs to obtain the trajectory prediction results of other vehicles, and calculate whether other vehicles will intersect with the trajectory of the vehicle, so as to trigger the collision warning and plan the driving route to avoid the possibility of vehicle collision.

[0101] In a possible implementation, the vehicle information includes location information, and selecting a target vehicle according to the vehicle information includes:

[0102] Acquire the signal light information within the detection area;

[0103] Determining a target lane that is in a passable state according to the signal light information;

[0104] A vehicle located in the target lane is screened according to the position information and is taken as a target vehicle.

[0105] Specifically, when a traffic light is set in the detection area, the traffic light information in the detection area (including but not limited to traffic driving status information and travel direction information) is obtained, and the target lane that is currently in a passable state (specifically, the traffic light is in a green light state) is determined based on the traffic light information. The lane where each vehicle is located is determined based on the position information of each vehicle, and the vehicle located in the target lane is screened based on the lane where each vehicle is located as the target vehicle.

[0106] It is understandable that the method for determining the target lane should comply with traffic laws and regulations. For example, in the absence of a right-running light indication, the vehicle's right turn is restricted by the straight-running light. Correspondingly, when the signal light information in the current detection area shows that the left-running light is green and the straight-running light is red, and there is no right-running light, the vehicle in the left lane is the target vehicle. Correspondingly, when the signal light information in the current detection area shows that the left-running light is red and the straight-running light is green, and there is no right-running light, the vehicles in the straight lane and the right lane are the target vehicles.

[0107] The target prediction objects are screened by combining the traffic light information with the vehicle's position information, and the number of target prediction objects input into the intention recognition model is limited by combining the periodic change state of the traffic light, thereby shortening the prediction time, improving the prediction efficiency of intention recognition, and thus improving the efficiency of trajectory prediction.

[0108] In one embodiment, after selecting a vehicle located in the target lane according to the position information and taking it as the target vehicle, the method further includes:

[0109] A prediction time domain is determined according to the traffic light information.

[0110] Specifically, in general, the traffic driving state and direction indicated by the traffic lights have certain time limits. According to traffic laws and regulations, vehicles can only pass through lanes in a passable state within the above time limits. Therefore, the trajectory prediction of the vehicle on the target lane is affected by the above time limit, which can also be understood as: the turning intention and the corresponding trajectory sequence of the vehicle on the target lane can only be predicted according to the above time limit. The prediction time domain can be specifically set according to actual needs. For example, the prediction time domain is set to the remaining time when the signal light information shows that it is in a passable state. For example, some users may not pass through the detection area in time during the state conversion process of the signal light, then the prediction time domain is set to the remaining time when the signal light information shows that it is in a passable state + the preset time length after the state conversion is completed. For another example, some users may rush to pass before the signal light has not changed its state, affecting the normal driving of other vehicles. It is necessary to predict the trajectory of the above vehicles in time to avoid collision problems as much as possible. Then the prediction time domain is set to the preset time length before the state conversion + all the time in the passable state in the signal light information + the preset time length after the state conversion is completed again.

[0111] For example, at the current moment, the traffic light information shows that it is in a green light passable state, and there are 5 seconds left before the state conversion (converted to a yellow light fast pass state), which can also be understood as the remaining time in the passable state is 5 seconds, and the corresponding prediction time domain includes 5 seconds from the current moment. For example, the preset time length is 3s, and at the current moment, the traffic light information shows that it is in a green light passable state, and there are 5 seconds left before the state conversion (converted to a yellow light fast pass state), and the corresponding prediction time domain includes 8 seconds from the current moment. For example, the preset time length is 3s, and at the current moment, the traffic light information has not been converted to a passable state, and the traffic light information shows that the total time in the green light passable state is 10s, and the corresponding prediction time domain is set to include 16 seconds from the current moment (preset time length 3s + total passable state time 10s + preset time length 3s).

[0112] The prediction time domain is determined by the dynamically changing traffic light signals to dynamically adjust the time series of vehicle trajectory prediction, thereby obtaining the turning intention and trajectory sequence corresponding to the dynamically changing time series, and improving the stability of vehicle trajectory prediction results under different complex road conditions.

[0113] In one embodiment, after selecting the target vehicle according to the vehicle information, the method further includes:

[0114] Determine the lane where the target vehicle is located according to the position information of the target vehicle;

[0115] The intention recognition result of the target vehicle is determined according to the travel direction of the lane.

[0116] Specifically, the lane where the target vehicle is located at the current moment is determined according to the position information of the target vehicle, the travel direction of the lane where the target vehicle is located is obtained, and the above travel direction is used as the intention recognition result of the target vehicle.

[0117] As an example and not a limitation, the lane's direction of travel generally includes left driving, right driving, and straight driving. Depending on different road conditions, the lane may include a left lane, a straight lane, and a right lane; or include a left straight lane, a straight lane, and a right straight lane. When a vehicle enters a straight lane, the vehicle's straight driving intention weight is set to 100%, and other intention weights are set to 0%; when the vehicle enters a left lane, the vehicle's left turn intention weight is set to 100%, and other intention weights are set to 0%; when the vehicle enters a right lane, the vehicle's right turn intention weight is set to 100%, and other intention weights are set to 0%; when the vehicle enters a left straight lane, the vehicle's left turn intention weight is set to 50%, the straight driving intention weight is set to 50%, and the right turn intention weight is set to 0%; when the vehicle enters a right straight lane, the vehicle's right turn intention weight is set to 50%, the straight driving intention weight is set to 50%, and the left turn intention weight is set to 0%. As an example and not a limitation, if the vehicle is located in an unrestricted lane, a left straight lane, or a right straight lane, it is determined that the maximum turning intention of the vehicle cannot be determined by the traffic direction of the lane, and the intention recognition result of the vehicle needs to be determined based on the intention recognition model.

[0118] like Figure 3 As shown, in a possible implementation, the step S103 of determining the intention recognition result of the target vehicle according to the vehicle information and the environmental information corresponding to the target vehicle includes:

[0119] S1031, determining an adjacent vehicle connected to the target vehicle according to the vehicle information, and using first vehicle information of the adjacent vehicle as environmental information of the target vehicle;

[0120] S1032, constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environmental information;

[0121] S1033. Input the adjacency matrix into a pre-trained intention recognition neural network model for processing to obtain the intention recognition result of the target vehicle; wherein the intention recognition result is used to characterize the driving direction of the target vehicle; the intention recognition neural network model is composed of at least one graph neural network layer, a fully connected layer, a long short-term memory network layer and a function layer connected in sequence, and the graph neural network layer, the fully connected layer and the long short-term memory network layer correspond to each other one by one.

[0122] Specifically, during the driving process of the target vehicle, the environmental factors that affect the target driving direction, driving speed, and driving trajectory mainly include vehicles in the same lane as the target vehicle, in front of and behind the target vehicle, and vehicles in adjacent lanes (including adjacent left lanes and adjacent right lanes) to the target vehicle, hereinafter referred to as adjacent vehicles connected to the target vehicle. Correspondingly, it is first necessary to determine the adjacent vehicles connected to the target vehicle based on the position information of each vehicle, and convert the first vehicle information of the adjacent vehicle (specifically the position information, in the form of vector e i (x i ,y i ) represents the environmental information of the target vehicle.

[0123] Specifically, according to the second vehicle information of the target vehicle (including position information, driving speed, heading angle, represented by vector e0 (x, y, v, g)) and environmental information (specifically the position information of adjacent vehicles, represented by vector e i (x i ,y i ) represents), construct the adjacency matrix of the target vehicle The above adjacency matrix E and the vector matrix ei of the target vehicle are concatenated to form an intention recognition matrix. Before concatenation, a zero vector is added to the vector matrix ei to expand its dimension so that its dimension is the same as that of the adjacency matrix. Then it is input into the intention recognition neural network model for processing to obtain the intention recognition result of the target vehicle; among which, the intention recognition result is used to characterize the target vehicle in the future t f Driving direction within the time period (also called turning intention).

[0124] like Figure 4 As shown, a structural schematic diagram of an intent recognition neural network model is provided;

[0125] like Figure 4As shown, the intention recognition neural network model is composed of at least one graph neural network layer (Graph Neural Networks, GNN), a fully connected layer, a long short-term memory network layer and a function layer connected in sequence, and the graph neural network layer, the fully connected layer and the long short-term memory network layer correspond to each other. Among them, the graph neural network layer, the fully connected layer and the long short-term memory network layer correspond to each other, and their number is directly proportional to the number n of the target vehicle and the adjacent vehicles connected to the target vehicle. The pooling layer in the intention recognition neural network model adopts maximum pooling maxpooling to extract the local node information that has the greatest impact on the turning intention of the target vehicle. The activation function adopts the Tanh function, and the graph neural network layer is stacked with 2 layers. The graph neural network layer is used to process the adjacency matrix E. The output social nodes, network nodes and edge features are spliced ​​with the vector matrix ei that has passed through the fully connected layer (FC) with the activation function of ReLU alone, and then input into the long short-term memory network (Long Short-Term Memory, LSTM) together. According to the input e at each moment n and the previous hidden state h n-1 Update the current hidden state h n , and finally the SoftMax function layer outputs the vehicle turning intention probability P. It can be understood that the vehicle's driving direction mainly includes one of left turning, right-left turning and straight driving. Therefore, the normalized exponential function (SoftMax) layer outputs the vehicle turning intention probability. The SoftMax function layer outputs the probability of three directions (left turning P1, straight driving P2, right turning P3), and selects the maximum probability value as the final intention output P.

[0126] In a possible implementation, determining, according to the vehicle information, an adjacent vehicle having a connection relationship with the target vehicle, and using first vehicle information of the adjacent vehicle as the environment information of the target vehicle includes:

[0127] Determine a neighborhood area centered on the target vehicle according to the location information of the target vehicle;

[0128] All other vehicles in the neighborhood area are regarded as adjacent vehicles having a connection relationship with the target vehicle; wherein the other vehicles are vehicles other than the target vehicle;

[0129] The first vehicle information of the adjacent vehicle is used as the environmental information of the target vehicle.

[0130] Specifically, according to the position information of the target vehicle, its position coordinates in the standard Cartesian coordinate system with the center of the detection area as the origin are determined. Based on the position coordinates of the target vehicle, a neighborhood area with the target vehicle as the center is determined in the detection area, and all other vehicles in the neighborhood area except the target vehicle are regarded as adjacent vehicles having a connection relationship with the target vehicle. The first vehicle information of the adjacent vehicle (specifically the position information of the adjacent vehicle, represented by the vector e i (x i ,y i ) represents) as the environmental information of the target vehicle. The size of the neighborhood area can be specifically set according to the preset area restriction data and the vehicle information of the target vehicle. The preset area restriction data includes but is not limited to the preset safe following distance D in the detection area and the vertical distance L between the center lines of two adjacent lanes in the detection area.

[0131] In a possible implementation, constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environment information includes:

[0132] Normalizing the second vehicle information of the target vehicle and the environmental information respectively to obtain a first feature vector of the target vehicle and a second feature vector of an adjacent vehicle;

[0133] Calculating the similarity between the target vehicle and the adjacent vehicles according to the first feature vector and the second feature vector;

[0134] An adjacency matrix corresponding to the target vehicle is constructed according to the similarity.

[0135] Specifically, by predetermining the environmental information of the target vehicle, the adjacent vehicles that partially affect the turning intention of the target vehicle can be screened out to obtain the adjacent vehicle cluster of the target vehicle (represented by v0) in, Represents the adjacent vehicle with id i at time t, and n is the number of adjacent vehicles in the neighborhood area. Vehicle information includes but is not limited to location information (x, y), speed v, and heading g. Correspondingly, the second vehicle information of the target vehicle is represented by the vector e0(x, y, v, g). (x, y) represents the absolute geographic location, v represents the absolute speed, and h represents the absolute angle. Therefore, before processing the above-mentioned first vehicle information, the first vehicle information and environmental information need to be normalized. The normalization formula is shown as follows:

[0136]

[0137] Where X represents the feature vector after normalization, and q represents the feature vector before normalization (including lateral position x, longitudinal position y, velocity v, and heading angle g); Represents the average value of the feature vector in the historical sequence; n is the number of adjacent vehicles in the neighborhood area, and the frequency is 10 Hz.

[0138] Specifically, an NxM adjacency matrix can be constructed based on the processed first eigenvector and the second eigenvector, where M=4 (including lateral position x, longitudinal position y, speed v, and heading angle g), and each row parameter in the adjacency matrix is ​​represented as an intention vector of a vehicle, and the similarity between each two vehicles is calculated by cosine similarity to characterize the intention relationship between the two vehicles.

[0139] Specifically, the similarity calculation formula is as follows:

[0140]

[0141] In the formula, represents the feature vector of vehicle with ID i, represents the feature vector of vehicle with ID j, e i ·e j Represents the result of vector inner product operation, ||e i || means e i The module of ; construct the intention adjacency matrix of the target vehicle with id i at time t

[0142] In a possible implementation, determining a neighborhood area centered on the target vehicle according to the location information of the target vehicle includes:

[0143] Convert the target vehicle's position information into a coordinate system to obtain converted position coordinates;

[0144] Get preset area restriction data;

[0145] A neighborhood area centered on the target vehicle is constructed according to the preset area restriction data and the position coordinates.

[0146] Specifically, after obtaining the location information of each vehicle (generally, expressed as WGS84 coordinate points), the location information of each vehicle is projected into the plane coordinate system, and a standard Cartesian coordinate system with the center of the detection area as the origin is constructed. The detection area is rotated by the above coordinate system to obtain the converted location coordinates of each vehicle. The preset area restriction data within the detection area is obtained, and the rectangular neighborhood area centered on the target vehicle is determined according to the location coordinates, driving speed and preset area restriction data of the target vehicle. The neighborhood area can be determined according to the following formula:

[0147] Range=rotate(S1,S2,S3,S4);

[0148] S1=(x-vt r ,yL),S2=(x-vt r ,y+L)

[0149] S3=(x+D r ,yL),S4=(x+D r ,y+L);

[0150] In the formula, Range represents the neighborhood area of ​​the target vehicle, rotate is the coordinate point rotation conversion operation, S1, S2, S3, S4 represent the coordinates of the four position points that constitute the neighborhood area, v represents the driving speed of the target vehicle, and D r It represents the preset safe following distance in the detection area, and L represents the vertical distance between the center lines of two adjacent lanes in the detection area.

[0151] Figure 5 A scene graph of a neighborhood area is provided.

[0152] like Figure 5 As shown, a standard Cartesian coordinate system is established with the center of the detection area as the origin, and the vehicle position information is rotated and transformed to obtain the converted position coordinates of each vehicle. The position coordinates of S1, S2, S3, and S4 are determined according to the position coordinates and driving speed of the target vehicle, the preset safe following distance in the detection area, and the vertical distance between the center lines of two adjacent lanes preset in the detection area, thereby obtaining a rectangular neighborhood area centered on the target vehicle.

[0153] It can be understood that in the neighborhood area, the environmental information corresponding to the target vehicle includes the vehicle information of at most six adjacent vehicles: two vehicles located in the same lane as the target vehicle, two vehicles located in front of and behind the target vehicle, and two vehicles located in the adjacent left lane and adjacent right lane of the target vehicle that are closest to the target vehicle. r(e1, e2, e3, e4, e5, e6) represents, among which, e1 (x1, y1) represents the position information of the adjacent vehicle that is in the same lane as the target vehicle and in front of the target vehicle, e2 (x2, y2) represents the position information of the adjacent vehicle that is in the same lane as the target vehicle and behind the target vehicle, e3 (x3, y3) represents the position information of the adjacent vehicle that is in the adjacent left lane as the target vehicle and in front of the left of the target vehicle, e4 (x4, y4) represents the position information of the adjacent vehicle that is in the adjacent left lane as the target vehicle and in the left rear of the target vehicle, e5 (x5, y5) represents the position information of the adjacent vehicle that is in the adjacent right lane as the target vehicle and in front of the right of the target vehicle, and e6 (x6, y6) represents the position information of the adjacent vehicle that is in the adjacent right lane as the target vehicle and in the right rear of the target vehicle. It should be noted that if the number of adjacent vehicles in the neighborhood area is less than 6, the position coordinates corresponding to the non-existent adjacent vehicles are dimensionally processed and set to e. i (0,0), i=1,2,3,…,6.

[0154] like Figure 6 As shown, a structural schematic diagram of a trajectory prediction neural network model is provided.

[0155] In a possible implementation, the intention recognition result and the environmental information corresponding to the target vehicle are input into a pre-trained trajectory prediction neural network model for processing to obtain the trajectory prediction result of the target vehicle, including:

[0156] The second vehicle information of the target vehicle, the intention recognition result and the environmental information are input into the pre-trained trajectory prediction neural network model for processing to obtain the trajectory prediction result of the target vehicle; wherein the trajectory prediction neural network model includes an encoder and a decoder, the encoder is composed of at least one long short-term memory network, a context connection layer, and a dimensionality reduction layer connected in sequence, and the decoder is composed of at least one long short-term memory network and a fully connected layer connected in sequence.

[0157] Specifically, the trajectory prediction neural network model is divided into an encoder and a decoder; considering the interaction between the target vehicle and the adjacent vehicles, the environmental vehicle information is added while the weights of each network layer are independent. Among them, the encoder is composed of at least one long short-term memory network, a context connection layer, and an embedding dimension reduction layer connected in sequence. The long short-term memory network is determined by the number of target vehicles and adjacent vehicles (generally, the number of adjacent vehicles in the neighborhood area is 6, and 7 long short-term memory networks are set). The second vehicle information and environmental information of the target vehicle are respectively input into the corresponding long short-term memory network in the encoder for processing. The dimension reduction layer is responsible for embedding the environmental information processed by the long short-term memory network, and then concatenated with the second vehicle information of the target vehicle through the context connection layer to encode into a context vector E (e0, e1,…e i ), and finally concatenates the intent recognition result P through the fully connected layer to form a complete input input(E,P). The decoder is composed of at least one long short-term memory network and a fully connected layer connected in sequence. The long short-term memory network is determined by the number of target vehicles and adjacent vehicles (generally, the number of adjacent vehicles in the neighborhood area is 6, corresponding to the setting of 7 long short-term memory networks). Among them, the long short-term memory network in the decoder is responsible for extracting feature information from the input(E,P) input vector obtained by the encoder and predicting the position of the target vehicle. The decoder can obtain the future t f The steering action taken by the target vehicle and the corresponding trajectory sequence within time. The initial state of the decoder is According to the output at the previous moment (t-1) and the hidden state Update the hidden state at the current time t According to and Update the output of the decoder at the current time t Finally, the fully connected layer adjusts the dimension and outputs the trajectory prediction result of the target vehicle. It can be understood that since the intention recognition model outputs the maximum intention probability P, the trajectory prediction neural network model only outputs the future t with the highest confidence. f The trajectory prediction result of the target vehicle within time.

[0158] In one embodiment, the method further comprises:

[0159] Obtain historical driving trajectories of multiple vehicles;

[0160] The historical driving trajectory is divided into a plurality of historical trajectory segments; the historical trajectory segments are composed of a first trajectory segment of a first preset duration and a second trajectory segment of a preset predicted duration, the first preset duration is before the preset predicted duration, and the first trajectory segment corresponds to the second trajectory segment one by one;

[0161] Adding a turning intention label and a trajectory sequence label to the corresponding first trajectory segment according to the second trajectory segment;

[0162] Inputting the first trajectory segment and the corresponding second trajectory segment into an intent recognition model for pre-training to obtain a pre-trained intent recognition model;

[0163] The first trajectory segment and the corresponding second trajectory segment are input into a trajectory prediction neural network model for pre-training to obtain a pre-trained trajectory prediction neural network model.

[0164] Specifically, the driving trajectory of the vehicle before the current moment is taken as the historical driving trajectory. The historical driving trajectories of multiple vehicles on different roads are obtained respectively, and the historical driving trajectory of each vehicle is divided into multiple historical trajectory segments according to the first preset duration and the preset predicted duration. Each historical trajectory segment is composed of a first trajectory segment of the first preset duration and a second trajectory segment of the preset predicted duration corresponding to the first trajectory segment one by one.

[0165] like Figure 7 As shown, a schematic diagram of a historical driving trajectory is provided;

[0166] Figure 7 In the figure, the first preset duration H is 5s, and the preset prediction duration F is 5s; vehicle A includes a 30s historical driving trajectory T, which is correspondingly divided into three historical trajectory segments (T1+T2+T3) with a duration of 10s, each of which is composed of a first trajectory segment of 5s and a second trajectory segment of 5s (T1=H1+F1, T2=H2+F2, T3=H3+F3).

[0167] Specifically, according to the driving direction of the vehicle in the second trajectory segment, a turning intention label is added to the first trajectory segment corresponding to the above second trajectory segment; according to the trajectory sequence of the vehicle in the second trajectory segment, a trajectory sequence label is added to the first trajectory segment corresponding to the above second trajectory segment. The first trajectory segment with the label (specifically, it can be the first trajectory segment with only the turning intention label added) and the corresponding second trajectory segment are input into the intention recognition model for pre-training to obtain a pre-trained intention recognition model; the first trajectory segment with the label and the corresponding second trajectory segment are input into the trajectory prediction neural network model for pre-training to obtain a pre-trained trajectory prediction neural network model. It should be understood that in the process of pre-training, the trajectory prediction neural network model can be trained and learned using the same network weights. At the same time, in order to avoid the neural network model from falling into a certain type of traffic light information or a certain direction of local learning, it is set that during the pre-training process of the trajectory prediction neural network model, the input data consists of a sequence of three turning intentions of left turn, straight, and right turn in the same number of four directions of east, west, south, and north and the same length.

[0168] This embodiment pre-identifies the intention recognition result of the target vehicle through the interactive relationship between multiple vehicles, and processes the intention recognition result of the target vehicle and the corresponding environmental information according to the pre-trained trajectory prediction neural network model, predicts the driving trajectory of the target vehicle in a certain time period, and improves the accuracy and efficiency of comprehensive trajectory prediction of multiple vehicles under complex road conditions.

[0169] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0170] Corresponding to the vehicle trajectory prediction method described in the above embodiment, Figure 8 A structural block diagram of a vehicle trajectory prediction device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0171] Reference Figure 8 , the vehicle trajectory prediction device 100 comprises:

[0172] The data acquisition module 101 is used to acquire vehicle information of each vehicle in the detection area;

[0173] A vehicle screening module 102, for selecting a target vehicle according to the vehicle information;

[0174] An intention recognition module 103, used to determine an intention recognition result of the target vehicle according to the vehicle information and environmental information corresponding to the target vehicle;

[0175] The trajectory prediction module 104 is used to input the intention recognition result and the environmental information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle;

[0176] The sending module 105 is used to send the trajectory prediction result to the target terminal.

[0177] In a possible implementation, the vehicle information includes location information, and the vehicle screening module includes:

[0178] A signal light information acquisition unit, used to acquire signal light information within the detection area;

[0179] A lane determination unit, configured to determine a target lane in a passable state according to the signal light information;

[0180] The vehicle screening unit is used to screen the vehicle located on the target lane according to the position information as the target vehicle.

[0181] In a possible implementation, the intention recognition module includes:

[0182] An environmental information determination submodule, used to determine an adjacent vehicle having a connection relationship with the target vehicle according to the vehicle information, and use the first vehicle information of the adjacent vehicle as the environmental information of the target vehicle;

[0183] A matrix construction submodule, used for constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environmental information;

[0184] An intention recognition submodule is used to input the adjacency matrix into a pre-trained intention recognition neural network model for processing to obtain the intention recognition result of the target vehicle; wherein the intention recognition result is used to characterize the driving direction of the target vehicle; the intention recognition neural network model is composed of at least one graph neural network layer, a fully connected layer, a long short-term memory network layer and a function layer connected in sequence, and the graph neural network layer, the fully connected layer and the long short-term memory network layer correspond to each other one by one.

[0185] In a possible implementation, the environment information determination submodule includes:

[0186] A neighborhood determination unit, configured to determine a neighborhood area centered on the target vehicle according to the location information of the target vehicle;

[0187] An adjacent vehicle determination unit, configured to determine all other vehicles in the neighborhood area as adjacent vehicles having a connection relationship with the target vehicle; wherein the other vehicles are vehicles other than the target vehicle;

[0188] The environmental information determining unit is used to use the first vehicle information of the adjacent vehicle as the environmental information of the target vehicle.

[0189] In a possible implementation, the matrix construction submodule includes:

[0190] A feature vector determination unit, used for normalizing the second vehicle information of the target vehicle and the environmental information respectively, to obtain a first feature vector of the target vehicle and a second feature vector of an adjacent vehicle;

[0191] A similarity calculation unit, configured to calculate the similarity between the target vehicle and the adjacent vehicle based on the first feature vector and the second feature vector;

[0192] A matrix construction unit is used to construct an adjacency matrix corresponding to the target vehicle according to the similarity.

[0193] In a possible implementation manner, the neighborhood determining unit includes:

[0194] A coordinate conversion subunit, used for converting the position information of the target vehicle into a coordinate system to obtain the converted position coordinates;

[0195] A data acquisition subunit, used to acquire preset area restriction data;

[0196] The neighborhood determination subunit is used to construct a neighborhood area centered on the target vehicle according to the preset area restriction data and the position coordinates.

[0197] In a possible implementation, the trajectory prediction module is specifically used to input the second vehicle information of the target vehicle, the intention recognition result and the environmental information into the pre-trained trajectory prediction neural network model for processing to obtain the trajectory prediction result of the target vehicle; wherein the trajectory prediction neural network model includes an encoder and a decoder, the encoder is composed of at least one long short-term memory network, a context connection layer, and a dimensionality reduction layer connected in sequence, and the decoder is composed of at least one long short-term memory network and a fully connected layer connected in sequence.

[0198] This embodiment pre-identifies the intention recognition result of the target vehicle through the interactive relationship between multiple vehicles, and processes the intention recognition result of the target vehicle and the corresponding environmental information according to the pre-trained trajectory prediction neural network model, predicts the driving trajectory of the target vehicle in a certain time period, and improves the accuracy and efficiency of comprehensive trajectory prediction of multiple vehicles under complex road conditions.

[0199] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0200] Fig. 9 This is a schematic diagram of the structure of the terminal device provided in this embodiment. Fig. 9 As shown, the terminal device 9 of this embodiment includes: at least one processor 90 ( Fig. 9 Only one is shown), a memory 91, and a computer program 92 stored in the memory 91 and executable on the at least one processor 90, wherein the processor 90 implements the steps in any of the above-mentioned vehicle trajectory prediction method embodiments when executing the computer program 92.

[0201] The terminal device 9 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud terminal device. The terminal device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will appreciate that Fig. 9 It is only an example of the terminal device 9 and does not constitute a limitation on the terminal device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0202] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0203] In some embodiments, the memory 91 may be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. In other embodiments, the memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 9. Further, the memory 91 may also include both an internal storage unit of the terminal device 9 and an external storage device. The memory 91 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0204] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0205] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0206] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0208] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0209] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0210] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A vehicle trajectory prediction method, characterized in that: include: Obtain vehicle information of each vehicle in the detection area; Selecting a target vehicle according to the vehicle information; Determining an intention recognition result of the target vehicle according to the vehicle information and environmental information corresponding to the target vehicle; Inputting the intention recognition result and the environmental information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle; The trajectory prediction result is sent to the target terminal.

2. The vehicle trajectory prediction method according to claim 1, characterized in that: The vehicle information includes location information, and selecting a target vehicle according to the vehicle information includes: Acquire the signal light information within the detection area; Determining a target lane that is in a passable state according to the signal light information; A vehicle located in the target lane is screened according to the position information and is taken as a target vehicle.

3. The vehicle trajectory prediction method according to claim 1, characterized in that: The determining the intention recognition result of the target vehicle according to the vehicle information and the environmental information corresponding to the target vehicle includes: Determine an adjacent vehicle connected to the target vehicle according to the vehicle information, and use the first vehicle information of the adjacent vehicle as the environmental information of the target vehicle; constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environmental information; The adjacency matrix is ​​input into a pre-trained intention recognition neural network model for processing to obtain the intention recognition result of the target vehicle; wherein the intention recognition result is used to characterize the driving direction of the target vehicle; the intention recognition neural network model is composed of at least one graph neural network layer, a fully connected layer, a long short-term memory network layer and a function layer connected in sequence, and the graph neural network layer, the fully connected layer and the long short-term memory network layer correspond to each other one by one.

4. The vehicle trajectory prediction method according to claim 3, characterized in that: The step of determining an adjacent vehicle connected to the target vehicle according to the vehicle information and using first vehicle information of the adjacent vehicle as the environmental information of the target vehicle includes: Determine a neighborhood area centered on the target vehicle according to the location information of the target vehicle; All other vehicles in the neighborhood area are regarded as adjacent vehicles having a connection relationship with the target vehicle; wherein the other vehicles are vehicles other than the target vehicle; The first vehicle information of the adjacent vehicle is used as the environmental information of the target vehicle.

5. The vehicle trajectory prediction method according to claim 3, characterized in that: The step of constructing an adjacency matrix according to the second vehicle information of the target vehicle and the environment information includes: Normalizing the second vehicle information of the target vehicle and the environmental information respectively to obtain a first feature vector of the target vehicle and a second feature vector of an adjacent vehicle; Calculating the similarity between the target vehicle and the adjacent vehicles according to the first feature vector and the second feature vector; An adjacency matrix corresponding to the target vehicle is constructed according to the similarity.

6. The vehicle trajectory prediction method according to claim 4, characterized in that: The step of determining a neighborhood area centered on the target vehicle according to the location information of the target vehicle includes: Convert the target vehicle's position information into a coordinate system to obtain converted position coordinates; Get preset area restriction data; A neighborhood area centered on the target vehicle is constructed according to the preset area restriction data and the position coordinates.

7. The vehicle trajectory prediction method according to any one of claims 1 to 6, characterized in that: The intention recognition result and the environment information corresponding to the target vehicle are input into a pre-trained trajectory prediction neural network model for processing to obtain the trajectory prediction result of the target vehicle, including: The second vehicle information of the target vehicle, the intention recognition result and the environmental information are input into the pre-trained trajectory prediction neural network model for processing to obtain the trajectory prediction result of the target vehicle; wherein the trajectory prediction neural network model includes an encoder and a decoder, the encoder is composed of at least one long short-term memory network, a context connection layer, and a dimensionality reduction layer connected in sequence, and the decoder is composed of at least one long short-term memory network and a fully connected layer connected in sequence.

8. A vehicle trajectory prediction device, characterized in that: include: A data acquisition module, used to acquire vehicle information of each vehicle in the detection area; A vehicle screening module, used for selecting a target vehicle according to the vehicle information; An intention recognition module, used to determine an intention recognition result of the target vehicle according to the vehicle information and environmental information corresponding to the target vehicle; A trajectory prediction module, used for inputting the intention recognition result and the environmental information corresponding to the target vehicle into a pre-trained trajectory prediction neural network model for processing to obtain a trajectory prediction result of the target vehicle; The sending module is used to send the trajectory prediction result to the target terminal.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images