Track prediction method based on scene change

Abstract

The invention relates to the technical field of trajectory prediction, and provides a trajectory prediction method based on scene change. The method comprises the following steps: determining a unique motion feature of each target agent in a current scene; constructing a spatial diagram based on the unique motion features of all target agents in the current scene; obtaining a node feature matrix, an edge feature matrix, a group generation matrix and a hyperedge feature matrix of all target agents in the current scene according to the spatial diagram; transforming the edge feature matrix and the node feature matrix by using a pairwise relation transformation model to obtain a pairwise node feature matrix; transforming the node feature matrix, the hyperedge feature matrix and the group generation matrix by using a super relation transformation model to obtain a final group feature matrix; and based on the paired node feature matrix and the final group feature matrix, predicting each target agent in the current scene by using a trajectory prediction model to obtain a prediction trajectory of the target agent. According to the method, the accuracy of trajectory prediction can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of trajectory prediction, and in particular to a trajectory prediction method based on scene change. Background Art

[0002] Trajectory prediction is a key issue in autonomous driving and intelligent transportation systems, which aims to predict the future motion path of intelligent agents (such as vehicles and pedestrians) based on their historical trajectories and environmental information. Accurate trajectory prediction is of great significance for improving traffic safety, optimizing traffic flow, and enhancing the decision-making ability of autonomous driving systems.

[0003] The main defects in trajectory prediction are: 1. Traditional methods use a unified feature representation and prediction strategy for all agents, without fully considering the differences between individuals and the impact of scene changes on agent behavior. 2. Traditional methods oversimplify the group relationship between agents, without fully considering the complex interactions within the group and the mutual influence between groups, resulting in low accuracy of trajectory prediction. It can be seen that the current trajectory prediction method has the problem of low accuracy of trajectory prediction. Summary of the invention

[0004] The present application provides a trajectory prediction method based on scene change, which can solve the problem of low accuracy of trajectory prediction.

[0005] In a first aspect, an embodiment of the present application provides a trajectory prediction method based on scene change, the trajectory prediction method comprising: according to T The trajectories of all target agents in the scene are used to determine the unique motion features of each target agent in the current scene; the unique motion features are used to describe the trajectory features of the target agent. T The scene is the current scene, and the trajectory includes the trajectory points of the target agent at each moment corresponding to the scene; A spatial graph is constructed based on the unique motion features of all target agents in the current scene; multiple nodes of the spatial graph correspond one-to-one to the unique motion features of multiple target agents, and the edges between the nodes are the neighboring relationships between the corresponding two target agents; Obtain the node feature matrix, edge feature matrix, group occurrence matrix and hyperedge feature matrix of all target agents in the current scene according to the spatial graph; the node feature matrix is ​​used to describe the unique motion features corresponding to all nodes in the spatial graph, the edge feature matrix is ​​used to describe all edges in the spatial graph, the group occurrence matrix is ​​used to describe the group to which each target agent in the current scene belongs, and the hyperedge feature matrix is ​​used to describe the features of each group; The edge feature matrix and the node feature matrix are transformed using the pairwise relationship transformation model to obtain a pairwise node feature matrix; The node feature matrix, the hyper-edge feature matrix, and the group occurrence matrix are transformed using the hyper-relation transformation model to obtain the final group feature matrix; Based on the paired node feature matrix and the final group feature matrix, the trajectory prediction model is used to predict each target agent in the current scene to obtain the predicted trajectory of each target agent in the current scene.

[0006] Optional, according to T The trajectories of all target agents in the scene are calculated to determine the unique motion characteristics of each target agent in the current scene, including: Calculate the average of unique motion features for each agent class in the current scene based on all trajectories in all other scenes except the current one; The unique motion features of each target agent in the current scene are obtained based on the average unique motion features of all agent categories and the trajectories of all target agents in the current scene.

[0007] Optionally, calculate the average of the unique motion features of the current scene based on all trajectories of all other scenes except the current scene, including: By formula: Calculate the In the scene The average of the unique motion features of the agent classes ; in, Indicated in The scene belongs to The number of target agents in the agent class, Indicated in The first scene moment, based on the of the agent class The unique motion features calculated from the trajectory points of the target agent, Indicates the last moment of each scene, , represents the number of agent categories, , Indicates the current scene. hour, Indicates In the scene The average of unique motion features for each agent class.

[0008] Optionally, build a spatial graph based on the unique motion features of all target agents in the current scene, including: Calculate the distance between the trajectories of every two target agents in the current scene. If the distance between the two target agents is less than the distance threshold, it is considered that the two target agents have a neighboring relationship.

[0009] A corresponding node is generated for the unique motion feature of each target agent in the current scene. If there is a proximity relationship between the two target agents corresponding to each two nodes, an edge between the two nodes is generated to obtain a spatial graph.

[0010] Optionally, the node feature matrix, edge feature matrix, group occurrence matrix and super edge feature matrix of all target agents in the current scene are obtained according to the spatial graph, including: For each two nodes with a connecting edge in the spatial graph, the two unique motion features corresponding to the two nodes are concatenated into a pair of edge features; Integrate all paired edge features into a matrix to obtain the node feature matrix; Construct an edge feature matrix based on all the edges in the spatial graph; Calculate the group relationship between the target agents corresponding to every two nodes in the spatial graph, obtain multiple groups based on all group relationships, integrate the groups to which all target agents belong into a matrix, and obtain the group occurrence matrix; The features of each group are calculated based on all group relationships and all unique motion features, and the features of all groups are integrated into a matrix to obtain the hyperedge feature matrix.

[0011] Optionally, the group relationship between target agents corresponding to every two nodes in the computation space graph includes: By formula: ; Calculate the The target agent and Group relations between target agents ; in, Indicates The target agent and The affinity between target agents, represents the threshold value, Indicates The unique motion characteristics of the target agent, Indicates The unique motion characteristics of the target agent, , , Represents the numbered set of target agents in the current scene, represents the unit step function.

[0012] Optionally, the pairwise relationship transformation model includes a plurality of pairwise relationship transformation modules connected in sequence; The input data of the first input terminal of the pairwise relationship transformation model is an edge feature matrix, the input data of the second input terminal of the pairwise relationship transformation model is a node feature matrix, and the output data of the output terminal of the pairwise relationship transformation model is a pairwise node feature matrix; The first input end of the first pairwise relationship transformation module is the first input end of the pairwise relationship transformation model, the second input end of the first pairwise relationship transformation module is the second input end of the pairwise relationship transformation model, and the second output end of the last pairwise relationship transformation module is the output end of the pairwise relationship transformation model; The pairwise relation transformation module includes a first relation attention layer, a message function layer, a first additive normalization layer, a second additive normalization layer, a third additive normalization layer, a fourth additive normalization layer, a first feedforward network layer, and a second feedforward network layer; The first input end of the first relational attention layer, the input end of the message function layer, and the input end of the second additive normalization layer are all the first input end of the pairwise relational transformation module, the second input end of the first relational attention layer and the input end of the first additive normalization layer are all the second input end of the pairwise relational transformation module, the output end of the third additive normalization layer is the first output end of the pairwise relational transformation module, and the output end of the fourth additive normalization layer is the second output end of the pairwise relational transformation module; The output end of the first relational attention layer is connected to the input end of the first additive normalization layer, the output end of the first additive normalization layer is connected to the input end of the second feedforward network layer, the output end of the second feedforward network layer is connected to the input end of the fourth additive normalization layer, the input end of the second additive normalization layer is respectively connected to the output end of the message function layer and the output end of the fourth additive normalization layer, the output end of the second additive normalization layer is respectively connected to the input end of the first feedforward network layer and the input end of the third additive normalization layer, and the output end of the first feedforward network layer is connected to the input end of the third additive normalization layer.

[0013] Optionally, the super-relation transformation model includes a plurality of super-relation transformation modules connected in sequence; The input data of the first input terminal of the hyper-relation transformation model is a node feature matrix, the input data of the second input terminal of the hyper-relation transformation model is a hyper-edge feature matrix, the input data of the third input terminal of the hyper-relation transformation model is a group occurrence matrix, and the output data of the output terminal of the hyper-relation transformation model is a final group feature matrix; The first input end of the first super-relation transformation module is the first input end of the super-relation transformation model, the second input end of the first super-relation transformation module is the second input end of the super-relation transformation model, the third input end of each super-relation transformation module is the third input end of the super-relation transformation model, and the first output end of the last super-relation transformation module is the output end of the super-relation transformation model; The super relation transformation module includes a second relation attention layer, a fifth additive normalization layer, a sixth additive normalization layer, a seventh additive normalization layer, an eighth additive normalization layer, a third feedforward network layer, a fourth feedforward network layer, and a fifth feedforward network layer; The first input end of the second relational attention layer is the first input end of the super-relational transformation module, the second input end of the second relational attention layer, the input end of the third feedforward network layer, and the input end of the sixth additive normalization layer are the second input end of the super-relational transformation module, the third input end of the second relational attention layer is the third input end of the super-relational transformation module, the output end of the seventh additive normalization layer is the first output end of the super-relational transformation module, and the output end of the eighth additive normalization layer is the second output end of the super-relational transformation module; The output end of the second relational attention layer is connected to the input end of the fifth additive normalization layer, the output end of the fifth additive normalization layer is respectively connected to the input end of the fourth feedforward network layer and the input end of the seventh additive normalization layer, the output end of the fourth feedforward network layer is connected to the input end of the seventh additive normalization layer, the input end of the sixth additive normalization layer is connected to the output end of the seventh additive normalization layer and the output end of the third feedforward network layer, the output end of the sixth additive normalization layer is respectively connected to the input end of the fifth feedforward network layer and the input end of the eighth additive normalization layer, and the output end of the fifth feedforward network layer is connected to the input end of the eighth additive normalization layer.

[0014] Optionally, based on the paired node feature matrix and the final group feature matrix, a trajectory prediction model is used to predict each target agent in the current scene to obtain a predicted trajectory of each target agent in the current scene, including: According to the paired node feature matrix and the final group feature matrix, the temporal features and spatial features of all target agents in the current scene are calculated; The trajectory prediction model is used to calculate the temporal features and spatial features to obtain the predicted trajectory of each target agent in the current scene.

[0015] Optionally, the trajectory prediction model includes a fusion module, a first fully connected module, a second fully connected module, a third fully connected module, a fourth fully connected module, a Gaussian mixture module, a long short-term memory module, a fifth fully connected module, and an addition module; The input end of the fusion module is the input end of the trajectory prediction model, and the output end of the addition module is the output end of the trajectory prediction model; The output end of the fusion module is connected to the input end of the first fully connected module, the input end of the second fully connected module, the input end of the third fully connected module, and the input end of the fourth fully connected module respectively; the output end of the first fully connected module, the output end of the second fully connected module, the output end of the third fully connected module, and the output end of the fourth fully connected module are all connected to the input end of the Gaussian mixture module; the output end of the Gaussian mixture module is connected to the input end of the long short-term memory module; the output end of the long short-term memory module is connected to the input end of the fifth fully connected module; and the output end of the fifth fully connected module is connected to the input end of the addition module.

[0016] In a second aspect, an embodiment of the present application provides a trajectory prediction device based on scene change, comprising: A determination module is used to determine the unique motion features of each target agent in the current scene based on the trajectories of all target agents in T scenes; the unique motion features are used to describe the trajectory features of the target agent, the Tth scene is the current scene, and the trajectory includes the trajectory points of the target agent at each moment corresponding to the scene; A construction module is used to construct a spatial graph based on the unique motion features of all target agents in the current scene; multiple nodes of the spatial graph correspond to the unique motion features of multiple target agents, and the edges between the nodes are the proximity relationships between the corresponding two target agents; An acquisition module is used to acquire the node feature matrix, edge feature matrix, group occurrence matrix and hyperedge feature matrix of all target agents in the current scene according to the spatial graph; the node feature matrix is ​​used to describe the unique motion features corresponding to all nodes in the spatial graph, the edge feature matrix is ​​used to describe all edges in the spatial graph, the group occurrence matrix is ​​used to describe the group to which each target agent in the current scene belongs, and the hyperedge feature matrix is ​​used to describe the features of each group; A first transformation module is used to transform the edge feature matrix and the node feature matrix using a paired relationship transformation model to obtain a paired node feature matrix; The second transformation module is used to transform the node feature matrix, the hyperedge feature matrix, and the group occurrence matrix using the hyperrelation transformation model to obtain the final group feature matrix; The prediction module is used to predict each target intelligent agent in the current scene based on the paired node feature matrix and the final group feature matrix using the trajectory prediction model to obtain the predicted trajectory of each target intelligent agent in the current scene.

[0017] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned scene change-based trajectory prediction method when executing the above-mentioned computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned scene change-based trajectory prediction method is implemented.

[0019] The above solution of the present application has the following beneficial effects: In some embodiments of the present application, the unique motion features of each target intelligent agent in the current scene are determined based on the trajectories of all target intelligent agents in T scenes, and then a spatial graph is constructed based on the unique motion features of all target intelligent agents in the current scene, and then the node feature matrix, edge feature matrix, group occurrence matrix and hyper-edge feature matrix of all target intelligent agents in the current scene are obtained based on the spatial graph, and then the edge feature matrix and the node feature matrix are transformed using a paired relationship transformation model to obtain a paired node feature matrix, and then the node feature matrix, the hyper-edge feature matrix and the group occurrence matrix are transformed using a hyper-relationship transformation model to obtain a final group feature matrix, and finally, based on the paired node feature matrix and the final group feature matrix, each target intelligent agent in the current scene is predicted using a trajectory prediction model to obtain a predicted trajectory for each target intelligent agent in the current scene. Among them, the unique motion characteristics of the target intelligent agent in the current scene are determined based on the trajectories of multiple scenes, the trajectory characteristics of the agent in different scenes can be obtained, the information content and accuracy of the unique motion characteristics can be improved, and the group patterns of the target intelligent agent can be analyzed to capture the target intelligent agent's internal collaborative behavior and the interaction between groups. Trajectory prediction is performed based on the paired node feature matrix and the final group feature matrix, which can effectively improve the accuracy of trajectory prediction.

[0020] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A flow chart of a trajectory prediction method based on scene change provided in an embodiment of the present application; Figure 2 A schematic diagram of scene change provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a pairwise relationship transformation module provided in one embodiment of the present application; Figure 4A schematic diagram of the structure of a super-relationship transformation module provided in one embodiment of the present application; Figure 5 A schematic diagram of the structure of a trajectory prediction model provided in one embodiment of the present application; Figure 6 A schematic diagram of the structure of a trajectory prediction device based on scene change provided in one embodiment of the present application; Figure 7 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0024] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0025] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0027] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0029] In response to the problem of low accuracy of existing trajectory prediction, an embodiment of the present application provides a trajectory prediction method based on scene transformation. The trajectory prediction method determines the unique motion characteristics of each target intelligent agent in the current scene based on the trajectories of all target intelligent agents in T scenes, and then constructs a spatial graph based on the unique motion characteristics of all target intelligent agents in the current scene. The node feature matrix, edge feature matrix, group occurrence matrix and super edge feature matrix of all target intelligent agents in the current scene are obtained according to the spatial graph. The edge feature matrix and the node feature matrix are then transformed using a paired relationship transformation model to obtain a paired node feature matrix. The node feature matrix, the hyper edge feature matrix and the group occurrence matrix are then transformed using a super relationship transformation model to obtain a final group feature matrix. Finally, based on the paired node feature matrix and the final group feature matrix, the trajectory prediction model is used to predict each target intelligent agent in the current scene to obtain a predicted trajectory for each target intelligent agent in the current scene. Among them, the unique motion characteristics of the target intelligent agent in the current scene are determined based on the trajectories of multiple scenes, the trajectory characteristics of the agent in different scenes can be obtained, the information content and accuracy of the unique motion characteristics can be improved, and the group patterns of the target intelligent agent can be analyzed to capture the target intelligent agent's internal collaborative behavior and the interaction between groups. Trajectory prediction is performed based on the paired node feature matrix and the final group feature matrix, which can effectively improve the accuracy of trajectory prediction.

[0030] Next, the trajectory prediction method based on scene change provided by the present application is exemplified.

[0031] like Figure 1 As shown, the trajectory prediction method based on scene change provided by the present application includes the following steps: Step 11, based on the trajectories of all target agents in T scenes, determine the unique motion features of each target agent in the current scene.

[0032] The above unique motion features are used to describe the trajectory features of the target agent, and the Tth scene is the current scene. The above scene is the trajectory of all target agents in the area where trajectory prediction is required at multiple times. The target agent is the agent in the area, such as a vehicle, a pedestrian, etc. The trajectory includes the trajectory points of the target agent at each time in the scene. For example, the scene can be the trajectory of all agents in a certain area at 8 o'clock, 9 o'clock, and 10 o'clock.

[0033] It should be noted that, if the scene is the current scene, the moment corresponding to the current scene only includes the current moment.

[0034] Exemplarily, the current scene is the third scene, the current time is 10 o'clock, the multiple times corresponding to the first scene are 4 o'clock, 5 o'clock, and 6 o'clock, the multiple times corresponding to the second scene are 7 o'clock, 8 o'clock, and 9 o'clock, and the third scene is the current scene, and the corresponding time is the current time of 10 o'clock.

[0035] In some embodiments of the present application, the trajectory of the target intelligent body can be obtained by using a global positioning system or the like. T The specific steps to determine the unique motion features of each target agent in the current scene are: In the first step, the average of the unique motion features of each agent class in the current scene is calculated based on all trajectories of all other scenes except the current scene.

[0036] The agent class is the physical type of the agent, such as vehicle, pedestrian, etc.

[0037] By formula: ; Calculate the In the scene The average of the unique motion features of the agent classes .

[0038] in, Indicated in The scene belongs to The number of target agents in each agent class, Indicated in The first scene moment, based on the of the agent class The unique motion features calculated from the trajectory points of the target agent, Indicates the last moment of each scene, , represents the number of agent categories, , Indicates the current scene. hour, Indicates In the scene The average of unique motion features for each agent class.

[0039] It should be noted that when hour, Indicates the first scene in the moments, based only on of the agent class The unique motion feature calculated by the trajectory point of the target intelligent body can be obtained by calculating the position of the trajectory point through a multi-layer perceptron. hour, Not only based on the trajectory points of the target intelligent agent, but also the unique motion feature average value of the intelligent agent category corresponding to the target intelligent agent calculated by the above formula needs to be introduced for calculation. Specifically, after obtaining the initial unique motion feature based on the trajectory points of the target intelligent agent (which can be obtained by using a multi-layer perceptron), the initial unique motion feature and the unique motion feature average value of the intelligent agent category corresponding to the target intelligent agent are weightedly summed to obtain the unique motion feature of the target intelligent agent in the scene.

[0040] In the second step, the unique motion features of each target agent in the current scene are obtained based on the average of the unique motion features of all agent categories and the trajectories of all target agents in the current scene.

[0041] Specifically, the current scene corresponds to the current moment, that is, the trajectory of the target intelligent agent in the current scene is the trajectory point of the target intelligent agent at the current moment; for each target intelligent agent in the current scene, the initial unique motion feature is obtained based on the trajectory point of the target intelligent agent at the current moment (which can be obtained using a multi-layer perceptron), and then the initial unique motion feature is weightedly summed with the average value of the unique motion feature of the intelligent agent category corresponding to the target intelligent agent to obtain the unique motion feature of the target intelligent agent in the current scene.

[0042] The following is an illustrative description of scene switching with reference to a specific example.

[0043] like Figure 2 As shown in the figure, for scene 0 and scene 1, in scene 0 and scene 1, the time transformation includes the 0th moment, the 1st moment to the At each moment, the preset unique motion feature average value is As the average of the unique motion features of scene 0 , and update the unique motion features of each target agent in scene 0 to obtain the unique motion features of the 0th target agent at time 0 , the unique motion characteristics of the 0th target agent at the first moment , the unique motion characteristics of the first target agent at the first moment , No. Unique motion characteristics of the third target agent at time ,...,No. Unique motion characteristics of the target agent at time a , and by the formula Calculate the average of unique motion features for scene 1 , and update the unique motion features of each target agent in scene 1 to obtain the unique motion features of the 0th target agent at the 0th time in scene 1 , the unique motion characteristics of the first target agent at time 0 , the unique motion characteristics of the 0th target agent at the first moment , No. Unique motion characteristics of the fourth target agent at time ,...,No. Unique motion characteristics of the target agent at time a , and then by the formula Calculate the average of the unique motion features for the next scene. Figure 2 The dashed circles represent newly emerged agents, and the solid circles represent agents that exist in both scenarios.

[0044] Step 12, construct a spatial graph based on the unique motion features of all target agents in the current scene.

[0045] The multiple nodes of the above-mentioned spatial graph correspond one-to-one to the unique motion features of the multiple target intelligent agents, and the edges between the nodes are the proximity relationships between the corresponding two target intelligent agents.

[0046] In some embodiments of the present application, the step of constructing a spatial graph based on the unique motion features of all target agents in the current scene includes: In the first step, the distance between the trajectories of every two target agents in the current scene is calculated. If the distance between the two target agents is less than the distance threshold, it is considered that the two target agents have a neighboring relationship.

[0047] Specifically, the Euclidean distance between the trajectory point positions of every two target agents in the current scene at the current moment can be calculated to obtain the distance between the trajectories of every two target agents.

[0048] It should be noted that if the distance between two target agents is greater than or equal to the distance threshold, it is considered that there is no proximity relationship between the two target agents.

[0049] The second step is to generate a corresponding node for the unique motion feature of each target agent in the current scene. If there is a proximity relationship between the two target agents corresponding to each two nodes, an edge between the two nodes is generated to obtain a spatial graph.

[0050] Step 13, obtain the node feature matrix, edge feature matrix, group occurrence matrix and hyperedge feature matrix of all target agents in the current scene according to the spatial graph.

[0051] The node feature matrix is ​​used to describe the unique motion features corresponding to all nodes in the spatial graph, the edge feature matrix is ​​used to describe all edges in the spatial graph, the group occurrence matrix is ​​used to describe the group to which each target agent in the current scene belongs (the group can be understood as an agent group, which refers to a group of directly or indirectly similar target agents combined together), and the hyperedge feature matrix is ​​used to describe the characteristics of each group.

[0052] In some embodiments of the present application, the step of obtaining the node feature matrix, edge feature matrix, group occurrence matrix and hyperedge feature matrix of all target agents in the current scene according to the spatial graph includes: In the first step, for each pair of nodes with an edge in the spatial graph, the two unique motion features corresponding to the two nodes are concatenated into paired edge features. Then all paired edge features are integrated into a matrix to obtain the node feature matrix.

[0053] In the second step, an edge feature matrix is ​​constructed based on all the edges in the spatial graph.

[0054] Specifically, if there is an edge between two nodes in the spatial graph, it is recorded as 1, and if there is no edge between the two nodes, it is recorded as 0. All 1s and 0s are integrated into a matrix to obtain the edge feature matrix.

[0055] The third step is to calculate the group relationship between the target agents corresponding to every two nodes in the spatial graph, obtain multiple groups based on all group relationships, integrate the groups to which all target agents belong into a matrix, and obtain the group occurrence matrix.

[0056] Specifically, through the formula: ; Calculate the The target agent and Group relations between target agents .

[0057] in, Indicates The target agent and The affinity between target agents, represents the threshold value, Indicates The unique motion characteristics of the target agent, Indicates The unique motion characteristics of the target agent, , , Represents the numbered set of target agents in the current scene, represents the unit step function.

[0058] It should be noted that the above is a unit step function, used for binarization. , then there is a similar relationship between the two target agents and they belong to the same group. , then there is no similarity relationship between the two target agents, and all target agents in the group have direct or indirect similarity relationships. For example, for target agents A, B, C, D, and E, there is a similarity relationship between A and B, between B and C, and between A and E. Then A, B, C, and E are a group. All groups are integrated into a matrix to obtain a group occurrence matrix, and the elements in this matrix are used to describe the group to which each target agent belongs.

[0059] For example, similarity relationships can be analyzed by forward and backward paths. Update to improve similarity accuracy.

[0060] The forward path passes through Right now , identify and estimate the target agent associated with the group. = 1, the i-th target agent belongs to the group of the j-th target agent; when = 0, it does not belong to. This function involves calculating the affinity between the target agents, and then according to a certain threshold ( ) to determine whether they belong to the same group.

[0061] remember is the forward propagation function: ; During back propagation, due to is not differentiable and its gradient cannot be calculated directly. Therefore, a differentiable function is used to approximate The gradient function is as follows: ; ; in For algebra, .

[0062] This differentiable function is defined to update the model parameters using the estimated partial derivatives. A straight-through estimator (STE) is used to estimate the derivatives of the unit step function for gradient updates during backpropagation. The gradient of the backward path estimate ( ): Based on the unit step function, the gradient of group division is estimated through back propagation. If the specific movement loss no longer decreases significantly after a certain number of iterations, the model can be considered to have converged and the specific movement can be stopped. This helps the model learn how to better divide groups based on the affinity between agents.

[0063] In the fourth step, the features of each group are calculated based on all group relationships and all unique motion features, and the features of all groups are integrated into a matrix to obtain the hyperedge feature matrix.

[0064] Specifically, through the formula: Calculate the Characteristics of the group .

[0065] in, Indicates The set of target agents in the group, Indicates Unique motion characteristics of each target agent.

[0066] Step 14: Use the paired relationship transformation model to transform the edge feature matrix and the node feature matrix to obtain a paired node feature matrix.

[0067] The above pairwise node feature matrix is ​​used to describe the interaction between every two target agents in the current scene.

[0068] The pairwise relationship transformation model includes a plurality of pairwise relationship transformation modules connected in sequence. For the pairwise relationship transformation modules other than the first and last pairwise relationship transformation modules, the first input end is connected to the first output end of the previous pairwise relationship transformation module, the second input end is connected to the second output end of the previous pairwise relationship transformation module, the first output end is connected to the first input end of the next pairwise relationship transformation module, and the second output end is connected to the second input end of the next pairwise relationship transformation module.

[0069] The input data of the first input terminal of the pairwise relation transformation model is an edge feature matrix, the input data of the second input terminal of the pairwise relation transformation model is a node feature matrix, and the output data of the output terminal of the pairwise relation transformation model is a pairwise node feature matrix.

[0070] The first input end of the first pairwise relationship transformation module is the first input end of the pairwise relationship transformation model, the second input end of the first pairwise relationship transformation module is the second input end of the pairwise relationship transformation model, and the second output end of the last pairwise relationship transformation module is the output end of the pairwise relationship transformation model. The output data of the first output end of the last pairwise relationship transformation module is not output, that is, the output data does not participate in the calculation of subsequent steps.

[0071] like Figure 3 As shown, the pairwise relation transformation module includes a first relation attention layer, a message function layer, a first additive normalization layer, a second additive normalization layer, a third additive normalization layer, a fourth additive normalization layer, a first feedforward network layer and a second feedforward network layer.

[0072] The first input end of the first relational attention layer, the input end of the message function layer, and the input end of the second additive normalization layer are all the first input end of the pairwise relational transformation module, the second input end of the first relational attention layer and the input end of the first additive normalization layer are all the second input end of the pairwise relational transformation module, the output end of the third additive normalization layer is the first output end of the pairwise relational transformation module, and the output end of the fourth additive normalization layer is the second output end of the pairwise relational transformation module.

[0073] The output end of the first relational attention layer is connected to the input end of the first additive normalization layer, the output end of the first additive normalization layer is connected to the input end of the second feedforward network layer, the output end of the second feedforward network layer is connected to the input end of the fourth additive normalization layer, the input end of the second additive normalization layer is respectively connected to the output end of the message function layer and the output end of the fourth additive normalization layer, the output end of the second additive normalization layer is respectively connected to the input end of the first feedforward network layer and the input end of the third additive normalization layer, and the output end of the first feedforward network layer is connected to the input end of the third additive normalization layer.

[0074] Figure 3 middle is the node feature matrix, is the edge feature matrix, is the pairwise edge feature matrix, is the pairwise node feature matrix, represents the number of pairwise relation transformation modules, Figure 3 What is described is: the pairwise relationship transformation model consists of L Figure 3 The pairwise relation transformation module shown in is composed of , and the input of the pairwise relation transformation model is and , the output is (output by the last pairwise relation transformation module, and the last pairwise relation transformation module also calculates , but the data is not output). For the first pairwise relationship transformation module, the input data of the first input terminal and the second input terminal are the edge feature matrix and the node feature matrix obtained in step 13, and for the other pairwise relationship transformation modules except the first pairwise relationship transformation module, the input data of the first input terminal and the second input terminal are the paired edge feature matrix and the paired node feature matrix obtained after the previous pairwise relationship transformation module performs operation, and for the other pairwise relationship transformation modules except the last pairwise relationship transformation module, the output data of the first output terminal of the pairwise relationship transformation module is the paired edge feature matrix, and the paired edge feature matrix is ​​transmitted to the first input terminal of the next pairwise relationship transformation module, and the output data of the second output terminal is the paired node feature matrix, and the paired node feature matrix is ​​transmitted to the second input terminal of the next pairwise relationship transformation module.

[0075] It should be noted that the above-mentioned first relational attention layer is used to perform self-attention operation on the input data, the message function layer is used to perform ReLU function operation on the input data, the first addition normalization layer, the second addition normalization layer, the third addition normalization layer, and the fourth addition normalization layer are all used to perform addition operation and normalization operation on the input data, and the first feedforward network layer and the second feedforward network layer are both used to perform feedforward network (FNN, Feedforward Neural Network) operation on the input data.

[0076] Relational attention is a key mechanism used to enhance the model’s understanding of relationships between agents. Specifically, relational attention improves the modeling capabilities of individual and group behaviors of agents by integrating edge features into the self-attention calculation of nodes.

[0077] The pairwise relation transformation focuses on the direct relationship between two nodes, representing the interaction or relationship between two agents.

[0078] The input of the pairwise relation transformation model is the node feature matrix and edge feature matrix The first relational attention model processes the pairwise relations between nodes. The final output data is the pairwise node feature matrix , where L is the number of layers of the pairwise relation transformation module. The specific formula is as follows: ; in, Represents operations on pairwise relational transformation models.

[0079] Step 15, using the hyper-relation transformation model to transform the node feature matrix, the hyper-edge feature matrix, and the group occurrence matrix to obtain the final group feature matrix.

[0080] The above-mentioned final group feature matrix is ​​the node feature matrix updated based on the group occurrence matrix and the hyperedge feature matrix.

[0081] The above-mentioned super-relation transformation model includes a plurality of super-relation transformation modules connected in sequence. For the other super-relation transformation modules except the first and the last super-relation transformation modules, the first input end is connected to the first output end of the previous super-relation transformation module, the second input end is connected to the second output end of the previous super-relation transformation module, the first output end is connected to the first input end of the next super-relation transformation module, and the second output end is connected to the second input end of the next super-relation transformation module.

[0082] The input data of the first input terminal of the hyperrelational transformation model is the node feature matrix, the input data of the second input terminal of the hyperrelational transformation model is the hyperedge feature matrix, the input data of the third input terminal of the hyperrelational transformation model is the group occurrence matrix, and the output data of the output terminal of the hyperrelational transformation model is the final group feature matrix.

[0083] The first input end of the first super-relational transformation module is the first input end of the super-relational transformation model, the second input end of the first super-relational transformation module is the second input end of the super-relational transformation model, the third input end of each super-relational transformation module is the third input end of the super-relational transformation model, and the first output end of the last super-relational transformation module is the output end of the super-relational transformation model. The output data of the second output end of the last super-relational transformation module is not output, that is, it does not participate in the calculation of subsequent steps.

[0084] like Figure 4 As shown, the super-relational transformation module includes a second relational attention layer, a fifth additive normalization layer, a sixth additive normalization layer, a seventh additive normalization layer, an eighth additive normalization layer, a third feedforward network layer, a fourth feedforward network layer, and a fifth feedforward network layer.

[0085] The first input end of the second relational attention layer is the first input end of the super-relational transformation module, the second input end of the second relational attention layer, the input end of the third feedforward network layer, and the input end of the sixth additive normalization layer are the second input end of the super-relational transformation module, the third input end of the second relational attention layer is the third input end of the super-relational transformation module, the output end of the seventh additive normalization layer is the first output end of the super-relational transformation module, and the output end of the eighth additive normalization layer is the second output end of the super-relational transformation module.

[0086] The output end of the second relational attention layer is connected to the input end of the fifth additive normalization layer, the output end of the fifth additive normalization layer is respectively connected to the input end of the fourth feedforward network layer and the input end of the seventh additive normalization layer, the output end of the fourth feedforward network layer is connected to the input end of the seventh additive normalization layer, the input end of the sixth additive normalization layer is connected to the output end of the seventh additive normalization layer and the output end of the third feedforward network layer, the output end of the sixth additive normalization layer is respectively connected to the input end of the fifth feedforward network layer and the input end of the eighth additive normalization layer, and the output end of the fifth feedforward network layer is connected to the input end of the eighth additive normalization layer.

[0087] Figure 4 middle is the node feature matrix, G is the population occurrence matrix, is the hyperedge feature matrix, is the final hyperedge feature matrix, is the final group feature matrix, represents the number of super-relation transformation modules, Figure 4 What is described is: The super-relational transformation model consists of L Figure 4 The super-relation transformation module shown in is composed of , and the input of the super-relation transformation model is , G and , the output is (output by the last super-relation transformation module, and the last super-relation transformation module also calculates , but the data is not output). The input data of the first input terminal and the second input terminal of the first super-relation transformation module are the node feature matrix and the hyper-edge feature matrix calculated in step 13, respectively. For other super-relation transformation modules other than the first super-relation transformation module, the input data of the first input terminal and the second input terminal are the hyper-edge feature matrix and the hyper-edge feature matrix calculated by the previous super-relation transformation module, respectively. For other super-relation transformation modules except the last super-relation transformation module, the output data of the second output terminal of the super-relation transformation module is the hyper-edge feature matrix, and the hyper-edge feature matrix is ​​transmitted to the second input terminal of the next super-relation transformation module, and the output data of the first output terminal is the hyper-edge feature matrix, and the hyper-edge feature matrix is ​​transmitted to the first input terminal of the next super-relation transformation module; the input data of the third input terminals of all super-relation transformation modules are the group occurrence matrix.

[0088] It should be noted that the above-mentioned second relational attention layer is used to perform self-attention operations on the input data, the fifth additive normalization layer, the sixth additive normalization layer, the seventh additive normalization layer, and the eighth additive normalization layer are all used to perform addition operations and normalization operations on the input data, and the third feedforward network layer, the fourth feedforward network layer, and the fifth feedforward network layer are all used to perform feedforward network operations on the input data.

[0089] The super-relationship transformation model inputs the population occurrence matrix G and the node feature matrix and the hyperedge feature matrix The group occurrence matrix G is used to indicate the group relationship between agents and affects the attention calculation in the super-relationship transformation model. The final output data is the final group feature matrix , the specific formula is: in Represents the processing of the hyperrelational transformation model.

[0090] Step 16, based on the paired node feature matrix and the final group feature matrix, use the trajectory prediction model to predict each target intelligent agent in the current scene to obtain the predicted trajectory of each target intelligent agent in the current scene.

[0091] The above predicted trajectory is the trajectory point position of the target agent at the next moment of the current moment.

[0092] In some embodiments of the present application, the step of predicting each target agent in the current scene using a trajectory prediction model based on the paired node feature matrix and the final group feature matrix to obtain a predicted trajectory of each target agent in the current scene includes: In the first step, the temporal and spatial features of all target agents in the current scene are calculated based on the paired node feature matrix and the final group feature matrix.

[0093] Specifically, for each node in the spatial graph, all elements related to the node are taken out from the paired node feature matrix and the final group feature matrix, and all elements are integrated into a vector; the temporal attention mechanism and graph convolution are used to operate on each vector, and all operation results are standardized to obtain the temporal characteristics of the target intelligent agent corresponding to the node; the spatial attention mechanism and graph convolution are used to operate on each vector, and all operation results are standardized to obtain the spatial characteristics of the target intelligent agent corresponding to the node.

[0094] For example, taking the time series feature as an example, the calculation expression is: in, Represents the time series characteristics, represents the normalization function, including nonlinear transformation and batch normalization, Indicates The graph convolution operation results corresponding to the nodes, represents the weight matrix, Indicates The query vector of nodes, Indicates The key vector of the nodes, Indicates The value vector of the nodes, Represents the dimension, Indicates The key vector of the nodes, Indicates The value vector of the node, The node is The neighbor node of the node or The node itself, Indicates The vector of nodes, , , represents the linear transformation function, represents the first A vector of nodes.

[0095] In the second step, the trajectory prediction model is used to calculate the temporal and spatial features to obtain the predicted trajectory of each target agent in the current scene.

[0096] like Figure 5 As shown, the trajectory prediction model includes a fusion module, a first fully connected module, a second fully connected module, a third fully connected module, a fourth fully connected module, a Gaussian mixture module, a long short-term memory module, a fifth fully connected module, and an addition module.

[0097] The input end of the fusion module is the input end of the trajectory prediction model, and the output end of the addition module is the output end of the trajectory prediction model.

[0098] The output end of the fusion module is connected to the input end of the first fully connected module, the input end of the second fully connected module, the input end of the third fully connected module, and the input end of the fourth fully connected module respectively; the output end of the first fully connected module, the output end of the second fully connected module, the output end of the third fully connected module, and the output end of the fourth fully connected module are all connected to the input end of the Gaussian mixture module; the output end of the Gaussian mixture module is connected to the input end of the long short-term memory module; the output end of the long short-term memory module is connected to the input end of the fifth fully connected module; and the output end of the fifth fully connected module is connected to the input end of the addition module.

[0099] It should be noted that the above-mentioned fusion module is used to fuse the temporal features and the spatial features to obtain the fused features, the first fully connected module is used to calculate the weight of the Gaussian kernel according to the input data, the second fully connected module is used to calculate the mean of the Gaussian kernel according to the input data, the third fully connected module is used to calculate the correlation coefficient of the Gaussian kernel according to the input data, the fourth fully connected module is used to calculate the original value of the correlation coefficient of the Gaussian kernel according to the input data, the Gaussian mixture module is used to perform Gaussian mixture operations on the input data to obtain the predicted destination, the long short-term memory module is used to perform long short-term memory network operations according to the predicted destination, the fused features, and the predicted trajectory point position at the current moment to obtain the hidden state, the fifth fully connected module is used to calculate the position change according to the hidden state, and the addition module is used to add the position change to the actual trajectory point position at the current moment to obtain the predicted trajectory point position at the next moment of the current moment.

[0100] Exemplarily, the expression of the above fusion module is: in, represents the fusion feature, represents the adaptive gating mechanism, Represents the time series characteristics, Represents spatial features.

[0101] The expressions of the first fully connected module, the second fully connected module, the third fully connected module, and the fourth fully connected module are respectively: ; ; ; ; in, Indicates The first to the second target agent The weights of the Gaussian kernels, Indicates The first to the second target agent The mean of the Gaussian kernels, Indicates The first to the second target agent The correlation coefficient of the Gaussian kernel is Indicates The first to the second target agent The original value of the correlation coefficient of the Gaussian kernel. Represents the output data of the first fully connected layer, i.e. The first to the second target agent The weights of the Gaussian kernel, is the function used for calculation in the first fully connected layer, represents the output data of the second fully connected layer, is the function used for calculation in the second fully connected layer, i.e. The first to the second target agent The mean of the Gaussian kernels, Indicates The first to the second target agent The initial correlation coefficient of the Gaussian kernel is calculated using The function converts the value of the initial correlation coefficient into a certain numerical range to obtain the correlation coefficient. is the function used for calculation in the third fully connected layer, Indicates The first to the second target agent The original value of the initial correlation coefficient of the Gaussian kernel is used The function converts the original value of the initial correlation coefficient into a certain numerical range to obtain the original value of the correlation coefficient. is the function used for calculation in the fourth fully connected layer.

[0102] The expression of the above Gaussian mixture module is: in, Indicates The probability density function of the predicted destination of the target agent, Indicates The target agent is in The covariance matrix of the Gaussian kernel, Indicates The target agent is in The weights in the Gaussian kernel, Indicates The target agent The mean of the Gaussian kernels.

[0103] The expression of the above long short-term memory network is: in, Indicates The hidden state of the target agent, represents the predicted destination, Indicates the predicted trajectory point position at the current moment (which can be calculated using a convolutional neural network, etc., or the previous moment before the current moment can be used as the current moment and obtained using the method of the present application).

[0104] The expression of the fifth fully connected module is: in, represents the position change, Represents a function.

[0105] The expression of the above addition module is: in, Indicates The predicted trajectory point position of the target agent at the next moment of the current moment, Indicates The trajectory point position of the target agent at the current moment.

[0106] It should be noted that if it is necessary to generate the trajectory point positions at multiple moments in the future, the next moment after the current moment is taken as the current moment, and the above-mentioned predicted trajectory point position is taken as the trajectory point position at the current moment, and returned to the long short-term memory network for cyclic calculation.

[0107] It is worth mentioning that determining the unique motion characteristics of the target intelligent agent in the current scene based on the trajectories of multiple scenes can obtain the trajectory characteristics of the agent in different scenes, improve the information content and accuracy of the unique motion characteristics, and analyze the group patterns of the target intelligent agent. It can capture the target intelligent agent's internal collaborative behavior and the interaction between groups. Trajectory prediction based on the paired node feature matrix and the final group feature matrix can effectively improve the accuracy of trajectory prediction.

[0108] In addition, the advantages of this application are: 1. By building unique motion features for each agent, and updating and transferring these features in different scenarios, it is possible to better adapt to the individual differences of different agents and changes in scenarios. For example, under different traffic flows and road conditions, the motion features of the agent may be different. By updating and transferring the features of a specific agent, its behavior in different scenarios can be more accurately predicted.

[0109] 2. By introducing the pairwise relation transformation model and the hyper-relation transformation model, the group relationship between agents can be modeled more complexly and flexibly. By calculating the affinity between agents and constructing hyper-edges, the model can capture the collaborative behavior within the group of agents and the interaction between groups, thereby more accurately predicting the movement trajectory of group agents.

[0110] The following is an exemplary description of the trajectory prediction device based on scene change provided by the present application.

[0111] like Figure 6As shown, the embodiment of the present application provides a trajectory prediction device based on scene change, and the trajectory prediction device based on scene change 600 includes: The determination module 601 is used to determine the unique motion feature of each target agent in the current scene according to the trajectories of all target agents in T scenes; the unique motion feature is used to describe the trajectory feature of the target agent, the Tth scene is the current scene, and the trajectory includes the trajectory points of the target agent at each moment corresponding to the scene; A construction module 602 is used to construct a spatial graph based on the unique motion features of all target agents in the current scene; multiple nodes of the spatial graph correspond to the unique motion features of multiple target agents one by one, and the edges between the nodes are the neighboring relationships between the corresponding two target agents; Acquisition module 603, used to acquire the node feature matrix, edge feature matrix, group occurrence matrix and hyperedge feature matrix of all target agents in the current scene according to the spatial graph; the node feature matrix is ​​used to describe the unique motion features corresponding to all nodes in the spatial graph, the edge feature matrix is ​​used to describe all edges in the spatial graph, the group occurrence matrix is ​​used to describe the group to which each target agent in the current scene belongs, and the hyperedge feature matrix is ​​used to describe the features of each group; A first transformation module 604 is used to transform the edge feature matrix and the node feature matrix using a paired relationship transformation model to obtain a paired node feature matrix; The second transformation module 605 is used to transform the node feature matrix, the hyperedge feature matrix, and the group occurrence matrix using the hyperrelation transformation model to obtain a final group feature matrix; The prediction module 606 is used to predict each target intelligent agent in the current scene based on the paired node feature matrix and the final group feature matrix using the trajectory prediction model to obtain the predicted trajectory of each target intelligent agent in the current scene.

[0112] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0113] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0114] like Figure 7 As shown, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 7 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0115] Specifically, when the processor D100 executes the computer program D102, it can obtain the trajectory characteristics of the agent in different scenes by determining the unique motion characteristics of the target agent in the current scene based on the trajectories of multiple scenes, improve the amount of information and accuracy of the unique motion characteristics, analyze the group pattern of the target agent, capture the target agent's internal collaborative behavior and the interaction between groups, and perform trajectory prediction based on the paired node feature matrix and the final group feature matrix, which can effectively improve the accuracy of trajectory prediction.

[0116] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0117] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0118] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0119] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0120] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the scene-based trajectory prediction method device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, disk or optical disk. Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, disk or optical disk.

[0121] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0122] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0123] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A trajectory prediction method based on scene change, characterized in that: include: Determine the unique motion characteristics of each target agent in the current scene based on the trajectories of all target agents in the T scenes; The unique motion feature is used to describe the trajectory feature of the target intelligent agent, the Tth scene is the current scene, and the trajectory includes the trajectory points of the target intelligent agent at each moment corresponding to the scene; Build a spatial graph based on the unique motion features of all target agents in the current scene; The multiple nodes of the spatial graph correspond one-to-one to the unique motion features of the multiple target intelligent agents, and the edges between the nodes are the neighboring relationships between the corresponding two target intelligent agents; According to the spatial graph, a node feature matrix, an edge feature matrix, a group occurrence matrix and a hyperedge feature matrix of all target agents in the current scene are obtained; the node feature matrix is ​​used to describe the unique motion features corresponding to all nodes in the spatial graph, the edge feature matrix is ​​used to describe all edges in the spatial graph, the group occurrence matrix is ​​used to describe the group to which each target agent in the current scene belongs, and the hyperedge feature matrix is ​​used to describe the characteristics of each group; The edge feature matrix and the node feature matrix are transformed by using a pairwise relationship transformation model to obtain a pairwise node feature matrix; The node feature matrix, the hyper-edge feature matrix, and the group occurrence matrix are transformed using a hyper-relation transformation model to obtain a final group feature matrix; Based on the paired node feature matrix and the final group feature matrix, a trajectory prediction model is used to predict each target intelligent agent in the current scene to obtain a predicted trajectory of each target intelligent agent in the current scene.

2. The trajectory prediction method according to claim 1, characterized in that: The basis T The trajectories of all target agents in the scene are determined to determine the unique motion characteristics of each target agent in the current scene, including: Calculate the average of unique motion features for each agent class in the current scene based on all trajectories of all other scenes except the current scene; According to the average value of the unique motion features of all agent categories and the trajectories of all target agents in the current scene, the unique motion features of each target agent in the current scene are obtained.

3. The trajectory prediction method according to claim 2, characterized in that: The calculating the unique motion feature average value of the current scene according to all trajectories of all other scenes except the current scene comprises: By formula: ； Calculate the In the scene The average of the unique motion features of the agent classes ; in, Indicated in The scene belongs to The number of target agents in each agent class, Indicated in The first scene moment, based on the of the agent class The unique motion features calculated from the trajectory points of the target agent, Indicates the last moment of each scene, , represents the number of agent categories, , Indicates the current scene. hour, Indicates In the scene The average of unique motion features for each agent class.

4. The trajectory prediction method according to claim 1, characterized in that: The spatial graph is constructed based on the unique motion features of all target agents in the current scene, including: Calculate the distance between the trajectories of every two target agents in the current scene. If the distance between the two target agents is less than a distance threshold, it is considered that the two target agents have a neighboring relationship. A corresponding node is generated for the unique motion feature of each target agent in the current scene. If there is a proximity relationship between the two target agents corresponding to each two nodes, an edge between the two nodes is generated to obtain a spatial graph.

5. The trajectory prediction method according to claim 4, characterized in that: The step of obtaining the node feature matrix, edge feature matrix, group occurrence matrix and super edge feature matrix of all target agents in the current scene according to the spatial graph includes: For each two nodes in the spatial graph having a connecting edge, two unique motion features corresponding to the two nodes are spliced ​​into a pair of edge features; Integrate all paired edge features into a matrix to obtain the node feature matrix; constructing an edge feature matrix according to all edges in the spatial graph; Calculate the group relationship between the target agents corresponding to every two nodes in the spatial graph, obtain multiple groups according to all group relationships, integrate the groups to which all target agents belong into a matrix, and obtain a group occurrence matrix; The features of each group are calculated based on all group relationships and all unique motion features, and the features of all groups are integrated into a matrix to obtain the hyperedge feature matrix.

6. The trajectory prediction method according to claim 5, characterized in that: The calculating the group relationship between target agents corresponding to every two nodes in the spatial graph includes: By formula: ； Calculate the The target agent and Group relations between target agents ; in, Indicates the The target agent and The affinity between target agents, represents the threshold value, Indicates the The unique motion characteristics of the target agent, Indicates the The unique motion characteristics of the target agent, , , Represents the numbered set of target agents in the current scene, represents a unit step function.

7. The trajectory prediction method according to claim 1, characterized in that: The pairwise relationship transformation model includes a plurality of pairwise relationship transformation modules connected in sequence; The input data of the first input terminal of the pairwise relationship transformation model is an edge feature matrix, the input data of the second input terminal of the pairwise relationship transformation model is a node feature matrix, and the output data of the output terminal of the pairwise relationship transformation model is a pairwise node feature matrix; The first input end of the first pairwise relationship transformation module is the first input end of the pairwise relationship transformation model, the second input end of the first pairwise relationship transformation module is the second input end of the pairwise relationship transformation model, and the second output end of the last pairwise relationship transformation module is the output end of the pairwise relationship transformation model; The pairwise relation transformation module includes a first relation attention layer, a message function layer, a first additive normalization layer, a second additive normalization layer, a third additive normalization layer, a fourth additive normalization layer, a first feedforward network layer and a second feedforward network layer; The first input end of the first relational attention layer, the input end of the message function layer, and the input end of the second additive normalization layer are all the first input end of the pairwise relational transformation module, the second input end of the first relational attention layer and the input end of the first additive normalization layer are all the second input end of the pairwise relational transformation module, the output end of the third additive normalization layer is the first output end of the pairwise relational transformation module, and the output end of the fourth additive normalization layer is the second output end of the pairwise relational transformation module; The output end of the first relational attention layer is connected to the input end of the first additive normalization layer, the output end of the first additive normalization layer is connected to the input end of the second feedforward network layer, the output end of the second feedforward network layer is connected to the input end of the fourth additive normalization layer, the input end of the second additive normalization layer is respectively connected to the output end of the message function layer and the output end of the fourth additive normalization layer, the output end of the second additive normalization layer is respectively connected to the input end of the first feedforward network layer and the input end of the third additive normalization layer, and the output end of the first feedforward network layer is connected to the input end of the third additive normalization layer.

8. The trajectory prediction method according to claim 1, characterized in that: The super-relation transformation model includes a plurality of super-relation transformation modules connected in sequence; The input data of the first input terminal of the super relation transformation model is a node feature matrix, the input data of the second input terminal of the super relation transformation model is a hyper-edge feature matrix, the input data of the third input terminal of the super relation transformation model is a group occurrence matrix, and the output data of the output terminal of the super relation transformation model is a final group feature matrix; The first input end of the first super-relation transformation module is the first input end of the super-relation transformation model, the second input end of the first super-relation transformation module is the second input end of the super-relation transformation model, the third input end of each super-relation transformation module is the third input end of the super-relation transformation model, and the first output end of the last super-relation transformation module is the output end of the super-relation transformation model; The super-relation transformation module includes a second relation attention layer, a fifth additive normalization layer, a sixth additive normalization layer, a seventh additive normalization layer, an eighth additive normalization layer, a third feedforward network layer, a fourth feedforward network layer, and a fifth feedforward network layer; The first input end of the second relational attention layer is the first input end of the super-relational transformation module, the second input end of the second relational attention layer, the input end of the third feedforward network layer, and the input end of the sixth additive normalization layer are the second input end of the super-relational transformation module, the third input end of the second relational attention layer is the third input end of the super-relational transformation module, the output end of the seventh additive normalization layer is the first output end of the super-relational transformation module, and the output end of the eighth additive normalization layer is the second output end of the super-relational transformation module; The output end of the second relational attention layer is connected to the input end of the fifth additive normalization layer, the output end of the fifth additive normalization layer is respectively connected to the input end of the fourth feedforward network layer and the input end of the seventh additive normalization layer, the output end of the fourth feedforward network layer is connected to the input end of the seventh additive normalization layer, the input end of the sixth additive normalization layer is connected to the output end of the seventh additive normalization layer and the output end of the third feedforward network layer, the output end of the sixth additive normalization layer is respectively connected to the input end of the fifth feedforward network layer and the input end of the eighth additive normalization layer, and the output end of the fifth feedforward network layer is connected to the input end of the eighth additive normalization layer.

9. The trajectory prediction method according to claim 1, characterized in that: The method of predicting each target agent in the current scene using a trajectory prediction model based on the paired node feature matrix and the final group feature matrix to obtain a predicted trajectory of each target agent in the current scene includes: Calculate the temporal features and spatial features of all target agents in the current scene according to the paired node feature matrix and the final group feature matrix; The temporal features and spatial features are calculated using a trajectory prediction model to obtain a predicted trajectory of each of the target intelligent agents in the current scene.

10. The trajectory prediction method according to claim 9, characterized in that: The trajectory prediction model includes a fusion module, a first fully connected module, a second fully connected module, a third fully connected module, a fourth fully connected module, a Gaussian mixture module, a long short-term memory module, a fifth fully connected module, and an addition module; The input end of the fusion module is the input end of the trajectory prediction model, and the output end of the addition module is the output end of the trajectory prediction model; The output end of the fusion module is respectively connected to the input end of the first fully connected module, the input end of the second fully connected module, the input end of the third fully connected module, and the input end of the fourth fully connected module; the output end of the first fully connected module, the output end of the second fully connected module, the output end of the third fully connected module, and the output end of the fourth fully connected module are all connected to the input end of the Gaussian mixture module; the output end of the Gaussian mixture module is connected to the input end of the long short-term memory module; the output end of the long short-term memory module is connected to the input end of the fifth fully connected module; and the output end of the fifth fully connected module is connected to the input end of the addition module.

Images