Track determination method and device and electronic equipment

Abstract

The invention discloses a trajectory determination method and device and electronic equipment. The method comprises the following steps: acquiring a current frame sensing result of trajectory detection points comprising N first targets; according to a matching result of the current frame sensing result and a first target trajectory set comprising trajectory information of a second target, modifying the first target trajectory set to obtain a second target trajectory set comprising trajectory information of Q third targets; according to the target distance and the course angle deviation between every two third targets, determining a relationship between the two corresponding third targets, and according to the relationship, determining target information of the two corresponding third targets; and according to the second target trajectory set and the target information of the Q third targets, determining trajectory prediction points of the Q third targets in the next frame, thereby comprehensively considering the relationship between the targets and the influence of the relationship between the targets and the scene on the trajectory points of the predicted targets in the next frame. And the accuracy of the trajectory prediction point of the target is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of monitoring technology, and in particular to a trajectory determination method, device and electronic equipment. Background Art

[0002] With the rapid development of science and technology, multi-target trajectory determination has also developed rapidly. Existing multi-target trajectory determination methods can be divided into multi-target trajectory determination in two-dimensional (English: Two Dimensions, abbreviated as 2D) image plane and multi-target trajectory determination in three-dimensional (English: Three Dimensions, abbreviated as 3D) space according to different perception results. Compared with multi-target trajectory determination in 2D image plane, multi-target trajectory determination in 3D space can obtain more realistic scale, more accurate target posture, more convenient for the use of scene information, and can effectively reduce the influence of perspective on trajectory determination. Therefore, multi-target trajectory determination in 3D space has been widely used in many fields.

[0003] Among them, the multi-target trajectory determination in 3D space includes the multi-target trajectory determination from a bird's eye view (English: Bird's Eye View, abbreviated as BEV).

[0004] The BEV multi-target trajectory determination method usually predicts the trajectory prediction points of the current frame based on the trajectory detection points of the previous frame of BEV; then, the trajectory prediction points of the current frame are matched with the trajectory detection points of the current frame of BEV to obtain a matching value. If the matching value is greater than the set threshold, it is determined that the target of the trajectory prediction point corresponding to the matching value is the same target as the target corresponding to the trajectory detection point of the current frame, thereby achieving the trajectory determination of the target.

[0005] However, when determining the target trajectory, only the motion characteristics of a single target are focused on, and the influence of other targets on the predicted target is ignored, thereby lacking the use of global information, resulting in low prediction accuracy of the trajectory prediction point corresponding to the focused target. Summary of the invention

[0006] The present application provides a trajectory determination method, device and electronic device to solve the problem that the prior art ignores the influence of other targets on the target when determining the target trajectory, resulting in low prediction accuracy. The specific implementation scheme is as follows:

[0007] In a first aspect, the present application provides a trajectory determination method, the method comprising:

[0008] Obtaining a current frame perception result including N trajectory detection points of the first target; wherein N is a positive integer;

[0009] According to the matching result between the current frame perception result and the first target trajectory set including the trajectory information of the second target, the first target trajectory set is modified to obtain a second target trajectory set including the trajectory information of Q third targets; wherein Q is a positive integer;

[0010] Determine the relationship between the corresponding two third targets according to the target distance and the heading angle deviation between each two third targets, and determine the target information of the corresponding two third targets according to the relationship;

[0011] According to the second target trajectory set and the target information of the Q third targets, trajectory prediction points of the Q third targets in the next frame are determined.

[0012] In the above manner, according to the current frame perception result including the trajectory detection points of N first targets, and the matching result between the first target trajectory set including the trajectory information of the second target, the first target trajectory set is modified so that the second target trajectory set obtained contains the information of the current frame perception result, and then according to the target distance and heading angle deviation between each two third targets in the second target trajectory set, the relationship between the corresponding two third targets is determined, thereby determining the target information of the corresponding two third targets, so that when the trajectory prediction point is obtained based on the target information, the influence of other third targets on the predicted third target can be considered, and then according to the target information and the second target trajectory set, the trajectory point of the third target in the next frame is predicted, so that the obtained trajectory prediction point takes into account the influence of other third targets on the predicted third target, thereby effectively utilizing the information between other third targets and the predicted third target, so that the accuracy of the obtained trajectory prediction point is greatly improved. In addition, since the heading angle in the heading angle deviation is a feature in the 3D scene, based on the above application embodiment, the problem of ignoring the influence of other targets on the target when determining the target trajectory in the 3D scene, resulting in low prediction accuracy, is solved.

[0013] In a possible implementation manner, obtaining a current frame perception result including N trajectory detection points of the first target includes:

[0014] Acquire a current frame image captured by a sensor; wherein the current frame image includes N first targets;

[0015] The current frame image is input into a bird's-eye view (BEV) perception algorithm to obtain the current frame perception result including N trajectory detection points of the first target.

[0016] In the above manner, based on the BEV perception algorithm, the current frame image including N first targets captured by the sensor is processed to obtain the current frame perception result including the trajectory detection points of the N first targets, thereby providing accurate data of the current frame for the prediction of the trajectory points of the target in the next frame. In addition, obtaining the current frame perception result by means of the BEV perception algorithm provides an intuitive and fusion-friendly way to represent the surrounding scene of the target, which is conducive to obtaining the environmental information of the target (such as road information).

[0017] In a possible implementation, the matching result between the current frame perception result and the first target trajectory set including the trajectory information of the second target is modified according to the first target trajectory set to obtain the second target trajectory set including the trajectory information of Q third targets, including:

[0018] Matching the current frame perception result with the first target trajectory set, determining the second target corresponding to each of the N first targets, and the first trajectory state corresponding to each of the N first targets and the second trajectory state corresponding to each of the second targets in the first target trajectory set; wherein the first trajectory state includes an initialization state and an update state; and the second trajectory state includes an update state, a loss state, and a deletion state;

[0019] The first target trajectory set is modified according to each of the first trajectory state and the second trajectory state and the current frame perception result to obtain the second target trajectory set including the trajectory information of the Q third targets.

[0020] In the above manner, the current frame perception result is matched with the first target trajectory set, and the second target corresponding to the first target in the first target trajectory set is determined, so that the trajectory information of the corresponding first target can be determined according to the second target, thereby providing historical data for the prediction of the trajectory point of the target in the next frame. At the same time, according to the matching of the current frame perception result with the first target trajectory set, the first trajectory state corresponding to each first target and the second trajectory state corresponding to each second target are determined, so that according to the first trajectory state and the second trajectory state, it is determined whether to directly add the information of the first target in the current frame perception result to the first target trajectory set or update it to the first target trajectory set, and it is determined whether to perform a deletion operation or an update operation on the second target in the first target trajectory set, so that after performing modification operations such as updating and deleting on the first target trajectory set, the second target trajectory set is obtained, so that the trajectory detection points in the current frame perception result and the trajectory information in the first target trajectory set are integrated, so that the information in the second target trajectory set can be directly used in the subsequent prediction of trajectory points.

[0021] In a possible implementation, matching the current frame perception result with the first target trajectory set to determine the second target corresponding to each of the N first targets, and the first trajectory state corresponding to each of the N first targets and the second trajectory state corresponding to each of the second targets in the first target trajectory set include:

[0022] Determine, according to the matching operation performed on each of the first targets, the second targets corresponding to each of the N first targets, and determine that the first trajectory state of the first targets among the N first targets that have not successfully matched the first target trajectory set is the initialization state;

[0023] Determine that the first trajectory state of the first target that successfully matches the first target trajectory set among the N first targets is the updated state, and the second trajectory state of the corresponding second target is the updated state;

[0024] Determine that the second trajectory state of the second target in the first target trajectory set that is not successfully matched with the N first targets is the lost state;

[0025] If the second target corresponding to the lost state in the first target trajectory set meets the deletion condition, it is determined that the second trajectory state of the corresponding second target is the deleted state.

[0026] In the above manner, according to the matching operation performed on each first target, the second target corresponding to each first target is determined, and the first trajectory state of the first target and the second trajectory state of the second target are determined, so that the specific modification operation of the first target trajectory set can be determined later according to the first trajectory state and the second trajectory state.

[0027] In a possible implementation manner, the matching operation is specifically:

[0028] Calculating a trajectory difference value between a trajectory detection point of the first target in the perception result of the current frame and each trajectory prediction point of the second target in the current frame in the first target trajectory set;

[0029] Among the calculated trajectory difference values, determining a minimum trajectory difference value;

[0030] Determining whether the minimum trajectory difference value is less than a trajectory difference threshold;

[0031] If so, determining that the target corresponding to the minimum trajectory difference value is the second target of the first target in the target trajectory set, and determining that the first target successfully matches the first target trajectory set;

[0032] If not, it is determined that the first target fails to match the first target trajectory set.

[0033] Through the above method, after calculating the trajectory difference value between the trajectory detection point of the first target in the perception result of the current frame and the trajectory prediction point of each second target in the current frame, it is determined whether the first target is matched successfully based on the comparison result of the minimum trajectory difference value in each trajectory difference value and the trajectory difference threshold, thereby providing a basis for determining the first trajectory state of the first target and the second trajectory state of the second target. In addition, when determining the matching result of the first target, the above method takes into account each second target in the first target trajectory set, so that the matching result is comprehensive. At the same time, when determining the matching result of the first target, the above method, after considering each second target in the first target trajectory set, also compares the minimum trajectory difference value with the trajectory difference threshold, thereby taking into account the situation that the first target is not in the first target trajectory set, so that the matching result of the first target is more accurate.

[0034] In a possible implementation, modifying the first target trajectory set according to each of the first trajectory state and the second trajectory state and the current frame perception result to obtain the second target trajectory set including the trajectory information of Q third targets includes:

[0035] For each of the first targets and each of the second targets, the following modification operations are performed:

[0036] If the first trajectory state of the first target is the initialization state, adding the corresponding trajectory detection point of the first target in the current frame perception result to the first target trajectory set;

[0037] If the first trajectory state of the first target is the update state, correcting the corresponding trajectory detection point of the first target in the current frame perception result, and updating the corrected trajectory correction point to the corresponding position in the first target trajectory set;

[0038] If the second trajectory state of the second target is the lost state, determining whether to modify the lost state to the deleted state according to a deletion condition;

[0039] If the second trajectory state of the second target is the deletion state, deleting the corresponding second target from the first target trajectory set;

[0040] Until each of the first targets and each of the second targets completes the modification operation, a second target trajectory set including trajectory information of Q third targets is obtained.

[0041] Through the above method, according to the various processing methods for the first target and the second target in the modification operation, the trajectory detection points in the current frame perception result are merged with the first target trajectory set, thereby realizing the modification of the first target trajectory set, so that the information in the obtained second target trajectory set includes both the information of the current frame perception result and the historical trajectory information of the target, thereby making the information in the second target trajectory set the latest and most accurate trajectory information. In addition, in the modification operation, when the first trajectory state of the first target is the update state, the corresponding trajectory detection point is first corrected, and then the corrected trajectory correction point is updated to the corresponding position in the first target trajectory set, so as to remove the noise in the trajectory detection point and avoid the influence of the noise on the trajectory point, so that the obtained trajectory correction point is more in line with the actual situation, that is, the data of the trajectory point is more accurate, which is conducive to further improving the accuracy of the trajectory prediction point.

[0042] In a possible implementation manner, determining the relationship between two corresponding third targets according to the target distance and the heading angle deviation between each two third targets includes:

[0043] For different third targets that are not on the lane, determine the relationship between the corresponding two third targets according to the first target distance and the heading angle deviation between each two third targets; wherein the relationship is any one of a predecessor relationship, a successor relationship, a left successor relationship, a right successor relationship, and a non-previous, non-left, and non-left-right successor relationship; and

[0044] For different third targets that are all on the lane, the relationship between the corresponding two third targets is determined according to the second target distance between every two third targets.

[0045] Through the above method, according to the first target distance and heading angle deviation between every two third targets that are not on the lane and according to the second target distance between every two third targets that are on the lane, the relationship between the corresponding two third targets (such as the predecessor relationship, the successor relationship, the left successor relationship, the right successor relationship, and the non-previous, subsequent, left and right successor relationship) is accurately determined, so that the target information of the corresponding two third targets can be accurately determined.

[0046] In addition, the lane or reference line where the third target is located is comprehensively considered to determine the relationship between the third target and other third targets, thereby taking into account the influence of other third targets and the lane or reference line on the third target, which is conducive to taking into account the global information when subsequently predicting the trajectory prediction point of the third target, thereby further increasing the prediction accuracy of the predicted trajectory point.

[0047] In a possible implementation, determining trajectory prediction points of the Q third targets in the next frame according to the second target trajectory set and the target information of the Q third targets includes:

[0048] Determine, according to the second target trajectory set, historical trajectory information corresponding to each of the Q third targets;

[0049] Determine, according to the target information and the historical trajectory information corresponding to each of the Q third targets, a behavior mode corresponding to each of the Q third targets; wherein the behavior mode is one of a free movement mode, a following mode, a lane keeping mode, a lane changing mode, and a turning mode;

[0050] According to the behavior patterns corresponding to the Q third targets, the trajectory prediction points of the Q third targets in the next frame are obtained.

[0051] Through the above method, since the second target trajectory set contains the trajectory information of the third target in the current frame and before the current frame, the historical trajectory information of each third target can be determined according to the second target trajectory set; then, the behavior pattern corresponding to each third target is determined according to the target information and historical trajectory information corresponding to each third target, so that the determination of the behavior pattern of the third target comprehensively considers the historical trajectory information of the corresponding third target and the influence of other third targets on its driving process, so that the determination of the behavior pattern is more accurate; then, the trajectory prediction point of the corresponding third target in the next frame is determined according to the behavior pattern, so as to adjust the determination method of the trajectory prediction point according to the behavior pattern, so that the determination of the trajectory prediction point is more flexible and accurate.

[0052] In a possible implementation, determining the behavior mode corresponding to each of the Q third targets according to the target information corresponding to each of the Q third targets and the historical trajectory information includes:

[0053] For each of the third targets, according to the corresponding target information and the historical trajectory information, if it is determined that the corresponding third target does not have a preceding target, a succeeding target, a left succeeding target, and a right succeeding target, and the corresponding third target is not on a lane, then determining that the behavior mode of the corresponding third target is the automatic motion mode;

[0054] If it is determined that the corresponding third target has the predecessor target and / or the successor target, then determining that the behavior mode of the corresponding third target is the following mode;

[0055] If it is determined that the current lane of the corresponding third target is the same lane as the lane of the previous moment, then determining that the behavior mode of the corresponding third target is the lane keeping mode;

[0056] If it is determined that the lane currently located by the corresponding third target is not the same lane as the lane located at the last moment, determining that the behavior mode of the corresponding third target is the lane changing mode;

[0057] If it is determined that the corresponding third target is traveling along the turning reference line, then the behavior mode of the corresponding third target is determined to be the turning mode.

[0058] Through the above method, based on the target information and historical trajectory information of the third target, it is determined whether the third target has a predecessor target, a successor target, a left successor target, and a right successor target, and it is determined whether the third target is on the lane, whether the current lane is the same lane as the lane at the previous moment, and whether it is traveling along the turning reference line, thereby determining the behavior pattern corresponding to the third target, so that the determination of the behavior pattern comprehensively considers the relationship between the third target and the third target, and the relationship between the third target and the road, thereby making the determination of the behavior pattern more accurate, which is conducive to further improving the accuracy of the trajectory prediction point.

[0059] In a possible implementation, obtaining, according to the behavior patterns corresponding to the Q third targets, respective trajectory prediction points of the Q third targets in the next frame includes:

[0060] For each of the third targets, the following prediction operations are performed:

[0061] If the behavior mode of the third target is the free motion mode, determining the speed and acceleration of the third target based on the historical trajectory information of the third target, and obtaining a trajectory prediction point of the third target in the next frame according to the speed and the acceleration;

[0062] If the behavior mode of the third target is not the free motion mode, encoding the environment information and the historical trajectory information of the third target to obtain first coded information corresponding to the historical trajectory information and second coded information corresponding to the environment information; fusing the first coded information and the second coded information to obtain fused information; and decoding the fused information to obtain a trajectory prediction point of the third target in the next frame;

[0063] Until each of the third targets completes the prediction operation, the trajectory prediction points of the Q third targets in the next frame are obtained.

[0064] Through the above method, when the behavior mode of the third target is a free motion mode, the corresponding trajectory prediction point of the third target in the next frame is determined directly based on the acceleration and speed determined by the historical trajectory information of the third target, and when the behavior mode of the third target is not a free motion mode, the trajectory prediction point of the third target in the next frame is determined according to the encoding, fusion, decoding and other operations of the historical trajectory information and environmental information of the third target, and the influence of different behavior modes on the trajectory point of the third target in the next frame is taken into consideration, so that the prediction of the trajectory prediction point of the third target is efficient and accurate.

[0065] In a possible implementation manner, if the behavior mode of the third target is the car-following mode, the environmental information is information of the predecessor target and the successor target of the third target;

[0066] If the behavior mode of the third target is any one of the lane keeping mode, the lane changing mode, and the turning mode, the environmental information is information about the lane in which the third target is currently located.

[0067] Through the above method, the content of the environmental information is determined according to the specific behavior pattern of the third target, which provides data for the prediction of the trajectory points of the third target in the next frame. The prediction comprehensively considers the influence of different behavior patterns on the prediction results, thereby considering the influence of the relationship between targets and the relationship between targets and lanes on the prediction results, so that the accuracy of the prediction results is further improved.

[0068] In a second aspect, the present application further provides a trajectory determination device, comprising:

[0069] An acquisition module, used to acquire a current frame perception result including N trajectory detection points of the first target; wherein N is a positive integer;

[0070] A matching module, configured to modify the first target trajectory set including trajectory information of the second target according to a matching result between the current frame perception result and the first target trajectory set, so as to obtain a second target trajectory set including trajectory information of Q third targets; wherein Q is a positive integer;

[0071] A determination module, configured to determine a relationship between two corresponding third targets according to a target distance and a heading angle deviation between each two third targets, and determine target information of the two corresponding third targets according to the relationship;

[0072] A processing module is used to determine trajectory prediction points of the Q third targets in the next frame according to the second target trajectory set and the target information of the Q third targets.

[0073] In a possible implementation, the acquisition module is specifically used to acquire a current frame image taken by a sensor; wherein the current frame image includes N first targets; the current frame image is input into a bird's-eye view BEV perception algorithm to obtain the current frame perception result including the trajectory detection points of the N first targets.

[0074] In a possible implementation, the matching module is specifically used to match the current frame perception result with the first target trajectory set, determine the second target corresponding to each of the N first targets, and the first trajectory state corresponding to each of the N first targets and the second trajectory state corresponding to each of the second targets in the first target trajectory set; wherein the first trajectory state includes an initialization state and an update state; the second trajectory state includes an update state, a loss state, and a deletion state; according to each of the first trajectory state and the second trajectory state and the current frame perception result, modify the first target trajectory set to obtain the second target trajectory set including the trajectory information of Q third targets.

[0075] In a possible implementation, the matching module is specifically configured to determine the second target corresponding to each of the N first targets according to the matching operation performed on each of the first targets, and determine that the first trajectory state of the first target that has not been successfully matched with the first target trajectory set among the N first targets is the initialization state; determine that the first trajectory state of the first target that has successfully matched the first target trajectory set among the N first targets is the update state, and the second trajectory state of the corresponding second target is the update state; determine that the second trajectory state of the second target that has not been successfully matched with the N first targets in the first target trajectory set is the lost state; if the second target corresponding to the lost state in the first target trajectory set meets the deletion condition, determine that the second trajectory state of the corresponding second target is the deleted state.

[0076] In a possible implementation, the matching operation is specifically as follows: calculating a trajectory difference value between a trajectory detection point of the first target in the perception result of the current frame and a trajectory prediction point of each of the second targets in the first target trajectory set in the current frame; determining a minimum trajectory difference value among the calculated trajectory difference values; judging whether the minimum trajectory difference value is less than a trajectory difference threshold; if so, determining that the target corresponding to the minimum trajectory difference value is the second target of the first target in the target trajectory set, and determining that the first target is successfully matched with the first target trajectory set; if not, determining that the first target is unmatched with the first target trajectory set.

[0077] In a possible implementation, the matching module is specifically configured to perform the following modification operation for each of the first targets and each of the second targets: if the first trajectory state of the first target is the initialization state, then the trajectory detection point of the corresponding first target in the current frame perception result is added to the first target trajectory set; if the first trajectory state of the first target is the update state, then the trajectory detection point of the corresponding first target in the current frame perception result is corrected, and the corrected trajectory correction point is updated to the corresponding position in the first target trajectory set; if the second trajectory state of the second target is the lost state, then according to the deletion condition, it is determined whether to modify the lost state to the deleted state; if the second trajectory state of the second target is the deleted state, then the corresponding second target is deleted from the first target trajectory set; until each of the first target and each of the second target completes the modification operation, and the second target trajectory set including the trajectory information of Q third targets is obtained.

[0078] In one possible implementation, the determination module is specifically used to determine, for different third targets that are not on the lane, the relationship between the corresponding two third targets based on the first target distance and the heading angle deviation between each two third targets; wherein the relationship is any one of a predecessor relationship, a successor relationship, a left-successor relationship, a right-successor relationship, and a non-front-back, left-right, and right-successor relationship; and for different third targets that are all on the lane, the relationship between the corresponding two third targets is determined based on the second target distance between each two third targets.

[0079] In a possible implementation, the processing module is specifically used to determine the historical trajectory information corresponding to each of the Q third targets based on the second target trajectory set; determine the behavior mode corresponding to each of the Q third targets based on the target information corresponding to each of the Q third targets and the historical trajectory information; wherein the behavior mode is one of a free movement mode, a following mode, a lane keeping mode, a lane changing mode, and a turning mode; and obtain the trajectory prediction point of each of the Q third targets in the next frame based on the behavior mode corresponding to each of the Q third targets.

[0080] In a possible implementation, the processing module is specifically used to, for each of the third targets, according to the corresponding target information and the historical trajectory information, if it is determined that the corresponding third target does not have a predecessor target, a successor target, a left successor target and a right successor target, and the corresponding third target is not on the lane, then determine that the behavior mode of the corresponding third target is the automatic movement mode; if it is determined that the corresponding third target has the predecessor target and / or the successor target, then determine that the behavior mode of the corresponding third target is the following mode; if it is determined that the lane where the corresponding third target is currently located is the same lane as the lane where it was located at the previous moment, then determine that the behavior mode of the corresponding third target is the lane keeping mode; if it is determined that the lane where the corresponding third target is currently located is not the same lane as the lane where it was located at the previous moment, then determine that the behavior mode of the corresponding third target is the lane changing mode; if it is determined that the corresponding third target is traveling along a turning reference line, then determine that the behavior mode of the corresponding third target is the turning mode.

[0081] In a possible implementation, the processing module is specifically used to perform the following prediction operations for each of the third targets: if the behavior mode of the third target is the free motion mode, then based on the historical trajectory information of the third target, determine the speed and acceleration of the third target, and obtain the trajectory prediction point of the third target in the next frame according to the speed and the acceleration; if the behavior mode of the third target is not the free motion mode, then encode the environmental information and the historical trajectory information of the third target to obtain first encoding information corresponding to the historical trajectory information and second encoding information corresponding to the environmental information; fuse the first encoding information and the second encoding information to obtain fused information; and decode the fused information to obtain the trajectory prediction point of the third target in the next frame; until each of the third targets has completed the prediction operation to obtain the trajectory prediction points of Q third targets in the next frame.

[0082] In one possible implementation, the processing module is specifically used to, if the behavior mode of the third target is the following mode, then the environmental information is the information of the predecessor target and the successor target of the third target; if the behavior mode of the third target is any one of the lane keeping mode, the lane changing mode, and the turning mode, then the environmental information is the information of the lane in which the third target is currently located.

[0083] In a third aspect, the present application provides an electronic device, including:

[0084] Memory, used to store computer programs;

[0085] The processor is used to implement the above-mentioned trajectory determination method steps when executing the computer program stored in the memory.

[0086] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the above-mentioned trajectory determination method steps are implemented.

[0087] For each aspect from the second to the fourth aspect and the technical effects that may be achieved by each aspect, please refer to the above description of the technical effects that can be achieved by the first aspect or various possible schemes in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A schematic diagram of a flow chart of a trajectory determination method provided in an embodiment of the present application;

[0089] Figure 2a The target embodiment of the present application provides Figure 1 ;

[0090] Figure 2b Target schematic diagram 2 provided for the embodiment of the present application;

[0091] Figure 2c The target embodiment of the present application provides Figure 3 ;

[0092] Figure 3 A schematic diagram of an interaction model provided in an embodiment of the present application;

[0093] Figure 4 A schematic diagram of the processing process of the trajectory determination method provided in an embodiment of the present application;

[0094] Figure 5 A schematic diagram of a trajectory determination device provided in an embodiment of the present application;

[0095] Figure 6 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0096] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. It should be noted that in the description of the present application, "multiple" is understood as "at least two". "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A is connected to B, which can represent: A is directly connected to B and A is connected to B through C. In addition, in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0097] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0098] The BEV multi-target trajectory determination method usually predicts the trajectory prediction points of the current frame based on the trajectory detection points of the previous frame of BEV; then, the trajectory prediction points of the current frame are matched with the trajectory detection points of the current frame of BEV to obtain a matching value. If the matching value is greater than the set threshold, the trajectory prediction point corresponding to the matching value is output as the trajectory detection point of the current frame. However, this method only focuses on the motion characteristics of a single target and ignores the influence of other targets on the predicted target, resulting in a lack of global information utilization in the trajectory determination of the predicted target, resulting in a low accuracy problem of the trajectory prediction point.

[0099] Therefore, the present application proposes a trajectory determination method, which modifies the first target trajectory set by acquiring the matching result of the current frame perception result and the first target trajectory set, and obtains a modified second target trajectory set, and then determines the relationship between the corresponding two third targets based on the target distance and heading angle deviation between each two third targets in the second target trajectory set, and then determines the target information of the corresponding two third targets based on the relationship, and then determines the trajectory prediction point of each third target in the next frame based on the second target trajectory set and the target information of the third target, thereby taking into account the influence of the relationship between different third targets on the trajectory prediction point, and then the trajectory determination of the third target takes into account the global information, which is beneficial to improving the accuracy of the trajectory prediction point of the third target.

[0100] Reference Figure 1 The figure is a flow chart of a trajectory determination method provided by an embodiment of the present application, the method comprising:

[0101] S101, obtaining a current frame perception result including N trajectory detection points of a first target.

[0102] Wherein, N is a positive integer.

[0103] In an embodiment of the present application, before determining the trajectory of a target, first obtain the current frame perception result including the trajectory detection point of the target (i.e., the first target). The first target may be a traffic participant, such as a pedestrian, a vehicle, etc. Specifically, first obtain the current frame image captured by the sensor. The current frame image includes N first targets and image information of the N first targets.

[0104] Then, the current frame image is input into the BEV perception algorithm to obtain the current frame perception result corresponding to the current frame image. The current frame perception result is the perception result of the first target under the BEV perspective, which may include the category (such as pedestrians, vehicles, etc.), 3D information (such as 3D coordinates, heading angle) and size (such as length, width and height) of the first target. Therefore, the trajectory detection point of the first target in the current frame perception result may be the corresponding 3D information and size information.

[0105] In the embodiment of the present application, the above-mentioned sensor can be a camera, a lidar or a combination of multiple sensors.

[0106] The above-mentioned BEV perception algorithm may be a BEVDepth algorithm, but is not limited thereto.

[0107] S102, modifying the first target trajectory set including trajectory information of the second target according to a matching result of a current frame perception result and a first target trajectory set including trajectory information of the second target, to obtain a second target trajectory set including trajectory information of Q third targets.

[0108] Among them, the first target trajectory set includes the trajectory information of M second targets, and the trajectory information may include the historical 3D information (such as 3D coordinates, heading angle, etc.), historical size (such as length, width and height), historical posture and other information of each second target. Among them, the historical 3D information and historical size information can be the historical trajectory points of the second target before the current frame or the trajectory prediction points in the current frame. Therefore, the trajectory information includes the trajectory prediction points of M second targets in the current frame, as well as the historical trajectory points before the current frame. M and Q are both positive integers, and M and Q may be equal or unequal, which needs to be determined according to the specific application scenario.

[0109] In the embodiment of the present application, the specific process of modifying the first target trajectory set to obtain the second target trajectory set according to the matching result between the current frame perception result and the first target trajectory set can be as follows:

[0110] First, the current frame perception result is matched with the first target trajectory set to determine the second targets corresponding to the N first targets in the first target trajectory set, as well as the first trajectory states corresponding to the N first targets and the second trajectory states corresponding to the M second targets in the first target trajectory set.

[0111] Specifically, for each first target in the current frame perception result, the following matching operations are performed:

[0112] First, the trajectory difference value between the trajectory detection point of the first target in the current frame perception result and the trajectory prediction point of each second target in the first target trajectory set in the current frame is calculated.

[0113] Then, among the calculated trajectory difference values, a minimum trajectory difference value is determined.

[0114] Then determine whether the minimum trajectory difference value is less than the trajectory difference threshold.

[0115] If it is determined that the minimum trajectory difference value is less than the trajectory difference threshold, the target corresponding to the minimum trajectory difference value is determined to be the second target of the first target in the first target trajectory set, and the first target is determined to be successfully matched with the first target trajectory set, that is, the matching result corresponding to the first target is a successful match. The first target and its corresponding second target in the first target trajectory set are the same target.

[0116] If it is determined that the minimum trajectory difference value is not less than the trajectory difference threshold, it is determined that there is no corresponding second target in the first target trajectory set for the first target. In other words, the second target of the first target in the first target trajectory set is empty. It is also determined that the first target fails to match the first target trajectory set, that is, the matching result corresponding to the first target is a matching failure.

[0117] After the above matching operation is performed on each first target in the current frame perception result, the second target corresponding to each first target in the first target trajectory set can be determined.

[0118] Further, when the first target fails to match the first target trajectory set, it is determined that the first trajectory state corresponding to the first target is an initialization state.

[0119] When the first target matches the first target trajectory set successfully, it is determined that the first trajectory state of the first target is an update state.

[0120] When there is a second target in the first target trajectory set that does not match any of the N first targets, the second target that does not match any of the N first targets is determined to be a lost target. In other words, the second target in the first target trajectory set that does not successfully match the N first targets is determined to be a lost target. And the second trajectory state of the lost target is determined to be a lost state.

[0121] When the lost object meets the deletion condition, the second track state of the lost object is determined to be a deletion state.

[0122] In the embodiment of the present application, the deletion condition may be that the number of lost frames of the lost target is greater than the frame loss threshold, or the lost duration of the lost target is greater than the duration loss threshold, or the lost displacement of the lost target is greater than the displacement loss threshold, where the lost displacement is the displacement after the loss, but is not limited thereto. Thus, according to the deletion condition, it can be determined whether the lost target has no meaning to continue the trajectory determination, and the accidental deletion of the second target in the first target trajectory set can be avoided.

[0123] In summary, the first track state of the first target may be an initialization state or an update state, and the second track state of the second target may be an update state, a lost state, or a deleted state.

[0124] Furthermore, after determining the second targets corresponding to each of the N first targets, and the first trajectory states corresponding to each of the N first targets and the second trajectory states corresponding to each of the M second targets, the first target trajectory set is modified according to the first trajectory states corresponding to each of the N first targets, the second trajectory states corresponding to each of the M second targets, and the current frame perception result to obtain a modified second target trajectory set.

[0125] Specifically, for each first target in the current frame perception result and each second target in the first target trajectory set, the following modification operation is performed:

[0126] If the first trajectory state of the first target is the initialization state, the trajectory detection point of the first target in the current frame perception result is added to the first target trajectory set, and the corresponding trajectory state is also added to the first target trajectory set.

[0127] If the first trajectory state of the first target is an update state or the second trajectory state of the second target in the first target trajectory set is an update state, then the trajectory detection point of the first target in the current frame perception result is corrected to obtain a corrected trajectory correction point. Then the trajectory correction point is updated to the corresponding position in the first target trajectory set, thereby reducing the impact of noise on the trajectory detection point of the first target to improve the data accuracy of the trajectory detection point. The corresponding position is the trajectory prediction point of the second target corresponding to the first target in the first target trajectory set in the current frame. In other words, the trajectory prediction point of the second target corresponding to the first target trajectory set in the current frame is modified to the trajectory correction point of the first target.

[0128] If the second track state of the second target in the first target track set is a lost state, determine whether to change the lost state to a deleted state according to the deletion condition. If the second target corresponding to the lost state (i.e., the lost target) meets the deletion condition, the second track state corresponding to the second target is changed to a deleted state; if it does not meet the deletion condition, the corresponding second track state is maintained.

[0129] If the second track state of the second target in the first target track set is a deletion state, and the second target has not been successfully matched in the current frame perception result, the second target is deleted from the first target track set.

[0130] After the N first targets and the M second targets in the first target trajectory set have all performed the above operations, a modified second target trajectory set is obtained.

[0131] The Q third targets in the second target trajectory set also include corresponding third trajectory states, which may be initialization states, update states, lost states, deleted states, and the like.

[0132] When the first trajectory state of the first target is the initialization state, the trajectory information of the corresponding third target in the second target trajectory set includes the trajectory detection point of the current frame.

[0133] When the first trajectory state of the first target is the updating state, the trajectory information of the corresponding third target in the second target trajectory set includes the trajectory correction point of the current frame.

[0134] When the second trajectory state of the second target is a lost state, the trajectory information of the corresponding third target in the second target trajectory set includes the trajectory prediction point of the current frame.

[0135] In the embodiment of the present application, the operation of correcting the trajectory detection point of the first target in the current frame perception result to obtain the corrected trajectory correction point may be as follows:

[0136] The trajectory detection point of the first target in the current frame perception result and the trajectory motion state of the second target corresponding to the first target in the first target trajectory set at the previous moment are input into the filter to obtain the corrected trajectory correction point.

[0137] The above-mentioned trajectory motion state may be velocity, acceleration, covariance, etc. in 3D space; the above-mentioned filter may be a Kalman filter, but is not limited thereto.

[0138] Through the above method, according to the trajectory motion state of the second target corresponding to the first target in the first target trajectory set at the previous moment, the trajectory detection point of the first target in the current frame is corrected through a filter, thereby reducing the influence of noise on the trajectory detection point, making the trajectory detection point more accurate, which is beneficial to improving the accuracy of the predicted trajectory prediction point.

[0139] S103, determining the relationship between the corresponding two third targets according to the target distance and the heading angle deviation between each two third targets, and determining the target information of the corresponding two third targets according to the relationship.

[0140] For any third target, the relationship formed between another third target and it includes predecessor relationship, successor relationship, left successor relationship, right successor relationship, etc.

[0141] When another third target is traveling in front of the third target, the relationship between the third target and the other third target is a predecessor-successor relationship, that is, the other third target is the predecessor-successor target of the third target.

[0142] When another third target is traveling behind the third target, the relationship between the third target and the other third target is a successor relationship, that is, the other third target is the successor target of the third target.

[0143] When another third target is traveling on the left side of the third target, the relationship between the third target and the other third target is a left successor relationship, that is, the other third target is the left successor target of the third target.

[0144] When another third object is traveling on the right side of the third object, the relationship between the third object and the other third object is a right subsequent object, that is, the other third object is the right subsequent object of the third object.

[0145] Therefore, the target information of the third target can be the vehicle identifiers corresponding to the predecessor, successor, left successor, and right successor of the third target. The vehicle identifier can be an identification code (English: Identity Document, abbreviated as ID). Therefore, according to the ID, the predecessor, successor, left successor, and right successor of the third target can be determined, and then the track information corresponding to the predecessor, successor, left successor, and right successor of the third target can be determined, so as to better predict the track points of the third target.

[0146] In the embodiment of the present application, for the third target, only one closest predecessor target and successor target are retained, and multiple left successor targets and right successor targets may be retained, but it is not limited thereto.

[0147] In the embodiment of the present application, the relationship between the two corresponding third targets is determined according to the target distance and the heading angle deviation between each two third targets, which can be determined by the following determination operation:

[0148] In a possible implementation manner, for each two of the Q third targets that are not on the lane, the following first determination operation is performed:

[0149] When both third targets are not on the lane, that is, for the case where there is no lane, the first target distance between the two third targets and the heading angle deviation between the two third targets are calculated. The lane includes the lane area composed of lane lines, the waiting area, the intersection area, and the smooth reference line composed of the entrances and exits of each lane in the intersection.

[0150] Then, it is determined whether the first target distance is less than the first distance threshold and the heading angle deviation is less than the first heading angle threshold are satisfied at the same time. And it is determined whether the first target distance is less than the second distance threshold and the heading angle deviation is less than the second heading angle threshold are satisfied at the same time.

[0151] If it is determined that the distance of the first target is less than the first distance threshold and the heading angle deviation is less than the first heading angle threshold are satisfied at the same time, then the relationship between the two third targets is determined to be a predecessor relationship or a successor relationship. Specifically, if the rear third target (i.e., the third target traveling behind the two third targets) is used as a reference, then the relationship between the two third targets is a predecessor relationship, and the front third target (i.e., the third target traveling in front of the two third targets) is the predecessor target of the rear third target. If the front third target is used as a reference, then the relationship between the two third targets is a successor relationship, and the rear third target is the successor target of the front third target. In this way, the ID of the predecessor target or the ID of the successor target in the target information of the two third targets can be determined.

[0152] If it is determined that the first target distance is less than the first distance threshold and the heading angle deviation is less than the first heading angle threshold are not satisfied at the same time, it is determined that the relationship between the two third targets is neither a predecessor relationship nor a successor relationship.

[0153] If it is determined that the distance of the first target is less than the second distance threshold and the heading angle deviation is less than the second heading angle threshold are satisfied at the same time, then the relationship between the two third targets is determined to be a left-successor relationship or a right-successor relationship. Specifically, if the third target on the left (i.e., the third target on the left of the two third targets) is taken as the benchmark, then the relationship between the two third targets is a right-successor relationship, and the third target on the right (i.e., the third target on the right of the two third targets) is the right-successor target of the third target on the left. If the third target on the right is taken as the benchmark, then the relationship between the two third targets is a left-successor relationship, and the third target on the left is the left-successor target of the third target on the right. In this way, the ID of the left-successor target or the ID of the right-successor target in the target information of the two third targets can be determined.

[0154] If it is determined that the first target distance is less than the second distance threshold and the heading angle deviation is less than the second heading angle threshold are not satisfied at the same time, it is determined that the relationship between the two third targets is neither a left-successive relationship nor a right-successive relationship.

[0155] If the conditions that the first target distance is less than the first distance threshold and the heading angle deviation is less than the first heading angle threshold are not met at the same time, and the conditions that the first target distance is less than the second distance threshold and the heading angle deviation is less than the second heading angle threshold are also not met at the same time, then it is determined that the relationship between the two third targets is not a front-end, back-end, left-right successor relationship, that is, the relationship between the two third targets is neither a predecessor relationship, nor a successor relationship, nor a left successor relationship or a right successor relationship.

[0156] In an embodiment of the present application, the first distance threshold is different from the second distance threshold, and the first heading angle threshold is also different from the second heading angle threshold, so as to distinguish the judgment of the left-right successor relationship and the front-back successor relationship according to different distance thresholds and heading angle thresholds.

[0157] The first target distance includes the lateral distance and the longitudinal distance between the two third targets, and the heading angle deviation is the angle between the two third targets.

[0158] For example, Figure 2a As shown, the third target is target a, and another third target is target b. dx is the lateral distance between target a and target b, dy is the longitudinal distance between target a and target b, and the angle θ between target a and target b is the heading angle deviation between target a and target b.

[0159] Optionally, a rectangular coordinate system is established with the center point of target a as the origin. Then, the horizontal coordinate of target b is the horizontal distance dx, the vertical coordinate of target b is the vertical distance dy, and the angle θ between the coordinate point of target b and the y-axis is the heading angle deviation.

[0160] Furthermore, the first target distance is less than the first distance threshold when the lateral distance dx is less than the first lateral distance threshold and the longitudinal distance dy is less than the first longitudinal distance threshold. The first lateral distance threshold may be consistent with the first longitudinal distance threshold or inconsistent with it, and can be adjusted according to specific application scenarios.

[0161] The above-mentioned first target distance is less than the second distance threshold, and the lateral distance dx is less than the second lateral distance threshold and the longitudinal distance dy is less than the second longitudinal distance threshold. The second lateral distance threshold and the second longitudinal distance threshold may be consistent or inconsistent, and may be adjusted according to the specific application scenario. However, the first lateral distance threshold is different from the second lateral distance threshold, and the first longitudinal distance threshold is different from the second longitudinal distance threshold, so as to distinguish the judgment of the left-right successor relationship from the front-back successor relationship.

[0162] In another possible implementation, for each two of the Q third targets that are not on the lane, the following second determination operation is performed:

[0163] When both the two third targets are on the lane, that is, in the case where there is a lane, the relationship between the corresponding two third targets is determined according to the second target distance between the two third targets.

[0164] In the embodiment of the present application, since the two third targets are both on the lane, the two third targets may be traveling along the same lane or the same reference line, or may be traveling along adjacent lanes or adjacent reference lines. Therefore, the second target distance between the two third targets is one of the third target distance (i.e., the target distance corresponding to when the two third targets are traveling along the same lane or the same reference line) and the fourth target distance (i.e., the target distance corresponding to when the two third targets are traveling along adjacent lanes or adjacent reference lines).

[0165] Optionally, if it is determined that the two third targets are traveling along the same lane or the same reference line, a third target distance between the two third targets along the reference line direction is calculated.

[0166] For example, Figure 2b As shown, target c and target d are two third targets. Target c and target d are traveling on the same reference line, and the distance d between target c and target d along the reference line is the third target distance between the two third targets.

[0167] Then, it is determined whether the third target distance is less than a third distance threshold.

[0168] If it is determined that the distance of the third target is less than the third distance threshold, the relationship between the two third targets is determined to be a predecessor relationship or a successor relationship, so that the ID of the predecessor target or the ID of the successor target in the target information of the two third targets can be determined.

[0169] If it is determined that the distance to the third target is not less than the third distance threshold, it is determined that the relationship between the two third targets is neither a predecessor relationship nor a successor relationship.

[0170] Optionally, if it is determined that the two third targets are traveling along adjacent lanes or adjacent reference lines, one third target is translated to the adjacent reference line (i.e., the reference line where the other third target is located), and the distance between the position after the translation and the position before the translation is calculated, and the distance is used as the fourth target distance between the two third targets.

[0171] For example, Figure 2c As shown, target c and target e are two third targets, and the two third targets travel along the adjacent reference line. Target c is translated to the adjacent reference line (i.e., the reference line where target e is located) to obtain target c'. The distance d between target c and target c' is the distance between the position after translation and the position before translation, and is also the fourth target distance between target c and target e.

[0172] Then, it is determined whether the fourth target distance is less than a fourth distance threshold.

[0173] If it is determined that the fourth target distance is less than the fourth distance threshold, the relationship between the two third targets is determined to be a left successor relationship or a right successor relationship, so that the ID of the left successor target or the ID of the right successor target in the target information of the two third targets can be determined.

[0174] If it is determined that the fourth target distance is not less than the fourth distance threshold, it is determined that the relationship between the two third targets is neither a left-successor relationship nor a right-successor relationship.

[0175] In an embodiment of the present application, before determining that the two third targets are traveling along the same lane or the same reference line, or determining that the two third targets are traveling along adjacent lanes or adjacent reference lines, it is necessary to first determine the lane in which the third targets are currently located, and then determine whether the two third targets are traveling along the same lane (or the same reference line) or adjacent lanes (or adjacent reference lines) based on whether the current lanes corresponding to the two third targets are the same lane (or the same reference line) or adjacent lanes (or adjacent reference lines).

[0176] In a possible implementation, the lane where the third target is currently located can be determined in the following manner:

[0177] When the lane distribution is determined and the lane where the third target was located at the last moment is determined, the fifth target distance between the current trajectory point of the third target and the lane where the third target was located at the last moment is first calculated.

[0178] If it is determined that the fifth target distance is less than the fifth distance threshold, the lane where the third target was at the last moment is determined to be the current lane.

[0179] If it is determined that the fifth target distance is not less than the fifth distance threshold, continue to calculate the left distance between the current trajectory point of the third target and the left lane of the lane where the third target was at the previous moment, and the right distance between the current trajectory point of the third target and the right lane of the lane where the third target was at the previous moment.

[0180] If it is determined that the left distance is less than the fifth distance threshold, it is determined that the lane where the third target is currently located is the left lane of the lane where the third target was located at the previous moment.

[0181] If it is determined that the right distance is less than the fifth distance threshold, it is determined that the lane where the third target is currently located is the right lane of the lane where the third target was located at the previous moment.

[0182] If it is determined that both the left distance and the right distance are not less than the fifth distance threshold, the minimum distance value between the left distance and the right distance is determined. Then, in the above manner, another adjacent lane of the lane corresponding to the minimum distance value is continuously calculated until the lane where the third target is currently located is determined.

[0183] In the embodiment of the present application, the target distance between the calculated trajectory point and the lane may be the distance between the calculated trajectory point and the center reference line of the lane.

[0184] Through the above method, based on the comparison result of the fifth target distance between the current trajectory point of the third target and the lane it was in at the last moment and the fifth distance threshold, the lane where the third target is currently located can be quickly determined, so that the determination of the current lane is both accurate and efficient.

[0185] In another possible implementation, when the distribution of lanes is not known, the lane where the third target is currently located may also be determined by:

[0186] First, the sixth target distance between the current frame trajectory point of the third target and each lane is calculated.

[0187] Then, among the sixth target distances, the sixth target distance that is smaller than the sixth distance threshold is determined, and the lane corresponding to the sixth target distance that is smaller than the sixth distance threshold is determined, thereby selecting a plurality of lanes closest to the third target.

[0188] Then, the average distance from the G historical trajectory points to each of the selected multiple lanes (ie, lanes corresponding to the sixth target distance less than the sixth distance threshold) is calculated. G is a non-negative integer.

[0189] Then, the minimum average distance is determined among the average distances, and the lane corresponding to the minimum average distance is determined to be the lane where the third target is currently located.

[0190] Through the above method, based on the sixth target distance between the current frame trajectory point of the third target and each lane, multiple lanes closest to the third target are determined, and then based on the average distance between the historical G trajectory points of the third target and the multiple selected lanes, the lane where the third target is currently located is more accurately determined, making the determination of the lane where the third target is currently located more accurate.

[0191] Optionally, after determining the target information of the third target, an interaction model of the third target may be constructed based on the target information, so as to more intuitively understand the relationship between different third targets. The interaction model may be a graph model.

[0192] Specifically, according to the target information of the third target, the predecessor target, successor target, left successor target and right successor target of the third target are determined. Then, each third target is used as a node and the relationship between every two third targets is used as an edge to construct an interaction model.

[0193] For example, the third target is target a, and the target information of target a indicates that the previous target of target a is target b, the next target is target c, the left target is target d, and the right target is target e. Then the interaction model of target a is as follows: Figure 3 shown.

[0194] S104, determining trajectory prediction points of the Q third targets in the next frame according to the second target trajectory set and target information of the Q third targets.

[0195] Specifically, first, the historical trajectory information corresponding to each third target is determined based on the second target trajectory set. Then, the behavior mode corresponding to each third target is determined based on the target information and historical trajectory information corresponding to each third target. Then, based on the behavior mode corresponding to each third target, the trajectory points of the corresponding third targets in the next frame are predicted to obtain the trajectory prediction points of each third target in the next frame.

[0196] However, it should be noted that if the third target is in a lost state in the current frame, the trajectory points of the third target in the current frame are first predicted based on the trajectory information in the second target trajectory set to obtain the current frame trajectory prediction points, so that the trajectory prediction points of the second target corresponding to the lost state in the current frame are more accurate. Then, based on the current frame trajectory prediction points, the trajectory points of the third target in the next frame are predicted to obtain the trajectory prediction points of the third target in the next frame. The prediction method of the current frame trajectory prediction points is consistent with the prediction method of the trajectory prediction points of the next frame.

[0197] If the trajectory state of the third target in the current frame is not in a lost state, the trajectory point of the third target in the next frame is directly predicted to obtain the trajectory prediction point of the third target in the next frame.

[0198] In the embodiment of the present application, the above-mentioned behavior mode can be one of the modes such as free movement mode, following mode, lane keeping mode, left or right lane change mode, and turning mode.

[0199] The free movement mode is a movement state of a third target without lane information and without forming a predecessor, successor, left successor, or right successor relationship with other third targets. The free movement mode indicates that after the third target disappears, it still moves according to the movement state before disappearance.

[0200] Therefore, for each of the N third targets, whether the behavior mode of the third target is a free motion mode can be determined based on the target information of the third target.

[0201] Specifically, firstly, according to the target information of the third target, it is determined whether the third target has a predecessor target, a successor target, a left successor target, and a right successor target.

[0202] If it is determined that the target information of the third target does not contain the ID of the predecessor target, the ID of the successor target, the ID of the left successor target, and the ID of the right successor target, then it is determined that the third target does not have a predecessor target, a successor target, a left successor target, and a right successor target.

[0203] At the same time, if it is determined that the third target is not in the lane, it can be determined that the behavior mode of the third target is a free movement mode.

[0204] When the behavior mode of the third target is the free motion mode, the trajectory prediction point of the third target in the next frame can be predicted by the following method:

[0205] First, the historical trajectory information of the third target is obtained from the second target trajectory set. Then, based on the historical trajectory information, the speed and acceleration of the third target are determined. When the time interval between the current frame and the next frame is determined, the trajectory prediction point of the third target in the next frame can be obtained based on the speed and acceleration.

[0206] In the embodiment of the present application, the speed and acceleration of the third target determined based on the historical trajectory information may be determined by a filter, such as a Kalman filter.

[0207] The above-mentioned following mode is a motion state of a third target having a predecessor target or a successor target.

[0208] Therefore, for each of the N third targets, whether the behavior mode of the third target is the car-following mode can be determined based on the target information of the third target.

[0209] Specifically, according to the target information of the third target, it is determined whether the third target has a predecessor target and / or a successor target.

[0210] If it is determined that the target information of the third target contains the ID of the predecessor target, or the ID of the successor target, or both the ID of the predecessor target and the ID of the successor target, then it is determined that the third target has a predecessor target and / or a successor target, and then the behavior mode of the third target is determined to be the following mode.

[0211] When the behavior mode of the third target is the car-following mode, the trajectory prediction point of the third target in the next frame can be predicted by the following method:

[0212] First, the historical trajectory information of the third target is obtained from the second target trajectory set. Then, the environmental information and historical trajectory information of the third target are encoded to obtain the first encoded information corresponding to the historical trajectory information and the second encoded information corresponding to the environmental information. The first encoded information and the second encoded information are then fused to obtain fused information. The fused information is decoded to obtain the trajectory prediction point of the third target in the next frame.

[0213] In the application embodiment, when the behavior mode of the third target is the car-following mode, the above-mentioned environmental information is the information of the predecessor target and the successor target of the third target.

[0214] If the third target has no predecessor target or successor target, the coding information corresponding to the non-existent predecessor target or non-existent successor target is represented by a set of fixed feature vectors. The feature value corresponding to the fixed feature vector can be 0, but is not limited thereto.

[0215] The encoding of the historical trajectory information and environmental information of the third target can be performed by a long short-term memory neural network (English: Long Short-Term Memory, abbreviated as LSTM) or a one-dimensional convolutional neural network (English: 1D Convolutional Neural Network, abbreviated as 1D-CNN), but is not limited to this.

[0216] The fusion of the first encoding information and the second encoding information may be performed by a multi-layer perceptron (English: MultilayerPerceptron, abbreviated as MLP), but is not limited thereto.

[0217] The lane keeping mode is a motion state of a third target that has a lane or reference line but has no other third target in a relationship with it (including a predecessor relationship, a successor relationship, a left successor relationship, and a right successor relationship). The lane keeping mode indicates that the vehicle continues to move along the lane direction while maintaining the current motion trend after the third target disappears.

[0218] In a possible implementation, for each of the Q third targets, whether the behavior mode of the third target is a lane keeping mode can be determined based on whether the lane the third target is currently in is the same lane as the lane it was in at the previous moment. If it is determined that the lane the third target is currently in is the same lane as the lane it was in at the previous moment, then the behavior mode of the third target is determined to be the lane keeping mode. In this way, it is possible to quickly and accurately determine whether the behavior mode of the third target is the lane keeping mode, so that the determination of the lane keeping mode is more efficient while being accurate, and lane information is provided for the subsequent prediction of the trajectory point of the third target in the next frame, so that the prediction of the trajectory point can take into account the influence of the lane, which is conducive to improving the prediction accuracy of the trajectory point.

[0219] In another possible implementation, for each of the Q third targets, determining whether the behavior mode of the third target is the lane keeping mode may be determined in the following manner:

[0220] First, the average distance from the historical W trajectory points to the lane where the third target is currently located is calculated, where W is a positive integer. Then, it is determined whether the average distance is less than the seventh distance threshold.

[0221] If it is determined that the average distance is less than the seventh distance threshold, it is determined that the third target is traveling along the current lane, and the behavior mode of the third target is determined to be a lane keeping mode.

[0222] Through the above method, whether the behavior mode of the third target is the lane keeping mode is determined based on the comparison result of the average distance from the historical W trajectory points to the lane where the third target is currently located and the seventh distance threshold, and the trajectory information of the third target before the current moment is comprehensively considered, so that the determination result of whether the behavior mode of the third target is the lane keeping mode is more accurate, and the lane information is provided for the subsequent prediction of the trajectory point of the third target in the next frame, so that the prediction of the trajectory point can take into account the influence of the lane, which is conducive to improving the prediction accuracy of the trajectory point.

[0223] When the behavior mode of the third target is the lane keeping mode, the prediction method of the trajectory prediction point of the third target in the next frame is the same as the prediction method when the behavior mode of the third target is the following mode, which will not be repeated here.

[0224] However, it should be noted that when the behavior mode of the third target is the lane keeping mode, the environmental information in the above encoding of environmental information is the information of the lane the third target is currently in. In addition, the above encoding of the historical trajectory information and environmental information of the third target can be performed by LSTM or 1D-CNN to encode the historical trajectory information and by 1D-CNN to encode the environmental information (i.e., the information of the lane the third target is currently in), but is not limited thereto.

[0225] The lane changing mode is a motion state in which the third target of lane keeping is to gradually deviate from the original lane (such as the lane at the last moment) and approach another lane, and the deviation of the third target heading angle from the lane tangent direction is greater than the third heading angle threshold.

[0226] In a possible implementation, for each of the Q third targets, whether the behavior mode of the third target is a lane change mode can be determined based on whether the lane where the third target is currently located is the same lane as the lane where the third target was located at the last moment, so that the determination of the lane change mode is efficient. If it is determined that the lane where the third target is currently located is not the same lane as the lane where the third target was located at the last moment, the behavior mode of the third target is determined to be a lane change mode. The lane change mode may include a right lane change mode and a left lane change mode.

[0227] When the current lane of the third target is the right lane of the lane it was in at the previous moment, the behavior mode of the third target is the right lane change mode.

[0228] When the current lane of the third target is the left lane of the lane it was in at the previous moment, the behavior mode of the third target is the left lane change mode.

[0229] In another possible implementation, for each of the Q third targets, whether the behavior mode of the third target is a lane change mode can be determined by the deviation of the heading angle from the lane tangent direction and the distance between the historical T trajectory points and the current lane, so that the lane change mode can be determined more accurately. T is a non-negative integer.

[0230] Specifically, the seventh target distance from each of the T historical trajectory points to the current lane is calculated, and the heading angle deviation of each of the T historical trajectory points in the direction of the tangent line of the lane is calculated. If the change trend of the T seventh target distances is increasing, and the change trend of the T heading angle deviations is increasing, and at the same time, the seventh target distance corresponding to the trajectory point closest to the current moment is less than the eighth distance threshold, and the heading angle deviation corresponding to the trajectory point closest to the current moment is less than the fourth heading angle deviation, then the behavior mode of the three targets is determined to be the lane change mode.

[0231] When the behavior mode of the third target is the lane changing mode, the prediction method of the trajectory prediction point of the third target in the next frame is the same as the prediction method when the behavior mode of the third target is the lane keeping mode, which will not be repeated here.

[0232] However, it should be noted that when the behavior mode of the third target is a lane change mode, if it is a left lane change mode, the environmental information in the above encoding of the environmental information is the information of the left lane of the lane where the third target is currently located. If it is a right lane change mode, the environmental information in the above encoding of the environmental information is the information of the right lane of the lane where the third target is currently located.

[0233] The above turning mode is a motion state of driving along a turning reference line for the third goal of lane keeping. The turning reference line can be drawn manually or obtained based on historical trajectory information.

[0234] For each of the Q third targets, the method for determining whether the behavior mode of the third target is the turning mode is the same as the method for determining whether the behavior mode of the third target is the lane keeping mode, which will not be repeated here.

[0235] When the behavior mode of the third target is the turning mode, the prediction method of the trajectory prediction point of the third target in the next frame is the same as the prediction method when the behavior mode of the third target is the lane keeping mode, which will not be repeated here.

[0236] Optionally, the above-mentioned behavior pattern can be embodied in the form of a model, so as to determine the behavior model corresponding to the third target, and then determine the trajectory prediction point of the third target in the next frame according to the third behavior model.

[0237] In addition, in the embodiment of the present application, the behavior pattern of the third target is different, the frame loss threshold in the aforementioned deletion condition is different, the duration loss threshold is different, and the displacement loss threshold is different.

[0238] In an embodiment of the present application, when determining the trajectory prediction point of the target in the next frame, in addition to considering the relationship between the target and the relationship between the target and the lane on the trajectory prediction point of the target in the next frame, the relationship between other scenes (such as buildings, etc.) and the target can also be added, so that the determination of the trajectory prediction point is more accurate. The specific implementation method of adding the relationship between other scenes (such as buildings, etc.) and the target is consistent with the specific implementation method of adding the relationship between the lane and the target or the relationship between the target and the target, and will not be repeated here.

[0239] In addition, when the scene is a 2D scene, the 2D data can be converted into 3D data first, and then the method provided in the embodiment of the present application is used to determine the trajectory prediction point of the target in the next frame.

[0240] In summary, the trajectory determination method proposed in the present application uses 3D information to match the current frame perception result including the trajectory detection point of the first target in the current frame with the first target trajectory set of the trajectory prediction point of the second target in the current frame, and then modifies the first target trajectory set according to the matching result to obtain the second target trajectory set, and then determines the relationship between the corresponding two third targets based on the target distance and heading angle deviation between each two third targets, and then determines the target information of the third target based on the relationship between the third target and the third target and the relationship between the third target and the lane, thereby realizing the prediction of the trajectory point of the third target in the second trajectory set based on the target information and the second target trajectory set, and making the obtained prediction result take into account the influence of the relationship between the target and the target and the relationship between the target and the lane on the trajectory point, thereby effectively utilizing the global information to predict the trajectory point, so that the accuracy of the prediction of the trajectory point is greatly improved.

[0241] Moreover, from the perspective of BEV, the lost target can also use its own and adjacent targets' (such as the predecessor target, the successor target, the left successor target, and the right successor target) trajectory information as well as road constraints to predict the position of the target, so that the lost target can be more easily matched when it appears again, thereby effectively improving the trajectory accuracy and stability.

[0242] The technical solution of this application is further explained below in conjunction with a specific application process.

[0243] like Figure 4The figure shows a processing diagram of the trajectory determination method. First, in the perception module, the current frame image captured by the sensor is obtained; then, the current frame image is input into the BEV perception algorithm to obtain the current frame perception result including the N first target trajectory detection points. The current frame perception result is input into the trajectory management module so that the trajectory management module manages the trajectory information based on the current frame perception result.

[0244] In the trajectory management module, the operations in the matching unit are performed for each first target in the perception result of the current frame.

[0245] Specifically, the trajectory difference value between the trajectory detection point of the first target in the current frame perception result and the trajectory prediction point of each second target in the first target trajectory set in the current frame is calculated. Then, among the calculated trajectory difference values, the minimum trajectory difference value is determined; if it is determined that the minimum trajectory difference value is less than the trajectory difference threshold, the target corresponding to the minimum trajectory difference value is determined to be the second target of the first target in the first target trajectory set, and the matching result of the first target is determined to be successfully matched with the first target trajectory set; if it is determined that the minimum trajectory difference value is not less than the trajectory difference threshold, the matching result of the first target is determined to be failed to match with the first target trajectory set.

[0246] After the first target performs the operation in the matching unit, the matching result of the first target is received in the trajectory state module in the trajectory management module. If the matching result indicates that the first target is successfully matched, the first trajectory state corresponding to the first target is determined to be an updated state, and the second trajectory state of the second target corresponding to the first target in the first target trajectory set is determined to be an updated state.

[0247] If the matching result indicates that the first target fails to match, it is determined that the first trajectory state corresponding to the first target is an initialization state.

[0248] After each first target in the current frame perception result has completed the operation in the matching unit, if there is a second target in the first target trajectory set that has not been successfully matched with any of the N first targets, it is determined that the second trajectory state of the second target is a lost state.

[0249] For the second target whose second track state is the lost state, if it is determined that the second target meets the deletion condition, the second track state of the second target is changed from the lost state to the deleted state. If it is determined that the second target meets the deletion condition, the second track state of the second target is kept as the lost state.

[0250] In the trajectory state module in the trajectory management module, after determining the second target corresponding to each first target in the current frame perception result, and the first trajectory state of each first target and the second trajectory state of each second target, the second target corresponding to each first target, and the first trajectory state of each first target and the second trajectory state of each second target are input into the set modification unit in the trajectory management module, so that the set modification unit modifies the first target trajectory set according to the second target corresponding to the first target, the first trajectory state and the second trajectory state.

[0251] In the set modification unit in the trajectory management module, for each first target and each second target, if the first trajectory state corresponding to the first target is an initialization state, the trajectory detection point of the first target in the current frame perception result is added to the first target trajectory set;

[0252] If the first trajectory state corresponding to the first target is the update state or the second trajectory state corresponding to the second target is the update state, the trajectory detection point is first corrected based on the trajectory motion state of the second target corresponding to the first target in the first target trajectory set at the previous moment and the trajectory detection point of the first target in the current frame perception result to obtain a corrected trajectory correction point; then the trajectory prediction point of the second target corresponding to the first target trajectory set in the current frame is modified to the trajectory correction point;

[0253] If the second trajectory state corresponding to the second target is a lost state, determining whether to modify the lost state to a deleted state according to the deletion condition;

[0254] If the second track state corresponding to the second target is a deletion state, the second target is deleted from the first target track set.

[0255] In the set modification unit in the trajectory management module, after each first target and each second target perform the above operations according to the first trajectory state and the second trajectory state, a modified second target trajectory set is obtained. The second target trajectory set is then transmitted to the interaction module so that the interaction module determines the target information of each third target according to the second target trajectory set; and the second target trajectory set is transmitted to the trajectory prediction module so that the trajectory prediction module predicts the trajectory points of the third target in the next frame based on the second target trajectory set.

[0256] In the interaction module, for different third targets that are not on the lane, the first target distance and the heading angle deviation between each two third targets are calculated according to the trajectory information of the third targets in the second target trajectory set; if the first target distance is less than the first distance threshold and the heading angle deviation is less than the first heading angle threshold are satisfied at the same time, the relationship between the corresponding two third targets is determined to be a predecessor relationship or a successor relationship, and according to the relationship, the ID of the predecessor target or the ID of the successor target in the target information of the two third targets is determined;

[0257] If the first target distance is less than the second distance threshold and the heading angle deviation is less than the second heading angle threshold are satisfied at the same time, the relationship between the corresponding two third targets is determined to be a left successor relationship or a right successor relationship, and based on the relationship, the ID of the left successor target or the ID of the right successor target in the target information of the two third targets is determined;

[0258] If the first target distance is less than the first distance threshold and the heading angle deviation is less than the first heading angle threshold are not met at the same time, and the first target distance is less than the second distance threshold and the heading angle deviation is less than the second heading angle threshold are also not met at the same time, then it is determined that the relationship between the corresponding two third targets is not a front-to-back, left-to-right relationship.

[0259] For different third targets that are all on the lane, according to the trajectory information of the third targets in the second target trajectory set, if it is determined that the two third targets are traveling along the same lane or the same reference line, then the third target distance between the two third targets along the reference line direction is calculated, and when it is determined that the third target distance is less than the third distance threshold, it is determined that the relationship between the corresponding two third targets is a predecessor relationship or a successor relationship, and according to the relationship, the ID of the predecessor target or the ID of the successor target in the target information of the two third targets is determined;

[0260] According to the trajectory information of the third target in the second target trajectory set, if it is determined that the two third targets are traveling along adjacent lanes or adjacent reference lines, one of the third targets is translated to the adjacent reference line. Then the distance between the position after translation and the position before translation is calculated, and the distance is used as the fourth target distance between the two third targets. If it is determined that the fourth target distance is less than the fourth distance threshold, the relationship between the corresponding two third targets is determined to be a left-successor relationship or a right-successor relationship, and based on the relationship, the ID of the left-successor target or the ID of the right-successor target in the target information of the two third targets is determined;

[0261] If it is determined that the distance to the third target is not less than the third distance threshold, or the distance to the fourth target is not less than the fourth distance threshold, then it is determined that the relationship between the corresponding two third targets is not a front-to-back relationship.

[0262] In the interaction module, after performing the above operations for different third targets not on the lane and for different third targets all on the lane, the target information of each third target is obtained. The target information of each third target is transmitted to the trajectory prediction module, so that the trajectory prediction module predicts the trajectory point of the third target in the next frame based on the target information.

[0263] First, in the behavior mode determination module in the trajectory prediction module, the historical trajectory information corresponding to each third target is determined according to the second target trajectory set transmitted by the set modification unit. Then, according to the target information of the third target transmitted by the interaction module and the determined historical trajectory information corresponding to the third target, the behavior mode corresponding to each third target is determined.

[0264] When the behavior mode is determined to be the free motion mode, in the free motion module, the historical trajectory information of the third target is obtained from the second target trajectory set. Then, based on the historical trajectory information, the speed and acceleration of the third target are determined through the Kalman filter. Then, based on the speed and acceleration, the trajectory prediction point of the third target in the next frame is obtained.

[0265] When the behavior mode is determined to be the car-following mode, in the car-following module, the historical trajectory information of the third target is obtained from the second target trajectory set. The historical trajectory information is encoded by LSTM or 1D-CNN to obtain the first encoded information. The information of the predecessor and successor targets of the third target is encoded by LSTM or 1D-CNN to obtain the second encoded information. Then, the first encoded information and the second encoded information are fused by MLP to obtain the fused information. The fused information is then decoded to obtain the trajectory prediction point of the third target in the next frame.

[0266] When the behavior mode is determined to be one of the lane keeping mode, lane changing mode, and turning mode, in the lane module, the historical trajectory information of the third target is obtained from the second target trajectory set. The historical trajectory information is encoded by LSTM or 1D-CNN to obtain the first encoded information. The target lane (the current lane, or the lane after the lane change) of the third target is encoded by 1D-CNN to obtain the second encoded information. Then, the first encoded information and the second encoded information are fused by MLP to obtain the fused information. The fused information is then decoded to obtain the trajectory prediction point of the third target in the next frame.

[0267] Through the above method, based on the comprehensive application of perception module, trajectory management module, interaction module and trajectory prediction module, the comprehensive utilization of the relationship between targets and between targets and roads is realized, which can better use global information to predict the target posture and even effectively predict targets that have been lost for multiple frames, making it easier to re-match the target when it appears again, greatly improving the trajectory accuracy and stability.

[0268] Based on the same inventive concept, the present application also provides a trajectory determination device, such as Figure 5 The figure is a schematic diagram of the structure of a trajectory determination device provided by the present application, comprising:

[0269] The acquisition module 501 is used to acquire the current frame perception result including N trajectory detection points of the first target; wherein N is a positive integer;

[0270] A matching module 502 is used to modify the first target trajectory set including trajectory information of the second target according to the matching result of the current frame perception result and the first target trajectory set to obtain a second target trajectory set including trajectory information of Q third targets; wherein Q is a positive integer; and the current frame perception result at least includes 3D information of the first target;

[0271] A determination module 503 is used to determine the relationship between the corresponding two third targets according to the target distance and the heading angle deviation between each two third targets, and determine the target information of the corresponding two third targets according to the relationship;

[0272] The processing module 504 is used to determine trajectory prediction points of the Q third targets in the next frame according to the second target trajectory set and the target information of the Q third targets.

[0273] In a possible implementation, the acquisition module 501 is specifically used to acquire a current frame image taken by a sensor; wherein the current frame image includes the N first targets; the current frame image is input into a bird's-eye view BEV perception algorithm to obtain a current frame perception result including trajectory detection points of the N first targets.

[0274] In a possible implementation, the matching module 502 is specifically configured to match the current frame perception result with the first target trajectory set, determine the second target corresponding to each of the N first targets, and the first trajectory state corresponding to each of the N first targets and the second trajectory state corresponding to each of the second targets in the first target trajectory set; wherein the first trajectory state includes an initialization state and an update state; the second trajectory state includes an update state, a loss state, and a deletion state; and modify the first target trajectory set according to each of the first trajectory state and the second trajectory state and the current frame perception result to obtain a second target trajectory set including trajectory information of Q third targets.

[0275] In a possible implementation, the matching module 502 is specifically configured to determine the second target corresponding to each of the N first targets according to the matching operation performed on each first target, and determine that the first trajectory state of the first target that has not been successfully matched with the first target trajectory set among the N first targets is an initialization state; determine that the first trajectory state of the first target that has been successfully matched with the first target trajectory set among the N first targets is an update state, and the second trajectory state of the corresponding second target is an update state; determine that the second trajectory state of the second target that has not been successfully matched with the N first targets in the first target trajectory set is a lost state; if the second target corresponding to the lost state in the first target trajectory set meets the deletion condition, determine that the second trajectory state of the corresponding second target is a deleted state.

[0276] In a possible implementation, the matching operation is specifically as follows: calculating a trajectory difference value between a trajectory detection point of the first target in the current frame perception result and a trajectory prediction point of each second target in the first target trajectory set in the current frame; determining a minimum trajectory difference value among the calculated trajectory difference values; judging whether the minimum trajectory difference value is less than a trajectory difference threshold; if so, determining that the target corresponding to the minimum trajectory difference value is the second target of the first target in the target trajectory set, and determining that the first target is successfully matched with the first target trajectory set; if not, determining that the first target is unmatched with the first target trajectory set.

[0277] In a possible implementation, the matching module 502 is specifically configured to perform the following modification operations for each first target and each second target: if the first trajectory state of the first target is an initialization state, then the trajectory detection point of the corresponding first target in the current frame perception result is added to the first target trajectory set; if the first trajectory state of the first target is an update state, then the trajectory detection point of the corresponding first target in the current frame perception result is corrected, and the corrected trajectory correction point is updated to the corresponding position in the first target trajectory set; if the second trajectory state of the second target is a lost state, then according to the deletion condition, it is determined whether to modify the lost state to a deleted state; if the second trajectory state of the second target is a deleted state, then the corresponding second target is deleted from the first target trajectory set; until the modification operation is performed on each first target and each second target, a second target trajectory set including trajectory information of Q third targets is obtained.

[0278] In one possible implementation, the determination module 503 is specifically used to determine, for different third targets that are not on the lane, the relationship between the corresponding two third targets based on the first target distance and the heading angle deviation between each two third targets; wherein the relationship is any one of a predecessor relationship, a successor relationship, a left-successor relationship, a right-successor relationship, and a non-previous, subsequent, left-and-right successor relationship; and for different third targets that are all on the lane, determine the relationship between the corresponding two third targets based on the second target distance between each two third targets.

[0279] In a possible implementation, the processing module 504 is specifically used to determine the historical trajectory information corresponding to each of the Q third targets based on the second target trajectory set; determine the behavior mode corresponding to each of the Q third targets based on the target information and historical trajectory information corresponding to each of the Q third targets; wherein the behavior mode is one of a free movement mode, a following mode, a lane keeping mode, a lane changing mode, and a turning mode; and obtain the trajectory prediction point of each of the Q third targets in the next frame based on the behavior mode corresponding to each of the Q third targets.

[0280] In one possible implementation, the processing module 504 is specifically used to, for each third target, based on the corresponding target information and historical trajectory information, if it is determined that the corresponding third target does not have a predecessor target, a successor target, a left successor target, and a right successor target, and the corresponding third target is not on the lane, then determine that the behavior mode of the corresponding third target is the automatic movement mode; if it is determined that the corresponding third target has a predecessor target and / or a successor target, then determine that the behavior mode of the corresponding third target is the following mode; if it is determined that the current lane of the corresponding third target is the same lane as the lane it was in at the previous moment, then determine that the behavior mode of the corresponding third target is the lane keeping mode; if it is determined that the current lane of the corresponding third target is not the same lane as the lane it was in at the previous moment, then determine that the behavior mode of the corresponding third target is the lane changing mode; if it is determined that the corresponding third target is traveling along a turning reference line, then determine that the behavior mode of the corresponding third target is the turning mode.

[0281] In a possible implementation, the processing module 504 is specifically used to perform the following prediction operations for each third target: if the behavior mode of the third target is a free motion mode, then based on the historical trajectory information of the third target, the speed and acceleration of the third target are determined, and according to the speed and acceleration, the trajectory prediction point of the third target in the next frame is obtained; if the behavior mode of the third target is not a free motion mode, then the environmental information and the historical trajectory information of the third target are encoded to obtain first encoding information corresponding to the historical trajectory information and second encoding information corresponding to the environmental information; the first encoding information and the second encoding information are fused to obtain fused information; and the fused information is decoded to obtain the trajectory prediction point of the third target in the next frame; until the prediction operation is completed for each third target, the trajectory prediction points of Q third targets in the next frame are obtained.

[0282] In one possible implementation, the processing module 504 is specifically used to, if the behavior mode of the third target is a following mode, then the environmental information is the information of the predecessor target and the successor target of the third target; if the behavior mode of the third target is any one of the lane keeping mode, lane changing mode, and turning mode, then the environmental information is the information of the lane in which the third target is currently located.

[0283] Based on the same inventive concept, an electronic device is also provided in the embodiment of the present application. The electronic device can realize the function of the aforementioned trajectory determination device. Figure 6 , the electronic equipment includes:

[0284] At least one processor 601, and a memory 602 connected to the at least one processor 601. The specific connection medium between the processor 601 and the memory 602 is not limited in the embodiment of the present application. Figure 6In the example, the processor 601 and the memory 602 are connected via a bus 600. Figure 6 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 600 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 601 can also be called a controller, and there is no limitation on the name.

[0285] In the embodiment of the present application, the memory 602 stores instructions that can be executed by at least one processor 601. The at least one processor 601 can execute the trajectory determination method discussed above by executing the instructions stored in the memory 602. The processor 601 can implement Figure 5 The functions of each module in the device shown.

[0286] Among them, the processor 601 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 602 and calling data stored in the memory 602, the various functions of the device and processing data, the device can be monitored as a whole.

[0287] In one possible design, the processor 601 may include one or more processing units, and the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0288] The processor 601 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the trajectory determination method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0289] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (English: Random Access Memory, abbreviated as RAM), a static random access memory (English: Static Random Access Memory, abbreviated as SRAM), a programmable read-only memory (English: Programmable Read Only Memory, abbreviated as PROM), a read-only memory (English: Read Only Memory, abbreviated as ROM), an electrically erasable programmable read-only memory (English: Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0290] By programming the processor 601, the code corresponding to the trajectory determination method described in the above embodiment can be fixed into the chip, so that the chip can execute the code when running. Figure 1 Steps of the trajectory determination method of the embodiment shown. How to design and program the processor 601 is a technique well known to those skilled in the art and will not be described in detail here.

[0291] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the trajectory determination method discussed above.

[0292] In some possible implementations, various aspects of the trajectory determination method provided in the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on a device, the program code is used to enable the control device to execute the steps of the trajectory determination method according to various exemplary embodiments of the present application described above in this specification.

[0293] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0294] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0295] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0296] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0297] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A trajectory determination method, characterized in that: include: Obtaining a current frame perception result including N trajectory detection points of the first target; wherein N is a positive integer; According to the matching result between the current frame perception result and the first target trajectory set including the trajectory information of the second target, modify the first target trajectory set to obtain a second target trajectory set including the trajectory information of Q third targets; wherein Q is a positive integer; Determine the relationship between the corresponding two third targets according to the target distance and the heading angle deviation between each two third targets, and determine the target information of the corresponding two third targets according to the relationship; According to the second target trajectory set and the target information of the Q third targets, trajectory prediction points of the Q third targets in the next frame are determined.

2. The method according to claim 1, characterized in that The method of modifying the first target trajectory set according to the matching result of the current frame perception result and the first target trajectory set including the trajectory information of the second target to obtain the second target trajectory set including the trajectory information of Q third targets includes: Matching the current frame perception result with the first target trajectory set, determining the second target corresponding to each of the N first targets, and the first trajectory state corresponding to each of the N first targets and the second trajectory state corresponding to each of the second targets in the first target trajectory set; wherein the first trajectory state includes an initialization state and an update state; and the second trajectory state includes an update state, a loss state, and a deletion state; The first target trajectory set is modified according to each of the first trajectory state and the second trajectory state and the current frame perception result to obtain the second target trajectory set including the trajectory information of the Q third targets.

3. The method according to claim 2, characterized in that The modifying the first target trajectory set according to each of the first trajectory state and the second trajectory state and the current frame perception result to obtain the second target trajectory set including the trajectory information of Q third targets includes: For each of the first targets and each of the second targets, the following modification operations are performed: If the first trajectory state of the first target is the initialization state, adding the corresponding trajectory detection point of the first target in the current frame perception result to the first target trajectory set; If the first trajectory state of the first target is the update state, correcting the corresponding trajectory detection point of the first target in the current frame perception result, and updating the corrected trajectory correction point to the corresponding position in the first target trajectory set; If the second trajectory state of the second target is the lost state, determining whether to modify the lost state to the deleted state according to a deletion condition; If the second trajectory state of the second target is the deletion state, deleting the corresponding second target from the first target trajectory set; Until each of the first targets and each of the second targets completes the modification operation, a second target trajectory set including trajectory information of Q third targets is obtained.

4. The method according to claim 1, characterized in that Determining the relationship between two corresponding third targets according to the target distance and the heading angle deviation between each two third targets includes: For different third targets that are not on the lane, determine the relationship between the corresponding two third targets according to the first target distance and the heading angle deviation between each two third targets; wherein the relationship is any one of a predecessor relationship, a successor relationship, a left successor relationship, a right successor relationship, and a non-previous, non-left, and non-left-right successor relationship; and For different third targets that are all on the lane, the relationship between the corresponding two third targets is determined according to the second target distance between every two third targets.

5. The method according to claim 1, characterized in that The step of determining trajectory prediction points of the Q third targets in the next frame according to the second target trajectory set and the target information of the Q third targets includes: Determine, according to the second target trajectory set, historical trajectory information corresponding to each of the Q third targets; Determine, according to the target information and the historical trajectory information corresponding to each of the Q third targets, a behavior mode corresponding to each of the Q third targets; wherein the behavior mode is one of a free movement mode, a following mode, a lane keeping mode, a lane changing mode, and a turning mode; According to the behavior patterns corresponding to the Q third targets, the trajectory prediction points of the Q third targets in the next frame are obtained.

6. The method according to claim 5, characterized in that The determining, according to the target information and the historical trajectory information respectively corresponding to the Q third targets, the behavior patterns respectively corresponding to the Q third targets, includes: For each of the third targets, according to the corresponding target information and the historical trajectory information, if it is determined that the corresponding third target does not have a preceding target, a succeeding target, a left succeeding target, and a right succeeding target, and the corresponding third target is not on a lane, then determining that the behavior mode of the corresponding third target is the automatic motion mode; If it is determined that the corresponding third target has the predecessor target and / or the successor target, then determining that the behavior mode of the corresponding third target is the following mode; If it is determined that the current lane of the corresponding third target is the same lane as the lane of the previous moment, then determining that the behavior mode of the corresponding third target is the lane keeping mode; If it is determined that the lane currently located by the corresponding third target is not the same lane as the lane located at the last moment, determining that the behavior mode of the corresponding third target is the lane changing mode; If it is determined that the corresponding third target is traveling along the turning reference line, then the behavior mode of the corresponding third target is determined to be the turning mode.

7. The method according to claim 5, characterized in that The step of obtaining, according to the behavior patterns corresponding to the Q third targets, trajectory prediction points of the Q third targets in the next frame respectively includes: For each of the third targets, the following prediction operations are performed: If the behavior mode of the third target is the free motion mode, determining the speed and acceleration of the third target based on the historical trajectory information of the third target, and obtaining a trajectory prediction point of the third target in the next frame according to the speed and the acceleration; If the behavior mode of the third target is not the free motion mode, encoding the environment information and the historical trajectory information of the third target to obtain first coded information corresponding to the historical trajectory information and second coded information corresponding to the environment information; fusing the first coded information and the second coded information to obtain fused information; and decoding the fused information to obtain a trajectory prediction point of the third target in the next frame; Until each of the third targets completes the prediction operation, the trajectory prediction points of the Q third targets in the next frame are obtained.

8. A trajectory determination device, characterized in that: include: An acquisition module, used to acquire a current frame perception result including N trajectory detection points of the first target; wherein N is a positive integer; A matching module, configured to modify the first target trajectory set including trajectory information of the second target according to a matching result between the current frame perception result and the first target trajectory set, so as to obtain a second target trajectory set including trajectory information of Q third targets; wherein Q is a positive integer; A determination module, configured to determine a relationship between two corresponding third targets according to a target distance and a heading angle deviation between each two third targets, and determine target information of the two corresponding third targets according to the relationship; A processing module is used to determine trajectory prediction points of the Q third targets in the next frame according to the second target trajectory set and the target information of the Q third targets.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to implement the method steps of any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images