Trajectory optimization method and device and storage medium

Abstract

The invention discloses a trajectory optimization method and apparatus, and a storage medium. The method comprises the steps of receiving first point cloud frame information and a first trajectory identifier corresponding to the first point cloud frame information; updating a first track corresponding to the first track identifier based on the first point cloud frame information; under the condition that the updated first track meets a preset first condition, screening a second track associated with the first track from a historical track set; predicting missing point cloud frame information between the first track and the second track based on the track and the second track to obtain first complemented point cloud frame information; and splicing the first track and the second track into a new track based on the first completion point cloud frame information. According to the embodiment of the invention, the second trajectory associated with the first trajectory is screened out from the historical trajectories, and the missing trajectory between the first trajectory and the second trajectory is predicted based on the first trajectory and the second trajectory, so that reconnection between different trajectories is realized, and a more complete trajectory is obtained.

Description

Technical Field

[0001] This application belongs to the technical field of target tracking, and particularly relates to a trajectory optimization method, device, and storage medium. Background Art

[0002] In the scenario of target tracking based on lidar, when the target object being tracked is occluded, the detection result of the current frame is usually optimized by performing target tracking on the historical detection results of the target object to complete the motion trajectory of the target object, thereby reducing target missed detection.

[0003] However, when the target in a trajectory is continuously occluded, it is difficult for the tracking algorithm to accurately determine the state of the target object. Therefore, the tracking algorithm usually sets a certain tracking time threshold and only tracks the target within this tracking time threshold. This means that when the occlusion time of the target object is greater than the tracking time threshold, the target object cannot be tracked during the period from when the occlusion time exceeds the tracking time threshold to when the target object drives out of the occlusion area, and the tracking is interrupted. When the target object reappears after driving out of the occlusion area, it will be regarded as a new trajectory, and the tracking identifier of the target object is updated.

[0004] The above method of performing target tracking based on historical detection results is difficult to achieve continuous tracking when the target object is continuously occluded, resulting in the interruption of the trajectory of the same target object within a continuous period of time and poor tracking performance. Summary of the Invention

[0005] Embodiments of this application provide a trajectory optimization method, device, and storage medium, which can improve trajectory integrity and enhance target tracking performance.

[0006] In a first aspect, embodiments of this application provide a trajectory optimization method, including:

[0007] Receiving first point cloud frame information and a first trajectory identifier corresponding to the first point cloud frame information;

[0008] Creating a first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information;

[0009] When the first trajectory meets a preset first condition, screening a second trajectory associated with the first trajectory from a set of historical trajectories;

[0010] Predicting missing point cloud frame information between the first trajectory and the second trajectory based on the first trajectory and the second trajectory to obtain first complementary point cloud frame information;

[0011] Splicing the first trajectory and the second trajectory into a new trajectory based on the first complementary point cloud frame information;

[0012] Among them, the first condition includes that the time length of the first trajectory is greater than the first preset duration.

[0013] In a second aspect, an embodiment of the present application provides a trajectory optimization device, including:

[0014] A receiving module, configured to receive first point cloud frame information and a first trajectory identifier corresponding to the first point cloud frame information;

[0015] A creating module, configured to create a first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information;

[0016] A screening module, configured to screen a second trajectory associated with the first trajectory from a set of historical trajectories when the first trajectory meets a preset first condition;

[0017] A predicting module, configured to predict missing point cloud frame information between the first trajectory and the second trajectory based on the first trajectory and the second trajectory, to obtain first complemented point cloud frame information;

[0018] A splicing module, configured to splice the first trajectory and the second trajectory into a new trajectory based on the first complemented point cloud frame information;

[0019] Among them, the first condition includes that the time length of the first trajectory is greater than the first preset duration.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a processor and a memory storing computer program instructions;

[0021] When the processor executes the computer program instructions, it implements the trajectory optimization method as in the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, they implement the trajectory optimization method as in the first aspect.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, and when the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is enabled to execute the trajectory optimization method as in the first aspect.

[0024] A trajectory optimization method, device, electronic device, storage medium, and program product according to an embodiment of the present application receive first point cloud frame information and a first trajectory identifier corresponding to the first point cloud frame information; update a first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information; when the updated first trajectory meets a preset first condition, screen a second trajectory associated with the first trajectory from a historical trajectory set; predict missing point cloud frame information between the first trajectory and the second trajectory based on the trajectory and the second trajectory to obtain first complementary point cloud frame information; and splice the first trajectory and the second trajectory into a new trajectory based on the first complementary point cloud frame information. According to the embodiment of the present application, a second trajectory associated with the first trajectory is screened from historical trajectories, and the missing trajectory between the first trajectory and the second trajectory is predicted based on the first trajectory and the second trajectory, so as to realize reconnection between different trajectories, and thus obtain a more complete trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required to be used in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0026] Figure 1 is a flowchart of a trajectory optimization method provided by some embodiments of the present application;

[0027] Figure 2 is a schematic diagram of a trajectory provided by some embodiments of the present application;

[0028] Figure 3 is a schematic diagram of a trajectory optimization device provided by some embodiments of the present application;

[0029] Figure 4 is a schematic diagram of the structure of an electronic device provided by some embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] The features and exemplary embodiments of various aspects of the present application will be described in detail below. To make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below in combination with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than limiting the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0031] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0032] Before elaborating on the technical solutions provided by the embodiments of the present application, for the convenience of understanding the embodiments of the present application, the present application first specifically describes the problems existing in the prior art:

[0033] Currently, for the target tracking technology based on lidar, mainly the lidar measures information such as the distance, angle, and reflection intensity of the target object by emitting laser beams and receiving the reflected light. Based on this information, a three-dimensional point cloud map of the surrounding environment is constructed. By analyzing the features in the point cloud map, the position, shape, size, etc. of the target object can be detected. On the basis of target detection, the detected target object is tracked to determine information such as the position and motion state of the target object at different time points, and the point cloud frame information of the target object is obtained. Among them, each piece of point cloud frame information corresponds to a time stamp respectively, and this time stamp is used to indicate the acquisition time of the point cloud corresponding to this piece of point cloud frame information.

[0034] Then, based on the point cloud frame information of the target object at different time points obtained by tracking, the motion trajectory of the target object is generated. Among them,

[0035] The methods of object detection are mainly divided into two types. One is the detection method based on geometric features, which uses the geometric shape features of the target object, such as edges, planes, cylinders, etc., to detect the target object. For example, by detecting the edge points in the point cloud map, the contour of the target object can be determined; by detecting the plane, the surface of the target object can be determined. The other is the detection method based on deep learning, which uses a deep neural network to process the lidar point cloud data and automatically learns the features of the target object to achieve object detection. The tracking methods are mainly divided into three types, namely the tracking method based on filtering, the tracking method based on data association, and the tracking method based on deep learning. Each of the three tracking methods has its own advantages and disadvantages. Among them, the tracking method based on filtering uses algorithms such as Kalman filtering and particle filtering to estimate and predict the state of the target object. This method is suitable for linear or approximately linear motion models, but the tracking effect for non-linear motion models is poor. The tracking method based on data association determines the identity and motion state of the target by associating the target detected at the current moment with the target at the previous moment. This method is suitable for complex motion scenarios, but the computational cost is large. The tracking method based on deep learning uses a deep neural network to extract and track the features of the target object. This method has a high tracking accuracy and robustness, but requires a large amount of training data and computing resources.

[0036] However, in practical applications, during the movement of the target object, it may be occluded by other objects. When the target object is occluded by other objects, the lidar will not be able to detect the point cloud information of the target object, resulting in a trajectory interruption and making the trajectory of the target object incomplete. And in practical applications, trajectory interruptions usually lead to various adverse effects. For example, in the context of autonomous driving, an accurate and complete target trajectory is crucial for vehicle decision-making and control. A missing trajectory may cause the vehicle to fail to react correctly in a timely manner, increasing the risk of accidents. In the context of security monitoring, a complete target trajectory helps analyze the behavior of suspicious persons or objects and timely detect potential security threats. If the trajectory is incomplete, important clues may be missed. In the context of robot navigation, the robot needs to rely on an accurate target trajectory for path planning and obstacle avoidance. A missing trajectory may cause the robot to lose its way or be unable to effectively complete the task. It can be seen that the integrity of the trajectory is very important. Therefore, when the target object is occluded, in order to improve the integrity of the trajectory, trajectory completion is usually carried out.

[0037] As described above, in the related art, the detection result of the current frame is mainly optimized by performing object tracking on the historical detection results of the target object to complete the motion trajectory of the target object. However, this method is only applicable to the case where the target object is temporarily occluded, and it is difficult to achieve continuous tracking in the case where the target object is continuously occluded, resulting in the interruption of the trajectory of the same target within a continuous period of time and poor tracking performance when the target object is continuously occluded.

[0038] In view of this, in order to solve the above technical problems and improve the tracking performance, an embodiment of the present application provides a trajectory optimization method, apparatus, electronic device, storage medium, and program product.

[0039] The trajectory optimization method provided by the embodiment of the present application can be applied to an object tracking scenario, and the object tracking scenario includes, but is not limited to, object tracking in scenarios such as autonomous driving, robot navigation, security monitoring, and industrial automation.

[0040] It should be noted that the trajectory optimization method provided by the embodiment of the present application can be executed by a trajectory optimization apparatus, and the trajectory optimization apparatus can be an electronic device or electronic component for generating and optimizing a trajectory based on point cloud frame information. For example, in an autonomous driving scenario, the trajectory optimization apparatus can be an in-vehicle computer; in a robot navigation scenario, the trajectory optimization apparatus can be a processor of the robot; in a security monitoring scenario, the trajectory optimization apparatus can be a monitoring device; and in an industrial automation scenario, the trajectory optimization apparatus can be a host computer. Hereinafter, taking the execution of the trajectory optimization method provided by the embodiment of the present application by the trajectory optimization apparatus as an example, the trajectory optimization method provided by the embodiment of the present application will be described.

[0041] See Figure 1 , which is a schematic flowchart of the trajectory optimization method provided by some embodiments of the present application. As Figure 1 shown, the method includes the following steps S110 - S150, which will be specifically described below.

[0042] S110. Receive first point cloud frame information and a first trajectory identifier corresponding to the first point cloud frame information.

[0043] As described above, the trajectory optimization method provided by the embodiment of the present application is executed by a trajectory optimization apparatus for generating and optimizing a trajectory in an application scenario. The trajectory optimization apparatus can receive point cloud frame information and a corresponding trajectory identifier sent by its upstream module, and then generate and optimize a trajectory based on the received point cloud frame information and trajectory identifier. Among them, the upstream module includes, but is not limited to, a module for performing object detection and tracking based on data collected by a lidar. The trajectory identifier is an identifier of the trajectory, used to distinguish different trajectories. Usually, different target objects correspond to different trajectory identifiers.

[0044] In some embodiments of the present application, the first point cloud frame information may be the point cloud frame information of any target object currently received by the trajectory optimization device. The point cloud frame information includes, but is not limited to, information such as the position information and size information of the target object, which can characterize the state of the target object. In an actual application scenario, the upstream module can simultaneously detect and track multiple different target objects. During the detection and tracking process, different point cloud frame information and trajectory identifiers are generated for different target objects. Based on this, the trajectory optimization device can simultaneously receive the point cloud frame information and trajectory identifiers of multiple different target objects, and then respectively generate or optimize the trajectories of each target object based on the point cloud frame information and trajectory identifiers of each target object. In this embodiment, the trajectory optimization methods for different target objects are the same. Therefore, for the sake of convenience of description, only the example of optimizing the trajectory of the target object corresponding to the first point cloud frame information based on the first point cloud frame information and the corresponding first trajectory identifier will be used for illustration.

[0045] S120. Create a first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information.

[0046] In some embodiments of the present application, after receiving the first point cloud frame information and the first trajectory identifier, the trajectory optimization device may create a trajectory corresponding to the first trajectory identifier based on the first point cloud frame information.

[0047] S130. When the first trajectory meets a preset first condition, screen a second trajectory associated with the first trajectory from the historical trajectory set.

[0048] In some embodiments of the present application, the upstream module receives the laser signal returned by the target object, performs target detection and target tracking based on the laser signal to obtain the point cloud frame information corresponding to the target object, assigns a corresponding trajectory identifier to the point cloud frame information, and sends the point cloud frame information and the trajectory identifier to the trajectory generation module to generate the motion trajectory corresponding to the target object through the trajectory generation module. In practical applications, the target object may be occluded. During the occlusion period of the target object, the upstream module cannot receive the laser signal returned by the target object, resulting in the upstream module being unable to continuously detect the target object during this period, and thus unable to continuously generate the point cloud frame information of the target object. When the duration for which the upstream module cannot obtain the point cloud frame information of the target object exceeds a certain preset duration, it is considered that the target object is lost, and thus the previous trajectory identifier is stopped being used. When the target object reappears after leaving the occlusion area later, it is considered a new target object, and thus a new trajectory identifier is assigned to it. Since the trajectory optimization device generates a trajectory based on the trajectory identifier, after the trajectory identifier changes, the trajectory optimization device will generate a new trajectory, which results in the same target object possibly corresponding to multiple different trajectories, and there are interruption gaps between different trajectories. In view of this, in order to reduce trajectory interruptions, when the first trajectory meets the first condition, the second trajectory associated with the first trajectory is screened from the historical trajectory set, so as to optimize the trajectory based on the first trajectory and the second trajectory and reduce trajectory interruptions. Among them, the first trajectory and the second trajectory associated with it are the motion trajectories corresponding to the same target object, that is, the second trajectory associated with the first trajectory is the historical trajectory corresponding to the same target object as the first trajectory.

[0049] In this embodiment, the first condition includes that the time length of the first trajectory is greater than the first preset duration. The limitation of setting the first preset duration is to be able to screen the second trajectory after the first trajectory has existed for a period of time. This is mainly considered because the first trajectory may be generated due to misdetection by the perception and tracking module. In order to ensure its credibility and avoid generating more misdetections by optimizing the misdetected trajectory.

[0050] In some embodiments of the present application, the first preset duration can be preset according to actual situations. Considering that when there is no missed detection of the target object in the upstream module, the period for generating point cloud frame information is fixed, that is, a point cloud frame information is generated every fixed period. That is, the time interval between any two adjacent point cloud frame information is the same and is a fixed period. Based on this, in this embodiment, the trajectory length, the first preset duration, etc. can all be represented by the step size, where the step size refers to the number of fixed periods. For example, if the fixed period is 10 seconds, and the first preset duration is 20 seconds, then the step size corresponding to the first preset duration is 2. If the first preset duration is 10 seconds, then the step size corresponding to the first preset duration is 1. Based on this, for the convenience of distinction, the step size corresponding to the first preset duration is called the minimum backtracking step size threshold, and the first condition can include: the length of the first trajectory is greater than the minimum backtracking step size threshold.

[0051] In some embodiments of the present application, a historical trajectory refers to a trajectory that has been interrupted, that is, a trajectory that is no longer updated currently. The trajectory optimization device can record information such as the point cloud frame information it receives and each generated trajectory. Based on this, the trajectory optimization device can directly screen out the historical trajectories from the recorded trajectories, and use the set composed of the screened historical trajectories as the historical trajectory set. The historical trajectory set includes one or more historical trajectories. The trajectory optimization device can calculate the matching degree between each historical trajectory in the historical trajectory set and the first trajectory respectively, so as to use the historical trajectory with a matching degree higher than the matching degree threshold with the first trajectory as the second trajectory associated with the first trajectory.

[0052] S140. Based on the first trajectory and the second trajectory, predict the missing point cloud frame information between the first trajectory and the second trajectory to obtain the first complemented point cloud frame information.

[0053] In some embodiments of the present application, the main purpose of trajectory optimization based on the first trajectory and the second trajectory is to obtain a complete trajectory corresponding to the target object, including the first trajectory and the second trajectory, based on the first trajectory and the second trajectory. As mentioned above, there is an interruption gap between the first trajectory and the second trajectory, that is, there is missing point cloud frame information. In order to enable the first trajectory and the second trajectory to be spliced into a complete trajectory, before forming the complete trajectory, first predict the missing point cloud frame information between the first trajectory and the second trajectory.

[0054] In some embodiments of the present application, the missing point cloud frame information between the first trajectory and the second trajectory is determined by forward predicting the second trajectory and backtracking and predicting the first trajectory, and the predicted point cloud frame information is referred to as the first complementary point cloud frame information. Among them, forward prediction refers to inferring and predicting the point cloud frame information after the second trajectory based on the point cloud frame information and the change trend of the point cloud frame information in the second trajectory. Backtracking prediction refers to inferring and predicting the point cloud frame information before the first trajectory based on the point cloud frame information and the change trend of the point cloud frame information in the first trajectory.

[0055] In some embodiments of the present application, the trajectory optimization device may maintain a trajectory information list TrackInfoList, which records all the information of each trajectory, including the trajectory identifier track_id of the trajectory, the point cloud frame information existing in the trajectory, and the point cloud frame information includes, but is not limited to, frame numbers and timestamps, etc. Based on this, the point cloud frame information of each trajectory can be obtained from the trajectory information list. Usually, there are multiple point cloud frame information in a trajectory. For the convenience of management and maintenance, the frame numbers and timestamps corresponding to the trajectory can be stored in the form of a list. The list corresponding to the frame numbers can be denoted as FrameIDList, and the list corresponding to the timestamps can be denoted as TimestampList. In addition, the trajectory information list may also include the trajectory interruption step length InteruptNum corresponding to each trajectory. The trajectory interruption step length of each trajectory is updated in real time, and it is used to indicate the step length of the interval between the timestamp of the last point cloud frame information in the trajectory and the current time. Here, the step length also refers to the number of fixed periods. In the initial state, the trajectory list information TrackInfoList is empty.

[0056] S150. Based on the first complementary point cloud frame information, splice the first trajectory and the second trajectory into a new trajectory.

[0057] In some embodiments of the present application, after obtaining the first complementary point cloud frame information, the missing trajectory between the first trajectory and the second trajectory is complemented based on the first complementary point cloud frame information to splice the first trajectory and the second trajectory, thereby obtaining a complete new trajectory. For example, as Figure 2 shown, the timestamp of the first point cloud frame information is t i , the first trajectory identifier is track id2 , the trajectory identifier of the second trajectory is track id1 , and the missing timestamp between the first trajectory and the second trajectory is located at t k+1 and t i-1 The point cloud frame information between them is predicted for the point cloud frame information with timestamps between t k+1 and t i-1 through the above step S140 to obtain the corresponding first complementary point cloud frame information, and then based on t k+1The corresponding point cloud frame information, t i-1 The corresponding point cloud frame information and the first completed point cloud frame information can generate one for t k+1 Taking as the starting time point, with t i-1 as the ending time point of the trajectory, that is, Figure 2 the dashed part in, and this trajectory can splice the first trajectory and the second trajectory respectively. In this way, a new trajectory containing the first trajectory and the second trajectory without interruption gaps in the middle can be obtained through splicing.

[0058] The trajectory optimization method provided by the embodiments of the present application receives the first point cloud frame information and the first trajectory identifier corresponding to the first point cloud frame information; updates the first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information; when the updated first trajectory meets the preset first condition, screens the second trajectory associated with the first trajectory from the historical trajectory set; predicts the missing point cloud frame information between the first trajectory and the second trajectory based on the trajectory and the second trajectory to obtain the first completed point cloud frame information; and splices the first trajectory and the second trajectory into a new trajectory based on the first completed point cloud frame information. According to the embodiments of the present application, the second trajectory associated with the first trajectory is screened from the historical trajectory, and the missing trajectory between the first trajectory and the second trajectory is predicted based on the first trajectory and the second trajectory, so as to realize the reconnection between different trajectories, and thus obtain a more complete trajectory.

[0059] In some embodiments, the first condition further includes that the time length of the first trajectory is less than a second preset duration, and the second preset duration is set according to the actual situation, but it is necessary to ensure that the second preset duration is greater than the first preset duration. Similar to the first preset duration, the second preset duration can also be represented by a step size. For the convenience of distinction, the step size corresponding to the second preset duration can be called the backtracking step size. Based on this, the above first condition can also be expressed as the time length of the first trajectory is less than the backtracking step size.

[0060] Here, setting the second preset duration is to facilitate determining the backtracking moment. When the time length of the first trajectory is less than the second preset duration, screening the historical trajectory can ensure that the first trajectory has not appeared at the backtracking moment, that is, the first moment is the moment before the first trajectory.

[0061] Based on this, the above step S130 of screening the second trajectory associated with the first trajectory from the historical trajectory set may include the following steps S131 - S135.

[0062] S131. When the first trajectory meets the first condition, subtract the second preset duration from the time stamp of the first point cloud frame information to obtain the backtracking moment.

[0063] In this embodiment, when the first trajectory meets the first condition, subtract the second preset duration from the timestamp of the first point cloud frame information to obtain a moment before the first trajectory, and use this moment as the retrospective moment.

[0064] S132. Based on the first trajectory, predict the target state information corresponding to the retrospective moment to obtain the first predicted target state information.

[0065] Here, the target state information refers to the state information of the target object corresponding to the trajectory. The target state information includes the position information and motion information of the target object. The motion information includes, but is not limited to, speed information and acceleration information. Hereinafter, an example will be given with the target state information including the position, speed information, and acceleration information of the target object.

[0066] In some embodiments of the present application, based on the currently acquired first point cloud frame information in the first trajectory, use the retrospective prediction method to predict the point cloud frame information corresponding to the retrospective moment, and call the predicted point cloud frame information the first predicted target state information.

[0067] In some embodiments of the present application, the first predicted target state information corresponding to the retrospective moment can be predicted based on the first point cloud frame information according to the interpolation algorithm shown in the following formulas (1)-(3).

[0068] X i =[x i y i vx i vy i ax i ay i ] T (1)

[0069] X ′ =A i *X i (2)

[0070]

[0071] In the formula, X i represents the position and motion information of the target object in the first point cloud frame information, where x i represents the horizontal coordinate of the position, y i represents the vertical coordinate of the position, vx i represents the horizontal speed, vy i represents the vertical speed, ax i represents the horizontal acceleration, ay i represents the vertical acceleration, X ′ represents the motion information of the target object at the retrospective moment deduced from the first point cloud frame information, and A iThe process matrix representing the recursion from the first point cloud frame information to the backtracking moment, P i The position covariance matrix corresponding to the first point cloud frame information, Q i The process noise representing the recursion from the first point cloud frame information to the backtracking moment, P ′ The estimated value of the position covariance representing the recursion from the first point cloud frame information to the backtracking moment. Based on X ′ and P ′ The determined position information and motion information are used as the first predicted target state information corresponding to the backtracking moment predicted based on the first trajectory.

[0072] S133. Respectively based on each historical trajectory in the historical trajectory set, predict the point cloud frame information corresponding to the backtracking moment, and obtain the second predicted target state information corresponding to each historical trajectory.

[0073] In some embodiments of the present application, the historical trajectory set includes one or more historical trajectories. Respectively based on each historical trajectory, by means of forward prediction, predict the point cloud frame information corresponding to the backtracking moment, and the point cloud frame information corresponding to the backtracking moment predicted based on the historical trajectory is called the second predicted target state information.

[0074] In some embodiments of the present application, for each historical trajectory in the historical trajectory set, based on the last point cloud frame information in the historical trajectory, according to the interpolation algorithm shown in the following formulas (4)-(6), predict the second predicted target state information corresponding to the backtracking moment. Among them, the last point cloud frame information refers to the point cloud frame information with the latest corresponding timestamp existing in the current recorded historical trajectory. Taking the Figure 2 track id1 corresponding trajectory as an example, the point cloud frame information with the timestamp of t k+1 is the last point cloud frame information of the track id1 corresponding trajectory.

[0075] X k =[x k y k vx k vy k ax k ay k T (4)

[0076] X″ = A k *X k (5)

[0077]

[0078] In the formula, X k ​Indicates the position and motion information of the target object in the last point cloud frame information in the historical trajectory, where x k Indicates the abscissa of the position, y k Indicates the ordinate of the position, vx k Indicates the horizontal velocity, vy k Indicates the vertical velocity, ax k Indicates the horizontal acceleration, ay k Indicates the vertical acceleration, X″ represents the motion information of the target object at the backtracking moment deduced from the last point cloud frame information, A k Indicates the process matrix from the last point cloud frame information to the backtracking moment, P k Indicates the position covariance matrix corresponding to the last point cloud frame information, Q k Indicates the process noise from the last point cloud frame information to the backtracking moment, P″ represents the position covariance estimate value from the last point cloud frame information to the backtracking moment. The position information and motion information determined based on X″ and P″ are used as the second predicted target state information corresponding to the backtracking moment predicted based on the historical trajectory.

[0079] Based on the above method, the second predicted target state information corresponding to each historical trajectory in the historical trajectory set can be obtained.

[0080] S134. Calculate the matching degree between the second predicted target state information corresponding to each historical trajectory and the first predicted target state information respectively.

[0081] In some embodiments of the present application, for each historical trajectory in the historical trajectory set, the matching degree between the second predicted target state information corresponding to the historical trajectory and the first predicted target state information is calculated to determine the matching degree between the historical trajectory and the first trajectory.

[0082] In some embodiments of the present application, the matching degree between the first predicted target state information and each second predicted target state information can be solved according to a general matching algorithm such as the Hungarian matching algorithm.

[0083] In some embodiments of the present application, when the trajectory generation module simultaneously receives the point cloud frame information of multiple target objects and optimizes the trajectories of multiple target objects at the same time, the first predicted target state information at the corresponding retrospective moment can be predicted respectively based on the point cloud frame information of each currently received target object. In this way, multiple pieces of first predicted target state information can be obtained. At this time, for the convenience of calculation, a list predNewObjList composed of multiple pieces of first predicted target state information and a list predInteruptObjList composed of all the second predicted target state information corresponding to the historical trajectory set are formed, and a similarity metric matrix between predNewObjList and predInteruptObjList is constructed. The similarity metric matrix includes, but is not limited to, similarity metric matrices such as Euclidean distance, Mahalanobis distance, shape similarity, or direction similarity. Then, the matching degree between the elements in predNewObjList and the elements in predInteruptObjList is solved according to a general matching algorithm such as the Hungarian matching algorithm.

[0084] S135. The historical trajectory corresponding to the second predicted target state information with a matching degree greater than the matching degree threshold is used as the second trajectory associated with the first trajectory.

[0085] In some embodiments of the present application, the matching degree threshold is set in advance according to actual requirements. After obtaining the matching degree between the first predicted target state information and the second predicted target state information, the matching degree is compared with the matching degree threshold. If the matching degree is greater than the matching degree threshold, it is determined that the historical trajectory corresponding to the matching degree is associated with the first trajectory, that is, it corresponds to the same target object. Therefore, the historical trajectory is used as the second trajectory associated with the first trajectory.

[0086] In some embodiments of the present application, when optimizing the trajectories of multiple target objects simultaneously, after solving the matching degree between the elements in predNewObjList and the elements in predInteruptObjList according to a general matching algorithm such as the Hungarian matching algorithm, an associated matching mapping table LinkIndexMap can be obtained, where LinkIndexMap records the index values of the first predicted target state information and the second predicted target state information with an associated matching relationship in their respective lists. So far, the associated matching result between the new trajectory and the historical trajectory corresponding to each target object can be obtained.

[0087] Through the above technical solution, the second trajectory associated with the first trajectory can be accurately determined.

[0088] In some embodiments, after determining the second trajectory associated with the first trajectory, in step S140 above, the missing point cloud frame information between the first trajectory and the second trajectory can be predicted based on the first predicted target state information, the second predicted target state information corresponding to the second trajectory, and a preset interpolation algorithm, to obtain the first completed point cloud frame information.

[0089] In some embodiments of the present application, the first completed point cloud frame information includes the point cloud frame information corresponding to the backtracking moment. When determining the second trajectory previously, the first predicted target state information at the backtracking moment predicted based on the first trajectory and the second predicted target state information at the backtracking moment predicted based on the second trajectory have been obtained. Based on this, the final target state information corresponding to the backtracking moment can be determined according to the following formulas (7)-(8) based on the first predicted target state information and the second predicted target state information predicted based on the second trajectory.

[0090] P=((P ′ ) -1 +(P″) -1 ) -1 (7)

[0091] X=P*((P ′ ) -1 *X ′ +(P″) -1 *X″) (8)

[0092] In the formula, P represents the position covariance matrix of the target object corresponding to the backtracking moment, and X represents the motion information of the target object corresponding to the backtracking moment. The position information and motion information determined based on P and X are used as the final target state information corresponding to the backtracking moment.

[0093] In addition, as mentioned above, in addition to the position information and motion information of the target object, the point cloud frame information also includes the size information of the target object. Therefore, when determining the point cloud frame information corresponding to the backtracking moment, in addition to determining the final target state information corresponding to the backtracking moment, the size information corresponding to the backtracking moment can also be determined according to the following formulas (9)-(10).

[0094]

[0095] Among them, and P dims respectively represent the size covariance matrices in the last point cloud frame information of the second trajectory, the first point cloud frame information, and the point cloud frame information corresponding to the backtracking moment, dims i and dims k respectively represent the size information in the last point cloud frame information of the second trajectory and the size information in the first point cloud frame information, dims jRepresents the size information corresponding to the backtracking moment.

[0096] The point cloud frame information composed of the final target state information and size information corresponding to the backtracking moment is used as the point cloud frame information corresponding to the backtracking moment.

[0097] In some embodiments of the present application, considering that there may be more than one target loss between the first trajectory and the second trajectory, in addition to the point cloud frame information corresponding to the backtracking moment, the first complemented point cloud frame information may also include the point cloud frame information of other target loss moments. Based on this, after obtaining the point cloud frame information corresponding to the backtracking moment, further, the point cloud frame information of other target loss moments can be predicted based on the point cloud frame information corresponding to the backtracking moment. The prediction method is the same as the method for predicting the point cloud frame information corresponding to the backtracking moment. Refer to the above formulas (1)-(10) and will not be elaborated here too much.

[0098] Finally, the obtained first complemented point cloud frame information is used to completely fill the interruption gap between the first trajectory and the second trajectory. The first trajectory is added to the end of the second trajectory to achieve the complete merger and update of the two matching trajectories.

[0099] Through the above method, the first complemented point cloud frame information can be accurately predicted.

[0100] In some embodiments, before performing the above step S130, candidate historical trajectories that meet the second condition can be first screened from the historical trajectories, and then the second trajectory associated with the first trajectory can be screened from the candidate historical trajectories in the above step S130.

[0101] Among them, the second condition includes: the interruption duration of the historical trajectory is greater than the second preset duration and less than the third preset duration, the third preset duration is greater than the second preset duration, and the interruption duration is the time difference between the timestamp of the last point cloud frame information in the historical trajectory and the current moment, that is, the duration corresponding to the interruption step of the historical trajectory. The purpose of setting the above conditions is to ensure that the candidate historical trajectory has been interrupted at the backtracking moment and the target object corresponding to the candidate historical trajectory cannot be lost for a long time. This is mainly considered that when the target object is lost for a long time, there will be a large error in predicting the point cloud frame information of the target object, which may cause incorrect matching, so this restriction is made.

[0102] Through the above method, the preliminary screening of the historical trajectories can be realized, the error of trajectory optimization can be reduced, and the quality of trajectory optimization can be improved.

[0103] In some embodiments, before performing the above step S120, the following steps S210-S230 can be first performed.

[0104] S210. Search for the target associated identifier pair containing the first trajectory identifier in the trajectory identifier mapping table. An associated identifier pair contains two associated trajectory identifiers.

[0105] In some embodiments of the present application, the trajectory optimization device maintains the trajectory identifier mapping table id_map in real time. The trajectory identifier mapping table is used to record the associated identifier pairs composed of trajectory identifiers with an associated relationship. In the initial state, the trajectory identifier mapping table id_map is empty. Based on this, after receiving the first point cloud frame information and the corresponding first trajectory identifier, the trajectory optimization device can query in the trajectory identifier mapping table id_map whether there is a trajectory identifier associated with the first trajectory identifier, and process it in different cases according to the query result.

[0106] S220. In the case where the target associated identifier pair is not found, it is determined that there is no trajectory identifier associated with the first trajectory identifier.

[0107] In some embodiments of the present application, if the target associated identifier pair is not found, it means that the target object corresponding to the first point cloud frame information appears for the first time. Therefore, in the above step S120, in the case where it is determined that there is no trajectory identifier associated with the first trajectory identifier, a first trajectory corresponding to the first trajectory identifier is created based on the first point cloud frame information.

[0108] Further, add the first trajectory to the trajectory information list TrackInfoList for maintenance and management.

[0109] In addition, in order to facilitate subsequent identification of whether the first trajectory has been created, an associated identifier pair containing only the first trajectory identifier can be added to the trajectory identifier mapping table. Taking the first trajectory identifier as track id2 as an example, add the key identifier pair track id2 —track id2 to id_map, indicating that this track id2 is associated with itself.

[0110] S230. In the case where the target associated identifier pair is found, it is determined that there is a trajectory identifier associated with the first trajectory identifier.

[0111] In some embodiments of the present application, if the target associated identifier pair is found, it means that the target object corresponding to the first point cloud frame information does not appear for the first time. The trajectory corresponding to the trajectory identifier associated with the first trajectory identifier is the trajectory corresponding to the target object. Therefore, at this time, there is no need to create a new trajectory. The target trajectory corresponding to the trajectory identifier associated with the first trajectory identifier can be updated based on the first point cloud frame information.

[0112] In some embodiments of the present application, updating the target trajectory corresponding to the trajectory identifier associated with the first trajectory identifier based on the first point cloud frame information may include adding the first point cloud frame information to the end of the target trajectory to extend the target trajectory, thereby achieving the update of the target trajectory.

[0113] By the above method, the creation of new trajectories can be effectively reduced, thereby improving the integrity of the trajectories corresponding to the same target object.

[0114] In some embodiments, considering that there may be an interruption gap between the target trajectory and the first point cloud frame information, the update of the target trajectory based on the first point cloud frame information can be achieved through the following steps S310 - S340.

[0115] S310. Determine whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory.

[0116] In some embodiments of the present application, to determine whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory, the following method can be adopted: determine whether the frame numbers of the first point cloud frame information and the last point cloud frame information in the target trajectory are continuous; in the case where the frame numbers of the first point cloud frame information and the last point cloud frame information in the target trajectory are not continuous, determine that there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory. When the target object can be continuously detected, the frame numbers of the point cloud frame information of the target object should be continuous. When the frame numbers corresponding to adjacent point cloud frame information are not continuous, it means that there is missing point cloud frame information between the adjacent point cloud frame information. Through this method, it can be accurately detected whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory.

[0117] In some other embodiments of the present application, it can also be determined whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory by judging whether the interruption step length of the target trajectory is 0. If the interruption step length of the target trajectory is not 0, it means that there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory. If the interruption step length of the target trajectory is 0, it means that there is no missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory. Through this method, it can be quickly detected whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory.

[0118] S320. In the case where there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory, based on the first point cloud frame information, the last point cloud frame information of the target trajectory, and a preset interpolation algorithm, predict the missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory to obtain the second complemented point cloud frame information.

[0119] In some embodiments of the present application, in the case where there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory, the target missing time between the two can be determined first, and then, for each target missing time, the point cloud frame information corresponding to the target missing time is predicted according to the interpolation algorithm shown in the above formulas (1)-(10), so as to obtain the second complemented point cloud frame information. For the specific prediction process, refer to the above relevant description. To avoid repetition, it will not be elaborated here too much.

[0120] S330. Update the target trajectory based on the second complemented point cloud frame information and the first point cloud frame information.

[0121] In some embodiments of the present application, when updating the target trajectory based on the second complemented point cloud frame information and the first point cloud frame information, the second complemented point cloud frame information and the first point cloud frame information can be added to the end of the target trajectory in the order of the time stamps of the point cloud frame information in the second complemented point cloud frame information and the first point cloud frame information from the earliest to the latest.

[0122] S340. In the case where there is no missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory, add the first point cloud frame information to the end of the target trajectory.

[0123] Through the above method, the missing point cloud frame information can be complemented to improve the integrity of the trajectory.

[0124] In some embodiments, in order to facilitate accurately identifying whether there is a trajectory identifier associated with the first trajectory based on the trajectory identifier mapping table, in the above step S150, the trajectory identifier of the second trajectory is used as the trajectory identifier of the new trajectory. For example, the first trajectory identifier is track in id2 , and the trajectory identifier of the second trajectory is track id1 , then the trajectory identifier of the spliced new trajectory is track id1 . And after the above step S150, the following steps are executed:

[0125] Update the associated identifier pair in the trajectory identifier mapping table that only contains the first trajectory identifier to the associated identifier pair composed of the first trajectory identifier and the trajectory identifier of the second trajectory.

[0126] In this way, when the point cloud frame information corresponding to the first trajectory identifier is received subsequently, the new trajectory can be directly updated based on the received point cloud frame information, thereby improving the integrity of the trajectory.

[0127] In some embodiments, after the above step S150, each trajectory in the maintained trajectory information list TrackInfoList can also be traversed and corresponding trajectory optimization can be performed, specifically as follows:

[0128] Count the mode of the categories of the target objects in the current trajectory and update the target category in the trajectory.

[0129] Count the sizes of the target objects in the current trajectory and sort them in descending order. Take the average of the top 20% of the sizes as the candidate size ref_dim_1. Obtain the reference size ref_dim_2 of the target objects of this category according to the updated target category, and take the larger of ref_dim_1 and ref_dim_2 as the final size of the target objects in the current trajectory.

[0130] According to the aforementioned final size and the original size of the target object, correspondingly update the center point position of the target object, which is mainly achieved based on the fixation of the target's nearest point and will not be elaborated here.

[0131] In addition, according to the time stamp, the complemented and optimized point cloud frame information FrameData at different times can be re-split from the trajectory information list TrackInfoList.

[0132] The trajectory optimization method provided by the embodiments of the present application, compared with the traditional tracking method based on historical data, maintains a trajectory information list, identifies historical trajectories for forward prediction, newly created trajectories for backward prediction, and correlates and matches the prediction results of the two to achieve reconnection between different trajectories, thereby obtaining a more complete target trajectory; on this basis, combining the information of the entire trajectory, optimizing the category, size, and center point position of the target object, realizing more accurate and stable target detection.

[0133] Based on the trajectory optimization method provided in the above embodiments, correspondingly, the present application also provides a specific implementation manner of a trajectory optimization device. Please refer to the following embodiments.

[0134] See Figure 3 FIG., a schematic diagram of the trajectory optimization device provided by the embodiments of the present application, as Figure 3 shown, the device 300 includes the following modules:

[0135] A receiving module 310, configured to receive first point cloud frame information and a first trajectory identifier corresponding to the first point cloud frame information;

[0136] A creation module 320, configured to create a first trajectory corresponding to a first trajectory identifier based on first point cloud frame information;

[0137] A screening module 330, configured to screen a second trajectory associated with the first trajectory from a set of historical trajectories when the first trajectory meets a preset first condition;

[0138] A prediction module 340, configured to predict missing point cloud frame information between the first trajectory and the second trajectory based on the first trajectory and the second trajectory, to obtain first complemented point cloud frame information;

[0139] A splicing module 350, configured to splice the first trajectory and the second trajectory into a new trajectory based on the first complemented point cloud frame information;

[0140] Wherein, the first condition includes that the time length of the first trajectory is greater than a first preset duration;

[0141] The trajectory identifier of the new trajectory is the trajectory identifier of the second trajectory.

[0142] In some embodiments, the first condition further includes that the time length of the first trajectory is less than a second preset duration, and the second preset duration is greater than the first preset duration; the screening module 330 is specifically configured to:

[0143] When the first trajectory meets the first condition, subtract the second preset duration from the timestamp of the first point cloud frame information to obtain a backtracking moment;

[0144] Based on the first trajectory, predict the target state information corresponding to the backtracking moment to obtain first predicted target state information;

[0145] Based on each historical trajectory in the set of historical trajectories respectively, predict the point cloud frame target state information corresponding to the backtracking moment to obtain second predicted target state information corresponding to each historical trajectory;

[0146] Calculate the matching degree between the second predicted target state information corresponding to each historical trajectory and the first predicted target state information respectively;

[0147] Use the historical trajectory corresponding to the second predicted target state information with a matching degree greater than the matching degree threshold as the second trajectory associated with the first trajectory.

[0148] In some embodiments, the prediction module 340 is specifically configured to:

[0149] Based on the first trajectory, the second trajectory and a preset interpolation algorithm, predict the missing point cloud frame information between the first trajectory and the second trajectory to obtain first complemented point cloud frame information.

[0150] In some embodiments, the apparatus 300 further includes: a primary selection module, configured to:

[0151] Before screening the second trajectory associated with the first trajectory from the historical trajectories, candidate historical trajectories that meet the second condition are screened from the historical trajectories;

[0152] A screening module 330, configured to:

[0153] Screen the second trajectory associated with the first trajectory from the candidate historical trajectories;

[0154] Wherein, the second condition includes:

[0155] The interruption duration of the historical trajectory is greater than a second preset duration and less than a third preset duration, the third preset duration is greater than the second preset duration, and the interruption duration is the time difference between the timestamp of the last point cloud frame information in the historical trajectory and the current moment.

[0156] In some embodiments, the apparatus 300 further includes: a lookup module, configured to:

[0157] Before creating the first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information, look up a target associated identifier pair containing the first trajectory identifier from the trajectory identifier mapping table, and one associated identifier pair contains two associated trajectory identifiers;

[0158] In the case where no target associated identifier pair is found, it is determined that there is no trajectory identifier associated with the first trajectory identifier;

[0159] In the case where a target associated identifier pair is found, it is determined that there is a trajectory identifier associated with the first trajectory identifier;

[0160] A creation module 320, configured to:

[0161] In the case where it is determined that there is no trajectory identifier associated with the first trajectory identifier, create the first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information.

[0162] In some embodiments, the apparatus 300 further includes: a trajectory update module, configured to:

[0163] In the case where there is a trajectory identifier associated with the first trajectory identifier, update the target trajectory corresponding to the trajectory identifier associated with the first trajectory identifier based on the first point cloud frame information.

[0164] In some embodiments, the trajectory update module is specifically configured to:

[0165] Determine whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory;

[0166] In the case where there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory, based on the first point cloud frame information, the last point cloud frame information of the target trajectory, and a preset interpolation algorithm, predict the missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory to obtain second complemented point cloud frame information;

[0167] Update the target trajectory based on the second complemented point cloud frame information and the first point cloud frame information.

[0168] In some embodiments, the trajectory update module is specifically configured to:

[0169] Determine whether the frame sequence number of the first point cloud frame information is continuous with the frame sequence number of the last point cloud frame information in the target trajectory;

[0170] In the case where the frame sequence number of the first point cloud frame information is not continuous with the frame sequence number of the last point cloud frame information in the target trajectory, determine that there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory.

[0171] In some embodiments, the apparatus 300 further includes: a mapping table update module, configured to:

[0172] In the case where the target association identifier pair is not found, add an association identifier pair containing only the first trajectory identifier to the trajectory identifier mapping table.

[0173] In some embodiments, the mapping table update module is further configured to:

[0174] The trajectory identifier of the new trajectory is the trajectory identifier of the second trajectory. After splicing the first trajectory and the second trajectory into a new trajectory, update the association identifier pair containing only the first trajectory identifier in the trajectory identifier mapping table to an association identifier pair composed of the first trajectory identifier and the trajectory identifier of the second trajectory.

[0175] The trajectory optimization apparatus provided by the embodiments of the present application can implement Figures 1 to 2 each process implemented by the method embodiments. To avoid repetition, it will not be elaborated here.

[0176] Figure 4 The hardware structure diagram of the electronic device provided by the embodiments of the present application is shown.

[0177] The electronic device 400 may include a processor 401 and a memory 402 storing computer program instructions.

[0178] Specifically, the above-mentioned processor 401 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present application.

[0179] The memory 402 may include a mass storage for data or instructions. By way of example and not limitation, the memory 402 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 402 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 402 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 402 is a non-volatile solid-state memory. The memory 402 may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory 402 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any of the trajectory optimization methods in the above embodiments.

[0180] The processor 401 reads and executes the computer program instructions stored in the memory 402 to implement any of the trajectory optimization methods in the above embodiments.

[0181] In one example, the electronic device 400 may further include a communication interface 403 and a bus 410. Among them, as Figure 4 shown, the processor 401, the memory 402, and the communication interface 403 are connected through the bus 410 to complete communication with each other.

[0182] The communication interface 403 is mainly used to implement communication between various modules, devices, units, and / or devices in the embodiments of the present application.

[0183] The bus 410 includes hardware, software, or both, and couples the components of the online data flow metering device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, the bus 410 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0184] In addition, in combination with the trajectory optimization method in the above embodiments, the embodiments of the present application can be implemented by providing a computer storage medium. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by a processor, any one of the trajectory optimization methods in the above embodiments is implemented.

[0185] The embodiments of the present application also provide a computer program product, including a computer program, and when the computer program is executed by a processor, any one of the methods for optimizing a trajectory in the above embodiments is implemented.

[0186] It should be clear that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0187] The functional blocks shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments for performing the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave. A "machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0188] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0189] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It should also be understood that each block in the block diagrams and / or flowcharts, and the combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware for performing the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0190] As described above, this is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. A trajectory optimization method, characterized in that, Including: Receiving the first point cloud frame information and the first trajectory identifier corresponding to the first point cloud frame information; Creating a first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information; When the first trajectory meets a preset first condition, screening a second trajectory associated with the first trajectory from a set of historical trajectories; Predicting missing point cloud frame information between the first trajectory and the second trajectory based on the first trajectory and the second trajectory to obtain first complemented point cloud frame information; Splicing the first trajectory and the second trajectory into a new trajectory based on the first complemented point cloud frame information; Wherein, the first condition includes that the time length of the first trajectory is greater than a first preset duration.

2. The method according to claim 1, wherein The first condition further includes that the time length of the first trajectory is less than a second preset duration, and the second preset duration is greater than the first preset duration; The step of, when the first trajectory meets a preset first condition, screening a second trajectory associated with the first trajectory from a set of historical trajectories includes: When the first trajectory meets the first condition, subtracting the second preset duration from the timestamp of the first point cloud frame information to obtain a retrospective moment; Predicting target state information corresponding to the retrospective moment based on the first trajectory to obtain first predicted target state information; Predicting point cloud frame target state information corresponding to the retrospective moment based on each historical trajectory in the set of historical trajectories respectively to obtain second predicted target state information corresponding to each historical trajectory; Calculating the matching degree between the second predicted target state information corresponding to each historical trajectory and the first predicted target state information respectively; Taking the historical trajectory corresponding to the second predicted target state information with a matching degree greater than a matching degree threshold as the second trajectory associated with the first trajectory.

3. The method according to claim 2, wherein The step of predicting missing point cloud frame information between the first trajectory and the second trajectory based on the first trajectory and the second trajectory to obtain first complemented point cloud frame information includes: Predicting missing point cloud frame information between the first trajectory and the second trajectory based on the first trajectory, the second trajectory and a preset interpolation algorithm to obtain first complemented point cloud frame information.

4. The method according to any one of claims 1 to 3, characterized in that, Before screening the second trajectory associated with the first trajectory from the historical trajectories, the method further includes: Screening candidate historical trajectories that meet a second condition from the historical trajectories; The step of screening the second trajectory associated with the first trajectory from the historical trajectories includes: Screening the second trajectory associated with the first trajectory from the candidate historical trajectories; Wherein, the second condition includes: The interruption duration of the historical trajectory is greater than a second preset duration and less than a third preset duration, the third preset duration is greater than the second preset duration, and the interruption duration is the time difference between the timestamp of the last point cloud frame information in the historical trajectory and the current moment.

5. The method according to any one of claims 1 to 4, characterized in that Before creating the first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information, the method further includes: Searching for a target associated identifier pair containing the first trajectory identifier from a trajectory identifier mapping table, and one associated identifier pair contains two associated trajectory identifiers; In the case where the target association identifier pair is not found, it is determined that there is no trajectory identifier associated with the first trajectory identifier; In the case where the target association identifier pair is found, it is determined that there is a trajectory identifier associated with the first trajectory identifier; The creating, based on the first point cloud frame information, of the first trajectory corresponding to the first trajectory identifier includes: In the case where it is determined that there is no trajectory identifier associated with the first trajectory identifier, creating, based on the first point cloud frame information, the first trajectory corresponding to the first trajectory identifier.

6. The method according to claim 5, characterized in that The method further includes: In the case where there is a trajectory identifier associated with the first trajectory identifier, determining whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory, where the target trajectory is the trajectory corresponding to the trajectory identifier associated with the first trajectory identifier; In the case where there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory, predicting, based on the first point cloud frame information, the last point cloud frame information of the target trajectory, and a preset interpolation algorithm, the missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory to obtain second complemented point cloud frame information; Updating the target trajectory based on the second complemented point cloud frame information and the first point cloud frame information.

7. The method according to claim 6, characterized in that, The determining whether there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory includes: Determining whether the frame sequence number of the first point cloud frame information is continuous with the frame sequence number of the last point cloud frame information in the target trajectory; In the case where the frame sequence number of the first point cloud frame information is not continuous with the frame sequence number of the last point cloud frame information in the target trajectory, determining that there is missing point cloud frame information between the first point cloud frame information and the last point cloud frame information of the target trajectory.

8. The method according to claim 5, characterized in that, In the case where the target association identifier pair is not found, the method further includes: Adding, to the trajectory identifier mapping table, an association identifier pair that only includes the first trajectory identifier.

9. The method according to claim 8, wherein The trajectory identifier of the new trajectory is the trajectory identifier of the second trajectory. After splicing the first trajectory and the second trajectory into a new trajectory, the method further includes: Updating, in the trajectory identifier mapping table, the association identifier pair that only includes the first trajectory identifier to an association identifier pair composed of the first trajectory identifier and the trajectory identifier of the second trajectory.

10. A trajectory optimization device, characterized in that, including: a receiving module, configured to receive first point cloud frame information and a first trajectory identifier corresponding to the first point cloud frame information; a creating module, configured to create a first trajectory corresponding to the first trajectory identifier based on the first point cloud frame information; a screening module, configured to screen, from a historical trajectory set, a second trajectory associated with the first trajectory in the case where the first trajectory meets a preset first condition; a predicting module, configured to predict, based on the first trajectory and the second trajectory, missing point cloud frame information between the first trajectory and the second trajectory to obtain first complemented point cloud frame information; A splicing module, configured to splice the first trajectory and the second trajectory into a new trajectory based on the first completed point cloud frame information; Wherein, the first condition includes that the time length of the first trajectory is greater than a first preset duration.

11. A computer storage medium, characterized in that, Computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, the trajectory optimization method according to any one of claims 1-9 is implemented.

Images