Track state generation method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a track state generation method and device, equipment, a medium and a program product. A specific embodiment of the method comprises the following steps: acquiring a historical track corresponding to a target moving object and a monitoring track state of the target moving object at a first target time; according to the historical trajectory, determining at least one first future trajectory state of the target moving object at the first target time by using a pre-trained trajectory prediction neural network model; according to the historical trajectory, determining at least one second future trajectory state of the target moving object at the first target time by using a kinematic model; and according to the at least one first future trajectory state, the at least one second future trajectory state and the monitoring trajectory state, generating an actual trajectory state of the target moving object at the first target time. The implementation mode is related to artificial intelligence, and the actual track state of the target moving object at the first target time can be accurately and efficiently generated.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of target detection, and more particularly to a trajectory state generation method, apparatus, device, medium, and program product. Background Art

[0002] Currently, multi-target tracking is a key technology for achieving environmental perception in autonomous driving systems. Determining the trajectory state of the tracked objects is crucial. Generating this trajectory state is typically done by directly using a kinematic model to predict the trajectory state of the target moving object at a target time.

[0003] However, the inventors have discovered that when the above method is adopted, the following technical problems often occur:

[0004] In the case of sudden changes in movement speed or direction, the kinematic model has limited predictive ability, resulting in a large deviation between the generated trajectory state and the actual trajectory state, making it impossible to effectively perform the target tracking task.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the Invention

[0006] The content of this disclosure is used to briefly introduce concepts that will be described in detail in the detailed description section below. The content of this disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure provide trajectory state generation methods, devices, equipment, media, and program products to solve the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a trajectory state generation method, including: obtaining a historical trajectory corresponding to a target moving object and a monitored trajectory state of the target moving object at a first target time; determining, based on the historical trajectory, at least one first future trajectory state of the target moving object at the first target time using a pre-trained trajectory prediction neural network model; determining, based on the historical trajectory, at least one second future trajectory state of the target moving object at the first target time using a kinematic model; and generating, based on the at least one first future trajectory state, the at least one second future trajectory state and the monitored trajectory state, an actual trajectory state of the target moving object at the first target time.

[0009] Optionally, the above-mentioned method of determining at least one first future trajectory state of the target moving object at a first target time based on the above-mentioned historical trajectory using a pre-trained trajectory prediction neural network model includes: performing data format conversion on the above-mentioned historical trajectory to generate trajectory conversion data under each historical segment; inputting each trajectory conversion data into the above-mentioned trajectory prediction neural network model to generate a first future trajectory state, thereby obtaining the above-mentioned at least one first future trajectory state.

[0010] Optionally, the trajectory prediction neural network model includes: a first multi-layer perceptron layer, a pooling layer, and a second multi-layer perceptron layer; and the step of inputting each of the trajectory conversion data into the trajectory prediction neural network model to generate a first future trajectory state includes: inputting the trajectory conversion data into the first multi-layer perceptron layer to generate a first output result; inputting the first output result into the pooling layer to generate a second output result; and inputting the second output result into the second multi-layer perceptron layer to generate the first future trajectory state.

[0011] Optionally, the kinematic model includes: a uniform linear motion model and a uniformly accelerated linear motion model; and the method of determining at least one second future trajectory state of the target moving object at the first target time based on the historical trajectory using the kinematic model includes: generating the at least one second future trajectory state based on the historical trajectory using the uniform linear motion model and the uniformly accelerated linear motion model.

[0012] Optionally, generating the actual trajectory state of the target moving object at the first target time based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state includes: determining, from the at least one first future trajectory state and the at least one second future trajectory state, a future trajectory state whose state similarity with the monitored trajectory state satisfies a target condition, as a target future trajectory state; combining the target future trajectory state and the monitored trajectory state to generate state combination information; and generating the actual trajectory state based on the state combination information.

[0013] Optionally, the above method also includes: based on the above actual trajectory state and the above historical trajectory, using the above trajectory prediction neural network model and kinematic model to generate the actual trajectory state of the above target moving object at a second target time, wherein the above second target time is the next future time of the above first target time.

[0014] Optionally, the method further includes: performing an obstacle avoidance operation for the target object at the first target time according to the actual trajectory state.

[0015] In a second aspect, some embodiments of the present disclosure provide a trajectory state generation device, including: an acquisition unit, configured to acquire a historical trajectory corresponding to a target moving object and a monitored trajectory state of the target moving object at a first target time; a first determination unit, configured to determine at least one first future trajectory state of the target moving object at the first target time based on the historical trajectory using a pre-trained trajectory prediction neural network model; a second determination unit, configured to determine at least one second future trajectory state of the target moving object at the first target time based on the historical trajectory using a kinematic model; a generation unit, configured to generate an actual trajectory state of the target moving object at the first target time based on the at least one first future trajectory state, the at least one second future trajectory state and the monitored trajectory state.

[0016] Optionally, the first determination unit can be configured to: perform data format conversion on the above-mentioned historical trajectory to generate trajectory conversion data under each historical segment; input each trajectory conversion data into the above-mentioned trajectory prediction neural network model to generate a first future trajectory state, thereby obtaining the above-mentioned at least one first future trajectory state.

[0017] Optionally, the above-mentioned trajectory prediction neural network model includes: a first multi-layer perceptron layer, a pooling layer and a second multi-layer perceptron layer; and the first determination unit can be configured to: input the above-mentioned trajectory conversion data into the above-mentioned first multi-layer perceptron layer to generate a first output result; input the above-mentioned first output result into the above-mentioned pooling layer to generate a second output result; input the above-mentioned second output result into the above-mentioned second multi-layer perceptron layer to generate the above-mentioned first future trajectory state.

[0018] Optionally, the kinematic model includes a uniform linear motion model and a uniformly accelerated linear motion model; and the second determination unit may be configured to generate the at least one second future trajectory state based on the historical trajectory using the uniform linear motion model and the uniformly accelerated linear motion model.

[0019] Optionally, the generation unit can be configured to: determine, from the at least one first future trajectory state and the at least one second future trajectory state, a future trajectory state whose state similarity with the monitored trajectory state satisfies a target condition, as a target future trajectory state; combine the target future trajectory state and the monitored trajectory state to generate state combination information; and generate the actual trajectory state based on the state combination information.

[0020] Optionally, the device also includes: based on the above-mentioned actual trajectory state and the above-mentioned historical trajectory, using the above-mentioned trajectory prediction neural network model and kinematic model to generate the actual trajectory state of the above-mentioned target moving object at a second target time, wherein the above-mentioned second target time is the next future time of the above-mentioned first target time.

[0021] Optionally, the device further includes: performing an object obstacle avoidance operation for the target object at the first target time according to the actual trajectory state.

[0022] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0023] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0024] In a fifth aspect, some embodiments of the present disclosure provide a computer program product, including a computer program, which implements the method described in any implementation manner in the first aspect when executed by a processor.

[0025] The above-described various embodiments of the present disclosure have the following beneficial effects: Through the trajectory state generation methods of some embodiments of the present disclosure, the actual trajectory state of a target moving object at a first target time can be accurately and efficiently generated. Specifically, the reason for the inaccuracy of the relevant trajectory state is that the kinematic model has limited predictive capabilities when there are sudden changes in motion speed or direction, resulting in a significant deviation between the generated trajectory state and the actual trajectory state, making it impossible to effectively perform the target tracking task. Based on this, the trajectory state generation methods of some embodiments of the present disclosure first obtain the historical trajectory corresponding to the target moving object and the monitored trajectory state of the target moving object at the first target time. Here, the historical trajectory is used to facilitate the subsequent prediction of the predicted trajectory state at the first target time. The obtained monitored trajectory state is used as detection data to participate in the subsequent generation of the actual trajectory state. Then, based on the historical trajectory, a pre-trained trajectory prediction neural network model can be used to accurately determine at least one first future trajectory state of the target moving object at the first target time. Here, the trajectory prediction neural network model can better explore the motion patterns of the trajectory and predict the potential trend of the trajectory. In addition, the at least one first future trajectory state obtained represents a possible state of the trajectory at the first target time. These predicted trajectory states are generated using a deep learning model and can cover a wider range of motion state variations. Furthermore, based on the historical trajectory, a kinematic model is used to determine at least one second future trajectory state of the target moving object at the first target time. This takes into account variations in the trajectory state predicted using the kinematic model, providing more possible states for the subsequent generation of the actual trajectory state. Finally, based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state, the actual trajectory state of the target moving object at the first target time can be accurately generated. By comprehensively considering the predicted trajectory states generated using different methods, the impact of the uncertainty of the object's motion state is mitigated, reducing the occurrence of trajectory state fragmentation. In summary, by combining a deep learning model with a kinematic model to predict trajectory states, it covers a wider range of motion state variations, mitigates the impact of uncertainty in the object's motion state, effectively reduces the occurrence of trajectory state fragmentation, and improves the accuracy of the generation of the actual trajectory state. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0027] Figure 1is a schematic diagram of an application scenario of the trajectory state generation method according to some embodiments of the present disclosure;

[0028] Figure 2 is a flow chart of some embodiments of the trajectory state generation method according to the present disclosure;

[0029] Figure 3 is a flow chart of other embodiments of the trajectory state generation method according to the present disclosure;

[0030] Figure 4 is a schematic diagram of various historical segments in some embodiments of the trajectory state generation method according to the present disclosure;

[0031] Figure 5 is a schematic structural diagram of some embodiments of the trajectory state generating device according to the present disclosure;

[0032] Figure 6 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0033] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0034] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other.

[0035] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0036] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0037] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0038] Before performing operations such as the collection, storage, and use of user personal information (such as historical tracks) involved in this disclosure, relevant organizations or individuals must fulfill their obligations, including conducting personal information security impact assessments, fulfilling their obligations to inform the personal information subjects, and obtaining the prior authorization and consent of the personal information subjects.

[0039] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0040] Figure 1 Schematic diagram of an application scenario of the trajectory state generation method according to some embodiments of the present disclosure.

[0041] exist Figure 1 In this application scenario, electronic device 101 can first obtain the historical trajectory corresponding to the target moving object and the monitored trajectory state 102 of the target moving object at a first target time. In this application scenario, the target moving object can be vehicle A. Monitored trajectory state 102 can be "position 1, direction angle A, 30 m / s." Then, based on the historical trajectory, electronic device 101 can use pre-trained trajectory prediction neural network model 103 to determine at least one first future trajectory state of the target moving object at the first target time. In this application scenario, the at least one first future trajectory state includes: first future trajectory state 104 and first future trajectory state 105. First future trajectory state 104 can be "position 2, direction angle B, 28 m / s." First future trajectory state 105 can be "position 3, direction angle C, 31 m / s." Next, based on the historical trajectory, electronic device 101 can use kinematic model 106 to determine at least one second future trajectory state of the target moving object at the first target time. In this application scenario, the at least one second future trajectory state includes: a second future trajectory state 107 and a second future trajectory state 108. Second future trajectory state 107 may be "position 2, direction angle B, 28 m / s." Second future trajectory state 108 may be "position 3, direction angle C, 31 m / s." Finally, electronic device 101 may generate an actual trajectory state 109 of the target moving object at the first target time based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state 102. In this application scenario, actual trajectory state 109 may be "position 1, direction angle B, 28 m / s."

[0042] It should be noted that the electronic device 101 can be hardware or software. When the electronic device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the electronic device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0043] It should be understood that Figure 1 The number of electronic devices in the embodiment is merely illustrative. Any number of electronic devices may be provided according to implementation requirements.

[0044] Continue to refer Figure 2 , shows a process 200 of some embodiments of the trajectory state generation method according to the present disclosure. The trajectory state generation method includes the following steps:

[0045] Step 201 : obtaining a historical trajectory corresponding to a target moving object and a monitoring trajectory state of the target moving object at a first target time.

[0046] In some embodiments, the execution subject of the above trajectory state generation method (for example Figure 1 The electronic device 101 shown can obtain the historical trajectory corresponding to the target moving object and the monitored trajectory status of the target moving object at a first target time via a wired or wireless connection. The target moving object may be a moving object whose corresponding trajectory status at the first target time is to be determined. The first target time may be a pre-set time point in the future. For example, if the current time is 6:00, the corresponding first target time may be 6:05. The trajectory status may be the motion status of the target moving object at the corresponding time point. In practice, the trajectory status may include, but is not limited to, at least one of the following: object motion direction, object speed, object acceleration, and object position. For different scenarios, the target moving object may be a corresponding entity. For example, in an autonomous driving scenario, the corresponding target moving object may be an autonomous vehicle. In a high-speed driving scenario, the corresponding target moving object may be a high-speed vehicle. The historical trajectory may be the trajectory information of the target moving object during a target historical time period. In practice, the historical trajectory includes the historical trajectory status at various historical time points. The monitored trajectory status may be the status result of monitoring the corresponding motion status of the target moving object at the first target time. In other words, the monitored trajectory status is the trajectory status detected by relevant status monitoring tools.

[0047] Step 202 : Based on the historical trajectory, a pre-trained trajectory prediction neural network model is used to determine at least one first future trajectory state of the target moving object at the first target time.

[0048] In some embodiments, the execution entity may determine at least one first future trajectory state of the target moving object at the first target time based on the historical trajectory using a pre-trained trajectory prediction neural network model. The first future trajectory state may be the trajectory state predicted by the target moving object at the first target time. The trajectory prediction neural network model may be a neural network model used to generate a trajectory prediction state (i.e., a future trajectory state). The trajectory prediction neural network model may be generated based on a conventional model training method. For example, the trajectory prediction neural network model may be a time series neural network model.

[0049] As an example, first, the above-mentioned execution entity can obtain the historical trajectory state of the historical trajectory at at least one historical time point to obtain at least one historical trajectory state. Then, obtain at least one trajectory prediction neural network model, wherein the model structure corresponding to each trajectory prediction neural network model is different. In practice, at least one trajectory prediction neural network model may include: a recurrent neural network (RNN) model, a long short-term memory network (LSTM) model, and a gated recurrent unit (GRU) network model. Finally, at least one historical trajectory state is input into each trajectory prediction neural network model in at least one trajectory prediction neural network model to generate a first future trajectory state, and obtain at least one first future trajectory state.

[0050] Step 203 : Based on the historical trajectory, at least one second future trajectory state of the target moving object at the first target time is determined using a kinematic model.

[0051] In some embodiments, the execution entity may determine at least one second future trajectory state of the target moving object at the first target time using a kinematic model based on the historical trajectory. The kinematic model may be a model related to kinematics. For example, the kinematic model may be one of the following: a uniform motion model or a uniformly accelerated motion model.

[0052] As an example, the execution entity may determine at least one second future trajectory state of the target moving object at a first target time based on the historical trajectory using a uniform acceleration model and a uniform acceleration motion model.

[0053] In some optional implementations of some embodiments, the above-mentioned kinematic model includes: a uniform linear motion model and a uniformly accelerated linear motion model. Among them, the uniform linear motion model can be a model for determining the trajectory state in a uniform linear motion scenario. The uniformly accelerated linear motion model can be a model for determining the trajectory state in a uniformly accelerated linear motion scenario. Here, in the uniform linear motion model, the speed and position at the next moment are predicted based on the current speed and position of the target moving object. In the uniformly accelerated motion model, the parameters in the kinematic model, such as speed and acceleration, are estimated based on the historical trajectory, and the speed and position at the next moment are predicted in combination with the current speed and position.

[0054] Optionally, based on the historical trajectory, the at least one second future trajectory state is generated using the uniform linear motion model and the uniformly accelerated linear motion model. The at least one second future trajectory state includes: a second future trajectory state predicted by the uniform linear motion model and a second future trajectory state predicted by the uniformly accelerated linear motion model.

[0055] Step 204 : generating an actual trajectory state of the target moving object at the first target time according to the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state.

[0056] In some embodiments, the execution entity may generate an actual trajectory state of the target moving object at the first target time based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state. The actual trajectory state may be a more accurate trajectory state of the target moving object at the first target time.

[0057] As an example, the execution entity may first combine at least one future trajectory state, at least one second future trajectory state, and the monitored trajectory state to generate a trajectory state set. Then, the trajectory state with the highest number of repetitions of the corresponding trajectory state is selected from the trajectory state set as the actual trajectory state.

[0058] In some optional implementations of some embodiments, the execution entity may generate the actual trajectory state of the target moving object at the first target time based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state, including the following steps:

[0059] In the first step, a future trajectory state whose state similarity with the monitored trajectory state satisfies a target condition is determined from the at least one first future trajectory state and the at least one second future trajectory state, and is used as the target future trajectory state. State similarity can represent the degree of similarity between two trajectory states. State similarity can be a value between 0 and 1. A higher value indicates greater similarity between the two trajectory states. The target condition can be the future trajectory state with the highest state similarity.

[0060] As an example, first, the execution entity may perform state vectorization on each of the at least one first future trajectory state and the at least one second future trajectory state to generate at least one first state vector and at least one second state vector. Then, the monitoring trajectory state is state vectorized to generate a monitoring state vector. Furthermore, the state vector having the highest cosine similarity with the monitoring state vector is selected from the at least one first state vector and the at least one second state vector as the target state vector. Finally, the first future trajectory state or the second future trajectory state corresponding to the target state vector is determined as the target future trajectory state.

[0061] In the second step, the target future trajectory state and the monitoring trajectory state are combined to generate state combination information.

[0062] The third step is to generate the above-mentioned actual trajectory state according to the above-mentioned state combination information.

[0063] As an example, the execution entity may generate the actual trajectory state according to the state combination information using a Kalman filter algorithm.

[0064] In some optional implementations of some embodiments, after step 204, the steps further include:

[0065] The execution subject can generate the actual trajectory state of the target moving object at the second target time based on the actual trajectory state and the historical trajectory using the trajectory prediction neural network model and the kinematic model. The second target time is the next future time after the first target time. For example, if the first target time is 6:05, the corresponding second target time is 6:10. The generation of the actual trajectory state at the second target time can refer to the generation of the actual trajectory state at the first target time.

[0066] In some optional implementations of some embodiments, after step 204, the steps further include:

[0067] The execution subject may perform an obstacle avoidance operation on the target object within the first target time according to the actual trajectory state.

[0068] As an example, the execution subject may adjust the trajectory state of the target object at the first target time according to the actual trajectory state to avoid a state conflict with the actual trajectory state, thereby performing an obstacle avoidance operation on the target object.

[0069] The above-described various embodiments of the present disclosure have the following beneficial effects: Through the trajectory state generation methods of some embodiments of the present disclosure, the actual trajectory state of a target moving object at a first target time can be accurately and efficiently generated. Specifically, the reason for the inaccuracy of the relevant trajectory state is that the kinematic model has limited predictive capabilities when there are sudden changes in motion speed or direction, resulting in a significant deviation between the generated trajectory state and the actual trajectory state, making it impossible to effectively perform the target tracking task. Based on this, the trajectory state generation methods of some embodiments of the present disclosure first obtain the historical trajectory corresponding to the target moving object and the monitored trajectory state of the target moving object at the first target time. Here, the historical trajectory is used to facilitate the subsequent prediction of the predicted trajectory state at the first target time. The obtained monitored trajectory state is used as detection data to participate in the subsequent generation of the actual trajectory state. Then, based on the historical trajectory, a pre-trained trajectory prediction neural network model can be used to accurately determine at least one first future trajectory state of the target moving object at the first target time. Here, the trajectory prediction neural network model can better explore the motion patterns of the trajectory and predict the potential trend of the trajectory. In addition, the at least one first future trajectory state obtained represents a possible state of the trajectory at the first target time. These predicted trajectory states are generated using a deep learning model and can cover a wider range of motion state variations. Furthermore, based on the historical trajectory, a kinematic model is used to determine at least one second future trajectory state of the target moving object at the first target time. This takes into account variations in the trajectory state predicted using the kinematic model, providing more possible states for the subsequent generation of the actual trajectory state. Finally, based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state, the actual trajectory state of the target moving object at the first target time can be accurately generated. By comprehensively considering the predicted trajectory states generated using different methods, the impact of the uncertainty of the object's motion state is mitigated, reducing the occurrence of trajectory state fragmentation. In summary, by combining a deep learning model with a kinematic model to predict trajectory states, it covers a wider range of motion state variations, mitigates the impact of uncertainty in the object's motion state, effectively reduces the occurrence of trajectory state fragmentation, and improves the accuracy of the generation of the actual trajectory state.

[0070] Further references Figure 3, shows a process 300 of another embodiment of the trajectory state generation method according to the present disclosure. The trajectory state generation method includes the following steps:

[0071] Step 301: Obtain a historical trajectory corresponding to a target moving object and a monitored trajectory state of the target moving object at a first target time.

[0072] Step 302 : performing data format conversion on the historical trajectory to generate trajectory conversion data for each historical segment.

[0073] In some embodiments, the execution entity (e.g. Figure 1 The electronic device 101 shown can convert the data format of the historical trajectory to generate trajectory conversion data for each historical segment. A historical segment can be a segment consisting of various historical time points. Each historical segment corresponds to a different time period. The trajectory conversion data for each historical segment can be the historical trajectory status at each historical time point in the historical segment.

[0074] In practice, Figure 4 Schematic diagrams of various historical segments are shown.

[0075] like Figure 4 As shown, for the current time t, the corresponding historical time period is [t-5, t]. Each historical segment can include: a first historical segment, a second historical segment, and a third historical segment. The first historical segment can be [t-5, t-3]. The second historical segment can be [t-4, t-2]. The third historical segment can be [t-3, t-1]. The first historical segment corresponds to the predicted trajectory state at t-2, the trajectory state at t-1, and the trajectory state at t. The second historical segment corresponds to the predicted trajectory state at t-1, the trajectory state at t, and the trajectory state at t+1. The third historical segment corresponds to the predicted trajectory state at t, the trajectory state at t+1, and the trajectory state at t+2.

[0076] Step 303 : Input each of the trajectory conversion data into the trajectory prediction neural network model to generate a first future trajectory state, thereby obtaining the at least one first future trajectory state.

[0077] In some embodiments, the execution entity may input each of the trajectory conversion data into the trajectory prediction neural network model to generate a first future trajectory state, thereby obtaining the at least one first future trajectory state. The at least one first future trajectory state is a future trajectory state at the first target time in each historical segment.

[0078] In some optional implementations of some embodiments, the trajectory prediction neural network model includes: a first multi-layer perceptron layer, a pooling layer, and a second multi-layer perceptron layer. In practice, the pooling layer may be a maximum pooling layer.

[0079] Optionally, the execution entity may input each of the trajectory conversion data into the trajectory prediction neural network model to generate a first future trajectory state, including the following steps:

[0080] In the first step, the trajectory conversion data is input into the first multi-layer perceptron layer to generate a first output result.

[0081] In the second step, the first output result is input to the pooling layer to generate a second output result. In practice, the second output result can be a maximum pooling result.

[0082] In the third step, the second output result is input into the second multi-layer perceptron layer to generate the first future trajectory state.

[0083] Step 304 : Determine at least one second future trajectory state of the target moving object at the first target time based on the historical trajectory and using a kinematic model.

[0084] Step 305 : generating an actual trajectory state of the target moving object at the first target time according to the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state.

[0085] In some embodiments, the specific implementation of steps 301, 304-305 and the technical effects thereof can be referred to in Figure 2 Steps 201, 203-204 in the corresponding embodiment will not be repeated here.

[0086] from Figure 3 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 3 In the process 300 of the trajectory state generation method in some corresponding embodiments, the state of the trajectory at a future moment can be more comprehensively predicted by using the historical trajectory state data in each historical segment.

[0087] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a trajectory state generating device. These device embodiments are similar to Figure 2 Corresponding to the method embodiments shown, the trajectory state generating device can be specifically applied to various electronic devices.

[0088] like Figure 5 As shown, a trajectory state generating device 500 includes: an acquisition unit 501, a first determination unit 502, a second determination unit 503, and a generation unit 504. The acquisition unit 501 is configured to acquire a historical trajectory corresponding to a target moving object and a monitored trajectory state of the target moving object at a first target time; the first determination unit 502 is configured to determine at least one first future trajectory state of the target moving object at the first target time based on the historical trajectory using a pre-trained trajectory prediction neural network model; the second determination unit 503 is configured to determine at least one second future trajectory state of the target moving object at the first target time based on the historical trajectory using a kinematic model; and the generation unit 504 is configured to generate an actual trajectory state of the target moving object at the first target time based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state.

[0089] In some optional implementations of some embodiments, the first determination unit 502 may be further configured to: perform data format conversion on the historical trajectory to generate trajectory conversion data for each historical segment; and input each trajectory conversion data into the trajectory prediction neural network model to generate a first future trajectory state, thereby obtaining the at least one first future trajectory state.

[0090] In some optional implementations of some embodiments, the trajectory prediction neural network model includes: a first multi-layer perceptron layer, a pooling layer, and a second multi-layer perceptron layer; and the first determination unit 502 can be further configured to: input the trajectory conversion data into the first multi-layer perceptron layer to generate a first output result; input the first output result into the pooling layer to generate a second output result; and input the second output result into the second multi-layer perceptron layer to generate the first future trajectory state.

[0091] In some optional implementations of some embodiments, the above-mentioned kinematic model includes: a uniform linear motion model and a uniformly accelerated linear motion model; and the second determination unit 503 can be further configured to: generate the above-mentioned at least one second future trajectory state based on the above-mentioned historical trajectory using the above-mentioned uniform linear motion model and the above-mentioned uniformly accelerated linear motion model.

[0092] In some optional implementations of some embodiments, the generation unit 504 may be further configured to: determine, from the at least one first future trajectory state and the at least one second future trajectory state, a future trajectory state whose state similarity with the monitored trajectory state satisfies a target condition, as a target future trajectory state; combine the target future trajectory state and the monitored trajectory state to generate state combination information; and generate the actual trajectory state based on the state combination information.

[0093] In some optional implementations of some embodiments, the apparatus 500 further includes a state generation unit (not shown). The state generation unit may be configured to generate, based on the actual trajectory state and the historical trajectory, the actual trajectory state of the target moving object at a second target time using the trajectory prediction neural network model and the kinematic model, where the second target time is a future time immediately after the first target time.

[0094] In some optional implementations of some embodiments, the apparatus 500 further includes an execution unit (not shown), wherein the execution unit may be configured to execute an obstacle avoidance operation for the target object at the first target time according to the actual trajectory state.

[0095] It is understandable that the various units described in the trajectory state generating device 500 are similar to those described in the reference Figure 2 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the trajectory state generating device 500 and the units included therein, and will not be repeated here.

[0096] Reference below Figure 6 , which shows an electronic device (eg, Figure 1 Schematic diagram of the structure of the electronic device 101)600. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0097] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory 602 or a program loaded from a storage device 608 into a random access memory 603. Various programs and data required for the operation of the electronic device 600 are also stored in the random access memory 603. The processing device 601, the read-only memory 602, and the random access memory 603 are connected to each other via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0098] Typically, the following devices may be connected to the input / output interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0099] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from a network via the communication device 609, or installed from the storage device 608, or installed from the read-only memory 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are performed.

[0100] It should be noted that in some embodiments of the present disclosure, the computer-readable medium mentioned above may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0101] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0102] The computer-readable medium may be included in the electronic device, or may exist independently and not be incorporated into the electronic device. The computer-readable medium carries one or more programs. When executed by the electronic device, the electronic device: obtains a historical trajectory corresponding to the target moving object and a monitored trajectory state of the target moving object at a first target time; determines at least one first future trajectory state of the target moving object at the first target time using a pre-trained trajectory prediction neural network model based on the historical trajectory; determines at least one second future trajectory state of the target moving object at the first target time using a kinematic model based on the historical trajectory; and generates an actual trajectory state of the target moving object at the first target time based on the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state.

[0103] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising an acquisition unit, a first determination unit, a second determination unit, and a generation unit. The names of these units do not, in some cases, constitute a limitation on the units themselves. For example, the acquisition unit may also be described as a "unit for acquiring the historical trajectory corresponding to the target moving object and the monitoring trajectory status of the above-mentioned target moving object at the first target time".

[0106] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0107] Some embodiments of the present disclosure further provide a computer program product, including a computer program, which implements any of the above-mentioned trajectory state generation methods when executed by a processor.

[0108] The above description is only an illustration of some preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A trajectory state generation method, comprising: Acquire a historical trajectory corresponding to a target moving object and a monitored trajectory state of the target moving object at a first target time; Determining at least one first future trajectory state of the target moving object at the first target time based on the historical trajectory using a pre-trained trajectory prediction neural network model; Determining at least one second future trajectory state of the target moving object at the first target time using a kinematic model based on the historical trajectory; An actual trajectory state of the target moving object at the first target time is generated according to the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state.

2. The method according to claim 1, wherein Determining at least one first future trajectory state of the target moving object at the first target time based on the historical trajectory using a pre-trained trajectory prediction neural network model includes: Performing data format conversion on the historical trajectory to generate trajectory conversion data for each historical segment; Each trajectory conversion data in each trajectory conversion data is input into the trajectory prediction neural network model to generate a first future trajectory state, thereby obtaining the at least one first future trajectory state.

3. The method according to claim 2, wherein: The trajectory prediction neural network model includes: a first multi-layer perceptron layer, a pooling layer, and a second multi-layer perceptron layer; and Inputting each of the trajectory conversion data into the trajectory prediction neural network model to generate a first future trajectory state includes: Inputting the trajectory conversion data into the first multi-layer perceptron layer to generate a first output result; Inputting the first output result into the pooling layer to generate a second output result; The second output result is input into the second multi-layer perceptron layer to generate the first future trajectory state.

4. The method according to claim 1, wherein The kinematic models include: a uniform linear motion model and a uniformly accelerated linear motion model; and Determining at least one second future trajectory state of the target moving object at the first target time using a kinematic model based on the historical trajectory includes: According to the historical trajectory, the at least one second future trajectory state is generated using the uniform linear motion model and the uniformly accelerated linear motion model.

5. The method according to claim 1, wherein The step of generating the actual trajectory state of the target moving object at the first target time according to the at least one first future trajectory state, the at least one second future trajectory state, and the monitored trajectory state includes: determining, from the at least one first future trajectory state and the at least one second future trajectory state, a future trajectory state whose state similarity with the monitored trajectory state satisfies a target condition as a target future trajectory state; Combining the target future trajectory state and the monitoring trajectory state to generate state combination information; The actual trajectory state is generated according to the state combination information.

6. The method according to claim 1, wherein The method further comprises: According to the actual trajectory state and the historical trajectory, the trajectory prediction neural network model and the kinematic model are used to generate the actual trajectory state of the target moving object at a second target time, wherein the second target time is a next future time after the first target time.

7. The method according to claim 1, wherein The method further comprises: An object obstacle avoidance operation is performed for a target object at the first target time according to the actual trajectory state.

8. A trajectory state generating device, comprising: An acquisition unit is configured to acquire a historical trajectory corresponding to a target moving object and a monitored trajectory state of the target moving object at a first target time; a first determining unit configured to determine, based on the historical trajectory and using a pre-trained trajectory prediction neural network model, at least one first future trajectory state of the target moving object at the first target time; a second determining unit configured to determine, based on the historical trajectory and using a kinematic model, at least one second future trajectory state of the target moving object at the first target time; The generating unit is configured to generate an actual trajectory state of the target moving object at the first target time according to the at least one first future trajectory state, the at least one second future trajectory state and the monitored trajectory state.

9. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images