Obstacle trajectory prediction method, device, equipment and medium
Through dynamic obstacle type and data-driven prediction model adjustment, the problem of real-time and accuracy of dynamic obstacle trajectory prediction in unmanned driving equipment is solved, and the accuracy and safety of path planning are improved.
Patent Information
- Application Number
- CN202011073105.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-09
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-10-09
AI Technical Summary
In the prediction of dynamic obstacle motion trajectory, existing unmanned driving equipment cannot take into account both the real-time and accuracy of prediction, resulting in unreasonable path planning.
By obtaining the type and image of dynamic obstacles or point cloud data, selecting the appropriate predictive model level and number, adjusting the prediction model to optimize motion trajectory prediction, ensuring a balance of prediction real-time and accuracy.
It realizes that the accuracy of dynamic obstacle motion trajectory prediction is improved while meeting the real-time prediction of prediction, optimizes the path planning of unmanned driving equipment, and reduces the risk of collision.
Smart Images

Figure CN112255628B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of unmanned driving technology, and in particular to an obstacle trajectory prediction method, device, equipment and medium. Background Art
[0002] In unmanned driving technology, the automatic driving system in the unmanned driving equipment will plan the path based on the acquired conditions around the equipment.
[0003] When planning the path of an unmanned vehicle, in order to enable the unmanned vehicle to avoid surrounding obstacles, it is necessary to predict the movement trajectory of the obstacles. Obstacles generally include static obstacles and dynamic obstacles. Dynamic obstacles can be roughly classified as vehicles, pedestrians or other objects in motion. At present, when predicting the movement trajectory of dynamic obstacles, a prediction model is usually used to predict the movement trajectory of dynamic obstacles based on the movement status of dynamic obstacles in historical time periods.
[0004] Since the prediction accuracy of the prediction model can be high or low, if the prediction accuracy of the prediction model is set relatively high, it will take a long time to predict the obstacle movement trajectory, thereby affecting the real-time path planning of the unmanned driving equipment; if the prediction accuracy of the prediction model is set relatively low, the obstacle movement trajectory prediction accuracy will be low, resulting in unreasonable path planning for the unmanned driving equipment. Summary of the invention
[0005] Embodiments of the present invention provide an obstacle trajectory prediction method, apparatus, device and medium to optimize the obstacle trajectory prediction method and take into account both the real-time prediction and the prediction accuracy of the obstacle trajectory.
[0006] In a first aspect, an embodiment of the present invention provides an obstacle trajectory prediction method, comprising:
[0007] Acquire the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device;
[0008] According to the type of the dynamic obstacle, a first level prediction model is selected from multiple level prediction models of the same type;
[0009] According to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, the level of the prediction model or the number of dynamic obstacles is adjusted to obtain the optimal level of the prediction model or the optimal number of the dynamic obstacles;
[0010] Control the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or control the prediction model to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
[0011] In a second aspect, an embodiment of the present invention further provides an obstacle trajectory prediction device, comprising:
[0012] An acquisition module, used to acquire the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device;
[0013] A selection module, configured to select a first-level prediction model from multiple level prediction models of the same type according to the type of the dynamic obstacle;
[0014] An adjustment module, configured to adjust the level of the prediction model or the number of dynamic obstacles according to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, so as to obtain an optimal level of the prediction model or an optimal number of the dynamic obstacles;
[0015] A prediction module is used to control the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or to control the prediction model to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
[0016] In a third aspect, an embodiment of the present invention further provides an electronic device, including:
[0017] one or more processors;
[0018] a storage device for storing one or more programs,
[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement any obstacle trajectory prediction method described in any one of the embodiments of the present invention.
[0020] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any obstacle trajectory prediction method described in any of the embodiments of the present invention.
[0021] The technical solution disclosed in the embodiment of the present invention has the following beneficial effects:
[0022] By obtaining the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device, a first-level prediction model is selected from multiple level prediction models of the same type according to the type of the dynamic obstacle, and the level of the prediction model or the number of dynamic obstacles is adjusted according to the relationship between the prediction duration and value of the first-level prediction and the duration threshold, so as to obtain the optimal level of the prediction model or the optimal number of dynamic obstacles, and then the optimal level prediction model is controlled to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or the prediction model is controlled to predict the motion trajectory of the optimal number of dynamic obstacles based on the image or point cloud data of the optimal number of dynamic obstacles. Thus, the problem that the existing dynamic obstacle motion trajectory prediction method is incompatible with the prediction real-time performance and prediction accuracy when dynamic obstacles appear during the driving process of the unmanned driving device is solved, and the prediction model matching the dynamic obstacle can be allowed to be upgraded to improve the prediction accuracy of the prediction model under the premise of satisfying the prediction real-time performance, thereby optimizing the obstacle trajectory prediction method and effectively taking into account the prediction real-time performance and prediction accuracy of the obstacle trajectory. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 is a flow chart of an obstacle trajectory prediction method provided by Embodiment 1 of the present invention;
[0024] Figure 2 is a flow chart of an obstacle trajectory prediction method provided by Embodiment 2 of the present invention;
[0025] Figure 2 (a) is a schematic diagram of a flow chart of determining an obstacle score of each dynamic obstacle provided in Embodiment 2 of the present invention;
[0026] Figure 3 is a flow chart of an obstacle trajectory prediction method provided by Embodiment 3 of the present invention;
[0027] Figure 4 is a flow chart of an obstacle trajectory prediction method provided by Embodiment 4 of the present invention;
[0028] Figure 4 (a) is a schematic diagram of a process for upgrading a first-level prediction model provided by Embodiment 4 of the present invention;
[0029] Figure 5 is a structural schematic diagram of an obstacle trajectory prediction device provided by Embodiment 5 of the present invention;
[0030] Figure 6 A schematic diagram of the structure of an electronic device provided in Embodiment 6 of the present invention. DETAILED DESCRIPTION
[0031] The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention, rather than to limit the embodiments of the present invention. It is also necessary to explain that, for ease of description, only parts related to the embodiments of the present invention are shown in the accompanying drawings, rather than all structures.
[0032] An obstacle trajectory prediction method, device, equipment and medium provided by an embodiment of the present invention are described below with reference to the accompanying drawings.
[0033] Embodiment 1
[0034] Figure 1 This is a flowchart of an obstacle trajectory prediction method provided by the first embodiment of the present invention. This embodiment is applicable to the scenario of predicting the motion trajectory of dynamic obstacles identified during the driving of unmanned driving equipment. The method can be executed by an obstacle trajectory prediction device, which can be composed of hardware and / or software and can be integrated into an electronic device. In this embodiment, the electronic device can be a vehicle-mounted terminal, an intelligent autopilot, or a server device. The method specifically includes the following:
[0035] S101, obtaining the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device.
[0036] In this embodiment, the unmanned driving equipment may be, but is not limited to: an autonomous driving vehicle, an unmanned vehicle, a drone, a robot, an automatic delivery equipment, and other equipment capable of autonomous driving.
[0037] The dynamic obstacles may include vehicles, pedestrians or other objects in motion, such as cats, dogs and other animals, or horse-drawn carriages.
[0038] Generally, the autonomous driving system of unmanned driving equipment can be divided into at least an obstacle perception module according to its functions. Among them, the obstacle perception module mainly identifies obstacles in the surrounding environment of the unmanned driving equipment according to the data collected by various sensors and the recognition algorithm, and can obtain information such as obstacle outline and position frame by frame.
[0039] Based on this, the embodiment of the present invention can detect whether there are dynamic obstacles during the driving process of the unmanned driving device through various sensors in the obstacle perception module of the automatic driving system. If a dynamic obstacle is detected, the type of the dynamic obstacle is obtained, and the image or point cloud data of the dynamic obstacle is obtained to lay the foundation for the subsequent prediction of the motion trajectory of the dynamic obstacle based on the determined prediction model.
[0040] The various sensors in the obstacle sensing module may include: cameras and different types of radars, such as laser radars and millimeter wave radars, etc., so as to collect the surrounding environment data of the unmanned driving equipment during driving through the above cameras and different types of radars, and obtain the dynamic obstacle type and the image or point cloud data with dynamic obstacles based on the surrounding environment data.
[0041] S102: Select a first-level prediction model from multiple level prediction models of the same type according to the type of the dynamic obstacle.
[0042] The prediction model may include multiple types, and the multiple types correspond to the types of dynamic obstacles. For example, if the types of dynamic obstacles include: vehicles, pedestrians, or others, then the types of the prediction model include vehicles, pedestrians, or others accordingly.
[0043] In order to obtain high-precision prediction results as much as possible under the premise of meeting the real-time requirements of predicting the motion trajectory of dynamic obstacles, this embodiment can train prediction models of multiple accuracy levels for each type. Among them, the multiple accuracy levels can be selected, for example, as follows: the first accuracy level, the second accuracy level, and the third accuracy level. In this embodiment, the first accuracy level can be a low accuracy level, the second accuracy level can be a medium accuracy level, and the third accuracy level can be a high accuracy level. For the specific training process of the prediction models of different accuracy levels in this embodiment, please refer to the existing implementation method, and it will not be described in detail here.
[0044] After obtaining multiple prediction models of different types and levels of accuracy, they can be configured in an electronic device, and each prediction model of each level of accuracy of each type can be run in the electronic device respectively, and the same number of frames of dynamic obstacle images or dynamic obstacle point cloud data can be input into each prediction model of each level of accuracy of each type to determine the predicted duration of the motion trajectory of the dynamic obstacle predicted by the prediction model of each level of accuracy of each type. Then, the prediction duration of the prediction model of each level of accuracy of each type is recorded to lay the foundation for the subsequent determination of the prediction duration of the prediction model. It can be understood that the shorter the prediction duration of the prediction model, the simpler the prediction model operation and the lower the corresponding prediction accuracy, and vice versa.
[0045] Furthermore, after obtaining the type of at least one dynamic obstacle, the electronic device may select the same type from multiple prediction model types according to the type of the dynamic obstacle, and select the first level prediction model from multiple level prediction models of the same type. Multiple level prediction models under multiple types can be selected by t i jIndicates, where t represents the prediction duration of each level prediction model under different types, and i and j represent the i-th level prediction model under the j-th type. In this embodiment, the first-level prediction model refers to the prediction model with the lowest accuracy level. In other words, by selecting a prediction model with the lowest accuracy level from the same type as the dynamic obstacle, the real-time prediction of the motion trajectory of the dynamic obstacle can be ensured.
[0046] S103: According to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, the prediction model level or the number of dynamic obstacles is adjusted to obtain the optimal level of the prediction model or the optimal number of dynamic obstacles.
[0047] S104, controlling the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or controlling the prediction model to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
[0048] The predicted duration and value are determined according to the number of dynamic obstacles during the driving process of the unmanned driving device. For example, when there is one dynamic obstacle, the predicted duration and value is the predicted duration of the prediction model corresponding to the dynamic obstacle; when there are multiple dynamic obstacles, the predicted duration and value is the sum of the predicted durations of the prediction models corresponding to multiple dynamic obstacles.
[0049] The duration threshold is a criterion for determining whether the time used to predict the motion trajectory of a dynamic obstacle meets the real-time requirements, and can be set to different values according to the type of electronic device. For example, when the electronic device is a vehicle-mounted terminal, the duration threshold can be set to 100 milliseconds (ms).
[0050] Optionally, the duration threshold set in the electronic device can be implemented according to formula (1):
[0051] S=s*k…………………………(1)
[0052] Among them, S represents the duration threshold set in different types of electronic devices, s represents the running time of the prediction model of each accuracy level under different types, and k represents the device coefficient of different types of electronic devices.
[0053] Since the prediction duration of each type of prediction model of each accuracy level is pre-recorded in the electronic device. To this end, this embodiment can determine the prediction duration of the first-level prediction model of the same type as the dynamic obstacle from the recorded prediction duration according to the type of the dynamic obstacle. Then, based on the prediction duration of the first-level prediction model, the prediction duration and value of the first-level prediction model are determined, and the prediction duration and value of the first-level prediction model are compared with the duration threshold to determine the size relationship between the prediction duration and value of the first-level prediction model and the duration threshold. For example, the prediction duration and value of the first-level prediction model is greater than the duration threshold; or, the prediction duration and value of the first-level prediction model is less than the duration threshold, etc.
[0054] Furthermore, according to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, the prediction model level or the number of dynamic obstacles is adjusted to determine the optimal level of the prediction model or the optimal number of dynamic obstacles.
[0055] Optionally, when the predicted duration and value of the first-level prediction model is greater than the duration threshold, the number of dynamic obstacles can be deleted to obtain an optimal number of dynamic obstacles, so that the predicted duration and value of the first-level prediction model of the optimal number of dynamic obstacles is less than or equal to the duration threshold, thereby using the first-level prediction model corresponding to the optimal number of dynamic obstacles to predict the motion trajectory of the optimal number of dynamic obstacles based on the image or point cloud data of the optimal number of dynamic obstacles; or, when the predicted duration and value of the first-level prediction model is less than the duration threshold, the first-level prediction model of the dynamic obstacles can be upgraded so that the predicted duration and value of the upgraded prediction model is less than or equal to the duration threshold, thereby obtaining an optimal level prediction model, thereby using the optimal level prediction model to predict the motion trajectory of the original dynamic obstacles based on the image or point cloud data of the original dynamic obstacles, so as to improve the accuracy of the prediction model of the dynamic obstacles, etc.
[0056] It is understandable that adjusting the level of the prediction model and / or the number of dynamic obstacles may be adjusting the level of the prediction model; or adjusting the number of dynamic obstacles; or adjusting the level of the prediction model and the number of dynamic obstacles, which are not specifically limited here.
[0057] The technical solution provided by the embodiment of the present invention obtains the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device, selects a first-level prediction model from multiple level prediction models of the same type according to the type of the dynamic obstacle, and adjusts the prediction model level or the number of dynamic obstacles according to the relationship between the prediction duration and value of the first-level prediction and the duration threshold, obtains the optimal level of the prediction model or the optimal number of dynamic obstacles, and then controls the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or controls the prediction model to predict the motion trajectory of the optimal number of dynamic obstacles based on the image or point cloud data of the optimal number of dynamic obstacles. Thus, the problem that the existing dynamic obstacle motion trajectory prediction method is incompatible with the prediction real-time performance and prediction accuracy when a dynamic obstacle appears during the driving process of the unmanned driving device is solved, and the prediction model matching the dynamic obstacle can be allowed to upgrade the prediction model on the premise of meeting the prediction real-time performance to improve the prediction accuracy of the prediction model, thereby optimizing the obstacle trajectory prediction method and effectively taking into account the prediction real-time performance and prediction accuracy of the obstacle trajectory.
[0058] Embodiment 2
[0059] It can be seen from the above description that the number of dynamic obstacles identified in the embodiment of the present invention is at least one. Figure 2 Taking the case where there are multiple dynamic obstacles as an example, the obstacle trajectory prediction of the embodiment of the present invention is further described. Figure 2 As shown, the method is as follows:
[0060] S201, obtaining the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device.
[0061] S202: If there are multiple dynamic obstacles, select a first-level prediction model for each dynamic obstacle from multiple level prediction models of the same type according to the type of each dynamic obstacle.
[0062] Optionally, when there are multiple dynamic obstacles and the types of each dynamic obstacle are different, the electronic device of this embodiment can select the same type as each dynamic obstacle from multiple prediction model types based on the type of each dynamic obstacle, and select the first-level prediction model from multiple level prediction models of the same type.
[0063] Since multiple dynamic obstacles have different effects on the normal driving of unmanned driving equipment, for example, some dynamic obstacles are relatively close to the unmanned driving equipment, so the impact on the unmanned driving equipment is relatively large; for another example, some dynamic obstacles are relatively far away from the unmanned driving equipment, so the impact on the unmanned driving equipment is relatively small. For this reason, before selecting the first-level prediction model for each dynamic obstacle in this embodiment, the obstacle score of each dynamic obstacle can be optionally determined to determine the importance of each dynamic obstacle according to the obstacle score, and then select prediction models for dynamic obstacles of different importance. Generally, the higher the obstacle score, the higher the importance, and vice versa.
[0064] Optionally, the obstacle score of each dynamic obstacle can be determined according to the obtained preset coefficient and the obstacle information of each dynamic obstacle. The preset coefficient includes: a position coefficient and a direction coefficient. Optionally, the preset coefficient is pre-configured in the configuration information of the electronic device, and when it is needed, the configuration information can be read from the electronic device, and then the position coefficient and the direction coefficient can be obtained from the configuration information.
[0065] In this embodiment, the obstacle information of each dynamic obstacle can be acquired through various sensors in the obstacle sensing module, wherein the obstacle information of each dynamic obstacle may include: the speed of the dynamic obstacle relative to the unmanned driving device, the straight-line distance between the dynamic obstacle and the unmanned driving device, and the vertical distance between the dynamic obstacle and the unmanned driving device;
[0066] As an optional implementation, Figure 2 As shown in (a), determining the obstacle score of each dynamic obstacle includes the following steps:
[0067] S21, determining a position score of each dynamic obstacle according to a straight-line distance and a position coefficient between each dynamic obstacle and the unmanned driving device.
[0068] The position coefficient is an adjustable coefficient, which is set according to actual application needs and is not limited here.
[0069] Specifically, the position score of each dynamic obstacle can be determined based on the following formula (2):
[0070] p m =k p *d m ……………………………………(2)
[0071] Among them, p m represents the position score of the mth dynamic obstacle, k p Denotes the position coefficient, d mIt represents the straight-line distance between the mth dynamic obstacle and the unmanned driving device, where m is an integer greater than or equal to 2.
[0072] Furthermore, since the obstacle information collected by various sensors in the obstacle sensing module may also include: the position information of each dynamic obstacle and the position information of the unmanned driving device, the position score of each dynamic obstacle in this embodiment may also be determined by the following formula (3):
[0073]
[0074] Among them, p m represents the position score of the mth dynamic obstacle, k p represents the location coefficient, x,y represents the location information of the unmanned driving device, x m ,y m Represents the location information of the mth dynamic obstacle, where m is an integer greater than or equal to 2.
[0075] S22, determining a direction score of each dynamic obstacle according to a speed of each dynamic obstacle relative to the unmanned driving device, a vertical distance between each dynamic obstacle and the unmanned driving device, and a direction coefficient.
[0076] The directional coefficient is an adjustable coefficient, which is set according to actual application needs and is not limited here.
[0077] Specifically, the direction score of each dynamic obstacle can be determined based on the following formula (4):
[0078]
[0079] Among them, D m represents the direction score of the mth dynamic obstacle, k d Indicates the directivity coefficient, L m represents the vertical distance between the mth dynamic obstacle and the unmanned driving device, Represents the speed of the mth dynamic obstacle relative to the driving direction of the unmanned driving device, where m is an integer greater than or equal to 2.
[0080] S23, determining an obstacle score of each obstacle according to the position score and the direction score of each dynamic obstacle.
[0081] Specifically, the obstacle score of each dynamic obstacle can be determined based on the following formula (5):
[0082] C m =p m +D m …………………………(5)
[0083] Among them, Cm represents the obstacle score of the mth dynamic obstacle, where m is an integer greater than or equal to 2, and p m represents the position score of the mth dynamic obstacle, D m Represents the direction score of the mth dynamic obstacle.
[0084] Then, the obstacle scores of each dynamic obstacle are sorted in descending order to obtain a sorting result. Then, according to the sorting result and the type of each dynamic obstacle, a first-level prediction model is selected for each dynamic obstacle from multiple level prediction models of the same type.
[0085] For example, if the dynamic obstacles include: vehicle 1, vehicle 2 and pedestrian 3, and the obstacle score of vehicle 1 is 50, the obstacle score of vehicle 2 is 85, and the obstacle score of pedestrian 3 is 62, then the ranking result of the dynamic obstacles is {vehicle 2, pedestrian 3, vehicle 1}. Then, according to the type of vehicle, the first-level prediction model is selected for vehicle 1 and vehicle 2 in the multiple-level prediction models of vehicles, and according to the type of pedestrian, the first-level prediction model is selected for pedestrian 3 in the multiple-level prediction models of pedestrians.
[0086] S203, when the predicted duration and value of the multiple first-level prediction models are less than the duration threshold, the prediction models of the dynamic obstacles in the sorting results are upgraded in sequence until the predicted duration and value of the multiple prediction models are greater than or equal to the duration threshold, or until the prediction model levels of all dynamic obstacles are the highest level, so as to obtain the optimal level of the prediction model and the optimal number of dynamic obstacles.
[0087] Optionally, after selecting the first-level prediction model for each dynamic obstacle, the electronic device can determine the prediction duration of the first-level prediction model of the same type as each dynamic obstacle from the recorded prediction duration according to the type of each dynamic obstacle. Then, the prediction durations of the first-level prediction models corresponding to each dynamic obstacle are added to obtain the predicted duration sum value. The predicted duration sum value is compared with the duration threshold to determine the size relationship between the predicted duration sum value and the duration threshold. If the predicted duration sum value is less than the duration threshold, it means that the prediction model of at least one of the multiple dynamic obstacles can be upgraded to improve the prediction accuracy of the motion trajectory of the dynamic obstacle.
[0088] Specifically, when the prediction model of the dynamic obstacle is upgraded, the prediction models of the dynamic obstacles in the sorting result may be upgraded in descending order according to the obstacle scores of the dynamic obstacles, until the prediction durations and values of multiple prediction models are greater than or equal to the duration threshold, or until the prediction model levels of all dynamic obstacles are at the highest level.
[0089] S204, when the sum of the predicted durations of the multiple first-level prediction models is greater than the duration threshold, delete the dynamic obstacle at the end of the sorting result and the first-level prediction model of the dynamic obstacle until the sum of the predicted durations of the remaining first-level prediction models is less than or equal to the duration threshold, so as to obtain the optimal level of the prediction model and the optimal number of dynamic obstacles.
[0090] Optionally, after selecting the first-level prediction model for each dynamic obstacle, the electronic device may add the predicted durations of the first-level prediction models corresponding to each dynamic obstacle to obtain a predicted duration sum value. Then, the predicted duration sum value is compared with the duration threshold to determine the magnitude relationship between the predicted duration sum value and the duration threshold. If the predicted duration sum value is greater than the duration threshold, it means that if the motion trajectory of each dynamic obstacle in the sorting result is predicted, the prediction may not meet the real-time requirements. Since the sorting results are sorted in descending order according to the obstacle scores, the dynamic obstacles at the end of the sorting results have little effect on the normal driving of the unmanned driving device and can be almost ignored.
[0091] To this end, this embodiment may preferentially delete the dynamic obstacle at the end of the sorting result and the first-level prediction model of the dynamic obstacle to reduce the number of dynamic obstacles in the sorting result, and then calculate the prediction duration and value of the remaining prediction models according to the prediction duration of each prediction model of the remaining dynamic obstacles, and then compare the prediction duration and value of the remaining prediction models with the duration threshold to determine the relationship between the prediction duration and value of the remaining prediction models and the duration threshold.
[0092] Among them, if the predicted duration and value of the remaining prediction models are less than or equal to the duration threshold, the best level prediction model of the remaining dynamic obstacles is determined to be the first level prediction model. If the predicted duration and value of the remaining prediction models are still greater than the duration threshold, the dynamic obstacles at the end of the current sorting result (latest sorting result) and the first level prediction model of the dynamic obstacles are deleted in sequence until the predicted duration and value of the latest remaining first level prediction model is less than or equal to the duration threshold, and the best level prediction model of the latest remaining dynamic obstacles is determined to be the first level prediction model.
[0093] S205: When the sum of the prediction durations of the plurality of first-level prediction models is equal to the duration threshold, determine that the best prediction model for each dynamic obstacle is the first-level prediction model.
[0094] Optionally, after selecting the first-level prediction model for each dynamic obstacle, the electronic device may add the predicted durations of the first-level prediction models corresponding to each dynamic obstacle to obtain a predicted duration sum value. Then, the predicted duration sum value is compared with the duration threshold to determine the magnitude relationship between the predicted duration sum value and the duration threshold. If the predicted duration sum value is equal to the duration threshold, it means that the first-level prediction model of each dynamic obstacle can be used to predict the motion trajectory of each dynamic obstacle in the sorting result to meet the real-time requirements. That is, the best prediction model for each dynamic obstacle is the first-level prediction model.
[0095] S206, controlling the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or controlling the prediction model to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
[0096] It can be understood that the embodiment of the present invention selects the prediction model with the lowest prediction accuracy under the same type of dynamic obstacle according to the type of dynamic obstacle, and when the prediction duration and value of the prediction model meet the real-time requirements, the level of the prediction model is upgraded to obtain a prediction model with higher prediction accuracy, so as to improve the prediction accuracy as much as possible while ensuring the real-time performance of the prediction; if the prediction duration and value of the upgraded prediction model cannot meet the real-time requirements, the prediction model with low prediction accuracy before the upgrade is used to predict the motion trajectory of the dynamic obstacle to ensure the real-time performance of the prediction.
[0097] The technical solution provided by the embodiment of the present invention solves the problem that the existing dynamic obstacle motion trajectory prediction method is incompatible with the prediction real-time performance and prediction accuracy when a dynamic obstacle appears during the driving process of the unmanned driving device, and realizes that the prediction model matched with the dynamic obstacle can be allowed to be upgraded to improve the prediction accuracy of the prediction model under the premise of satisfying the prediction real-time performance, thereby optimizing the obstacle trajectory prediction method and effectively taking into account the prediction real-time performance and prediction accuracy of the obstacle trajectory. In addition, when there are multiple dynamic obstacles, the prediction model of the dynamic obstacle can be selectively upgraded according to the obstacle score of each dynamic obstacle when the prediction duration and value of the multiple dynamic obstacle prediction models are less than the duration threshold, so as to quickly and accurately predict the motion trajectory of the dynamic obstacle that has a serious impact on the normal driving of the unmanned driving device, so as to re-plan the driving trajectory of the unmanned driving device according to the dynamic obstacle motion trajectory, so as to reduce the occurrence of collisions and provide conditions for improving driving safety.
[0098] Embodiment 3
[0099] Figure 3: is a flowchart of an obstacle trajectory prediction method provided in the third embodiment of the present invention. Based on the above embodiment, the method of "upgrading the prediction models of dynamic obstacles in the sorting results in sequence until the prediction duration and value of the multiple prediction models are greater than or equal to the duration threshold" in this embodiment is further optimized. Figure 3 As shown, the method is as follows:
[0100] S301, upgrading the first-level prediction model of the dynamic obstacle located at the first position in the sorting result to a second-level prediction model, and determining the relationship between the prediction duration and value of the second-level prediction model and other first-level prediction models and the duration threshold.
[0101] Since the dynamic obstacles in the sorting results are sorted in descending order according to the obstacle scores, the dynamic obstacle at the first position has the greatest impact on the unmanned driving device compared to the dynamic obstacles at other positions. Therefore, when determining that the prediction model of at least one of the multiple obstacles can be upgraded, this embodiment prioritizes upgrading the prediction model level corresponding to the dynamic obstacle that has the greatest impact on the unmanned driving device, thereby providing conditions for the path planning of the unmanned driving device. In specific implementation, the first-level prediction model of the dynamic obstacle at the first position in the sorting results can be preferentially upgraded to the second-level prediction model.
[0102] Among them, after the first level prediction model of the dynamic obstacle at the first position is upgraded, this embodiment may also upgrade the obstacle score of the dynamic obstacle at the first position to lay a foundation for subsequent updating of the sorting result.
[0103] Furthermore, after the first-level prediction model of the dynamic obstacle at the first position in the sorting result is upgraded, the optional embodiment can add the predicted duration of the second-level prediction model and other first-level prediction models to obtain the predicted duration sum value. The predicted duration sum value is compared with the duration threshold to determine the size relationship between the predicted duration sum value and the duration threshold.
[0104] S302: If the predicted duration and value are greater than the duration threshold, the best level prediction model of each dynamic obstacle is determined to be the first level prediction model.
[0105] In this embodiment, when the predicted duration and value of the second-level prediction model and other first-level prediction models are greater than the duration threshold, it means that although the second-level prediction model of the dynamic obstacle at the first position in the sorting result meets the high prediction accuracy, the predicted duration and value of multiple dynamic obstacle prediction models cannot meet the prediction real-time requirements. Therefore, in order to ensure the real-time prediction, this embodiment gives priority to the first-level prediction model of each dynamic model, based on the image or point cloud data of each dynamic obstacle, to predict the motion trajectory of each dynamic obstacle. That is, in this embodiment, the best level model for each dynamic obstacle is determined to be the first-level prediction model.
[0106] Each dynamic obstacle corresponds to a prediction model. When the prediction model is used to predict the motion trajectory of the dynamic obstacle, the motion trajectory of the dynamic obstacle is predicted based on the prediction model corresponding to the dynamic obstacle.
[0107] S303: If the predicted duration and value are equal to the duration threshold, determine that the best level prediction model of the dynamic obstacle at the first position in the sorting result is the second level prediction model, and the best level prediction models of the dynamic obstacles at other positions are the first level prediction models.
[0108] In this embodiment, when the sum of the prediction durations of the second-level prediction model and other first-level prediction models is equal to the duration threshold, it means that when the trajectory prediction of the dynamic obstacle at the first position is performed using the second-level prediction model, not only the prediction real-time requirement is met, but also the high prediction accuracy requirement can be met. At this time, the electronic device can use the second-level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle at the first position, and use the first-level prediction model to predict the motion trajectory of the dynamic obstacles at other positions based on the image or point cloud data of the dynamic obstacles at other positions in the sorting results. That is, in this embodiment, the best level model for determining the dynamic obstacle at the first position is the second-level prediction model, and the best level prediction model for the dynamic obstacles at other positions is the first-level prediction model.
[0109] S304, if the predicted duration and value are less than the duration threshold, the first-level prediction models of the dynamic obstacles at other positions in the sorting result except the first position are upgraded to second-level prediction models in sequence, until the predicted duration and value of all prediction models are greater than or equal to the duration threshold, and the prediction model before the last upgrade is determined as the best-level prediction model for each dynamic obstacle, or the prediction model upgraded for the last time is determined as the best-level prediction model for each dynamic obstacle.
[0110] In this embodiment, when the predicted duration and value of the second-level prediction model and other first-level prediction models are less than the duration threshold, it means that when the trajectory prediction of the dynamic obstacle at the first position is performed using the second-level prediction model, it not only meets the prediction real-time requirements, but also meets the requirements of higher prediction accuracy, and there is still room for upgrading the prediction models of dynamic obstacles at other positions. In order to improve the overall prediction accuracy, this optional embodiment can upgrade the first-level prediction models of the dynamic obstacles at the remaining positions except the first position in the sorting results to the second-level prediction models in sequence according to the sorting position from front to back, or according to the obstacle score from high to low, and perform different prediction operations according to the relationship between the predicted duration and value of the upgraded prediction model and the duration threshold.
[0111] Specifically, if the first-level prediction model of the second-position dynamic obstacle in the sorting result is upgraded to the second-level prediction model, and the sum of the prediction durations of the two second-level prediction models and the other first-level prediction models is less than the duration threshold, the first-level prediction model of the third-position dynamic obstacle is upgraded to the second-level prediction model. When the sum of the prediction durations of the three second-level prediction models and the other first-level prediction models is greater than the duration threshold, it means that although the second-level prediction model of the third-position dynamic obstacle can determine a higher prediction accuracy, the trajectory prediction of all dynamic obstacles cannot meet the real-time requirements. Therefore, in this embodiment, the second-level prediction model of the first-position dynamic obstacle is preferentially used to predict the motion trajectory of the first-position dynamic obstacle, the second-level prediction model of the second-position dynamic obstacle is used to predict the motion trajectory of the second-position dynamic obstacle, and the first-level prediction model of the other-position dynamic obstacles is used to predict the motion trajectory of the other-position dynamic obstacles. That is, the best-level model of the first-position dynamic obstacle and the second-position dynamic obstacle is determined to be the second-level prediction model, and the best-level prediction model of the other-position dynamic obstacles is the first-level prediction model.
[0112] If the first-level prediction model of the second-position dynamic obstacle in the sorting result is upgraded to the second-level prediction model, and the sum of the prediction durations of the two second-level prediction models and other first-level prediction models is equal to the duration threshold, it means that the second-level prediction model of the second-position dynamic obstacle can not only determine a higher prediction accuracy, but also meet the prediction real-time requirements. Therefore, this embodiment uses the second-level prediction model of the first-position dynamic obstacle to predict the motion trajectory of the first-position dynamic obstacle, uses the second-level prediction model of the second-position dynamic obstacle to predict the motion trajectory of the second-position dynamic obstacle, and uses the first-level prediction model of other-position dynamic obstacles to predict the motion trajectory of other-position dynamic obstacles. That is, the best-level model of the first-position dynamic obstacle and the second-position dynamic obstacle is determined to be the second-level prediction model, and the best-level prediction model of other-position dynamic obstacles is the first-level prediction model.
[0113] If the first-level prediction model of the dynamic obstacle at the second position in the sorting result is upgraded to the second-level prediction model, and the sum of the prediction durations of the two second-level prediction models and other first-level prediction models is greater than the duration threshold, it means that although the second-level prediction model of the dynamic obstacle at the second position improves the prediction accuracy, the trajectory prediction of all dynamic obstacles cannot meet the real-time prediction requirements. Therefore, this embodiment adopts the second-level prediction model of the dynamic obstacle at the first position to predict the motion trajectory of the dynamic obstacle at the first position, and adopts the first-level prediction model of the dynamic obstacles at other positions to predict the motion trajectory of the dynamic obstacles at other positions. That is, the best level model of the dynamic obstacle at the first position is determined to be the second-level prediction model, and the best level prediction model of the dynamic obstacles at other positions is the first-level prediction model.
[0114] Continuing with the above example, if the duration threshold is 100ms, the prediction model of vehicle 2 in the sorting result is the second-level prediction model, the prediction model of pedestrian 3 is the first-level prediction model, and the prediction model of vehicle 1 is the first-level prediction model, then according to the prediction duration of the respective prediction models of vehicle 2, pedestrian 3 and vehicle 1, the prediction duration and value are determined to be 90ms. If it is determined that the prediction duration and value are less than 100ms, the first-level prediction model of pedestrian 3 can be upgraded to the second-level prediction model at this time, and the prediction duration and value of the corresponding prediction models of vehicle 2, pedestrian 3 and vehicle 1 can be calculated. If the prediction duration and value are 105ms, it is determined to be greater than 100ms. Then the electronic device can perform motion trajectory prediction for vehicle 2 according to the second-level prediction model, for pedestrian 3 according to the first-level prediction model, and for vehicle 1 according to the first-level prediction model.
[0115] S305, if the predicted duration and value are less than the duration threshold, the first-level prediction models of the dynamic obstacles at other positions in the sorting results except the first position are upgraded to second-level prediction models in sequence, until the prediction model levels of all dynamic obstacles are at the highest level, and the best level prediction model of each dynamic obstacle is determined to be the highest level prediction model.
[0116] Optionally, when the predicted duration and value of the second-level prediction model and other first-level prediction models are less than the duration threshold, it means that when the trajectory prediction of the dynamic obstacle at the first position is performed using the second-level prediction model, it not only meets the prediction real-time requirements, but also meets the higher prediction accuracy requirements, and there is still room for upgrading the prediction models of dynamic obstacles at other positions. To this end, this embodiment can upgrade the first-level prediction models of dynamic obstacles at other positions in the sorting results in sequence, and when the prediction models of dynamic obstacles at all positions are upgraded to the highest-level prediction models, and the predicted duration and value of all the highest-level prediction models are still less than the duration threshold, the highest-level prediction models of the dynamic obstacles at each position are respectively used to predict the motion trajectory of the dynamic obstacles based on the image or point cloud data of each dynamic obstacle. Thereby, not only the real-time prediction of the dynamic obstacle is ensured, but also the high-precision prediction of the dynamic obstacle is ensured. That is, the best-level model of each dynamic obstacle is determined to be the highest-level prediction model.
[0117] The technical solution provided by the embodiment of the present invention solves the problem that the existing dynamic obstacle motion trajectory prediction method is incompatible with the prediction real-time performance and prediction accuracy when a dynamic obstacle appears during the driving process of the unmanned driving device, and realizes that the prediction model matched with the dynamic obstacle can be allowed to be upgraded to improve the prediction accuracy of the prediction model under the premise of satisfying the prediction real-time performance, thereby optimizing the obstacle trajectory prediction method and effectively taking into account the prediction real-time performance and prediction accuracy of the obstacle trajectory. In addition, when there are multiple dynamic obstacles, the prediction model of the dynamic obstacle can be selectively upgraded according to the obstacle score of each dynamic obstacle when the prediction duration and value of the multiple dynamic obstacle prediction models are less than the duration threshold, so as to quickly and accurately predict the motion trajectory of the dynamic obstacle that has a serious impact on the normal driving of the unmanned driving device, so as to re-plan the driving trajectory of the unmanned driving device according to the dynamic obstacle motion trajectory, so as to reduce the occurrence of collisions and provide conditions for improving driving safety.
[0118] Embodiment 4
[0119] It can be seen from the above description that the number of dynamic obstacles identified in the embodiment of the present invention is at least one. Figure 4 Taking the number of dynamic obstacles as one as an example, the obstacle trajectory prediction of the embodiment of the present invention is further described. Figure 4As shown, the method is as follows:
[0120] S401, obtaining the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device.
[0121] S402: If the number of the dynamic obstacle is one, a first-level prediction model is selected from multiple level prediction models of the same type according to the type of the dynamic obstacle.
[0122] S403: When the predicted duration and value of the first-level prediction model is less than the duration threshold, the first-level prediction model is upgraded according to the multiple-level prediction models to obtain the best level of the prediction model.
[0123] Optionally, according to the type of the dynamic obstacle, the prediction duration of the first-level prediction model of the same type as the dynamic obstacle can be determined from the recorded prediction duration to obtain the prediction duration and value of the first-level prediction model. Then, the prediction duration and value of the first-level prediction model are compared with the duration threshold to determine whether the prediction duration and value of the first-level prediction model is less than the duration threshold.
[0124] Generally, when the number of dynamic obstacles is one and the prediction model of the dynamic obstacle is a first-level prediction model, the prediction duration and value of the first-level prediction model are both less than the duration threshold, but since the accuracy of the first-level prediction model is relatively low, the electronic device can upgrade the first-level prediction model of the dynamic obstacle, that is, upgrade the prediction accuracy level, so as to improve the prediction accuracy as much as possible while ensuring the real-time requirements of the prediction.
[0125] Optionally, when the first-level prediction model is upgraded according to the multiple-level prediction models, the present embodiment specifically includes the following steps:
[0126] S41, upgrading the first-level prediction model to a second-level prediction model, and determining the relationship between the prediction duration and value of the second-level prediction model and the duration threshold.
[0127] In specific implementation, the first-level prediction model of the dynamic obstacle can be upgraded to a second-level prediction model. After the first-level prediction model is upgraded to the second-level prediction model, the electronic device can determine the prediction duration of the second-level prediction model of the same type as the dynamic obstacle from the recorded prediction duration to obtain the prediction duration and value of the second-level prediction model. Then, the prediction duration and value of the second-level prediction model are compared with the duration threshold to determine the size relationship between the prediction duration and value of the second-level prediction model and the duration threshold, and adjust the prediction model level according to the size relationship. See S42-S45 below for details.
[0128] S42: If the predicted duration and value is greater than the duration threshold, determining that the best level prediction model of the dynamic obstacle is the first level prediction model.
[0129] In this embodiment, when it is determined that the predicted duration and value of the second-level prediction model is greater than the duration threshold, it means that the upgraded second-level prediction model meets the higher prediction accuracy requirements, but cannot meet the prediction real-time requirements. Therefore, in order to ensure the real-time prediction, when the predicted duration and value of the upgraded second-level prediction model is greater than the duration threshold, the first-level prediction model before the upgrade is preferentially used in this embodiment to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle. That is, in this embodiment, the best level model for determining the dynamic obstacle is the first-level prediction model.
[0130] S43: If the predicted duration and value are equal to the duration threshold, determining that the best level prediction model of the dynamic obstacle is the second level prediction model.
[0131] In this embodiment, when it is determined that the predicted duration and value of the second-level prediction model are equal to the duration threshold, it means that the second-level prediction model can meet the requirements of higher prediction accuracy on the basis of meeting the requirements of prediction real-time performance. At this time, the electronic device can use the second-level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle, so as to take into account the prediction real-time performance and prediction accuracy of the obstacle trajectory. That is, in this embodiment, the best level model for determining the dynamic obstacle is the first-level prediction model.
[0132] S44, if the predicted duration and value are less than the duration threshold, the second-level prediction model is upgraded step by step until the predicted duration and value of the upgraded prediction model is greater than or equal to the duration threshold, and the prediction model before the last upgrade is determined as the optimal level prediction model of the dynamic obstacle, or the prediction model of the last upgrade is determined as the optimal level prediction model of the dynamic obstacle.
[0133] In this embodiment, when it is determined that the predicted duration and value of the second-level prediction model are less than the duration threshold, it means that the second-level prediction model can not only meet the prediction real-time requirements, but also meet the higher prediction accuracy requirements, and there is still room for upgrading the prediction model level. In order to further improve the prediction accuracy, this embodiment can also upgrade the second-level prediction model to a third-level prediction model, and perform different prediction operations according to the relationship between the predicted duration and value of the third-level prediction model and the duration threshold.
[0134] Specifically, if the predicted duration and value of the third-level prediction model is less than the duration threshold, it means that the prediction accuracy can be further improved in this embodiment, so that the third-level prediction model can be upgraded to the fourth-level prediction model, and when the predicted duration and value of the fourth-level prediction model is greater than the duration threshold, it means that although the fourth-level prediction model can ensure high prediction accuracy, it cannot meet the real-time prediction requirements. Therefore, in this embodiment, the third-level prediction model before the upgrade is preferentially used to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle. That is, the best level prediction model for the dynamic obstacle in this embodiment is the third-level prediction model.
[0135] If the predicted duration and value of the third-level prediction model is greater than the duration threshold, it means that although the third-level prediction model meets the high prediction accuracy, it cannot meet the prediction real-time requirement. Therefore, this embodiment gives priority to using the second-level prediction model before upgrading, based on the image or point cloud data of the dynamic obstacle, to predict the motion trajectory of the dynamic obstacle. That is, the best level prediction model for the dynamic obstacle in this embodiment is the second-level prediction model.
[0136] If the predicted duration and value of the third-level prediction model are equal to the duration threshold, it means that the third-level prediction model can meet the prediction real-time requirements and high prediction accuracy requirements. At this time, the third-level prediction model can be used in this embodiment to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle. That is, the best level prediction model for the dynamic obstacle in this embodiment is the third-level prediction model.
[0137] S45, if the predicted duration and value are less than the duration threshold, the second-level prediction model is upgraded step by step until it is upgraded to the highest level, and the best-level prediction model of the dynamic obstacle is determined to be the highest-level prediction model.
[0138] Optionally, when it is determined that the predicted duration and value of the second-level prediction model are less than the duration threshold, the present embodiment can upgrade the second-level prediction model, and when the predicted duration and value of the upgraded prediction model are still less than the duration threshold, the prediction model is continuously upgraded. When the prediction model is upgraded to the highest-level prediction model, and the predicted duration and value of the highest-level prediction model are still less than the duration threshold, the highest-level prediction model is used to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle. This ensures not only the real-time prediction of the dynamic obstacle, but also the high accuracy of the prediction of the dynamic obstacle. That is, the best-level prediction model for the dynamic obstacle in this embodiment is the highest-level prediction model.
[0139] S404, controlling the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle.
[0140] The technical solution provided by the embodiment of the present invention solves the problem that the existing dynamic obstacle motion trajectory prediction method is incompatible with prediction real-time performance and prediction accuracy when a dynamic obstacle appears during the driving process of an unmanned driving device. It realizes that the prediction model matching the dynamic obstacle can be upgraded to improve the prediction accuracy of the prediction model while meeting the prediction real-time performance, thereby optimizing the obstacle trajectory prediction method and effectively taking into account the prediction real-time performance and prediction accuracy of the obstacle trajectory.
[0141] Embodiment 5
[0142] Figure 5 is a schematic diagram of the structure of an obstacle trajectory prediction device provided in Embodiment 5 of the present invention. The obstacle trajectory prediction device of this embodiment may be composed of hardware and / or software and may be integrated into an electronic device. In this embodiment, the electronic device may be a vehicle-mounted terminal, an intelligent autopilot, or a server device. Figure 5 As shown, the obstacle trajectory prediction device 500 provided by the embodiment of the present invention includes: an acquisition module 510, a selection module 520, an adjustment module 530 and a prediction module 540.
[0143] The acquisition module 510 is used to acquire the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device;
[0144] A selection module 520, configured to select a first level prediction model from multiple level prediction models of the same type according to the type of the dynamic obstacle;
[0145] An adjustment module 530, configured to adjust the level of the prediction model or the number of dynamic obstacles according to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, to obtain an optimal level of the prediction model or an optimal number of the dynamic obstacles;
[0146] The prediction module 540 is used to control the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or to control the prediction model to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
[0147] As an optional implementation of the embodiment of the present invention, if there are multiple dynamic obstacles, the device 500 further includes: an obstacle score determination module;
[0148] The obstacle score determination module is used to determine the obstacle score of each dynamic obstacle according to the obstacle information of each dynamic obstacle, and to sort the obstacle scores of multiple dynamic obstacles;
[0149] Accordingly, the selection module 520 is specifically the same as:
[0150] According to the sorting result and the type of each of the dynamic obstacles, a first level prediction model is selected for each dynamic obstacle from multiple level prediction models of the same type.
[0151] As an optional implementation of an embodiment of the present invention, the obstacle information includes: a speed of the dynamic obstacle relative to the unmanned driving device, a straight-line distance between the dynamic obstacle and the unmanned driving device, and a vertical distance between the dynamic obstacle and the unmanned driving device;
[0152] The acquisition module 510 is further used for:
[0153] Get the position coefficient and direction coefficient;
[0154] Accordingly, the obstacle score determination module includes: a position score determination unit, a direction score determination unit and an obstacle score determination unit;
[0155] The position score determination unit is used to determine the position score of each dynamic obstacle according to the straight-line distance and the position coefficient between each dynamic obstacle and the unmanned driving device;
[0156] a direction score determination unit, configured to determine a direction score of each dynamic obstacle according to a speed of each dynamic obstacle relative to the unmanned driving device, a vertical distance between each dynamic obstacle and the unmanned driving device, and a direction coefficient;
[0157] The obstacle score determination unit is used to determine the obstacle score of each obstacle according to the position score and the direction score of each dynamic obstacle.
[0158] As an optional implementation of the embodiment of the present invention, the adjustment module 530 is specifically configured to:
[0159] When the predicted durations and values of the multiple first-level prediction models are less than the duration threshold, the prediction models of the dynamic obstacles in the sorting results are upgraded in sequence until the predicted durations and values of the multiple prediction models are greater than or equal to the duration threshold, or until the prediction model levels of all dynamic obstacles are at the highest level;
[0160] When the predicted duration and value of the multiple first-level prediction models are greater than the duration threshold, the dynamic obstacle at the end of the sorting result and the first-level prediction model of the dynamic obstacle are deleted until the predicted duration and value of the remaining first-level prediction models are less than or equal to the duration threshold.
[0161] As an optional implementation of the embodiment of the present invention, the adjustment module 530 is further used for:
[0162] Upgrading the first-level prediction model of the dynamic obstacle at the first position in the sorting result to a second-level prediction model, and determining the relationship between the prediction duration and value of the second-level prediction model and other first-level prediction models and the duration threshold;
[0163] If the predicted duration and value are greater than the duration threshold, determining that the best level prediction model for each dynamic obstacle is the first level prediction model;
[0164] If the predicted duration and value are equal to the duration threshold, determining that the best level prediction model of the dynamic obstacle at the first position in the sorting result is the second level prediction model, and the best level prediction models of the dynamic obstacles at other positions are the first level prediction models;
[0165] If the predicted duration and value are less than the duration threshold, the first-level prediction models of the dynamic obstacles at other positions in the sorting result except the first position are upgraded to second-level prediction models in sequence, until the predicted duration and values of all prediction models are greater than or equal to the duration threshold, and the prediction model before the last upgrade is determined as the best-level prediction model for each dynamic obstacle, or the prediction model upgraded for the last time is determined as the best-level prediction model for each dynamic obstacle;
[0166] If the predicted duration and value is less than the duration threshold, the first-level prediction models of the dynamic obstacles at other positions in the sorting results except the first position are upgraded to second-level prediction models in sequence until the prediction model levels of all dynamic obstacles are at the highest level, and the best level prediction model of each dynamic obstacle is determined to be the highest level prediction model.
[0167] As an optional implementation of the embodiment of the present invention, if the number of the dynamic obstacle is one, the adjustment module 530 is specifically configured to:
[0168] When the predicted duration and value of the first-level prediction model is less than the duration threshold, the first-level prediction model is upgraded according to the multiple-level prediction models.
[0169] As an optional implementation of the embodiment of the present invention, the adjustment module 530 is further configured to:
[0170] Upgrading the first-level prediction model to a second-level prediction model, and determining a relationship between the prediction duration and value of the second-level prediction model and the duration threshold;
[0171] If the predicted duration and value are greater than the duration threshold, determining that the best level prediction model of the dynamic obstacle is the first level prediction model;
[0172] If the predicted duration and value are equal to the duration threshold, determining that the best level prediction model of the dynamic obstacle is the second level prediction model;
[0173] If the predicted duration and value are less than the duration threshold, the second-level prediction model is upgraded step by step until the predicted duration and value of the upgraded prediction model is greater than or equal to the duration threshold, and the prediction model before the last upgrade is determined as the best-level prediction model for the dynamic obstacle, or the prediction model upgraded for the last time is determined as the best-level prediction model for the dynamic obstacle;
[0174] If the predicted duration and value are less than the duration threshold, the second-level prediction model is upgraded step by step until it is upgraded to the highest level, and the best-level prediction model of the dynamic obstacle is determined to be the highest-level prediction model.
[0175] It should be noted that the above explanation of the obstacle trajectory prediction method embodiment is also applicable to the obstacle trajectory prediction device of this embodiment, and the implementation principle is similar, which will not be repeated here.
[0176] The technical solution provided by the embodiment of the present invention solves the problem that the existing dynamic obstacle motion trajectory prediction method is incompatible with prediction real-time performance and prediction accuracy when a dynamic obstacle appears during the driving process of an unmanned driving device. It realizes that the prediction model matching the dynamic obstacle can be upgraded to improve the prediction accuracy of the prediction model while meeting the prediction real-time performance, thereby optimizing the obstacle trajectory prediction method and effectively taking into account the prediction real-time performance and prediction accuracy of the obstacle trajectory.
[0177] Embodiment 6
[0178] Figure 6 A schematic diagram of the structure of an electronic device provided in Embodiment 6 of the present invention. Figure 6 A block diagram of an exemplary electronic device 600 suitable for implementing embodiments of the present invention is shown. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0179] like Figure 6As shown, the electronic device 600 is in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
[0180] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
[0181] The electronic device 600 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 600, including volatile and non-volatile media, removable and non-removable media.
[0182] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 600 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.
[0183] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.
[0184] The electronic device 600 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 600 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0185] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the obstacle trajectory prediction method provided in the embodiment of the present invention, including:
[0186] Acquire the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device;
[0187] According to the type of the dynamic obstacle, a first level prediction model is selected from multiple level prediction models of the same type;
[0188] According to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, the level of the prediction model or the number of dynamic obstacles is adjusted to obtain the optimal level of the prediction model or the optimal number of the dynamic obstacles;
[0189] Control the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or control the prediction model to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
[0190] It should be noted that the above explanation of the obstacle trajectory prediction method embodiment is also applicable to the electronic device of this embodiment, and its implementation principle is similar, which will not be repeated here.
[0191] The electronic device provided by the embodiment of the present invention solves the problem that the existing dynamic obstacle motion trajectory prediction method is incompatible with the prediction real-time performance and prediction accuracy when a dynamic obstacle appears during the driving process of the unmanned driving device. It realizes that the prediction model matching the dynamic obstacle can be allowed to be upgraded to improve the prediction accuracy of the prediction model while meeting the prediction real-time performance, thereby optimizing the obstacle trajectory prediction method and effectively taking into account the prediction real-time performance and prediction accuracy of the obstacle trajectory.
[0192] Embodiment 7
[0193] In order to achieve the above objective, embodiment 7 of the present invention further proposes a computer-readable storage medium.
[0194] The computer-readable storage medium provided by the embodiment of the present invention stores a computer program, and when the program is executed by a processor, the obstacle trajectory prediction method according to the embodiment of the present invention is implemented, and the method includes:
[0195] Acquire the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device;
[0196] According to the type of the dynamic obstacle, a first level prediction model is selected from multiple level prediction models of the same type;
[0197] According to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, the level of the prediction model or the number of dynamic obstacles is adjusted to obtain the optimal level of the prediction model or the optimal number of the dynamic obstacles;
[0198] Control the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; or control the prediction model to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
[0199] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.
[0200] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0201] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0202] Computer program code for performing the operation of the present invention may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0203] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A method for predicting obstacle trajectories, characterized in that: include: Acquire the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device; If there are multiple dynamic obstacles, obtain a position coefficient and a direction coefficient; determine the position score of each dynamic obstacle according to the straight-line distance and the position coefficient between each dynamic obstacle and the unmanned driving device; determine the direction score of each dynamic obstacle according to the speed of each dynamic obstacle relative to the unmanned driving device, the vertical distance between each dynamic obstacle and the unmanned driving device, and the direction coefficient; determine the obstacle score of each obstacle according to the position score and the direction score of each dynamic obstacle, and sort the obstacle scores of multiple dynamic obstacles; select a first-level prediction model for each dynamic obstacle from multiple level prediction models of the same type according to the sorting result and the type of each dynamic obstacle; According to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, the level of the prediction model or the number of dynamic obstacles is adjusted to obtain the optimal level of the prediction model or the optimal number of the dynamic obstacles; Controlling the optimal level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; Alternatively, the prediction model is controlled to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
2. The method according to claim 1, characterized in that The step of adjusting the prediction model level or the number of dynamic obstacles according to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold comprises: When the predicted durations and values of the multiple first-level prediction models are less than the duration threshold, the prediction models of the dynamic obstacles in the sorting results are upgraded in sequence until the predicted durations and values of the multiple prediction models are greater than or equal to the duration threshold, or until the prediction model levels of all dynamic obstacles are at the highest level; When the predicted duration and value of the multiple first-level prediction models are greater than the duration threshold, the dynamic obstacle at the end of the sorting result and the first-level prediction model of the dynamic obstacle are deleted until the predicted duration and value of the remaining first-level prediction models are less than or equal to the duration threshold.
3. The method according to claim 2, characterized in that The step of sequentially upgrading the prediction models of the dynamic obstacles in the sorting results until the prediction durations and values of the multiple prediction models are greater than or equal to the duration threshold comprises: Upgrading the first-level prediction model of the dynamic obstacle at the first position in the sorting result to a second-level prediction model, and determining the relationship between the prediction duration and value of the second-level prediction model and other first-level prediction models and the duration threshold; If the predicted duration and value are greater than the duration threshold, determining that the best level prediction model for each dynamic obstacle is the first level prediction model; If the predicted duration and value are equal to the duration threshold, determining that the best level prediction model of the dynamic obstacle at the first position in the sorting result is the second level prediction model, and the best level prediction models of the dynamic obstacles at other positions are the first level prediction models; If the predicted duration and value are less than the duration threshold, the first-level prediction models of the dynamic obstacles at other positions in the sorting result except the first position are upgraded to second-level prediction models in sequence, until the predicted duration and values of all prediction models are greater than or equal to the duration threshold, and the prediction model before the last upgrade is determined as the best-level prediction model for each dynamic obstacle, or the prediction model upgraded for the last time is determined as the best-level prediction model for each dynamic obstacle; If the predicted duration and value is less than the duration threshold, the first-level prediction models of the dynamic obstacles at other positions in the sorting results except the first position are upgraded to second-level prediction models in sequence until the prediction model levels of all dynamic obstacles are at the highest level, and the best level prediction model of each dynamic obstacle is determined to be the highest level prediction model.
4. The method according to claim 1, characterized in that If the number of the dynamic obstacles is one, adjusting the prediction model level or the number of dynamic obstacles according to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, including: When the predicted duration and value of the first-level prediction model is less than the duration threshold, the first-level prediction model is upgraded according to the multiple-level prediction models.
5. The method according to claim 4, characterized in that The step of upgrading the first-level prediction model according to the multiple-level prediction models includes: Upgrading the first-level prediction model to a second-level prediction model, and determining a relationship between the prediction duration and value of the second-level prediction model and the duration threshold; If the predicted duration and value are greater than the duration threshold, determining that the best level prediction model of the dynamic obstacle is the first level prediction model; If the predicted duration and value are equal to the duration threshold, determining that the best level prediction model of the dynamic obstacle is the second level prediction model; If the predicted duration and value are less than the duration threshold, the second-level prediction model is upgraded step by step until the predicted duration and value of the upgraded prediction model is greater than or equal to the duration threshold, and the prediction model before the last upgrade is determined as the best-level prediction model for the dynamic obstacle, or the prediction model upgraded for the last time is determined as the best-level prediction model for the dynamic obstacle; If the predicted duration and value are less than the duration threshold, the second-level prediction model is upgraded step by step until it is upgraded to the highest level, and the best-level prediction model of the dynamic obstacle is determined to be the highest-level prediction model.
6. An obstacle trajectory prediction device, characterized in that: include: An acquisition module, used to acquire the type and image or point cloud data of at least one dynamic obstacle identified during the driving process of the unmanned driving device; A selection module is used to obtain a position coefficient and a direction coefficient if there are multiple dynamic obstacles; determine the position score of each dynamic obstacle according to the straight-line distance and the position coefficient between each dynamic obstacle and the unmanned driving device; determine the direction score of each dynamic obstacle according to the speed of each dynamic obstacle relative to the unmanned driving device, the vertical distance between each dynamic obstacle and the unmanned driving device, and the direction coefficient; determine the obstacle score of each obstacle according to the position score and the direction score of each dynamic obstacle, and sort the obstacle scores of multiple dynamic obstacles; select a first-level prediction model for each dynamic obstacle from multiple level prediction models of the same type according to the sorting result and the type of each dynamic obstacle; An adjustment module, configured to adjust the level of the prediction model or the number of dynamic obstacles according to the relationship between the prediction duration and value of the first-level prediction model and the duration threshold, so as to obtain an optimal level of the prediction model or an optimal number of the dynamic obstacles; A prediction module, used for controlling the best level prediction model to predict the motion trajectory of the dynamic obstacle based on the image or point cloud data of the dynamic obstacle; Alternatively, the prediction model is controlled to predict the motion trajectory of the optimal dynamic obstacle based on the image or point cloud data of the optimal number of dynamic obstacles.
7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the obstacle trajectory prediction method as described in any one of claims 1-5.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the obstacle trajectory prediction method as described in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Medium to long-term predication method in self-adaptive telephone traffic based on prior knowledge
CN101801004A
Movement region prediction apparatus
CN102307769A
Systems and methods for prioritized driver alerts
CN105989749A
Obstacle detection method, device, apparatus and storage medium
CN106951847A
Dubins path based obstacle avoidance control device and method for driverless car
CN108536149A
