Obstacle trajectory prediction method and device, electronic equipment and storage medium

Abstract

The invention relates to an obstacle trajectory prediction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring self-vehicle sensing information and obstacle sensing information corresponding to a current frame; determining the priority of each obstacle in the current frame according to the self-vehicle sensing information and the obstacle sensing information; and predicting the barrier with the highest priority in the current frame by using the rule prediction model, and predicting the remaining barriers except the barrier with the highest priority in the current frame by using the learning prediction model to obtain prediction tracks corresponding to the barriers in the current frame. Therefore, the obstacle with the highest priority can be predicted preferentially by using the rule prediction model, so that the number of the obstacles needing to be predicted by the learning prediction model is reduced, and high trajectory prediction efficiency and accuracy of each obstacle can be ensured even under the condition of limited computing power. Therefore, the efficiency and accuracy of trajectory prediction are compatible at the same time.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an obstacle trajectory prediction method, device, electronic device and storage medium. Background Art

[0002] Trajectory prediction is one of the important stages of vehicle autonomous driving. The efficiency and accuracy of trajectory prediction will directly affect the safety and stability of vehicle autonomous driving. Currently, trajectory prediction mainly adopts the following two methods:

[0003] (1) Trajectory prediction based on rule-based prediction models. This method usually uses human understanding of driving behavior and driving experience to establish a rule model, and determines whether the vehicle is in a lane or intersection based on the vehicle's movement information such as speed, direction, and position, as well as surrounding map elements, and then further generates the corresponding trajectory. This method requires low computing power, but the prediction accuracy is also low in complex scenarios.

[0004] (2) Trajectory prediction based on learning prediction models. This method usually uses deep neural networks to build trajectory prediction models, relies on a large amount of data to train the model, and can adaptively adjust the prediction model to adapt to new environments. This method can meet the prediction accuracy in complex scenarios, but it has high requirements on computing power, so the prediction efficiency is low under conditions of limited computing power.

[0005] Therefore, if these two methods are used alone for trajectory prediction, it is difficult to take into account both the efficiency and accuracy of trajectory prediction. Summary of the invention

[0006] The present application provides an obstacle trajectory prediction method, device, electronic device and storage medium to solve the problem that the existing technology uses only rule-based prediction models and learning-based prediction models for trajectory prediction, which makes it difficult to take into account both the efficiency and accuracy of trajectory prediction.

[0007] In a first aspect, the present application provides an obstacle trajectory prediction method, the method comprising:

[0008] Obtain the vehicle perception information and obstacle perception information corresponding to the current frame;

[0009] Determining the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information;

[0010] The obstacle with the highest priority in the current frame is predicted using a rule prediction model, and the remaining obstacles in the current frame except the obstacle with the highest priority are predicted using a learning prediction model to obtain a predicted trajectory corresponding to each obstacle in the current frame.

[0011] Further, determining the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information includes:

[0012] According to the vehicle perception information and the obstacle perception information, determining whether the target obstacle meets a preset condition for adjusting the region of interest, wherein the target obstacle is any obstacle in the current frame, and the initial region range of the region of interest is preset;

[0013] Determine the target area range of the region of interest according to the judgment result;

[0014] The priority of the target obstacle is determined according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information.

[0015] Further, the region of interest includes a forward first rectangular region and a backward first rectangular region, and the adjustment condition includes a first condition for adjusting the widths of the forward first rectangular region and the backward first rectangular region and a second condition for adjusting the length of the backward first rectangular region;

[0016] The determining, based on the vehicle perception information and the obstacle perception information, whether the target obstacle satisfies a preset condition for adjusting the region of interest includes:

[0017] Determining whether the vehicle is at an intersection or the target obstacle is at an intersection based on the vehicle perception information and the obstacle perception information;

[0018] When it is determined that the vehicle is at an intersection or the target obstacle is at an intersection, determining that the target obstacle satisfies the first condition;

[0019] According to the vehicle perception information and the obstacle perception information, determining whether the target obstacle is located in a rearward second rectangular area and whether the type of the target obstacle is a non-motor vehicle type, wherein the rearward second rectangular area includes the rearward first rectangular area;

[0020] When it is determined that the target obstacle is located in the rearward second rectangular area and the type of the target obstacle is a non-motor vehicle type, it is determined that the target obstacle satisfies the second condition.

[0021] Further, determining the priority of the target obstacle according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information includes:

[0022] Determining whether the target obstacle is in a stationary state according to the target area range of the area of ​​interest, the vehicle perception information, and the obstacle perception information;

[0023] In the case where it is determined that the target obstacle is in a stationary state, determining the priority of the target obstacle to be a first priority, wherein the first priority is the lowest priority;

[0024] In the case where it is determined that the target obstacle is in a non-stationary state, determining whether the target obstacle is located in the rearward second rectangular area or the forward second rectangular area, wherein the forward second rectangular area includes the forward first rectangular area;

[0025] When it is determined that the target obstacle is not located in the second rearward rectangular area and is not located in the second forward rectangular area, determining the priority of the target obstacle to be the first priority;

[0026] In the case where it is determined that the target obstacle is located within the rearward second rectangular area or the forward second rectangular area, determining whether the type of the target obstacle is a motor vehicle;

[0027] In the case where the type of the target obstacle is determined to be a motor vehicle, determining in sequence whether the target obstacle satisfies a rearward region of interest judgment condition, a rearward away judgment condition, a forward reverse judgment condition, and a forward away judgment condition;

[0028] When the target obstacle does not satisfy the backward region of interest judgment condition, the backward away judgment condition, the forward reverse judgment condition, and the forward away judgment condition, determining the priority of the target obstacle to be the second priority, wherein the second priority is the highest priority;

[0029] When the target obstacle satisfies the backward region of interest judgment condition, the backward away judgment condition, the forward reverse judgment condition or the forward away judgment condition, the priority of the target obstacle is determined to be the first priority.

[0030] Furthermore, the rearward area of ​​interest judgment condition refers to that the target obstacle is located in the rearward second rectangular area and is not located in the rearward first rectangular area; the rearward away judgment condition refers to that the target obstacle is located in the rearward first rectangular area and is away from the vehicle; the forward reverse judgment condition refers to that the target obstacle is in a forward reverse state and is not located in the forward first rectangular area; the forward away judgment condition refers to that the target obstacle is away from the vehicle and is not located in the forward first rectangular area.

[0031] Further, after determining whether the type of the target obstacle is a motor vehicle, the method further includes:

[0032] In the case where it is determined that the type of the target obstacle is a non-motor vehicle, determining whether the target obstacle is located in a passable area of ​​the vehicle;

[0033] In a case where it is determined that the target obstacle is not located in a passable area of ​​the vehicle, determining the priority of the target obstacle to be the first priority;

[0034] In the case where it is determined that the target obstacle is located in a passable area of ​​the vehicle, determining whether the target obstacle satisfies a first preset condition;

[0035] In the case where it is determined that the target obstacle meets the first preset condition, the steps of: sequentially determining whether the target obstacle meets a backward region of interest determination condition, a backward away determination condition, a forward retrograde determination condition, and a forward away determination condition;

[0036] When it is determined that the target obstacle does not satisfy the first preset condition, the priority of the target obstacle is determined to be a third priority, wherein the priority of the third priority is higher than the first priority and lower than the second priority.

[0037] Further, the determining whether the target obstacle is in a stationary state includes:

[0038] Acquire the state information of the target obstacle in the historical frame before the current frame;

[0039] When the state information of the historical frames reaches a preset number, determining whether the cumulative proportion of the static state of the historical frames of the target obstacle reaches a first preset threshold;

[0040] When it is determined that the cumulative proportion of the static state of the historical frames of the target obstacle reaches the first preset threshold, determining that the target obstacle is in a static state;

[0041] If it is determined that the cumulative proportion of the static state in the historical frames of the target obstacle does not reach the first preset threshold, determining whether the cumulative proportion of the non-static state in the historical frames of the target obstacle reaches a second preset threshold;

[0042] When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle reaches the second preset threshold, determining that the target obstacle is in a non-stationary state;

[0043] When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle does not reach the second preset threshold or the state information of the historical frames does not reach the preset quantity, the state of the target obstacle is determined according to the state information of the previous frame of the current frame.

[0044] Furthermore, after predicting the obstacle with the highest priority in the current frame using the rule prediction model, the method further includes:

[0045] Obtaining the rule prediction coverage corresponding to each obstacle among the obstacles with the highest priority in the current frame;

[0046] Obstacles whose rule prediction coverage is less than a third preset threshold are predicted again using the learning prediction model.

[0047] In a second aspect, the present application further provides an obstacle trajectory prediction device, the device comprising:

[0048] The first acquisition module is used to acquire the vehicle perception information and obstacle perception information corresponding to the current frame;

[0049] A determination module, configured to determine the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information;

[0050] The first prediction module is used to predict the obstacle with the highest priority in the current frame using a rule prediction model, and to predict the remaining obstacles in the current frame except the obstacle with the highest priority using a learning prediction model to obtain a predicted trajectory corresponding to each obstacle in the current frame.

[0051] In a third aspect, the present application further provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0052] Memory, used to store computer programs;

[0053] The processor is used to implement the steps of the obstacle trajectory prediction method described in any one of the first aspects when executing the program stored in the memory.

[0054] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the obstacle trajectory prediction method described in any one of the first aspects are implemented.

[0055] In a fifth aspect, the present application also provides a mobile tool, comprising the electronic device described in the third aspect.

[0056] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application obtains the vehicle perception information and obstacle perception information corresponding to the current frame; determines the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information; predicts the obstacle with the highest priority in the current frame using the rule prediction model, and predicts the remaining obstacles in the current frame except the obstacle with the highest priority using the learning prediction model, and obtains the predicted trajectory corresponding to each obstacle in the current frame. In the above manner, obstacles can be classified according to their priority, and obstacles with the highest priority can be predicted using the rule prediction model first, thereby reducing the number of obstacles that need to be predicted by the learning prediction model, and even under the condition of limited computing power, each obstacle can be guaranteed to have a high trajectory prediction efficiency and accuracy, thereby achieving the compatibility of trajectory prediction efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0059] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0060] Figure 1 A flowchart of an obstacle trajectory prediction method provided in an embodiment of the present application;

[0061] Figure 2 A schematic diagram of an initial area range of each area provided in an embodiment of the present application;

[0062] Figure 3 A flowchart of obstacle priority flag generation provided in an embodiment of the present application;

[0063] Figure 4 A flowchart for determining whether a target obstacle is in a stationary state provided in an embodiment of the present application;

[0064] Figure 5A schematic diagram of the structure of a trajectory prediction and reasoning optimization system provided in an embodiment of the present application;

[0065] Figure 6 A schematic diagram of the structure of an obstacle trajectory prediction device provided in an embodiment of the present application;

[0066] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0068] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0069] See also Figure 1 , Figure 1 A flowchart of an obstacle trajectory prediction method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the obstacle trajectory prediction method may include the following steps:

[0070] Step 101: Obtain vehicle perception information and obstacle perception information corresponding to the current frame.

[0071] Specifically, the above-mentioned ego vehicle perception information can be generated by modeling the ego vehicle environment data based on the ego vehicle positioning information, HD map (i.e., high-definition map) information, and ego vehicle planning information. The constituent fields of the ego vehicle perception information may include, but are not limited to: timestamp ego_timestamp, frame number frame_id, tracking number ego_id (such as -1), life cycle age, three-dimensional coordinates [ego_x, ego_y, ego_z] in the Universal Transverse Mercator Grid System (UTM) coordinate system, heading angle ego_yaw, driving scenario type scenarios_type (such as 0 for a straight road, 1 for a curve, 2 for an intersection, and 3 for others), etc. The above-mentioned obstacle perception information can be generated by modeling the obstacle environment data based on the ego vehicle perception information and HD map information. The fields constituting the obstacle perception information may include, but are not limited to: timestamp obj_timestamp, frame number frame_id, tracking number obj_id, target category type_id (such as 0 for motor vehicles, 1 for pedestrians, 2 for cyclists, 3 for other categories, etc.), life cycle age, three-dimensional coordinates [obj_x, obj_y, obj_z] in the UTM coordinate system, heading angle obj_yaw, driving scenario type scenarios_type (such as 0 for straight roads, 1 for curves, 2 for intersections, and 3 for others), motion status move_status, and motion status confidence conf_move_status (the value range is [0,1], the larger the value, the better).

[0072] It should be noted that the acquisition frequency of the above-mentioned ego-vehicle perception information and obstacle perception information is related to the acquisition frame rate of the sensor equipped on the ego-vehicle. Assuming the acquisition frame rate is 10fps, that is, 10 frames of image data are collected in 1 second, the ego-vehicle perception information and obstacle perception information corresponding to 10 frames of image data can be obtained in 1 second, and the frame numbers frame_id are accumulated in sequence. The tracking number ego_id is set to a fixed value that can be distinguished from the target category type_id. The target category type_id starts counting from 1, and the ego-vehicle number can be set to -1, -100 or other values. The role of the life cycle age: for obstacles, it is used to count the cumulative number of occurrences of each obstacle; for ego-vehicles, it is used to count the running time of the ego-vehicle.

[0073] Step 102: Determine the priority of each obstacle in the current frame based on the vehicle perception information and the obstacle perception information.

[0074] Specifically, the priority of each obstacle in the current frame can be divided according to actual needs, such as dividing the priority into 2 priorities, 3 priorities or 4 priorities, etc.

[0075] In this step, the degree of influence of each obstacle on the driving of the ego vehicle in the current frame can be determined based on the acquired ego vehicle perception information and obstacle perception information, and then each obstacle can be divided into different priorities according to the degree of influence of each obstacle on the driving of the ego vehicle, so as to facilitate the subsequent prediction using different trajectory prediction methods according to different priorities.

[0076] Step 103: predict the obstacle with the highest priority in the current frame using the rule prediction model, and predict the remaining obstacles except the obstacle with the highest priority in the current frame using the learning prediction model to obtain the predicted trajectory corresponding to each obstacle in the current frame.

[0077] Specifically, the above-mentioned rule prediction model refers to a prediction model for trajectory prediction based on rules. The rule prediction model has low requirements for computing power, but the prediction accuracy in complex scenarios is also low. The above-mentioned learning prediction model refers to a prediction model for trajectory prediction based on learning, such as Convolutional Neural Networks (CNN) model, Recurrent Neural Networks (RNN) model, transformer model (a neural network model based on self-attention mechanism), etc. The learning prediction model can meet the prediction accuracy in complex scenarios, but has high requirements for computing power, so the prediction efficiency is low under conditions of limited computing power.

[0078] In order to prioritize the impact of the highest priority obstacle on the driving of the vehicle, the highest priority obstacle in the current frame can be predicted using the rule prediction model, and the remaining obstacles in the current frame except the highest priority obstacle can be predicted using the learning prediction model. This can not only ensure the efficiency and accuracy of the trajectory prediction of the highest priority obstacle, but also use the remaining computing power to predict the trajectory of obstacles with lower priorities, so that the rule prediction model and the learning prediction model are integrated to achieve fast and effective prediction of the trajectories of multiple categories of obstacles.

[0079] In this embodiment, obstacles can be classified according to their priority, and obstacles with the highest priority are preferentially predicted using the rule prediction model, thereby reducing the number of obstacles that need to be predicted by the learning prediction model. Even under conditions of limited computing power, each obstacle can be guaranteed to have a high trajectory prediction efficiency and accuracy, thereby achieving simultaneous compatibility of trajectory prediction efficiency and accuracy.

[0080] Furthermore, the above step 102, determining the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information, specifically includes:

[0081] According to the vehicle perception information and the obstacle perception information, it is determined whether the target obstacle meets the preset conditions for adjusting the region of interest, wherein the target obstacle is any obstacle in the current frame, and the initial region range of the region of interest is preset;

[0082] According to the judgment result, the target area range of the area of ​​interest is determined;

[0083] The priority of the target obstacles is determined based on the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information.

[0084] Specifically, the above region of interest (ROI) refers to an area to be processed in the image being processed in the form of a box, circle, ellipse, irregular polygon, etc. in machine vision and image processing. The number and shape of the ROI can be set according to actual needs and are not specifically limited here.

[0085] In one embodiment, the initial area range of the area of ​​interest can be pre-set, and then according to the vehicle perception information and the obstacle perception information, it is determined whether the target obstacle meets the preset conditions for adjusting the area of ​​interest. If the target obstacle meets the preset conditions for adjusting the area of ​​interest, the area range of the area of ​​interest is adjusted, and the adjusted area range is determined as the target area range of the area of ​​interest; if the target obstacle does not meet the preset conditions for adjusting the area of ​​interest, the area range of the area of ​​interest is not adjusted, and the initial area range of the area of ​​interest is determined as the target area range of the area of ​​interest. Then, according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information, the priority of the target obstacle is determined. By using the same technical means as above, each obstacle in the current frame is traversed in turn, and the priority of each obstacle in the current frame can be determined.

[0086] In this embodiment, the area range of the region of interest corresponding to different obstacles can be dynamically adjusted, and then priorities can be determined for different obstacles, so that the determined priorities of each obstacle can better meet actual application requirements.

[0087] Further, the region of interest includes a forward first rectangular region and a backward first rectangular region, and the adjustment condition includes a first condition for adjusting the widths of the forward first rectangular region and the backward first rectangular region and a second condition for adjusting the length of the backward first rectangular region;

[0088] The above steps, based on the vehicle perception information and the obstacle perception information, determine whether the target obstacle meets the preset conditions for adjusting the region of interest, specifically include:

[0089] Based on the vehicle perception information and obstacle perception information, determine whether the vehicle is at the intersection or whether the target obstacle is at the intersection;

[0090] When it is determined that the vehicle is at the intersection or the target obstacle is at the intersection, determining that the target obstacle satisfies the first condition;

[0091] According to the vehicle perception information and the obstacle perception information, determine whether the target obstacle is located in the rearward second rectangular area and whether the type of the target obstacle is a non-motor vehicle type, wherein the rearward second rectangular area includes the rearward first rectangular area;

[0092] When it is determined that the target obstacle is located in the rearward second rectangular area and the type of the target obstacle is a non-motor vehicle type, it is determined that the target obstacle satisfies the second condition.

[0093] It should be noted that the above-mentioned region of interest may include a first front rectangular region FrontROI (abbreviated as FR) and a first rear rectangular region BackROI (abbreviated as BR), wherein the first front rectangular region FR is contained in the second front rectangular region FrontArea (abbreviated as FA), and the first rear rectangular region BR is contained in the second rear rectangular region BackArea (abbreviated as BA). The initial area ranges of the first front rectangular region FR, the second front rectangular region FA, the first rear rectangular region BR, and the second rear rectangular region BA can be set according to the vehicle position, heading angle, and perception range, and the axial direction of each area is consistent with the heading angle of the vehicle. Each area is a rectangular box (Bounding Box, abbreviated as BB), and each rectangular box can be represented by a 5-dimensional vector [center_x, center_y, axis_heading, height, width], wherein center_x and center_y represent the center point coordinates, axis_heading represents the axial direction, length represents the area length, and width represents the area width.

[0094] In this way, the initial area range of each area can be obtained as follows Figure 2 As an optional implementation, the effective sensing range can be SCAN_LENGTH=80, SCAN_WIDTH=12, and the vector expressions of each area are as follows:

[0095]

[0096] In one embodiment, it is possible to determine whether the vehicle is at an intersection or a target obstacle is at an intersection based on the vehicle perception information and the obstacle perception information. If it is determined that the vehicle is at an intersection or the target obstacle is at an intersection, it is possible to determine that the target obstacle satisfies the first condition, and the widths of the forward first rectangular area and the rearward first rectangular area can be adjusted. If it is determined that the vehicle is not at an intersection and the target obstacle is not at an intersection, it is possible to determine that the target obstacle does not satisfy the first condition, and the widths of the forward first rectangular area and the rearward first rectangular area cannot be adjusted.

[0097] In addition, it is also possible to determine whether the target obstacle is located in the rearward second rectangular area and whether the type of the target obstacle is a non-motor vehicle type based on the vehicle perception information and the obstacle perception information. If it is determined that the target obstacle is located in the rearward second rectangular area and the type of the target obstacle is a non-motor vehicle type, it can be determined that the target obstacle meets the second condition and the length of the rearward first rectangular area can be adjusted. If it is determined that the target obstacle is not located in the rearward second rectangular area, or the type of the target obstacle is a motor vehicle type, it can be determined that the target obstacle does not meet the second condition and the length of the rearward first rectangular area cannot be adjusted.

[0098] In this embodiment, the region of interest of each obstacle can be adaptively updated according to the driving behavior and scene of the vehicle and surrounding obstacles, so that the priority of the obstacle determined according to the updated region of interest is more accurate.

[0099] Furthermore, the above steps, based on the judgment result, determine the target area range of the region of interest, specifically including:

[0100] When it is determined that the target obstacle meets the first condition, the widths of the first forward rectangular area and the first backward rectangular area are expanded to N times of the original widths to obtain the target area range of the area of ​​interest, where N is a positive number greater than 1;

[0101] When it is determined that the target obstacle satisfies the second condition, the length of the backward first rectangular area is reduced to M times of the original length to obtain the target area range of the region of interest, where M is a positive number less than 1.

[0102] In one embodiment, when the vehicle or obstacles are in different scenes such as multiple lanes or intersections, the size of the regions FR and BR will also change accordingly. With respect to the width SCAN_WIDTH=12 (12 / 3.75=3.2), that is, one lane is considered on the left and right of the vehicle, the collision risk is relatively high in the intersection scene (such as the vehicle turning right at the intersection or unprotected left turn scene), so the adaptive change of the region of interest refers to the change of the width width of the regions FR and BR, which can be expressed by the following formula.

[0103]

[0104] As an optional implementation, the value of N may be 3, that is, when the vehicle is at an intersection or the target obstacle is at an intersection, the widths of the regions FR and BR are expanded to three times their original widths to reduce the risk of collision.

[0105] If the obstacle target category in the rearward area is not a non-motor vehicle, the size of the area BR will also change accordingly. The area BR becomes smaller in the longitudinal direction, that is, the length of the area BR changes adaptively, which can be expressed by the following formula:

[0106]

[0107] As an optional implementation, the value of M can be 0.5, that is, when the target obstacle is located in the second rear rectangular area and the type of the target obstacle is a non-motor vehicle type, the length of area BR is reduced to 0.5 times the original length to reduce the risk of collision.

[0108] In this embodiment, the region of interest of each obstacle can be adaptively updated according to the driving behavior and scene of the vehicle and surrounding obstacles, so that the priority of the obstacle determined according to the updated region of interest is more accurate.

[0109] Further, see Figure 3 , Figure 3 A flowchart of an obstacle priority flag generation provided in an embodiment of the present application. Figure 3 As shown, the above steps determine the priority of the target obstacle according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information, and specifically include:

[0110] According to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information, determine whether the target obstacle is stationary;

[0111] When it is determined that the target obstacle is in a stationary state, determining the priority of the target obstacle to be a first priority, wherein the first priority is the lowest priority;

[0112] In the case where it is determined that the target obstacle is in a non-stationary state, determining whether the target obstacle is located in the rearward second rectangular area or the forward second rectangular area, wherein the forward second rectangular area includes the forward first rectangular area;

[0113] When it is determined that the target obstacle is not located in the second rearward rectangular area and is not located in the second forward rectangular area, determining the priority of the target obstacle to be the first priority;

[0114] In the case where it is determined that the target obstacle is located within the rearward second rectangular area or the forward second rectangular area, determining whether the type of the target obstacle is a motor vehicle;

[0115] When it is determined that the type of the target obstacle is a motor vehicle, it is sequentially determined whether the target obstacle satisfies a rearward region of interest determination condition, a rearward away determination condition, a forward reverse determination condition, and a forward away determination condition;

[0116] When the target obstacle does not meet the backward region of interest judgment condition, the backward away judgment condition, the forward reverse judgment condition and the forward away judgment condition, the priority of the target obstacle is determined to be the second priority, wherein the second priority is the highest priority;

[0117] When the target obstacle satisfies the backward region of interest judgment condition, the backward away judgment condition, the forward reverse judgment condition or the forward away judgment condition, the priority of the target obstacle is determined to be the first priority.

[0118] Specifically, for each obstacle at the current moment (or current frame), firstly, its default priority flag is set to priority=0 (such as 0 for normal, 1 for alert, and 2 for ignore); secondly, a series of logical condition judgments are performed in a cascade manner; finally, the final priority flag of each obstacle is obtained.

[0119] The cascade determination rules for the priority flag bits are shown in the following table:

[0120]

[0121]

[0122] It should be noted that the obstacle stationary judgment needs to be integrated based on the following two points: (1) the confidence and flag of the motion state given by the upstream; (2) the current speed of the obstacle and the accumulation of historical frame speed. The passable area judgment can be implemented based on the map application program interface (Application Program Interface, referred to as API) or based on the perception semantic segmentation method.

[0123] In this embodiment, the priority of each obstacle can be determined according to the updated region of interest, so as to achieve classification of the obstacles.

[0124] Furthermore, the rearward region of interest judgment condition refers to that the target obstacle is located in the rearward second rectangular area and is not located in the rearward first rectangular area; the rearward away judgment condition refers to that the target obstacle is located in the rearward first rectangular area and is away from the vehicle; the forward reverse judgment condition refers to that the target obstacle is in a forward reverse state and is not located in the forward first rectangular area; the forward away judgment condition refers to that the target obstacle is away from the vehicle and is not located in the forward first rectangular area.

[0125] Specifically, the heading angle difference between the vehicle and the obstacle, diff_yaw (in degrees), can be used to determine whether the obstacle is far away from the vehicle. The following formula can be used to express it:

[0126] diff_yaw=|NormalizeAngle(ego_yaw-obj_yaw)|*180 / π

[0127] The NormalizeAngle function is used to convert the angle value into a standardized angle between [-π,π]. For different judgment conditions, the heading angle difference diff_yaw has different value requirements, as shown in the following table:

[0128]

[0129] In this embodiment, the difference in heading angles between the ego vehicle and the obstacle can be used to determine whether the obstacle is moving backward, forward, or in the opposite direction relative to the ego vehicle, thereby determining the risk of a collision between the obstacle and the ego vehicle, which can be used as one of the judgment conditions for measuring priority, thereby improving the accuracy of obstacle priority.

[0130] Further, see Figure 3 After determining whether the type of the target obstacle is a motor vehicle, the method further includes:

[0131] When it is determined that the type of the target obstacle is a non-motor vehicle, determining whether the target obstacle is located in a passable area of ​​the vehicle;

[0132] When it is determined that the target obstacle is not located in the passable area of ​​the vehicle, the priority of the target obstacle is determined to be the first priority;

[0133] In the case where it is determined that the target obstacle is located in the passable area of ​​the vehicle, determining whether the target obstacle meets a first preset condition;

[0134] When it is determined that the target obstacle meets the first preset condition, the steps of: sequentially determining whether the target obstacle meets the backward region of interest determination condition, the backward away determination condition, the forward reverse determination condition, and the forward away determination condition;

[0135] When it is determined that the target obstacle does not satisfy the first preset condition, the priority of the target obstacle is determined to be a third priority, wherein the priority of the third priority is higher than the first priority and lower than the second priority.

[0136] In this embodiment, for non-motor vehicle targets, it can be first determined whether they are within the passable area of ​​the vehicle. If they are not within the passable area of ​​the vehicle, they can be ignored, thereby reducing the number of obstacles to be predicted and reducing the processing load. If they are within the passable area of ​​the vehicle, it is further determined whether the obstacle meets the first preset condition. If the obstacle meets the first preset condition, it is necessary to sequentially determine whether the target obstacle meets the rearward area of ​​interest judgment condition, the rearward away judgment condition, the forward reverse judgment condition and the forward away judgment condition, and then determine the priority according to the judgment result; if the obstacle does not meet the first preset condition, it means that its priority is low and the degree of collision risk is low.

[0137] Furthermore, the first preset condition means that the target obstacle is not located at an intersection and in a lane, and the target obstacle is not located in a neighborhood of a front lane, wherein the neighborhood of a front lane means that the target obstacle is located in front of the vehicle and near the lane.

[0138] It should be noted that, from the driving scenario type scenarios_type (0: straight road, 1: curve, 2: intersection, 3: other), the flag bit is_in_junction indicating whether the vehicle is at an intersection and the flag bit is_on_lane indicating whether the vehicle is in a lane can be obtained, as shown in the following formula:

[0139]

[0140]

[0141] In addition, the relative position relationship between the obstacle and the vehicle in the s direction (Frenet coordinate system) can be determined by calculating the projection of the vehicle's heading angle unit vector on the relative position vector of the vehicle and the obstacle (i.e., vector inner product) (s<0 means the obstacle is behind the vehicle; s>0 means the obstacle is in front of the vehicle). The specific formula is as follows:

[0142]

[0143] In an optional embodiment, the threshold value S_THR=-2.

[0144] In addition, you can also use the map API to search whether there is a lane within a certain radius (such as 5 meters) from the obstacle to determine whether the obstacle is near the lane.

[0145] In this embodiment, whether an obstacle is the first preset condition can be determined based on the driving scene type, the projection of the unit vector of the vehicle's heading angle on the relative position vector of the vehicle and the obstacle, and whether there is a lane within a certain radius from the obstacle. This can be used as one of the judgment conditions for measuring priority, thereby improving the accuracy of obstacle priority.

[0146] Furthermore, the above step of determining whether the target obstacle is in a stationary state includes:

[0147] Get the state information of the target obstacle in the historical frame before the current frame;

[0148] When the state information of the historical frames reaches a preset number, determining whether the cumulative proportion of the static state of the historical frames of the target obstacle reaches a first preset threshold;

[0149] When it is determined that the cumulative proportion of the static state of the historical frames of the target obstacle reaches a first preset threshold, determining that the target obstacle is in a static state;

[0150] When it is determined that the cumulative proportion of the static state in the historical frames of the target obstacle has not reached the first preset threshold, determining whether the cumulative proportion of the non-static state in the historical frames of the target obstacle has reached the second preset threshold;

[0151] When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle reaches a second preset threshold, determining that the target obstacle is in a non-stationary state;

[0152] When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle does not reach a second preset threshold or the state information of the historical frames does not reach a preset number, the state of the target obstacle is determined according to the state information of the previous frame of the current frame.

[0153] Specifically, the above-mentioned preset number, the first preset threshold and the second preset threshold can be set according to actual needs, and the embodiments of the present application do not make specific limitations.

[0154] In one embodiment, see Figure 4 , Figure 4 The present invention provides a flowchart for determining whether a target obstacle is in a stationary state. Figure 4 As shown, the specific steps are as follows:

[0155] Determine the total number of obstacles in the current frame. If there are obstacles, iterate all the remaining obstacles, otherwise proceed to the next frame.

[0156] Each obstacle is classified and different minimum speed thresholds SPE_THR are assigned to different categories of obstacles (such as vehicles are represented by CAR_SPE_THR, pedestrians are represented by PED_SPE_THR, cyclists and others are represented by CYC_SPE_THR, etc.). In an optional embodiment, CAR_SPE_THR, PED_SPE_THR and CYC_SPE_THR can be 1m / s, 0.5m / s, and 0.75m / s respectively;

[0157] The motion state confidence conf_move_status given by the upstream is judged. When the motion state confidence conf_move_status is greater than 0.5 (the actual value can be determined according to the upstream performance), the upstream motion state move_status is considered feasible; otherwise, the current speed v_cur is judged. If the current speed v_cur < the minimum speed threshold, is_static_cur = 1. is_static_his = 1 indicates the final static flag of the obstacle (the default is 0, 1 indicates static), and is_static_cur = 1 is the static flag at the current moment (this flag is a temporary local variable and does not need to be released to the downstream);

[0158] Cumulative frame number judgment and historical frame static cumulative proportion judgment, historical frame non-static cumulative proportion judgment. If the cumulative number of frames of the target obstacle at the current moment (i.e., the sum of the current frame and the historical frame) reaches the specified number, the historical frame static cumulative proportion and the historical frame non-static cumulative proportion judgment are performed in sequence. In an optional embodiment, the cumulative frame threshold NUM_HIS_THR=4, the historical frame static cumulative ratio threshold (i.e., the first preset threshold) RAT_THR=0.6, and the historical frame non-static cumulative ratio threshold (i.e., the second preset threshold) RAT_NO_THR=0.6. Among them, the historical frame static cumulative proportion = static frame number / cumulative frame number (∑is_static_cur==1) / NUM_HIS_THR, similarly, the historical frame non-static cumulative proportion = non-static frame number / cumulative frame number (∑is_nostatic_cur==1) / NUM_HIS_THR. If the historical frame static state cumulative ratio threshold or the historical frame non-static state cumulative ratio threshold is not reached, the is_static_his of the target obstacle in the previous frame is used as the value of is_static_his in the current frame, that is, is_static_his(t)=is_static_his(t-1).

[0159] In this embodiment, the state of the target obstacle in the current frame is determined by accumulating the cumulative proportion of static states in historical frames and the cumulative proportion of non-static states in historical frames. This is more accurate than directly judging the state based on the speed of the target obstacle in the current frame.

[0160] Further, see Figure 4 After determining whether the target obstacle is in a stationary state, the method further includes:

[0161] Get the moving speed of the target obstacle in the current frame;

[0162] When it is determined that the target obstacle is stationary and the moving speed is greater than zero, the moving speed is updated to the minimum speed threshold corresponding to the type of the target obstacle.

[0163] Specifically, it can be determined whether the current speed of the target obstacle needs to be updated according to the values ​​of the is_static_his flag and the is_static_cur flag. The following formula can be used to express it:

[0164]

[0165] When is_static_his=0 and is_static_cur=1, it means that the target obstacle is static according to the accumulated frames (is_static_his=0), but is not static according to the current speed (is_static_cur=1), which may be due to inaccurate current frame speed, so the current speed v_cur needs to be updated to SPE_THR. In this way, the data in the system can be corrected in time to provide more accurate data for the downstream.

[0166] Furthermore, after the above step 103, predicting the obstacle with the highest priority in the current frame using the rule prediction model, the method further includes:

[0167] Obtain the rule prediction coverage corresponding to each obstacle among the obstacles with the highest priority in the current frame;

[0168] Obstacles whose rule prediction coverage is less than the third preset threshold are predicted again using the learning prediction model.

[0169] Specifically, the third preset threshold can be set according to actual needs, and the embodiment of the present application does not make specific limitations. In one embodiment, the third preset threshold can be set to 0.3. The rule prediction coverage indicates the degree to which the rule prediction model can generate a better future trajectory for the obstacle, and its value range is [0,1].

[0170] In one embodiment, after the highest priority obstacle in the current frame is predicted using the rule prediction model, the rule prediction coverage corresponding to each obstacle in the highest priority obstacle in the current frame can also be obtained; obstacles with rule prediction coverage less than the third preset threshold are predicted again using the learning prediction model. In this way, when the prediction effect of the rule prediction model is not good enough, the learning prediction model can be used again for prediction, thereby improving the accuracy of obstacle trajectory prediction.

[0171] In one embodiment, the Figure 5 The trajectory prediction reasoning optimization system shown implements the obstacle trajectory prediction method in the embodiment of the present application. The trajectory prediction reasoning optimization system may include: an environment data processing layer, an obstacle priority setting layer, a trajectory predictor scheduling layer, and a trajectory result output layer.

[0172] Among them, the environmental data processing layer provides input information for the entire trajectory prediction and inference optimization system, which includes the modeling of ego vehicle perception information and obstacle perception information. The modeling of ego vehicle perception information refers to the modeling of ego vehicle environment data based on ego vehicle positioning information, HD map information and ego vehicle planning information to generate ego vehicle perception information. The constituent fields of ego vehicle perception information include but are not limited to: timestamp ego_timestamp, frame number frame_id, tracking number ego_id, life cycle age, three-dimensional coordinates [ego_x, ego_y, ego_z] in the UTM coordinate system, heading angle ego_yaw, and driving scenario type scenarios_type. The modeling of obstacle perception information refers to the modeling of obstacle environment data based on the vehicle perception information and HD map information to generate various obstacle perception information. The components of the obstacle perception information include but are not limited to: timestamp obj_timestamp, frame number frame_id, tracking number obj_id, target category type_id, life cycle age, three-dimensional coordinates in the UTM coordinate system [obj_x, obj_y, obj_z], heading angle obj_yaw, driving scenario type scenarios_type, motion status move_status, and motion status confidence conf_move_status.

[0173] The obstacle priority setting layer can load the ego vehicle perception information and obstacle perception information in the environment data processing layer, and after a series of automated operations, generate the obstacle priority flag required by the trajectory predictor scheduling layer. The obstacle priority setting layer can include three parts: initial setting of the region of interest, adaptive update according to the ego vehicle driving behavior and scene, and adaptive update according to the obstacle driving behavior and scene. Among them, the initial setting of the region of interest sets the area around the ego vehicle as the first forward rectangular area, the second forward rectangular area, the first backward rectangular area, and the second backward rectangular area according to the ego vehicle position, heading angle, and perception range, respectively recorded as FrontROI (FR), FrontArea (FA), BackROI (BR), BackArea (BA), and the axial direction of each area is consistent with the heading angle of the ego vehicle. Each area is a rectangular box (Bounding Box, BB), which can be represented by the following 5-dimensional vector [center_x, center_y, axis_heading, height, width]. The adaptive update according to the driving behavior and scene of the ego vehicle and the adaptive update according to the driving behavior and scene of the obstacle are used to adaptively update the initial area of ​​the area of ​​interest according to the driving behavior and scene of the ego vehicle and the surrounding obstacles, and generate the obstacle priority flag priority.

[0174] The trajectory prediction scheduling layer mainly involves three parts: prediction priority sorting, rule prediction coverage generation, and learning prediction model adaptation. Specifically, the obstacle priority flag generated by the obstacle priority setting layer can be used for sorting, and the rule-based prediction model is used to process the obstacles with the highest priority (i.e., priority = 0). The category priority is pedestrians, cyclists, and vehicles. Multi-threaded parallel computing can also be used. The rule-based prediction model will attach a cover flag to each obstacle (the cover flag can be set based on the satisfaction of the cascade judgment logic within the rule prediction model). When the cover is less than a certain threshold (such as 0.3), the obstacle is sent to the learning-based prediction model for trajectory prediction again, such as the VectorNet model (a neural network model for processing sequence data), the TNT (abbreviation for Transformer-In-Transformer) model, the DenseTNT model (an anchor-free target-based model that can be learned in an end-to-end manner), the Multipath model (i.e., the Multipath l model), the SceneTransformer model, etc. After processing the highest priority obstacle (priority=0), if it is within the calculation cycle (assuming that the prediction results are sent downstream at a certain cycle, the time taken to process the obstacle with priority=0 is within 100ms of the calculation cycle), the obstacles with lower priority (priority=1) can continue to be processed based on the learned prediction model, and there is no need to process the obstacles with the lowest priority (priority=2). If it exceeds the calculation cycle, there is no need to process the obstacles with priority=1 and priority=2.

[0175] The trajectory prediction output layer is the output layer of the whole system, which is used to output the obstacle trajectory prediction results for downstream services. The trajectory prediction information of each obstacle may include but is not limited to: timestamp obj_timestamp, frame number frame_id, tracking number obj_id, target category type_id, life cycle age, three-dimensional coordinates in the UTM coordinate system [obj_x, obj_y, obj_z], heading angle obj_yaw, driving scenario type scenarios_type, motion state move_status, and motion state confidence conf_move_status, priority flag priority, and rule coverage cover.

[0176] The trajectory prediction information of each obstacle is visualized as a trajectory. Each trajectory consists of multiple waypoints. Assuming that 5 waypoints can be used per second, N seconds is 5N waypoints. In this way, the vectorization of each waypoint in the trajectory of the next N seconds can be obtained, which is expressed as follows:

[0177] [obj_timestamp,frame_id,obj_id,type_id,age,obj_x,obj_y,obj_z,obj_yaw,scenarios_type,move_status,conf_move_status,priority,cover]

[0178] In this embodiment, by studying the impact of different obstacles on the driving of the vehicle, an adaptive update strategy of the region of interest that changes with the driving scenario and driving behavior is designed, and the adaptive region of interest and cascade judgment rules are used to generate obstacle priorities to solve the fusion problem of rule prediction and learning prediction, alleviate the time-consuming problem of the trajectory prediction model based on deep learning under the condition of limited computing power, and finally realize the fast and effective prediction of the trajectories of multiple categories of obstacles.

[0179] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an obstacle trajectory prediction device provided in an embodiment of the present application. Figure 6 As shown, the obstacle trajectory prediction device 600 includes:

[0180] The first acquisition module 601 is used to acquire the vehicle perception information and obstacle perception information corresponding to the current frame;

[0181] A determination module 602 is used to determine the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information;

[0182] The first prediction module 603 is used to predict the obstacle with the highest priority in the current frame using the rule prediction model, and predict the remaining obstacles except the obstacle with the highest priority in the current frame using the learning prediction model to obtain the predicted trajectory corresponding to each obstacle in the current frame.

[0183] Further, the determination module 602 includes:

[0184] A judgment submodule, used to judge whether the target obstacle meets the preset conditions for adjusting the region of interest based on the vehicle perception information and the obstacle perception information, wherein the target obstacle is any obstacle in the current frame, and the initial region range of the region of interest is preset;

[0185] A first determination submodule is used to determine the target area range of the region of interest according to the judgment result;

[0186] The second determination submodule is used to determine the priority of the target obstacle according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information.

[0187] Further, the region of interest includes a forward first rectangular region and a backward first rectangular region, and the adjustment condition includes a first condition for adjusting the widths of the forward first rectangular region and the backward first rectangular region and a second condition for adjusting the length of the backward first rectangular region;

[0188] The judgment submodule includes:

[0189] A first judgment unit, configured to judge whether the ego vehicle is at an intersection or whether a target obstacle is at an intersection according to the ego vehicle perception information and the obstacle perception information;

[0190] A first determination unit, configured to determine that the target obstacle satisfies a first condition when it is determined that the vehicle is at an intersection or the target obstacle is at an intersection;

[0191] a second judgment unit, configured to judge whether the target obstacle is located in a rearward second rectangular area and whether the type of the target obstacle is a non-motor vehicle type according to the vehicle perception information and the obstacle perception information, wherein the rearward second rectangular area includes the rearward first rectangular area;

[0192] The second determination unit is used to determine that the target obstacle meets the second condition when it is determined that the target obstacle is located in the rear second rectangular area and the type of the target obstacle is a non-motor vehicle type.

[0193] Furthermore, the first determining submodule includes:

[0194] An expansion unit, configured to expand the widths of the forward first rectangular area and the backward first rectangular area to N times of the original widths to obtain a target area range of the area of ​​interest when it is determined that the target obstacle meets the first condition, where N is a positive number greater than 1;

[0195] The reduction unit is used to reduce the length of the backward first rectangular area to M times the original length when it is determined that the target obstacle meets the second condition, so as to obtain the target area range of the area of ​​interest, where M is a positive number less than 1.

[0196] Furthermore, the second determining submodule includes:

[0197] A third judgment unit is used to judge whether the target obstacle is in a stationary state according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information;

[0198] A third determining unit is used to determine the priority of the target obstacle as a first priority when it is determined that the target obstacle is in a stationary state, wherein the first priority is the lowest priority;

[0199] a fourth judgment unit, configured to judge whether the target obstacle is located in the rearward second rectangular area or the forward second rectangular area when the target obstacle is determined to be in a non-stationary state, wherein the forward second rectangular area includes the forward first rectangular area;

[0200] a fourth determining unit, configured to determine that the priority of the target obstacle is the first priority when it is determined that the target obstacle is not located in the rearward second rectangular area and is not located in the forward second rectangular area;

[0201] A fifth judgment unit, configured to judge whether the type of the target obstacle is a motor vehicle when it is determined that the target obstacle is located within the rearward second rectangular area or the forward second rectangular area;

[0202] a sixth judgment unit, configured to, when determining that the type of the target obstacle is a motor vehicle, sequentially judge whether the target obstacle satisfies a rearward region of interest judgment condition, a rearward away judgment condition, a forward reverse judgment condition, and a forward away judgment condition;

[0203] a fifth determining unit, configured to determine the priority of the target obstacle as a second priority when the target obstacle does not satisfy the backward region of interest judgment condition, the backward away judgment condition, the forward reverse judgment condition, and the forward away judgment condition, wherein the second priority is the highest priority;

[0204] The sixth determination unit is used to determine that the priority of the target obstacle is the first priority when the target obstacle meets the backward area of ​​interest judgment condition, the backward away judgment condition, the forward reverse judgment condition or the forward away judgment condition.

[0205] Furthermore, the rearward region of interest judgment condition refers to that the target obstacle is located in the rearward second rectangular area and is not located in the rearward first rectangular area; the rearward away judgment condition refers to that the target obstacle is located in the rearward first rectangular area and is away from the vehicle; the forward reverse judgment condition refers to that the target obstacle is in a forward reverse state and is not located in the forward first rectangular area; the forward away judgment condition refers to that the target obstacle is away from the vehicle and is not located in the forward first rectangular area.

[0206] Furthermore, the second determining submodule further includes:

[0207] a seventh determination unit, configured to determine whether the target obstacle is located in a passable area of ​​the vehicle when the target obstacle is determined to be a non-motor vehicle;

[0208] a seventh determination unit, configured to determine the priority of the target obstacle as the first priority when it is determined that the target obstacle is not located in the passable area of ​​the vehicle;

[0209] an eighth judgment unit, configured to judge whether the target obstacle satisfies a first preset condition when it is determined that the target obstacle is located in a passable area of ​​the vehicle;

[0210] The execution unit is used to, when it is determined that the target obstacle meets the first preset condition, execute the steps of: sequentially determining whether the target obstacle meets the backward region of interest determination condition, the backward away determination condition, the forward reverse determination condition and the forward away determination condition;

[0211] The eighth determination unit is used to determine that the priority of the target obstacle is a third priority when it is determined that the target obstacle does not meet the first preset condition, wherein the priority of the third priority is higher than the first priority and lower than the second priority.

[0212] Furthermore, the first preset condition means that the target obstacle is not located at an intersection and in a lane, and the target obstacle is not located in a neighborhood of a front lane, wherein the neighborhood of a front lane means that the target obstacle is located in front of the vehicle and near the lane.

[0213] Furthermore, the third judgment unit is specifically used for:

[0214] Get the state information of the target obstacle in the historical frame before the current frame;

[0215] When the state information of the historical frames reaches a preset number, determining whether the cumulative proportion of the static state of the historical frames of the target obstacle reaches a first preset threshold;

[0216] When it is determined that the cumulative proportion of the static state of the historical frames of the target obstacle reaches a first preset threshold, determining that the target obstacle is in a static state;

[0217] When it is determined that the cumulative proportion of the static state in the historical frames of the target obstacle has not reached the first preset threshold, determining whether the cumulative proportion of the non-static state in the historical frames of the target obstacle has reached the second preset threshold;

[0218] When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle reaches a second preset threshold, determining that the target obstacle is in a non-stationary state;

[0219] When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle does not reach a second preset threshold or the state information of the historical frames does not reach a preset number, the state of the target obstacle is determined according to the state information of the previous frame of the current frame.

[0220] Furthermore, the third judgment unit is further used for:

[0221] Get the moving speed of the target obstacle in the current frame;

[0222] When it is determined that the target obstacle is stationary and the moving speed is greater than zero, the moving speed is updated to the minimum speed threshold corresponding to the type of the target obstacle.

[0223] Furthermore, the device 600 further includes:

[0224] The second acquisition module is used to obtain the rule prediction coverage corresponding to each obstacle among the obstacles with the highest priority in the current frame;

[0225] The second prediction module is used to re-predict obstacles whose rule prediction coverage is less than a third preset threshold value using a learning prediction model.

[0226] It should be noted that the obstacle trajectory prediction device 600 can implement the steps of the obstacle trajectory prediction method provided by any of the aforementioned method embodiments and can achieve the same technical effects, which will not be described in detail herein.

[0227] like Figure 7 As shown, the embodiment of the present application further provides an electronic device, including a processor 711, a communication interface 712, a memory 713 and a communication bus 714, wherein the processor 711, the communication interface 712, and the memory 713 communicate with each other through the communication bus 714.

[0228] Memory 713, used for storing computer programs;

[0229] In one embodiment of the present application, the processor 711 is used to implement the steps of the obstacle trajectory prediction method provided by any of the aforementioned method embodiments when executing the program stored in the memory 713.

[0230] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the obstacle trajectory prediction method provided by any of the aforementioned method embodiments are implemented.

[0231] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer implements the steps of the obstacle trajectory prediction method provided by any of the aforementioned method embodiments.

[0232] The present application also provides a mobile tool, which includes Figure 7 The electronic device shown in the figure can realize the above-mentioned Figure 7The functions of the electronic devices shown in the figure can achieve the same technical effects, which will not be described one by one here. The mobile tool can be all devices with mobile capabilities, including vehicles with automatic driving or intelligent driving (including manned vehicles (such as cars, buses, buses, minibuses, etc.), cargo vehicles (such as ordinary trucks, vans, trailers, closed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, special structure trucks), special vehicles (such as logistics distribution vehicles, automatic guided transport vehicles AGV, patrol cars, cranes, cranes, excavators, bulldozers, forklifts, rollers, loaders, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor washing vehicles, water trucks, sweeping robots, food delivery robots, shopping guide robots, lawn mowers, golf carts, etc.), vehicles with entertainment functions (such as entertainment vehicles, playground automatic driving devices, balance cars, etc.), rescue vehicles (such as fire trucks, ambulances, power repair vehicles, engineering rescue vehicles, etc.)) and robots (such as sweeping robots, food delivery robots, etc.).

[0233] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0234] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for predicting obstacle trajectories, characterized in that: The method comprises: Obtain the vehicle perception information and obstacle perception information corresponding to the current frame; Determining the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information; The obstacle with the highest priority in the current frame is predicted using a rule prediction model, and the remaining obstacles in the current frame except the obstacle with the highest priority are predicted using a learning prediction model to obtain a predicted trajectory corresponding to each obstacle in the current frame.

2. The method according to claim 1, characterized in that: The determining the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information includes: According to the vehicle perception information and the obstacle perception information, determining whether the target obstacle meets a preset condition for adjusting the region of interest, wherein the target obstacle is any obstacle in the current frame, and the initial region range of the region of interest is preset; Determine the target area range of the region of interest according to the judgment result; The priority of the target obstacle is determined according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information.

3. The method according to claim 2, characterized in that The region of interest includes a forward first rectangular region and a backward first rectangular region, and the adjustment condition includes a first condition for adjusting the widths of the forward first rectangular region and the backward first rectangular region and a second condition for adjusting the length of the backward first rectangular region; The determining, based on the vehicle perception information and the obstacle perception information, whether the target obstacle satisfies a preset condition for adjusting the region of interest includes: Determining whether the vehicle is at an intersection or the target obstacle is at an intersection based on the vehicle perception information and the obstacle perception information; When it is determined that the vehicle is at an intersection or the target obstacle is at an intersection, determining that the target obstacle satisfies the first condition; According to the vehicle perception information and the obstacle perception information, determining whether the target obstacle is located in a rearward second rectangular area and whether the type of the target obstacle is a non-motor vehicle type, wherein the rearward second rectangular area includes the rearward first rectangular area; When it is determined that the target obstacle is located in the rearward second rectangular area and the type of the target obstacle is a non-motor vehicle type, it is determined that the target obstacle satisfies the second condition.

4. The method according to claim 3, characterized in that The determining the priority of the target obstacle according to the target area range of the area of ​​interest, the vehicle perception information and the obstacle perception information includes: Determining whether the target obstacle is in a stationary state according to the target area range of the area of ​​interest, the vehicle perception information, and the obstacle perception information; In the case where it is determined that the target obstacle is in a stationary state, determining the priority of the target obstacle to be a first priority, wherein the first priority is the lowest priority; In the case where it is determined that the target obstacle is in a non-stationary state, determining whether the target obstacle is located in the rearward second rectangular area or the forward second rectangular area, wherein the forward second rectangular area includes the forward first rectangular area; When it is determined that the target obstacle is not located in the second rearward rectangular area and is not located in the second forward rectangular area, determining the priority of the target obstacle to be the first priority; In the case where it is determined that the target obstacle is located within the rearward second rectangular area or the forward second rectangular area, determining whether the type of the target obstacle is a motor vehicle; In the case where the type of the target obstacle is determined to be a motor vehicle, determining in sequence whether the target obstacle satisfies a rearward region of interest judgment condition, a rearward away judgment condition, a forward reverse judgment condition, and a forward away judgment condition; When the target obstacle does not satisfy the backward region of interest judgment condition, the backward away judgment condition, the forward reverse judgment condition, and the forward away judgment condition, determining the priority of the target obstacle to be the second priority, wherein the second priority is the highest priority; When the target obstacle satisfies the backward region of interest judgment condition, the backward away judgment condition, the forward reverse judgment condition or the forward away judgment condition, the priority of the target obstacle is determined to be the first priority.

5. The method according to claim 4, characterized in that The rearward region of interest judgment condition means that the target obstacle is located in the rearward second rectangular area and is not located in the rearward first rectangular area; the rearward distance judgment condition means that the target obstacle is located in the rearward first rectangular area and is far away from the vehicle; The forward reverse judgment condition means that the target obstacle is in a forward reverse state and the target obstacle is not located in the forward first rectangular area; the forward away judgment condition means that the target obstacle is away from the vehicle and the target obstacle is not located in the forward first rectangular area.

6. The method according to claim 4, characterized in that After determining whether the type of the target obstacle is a motor vehicle, the method further includes: In the case where it is determined that the type of the target obstacle is a non-motor vehicle, determining whether the target obstacle is located in a passable area of ​​the vehicle; In a case where it is determined that the target obstacle is not located in a passable area of ​​the vehicle, determining the priority of the target obstacle to be the first priority; In the case where it is determined that the target obstacle is located in a passable area of ​​the vehicle, determining whether the target obstacle satisfies a first preset condition; In the case where it is determined that the target obstacle meets the first preset condition, the steps of: sequentially determining whether the target obstacle meets a backward region of interest determination condition, a backward away determination condition, a forward retrograde determination condition, and a forward away determination condition; When it is determined that the target obstacle does not satisfy the first preset condition, the priority of the target obstacle is determined to be a third priority, wherein the priority of the third priority is higher than the first priority and lower than the second priority.

7. The method according to claim 4, characterized in that The determining whether the target obstacle is in a stationary state includes: Obtaining state information of the target obstacle in the historical frame before the current frame; When the state information of the historical frames reaches a preset number, determining whether the cumulative proportion of the static state of the historical frames of the target obstacle reaches a first preset threshold; When it is determined that the cumulative proportion of the static state of the historical frames of the target obstacle reaches the first preset threshold, determining that the target obstacle is in a static state; If it is determined that the cumulative proportion of the static state in the historical frames of the target obstacle does not reach the first preset threshold, determining whether the cumulative proportion of the non-static state in the historical frames of the target obstacle reaches a second preset threshold; When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle reaches the second preset threshold, determining that the target obstacle is in a non-stationary state; When it is determined that the cumulative proportion of the non-stationary state of the historical frames of the target obstacle does not reach the second preset threshold or the state information of the historical frames does not reach the preset quantity, the state of the target obstacle is determined according to the state information of the previous frame of the current frame.

8. The method according to claim 1, characterized in that After predicting the obstacle with the highest priority in the current frame using the rule prediction model, the method further includes: Obtaining the rule prediction coverage corresponding to each obstacle among the obstacles with the highest priority in the current frame; Obstacles whose rule prediction coverage is less than a third preset threshold are predicted again using the learning prediction model.

9. An obstacle trajectory prediction device, characterized in that: The device comprises: The first acquisition module is used to acquire the vehicle perception information and obstacle perception information corresponding to the current frame; A determination module, configured to determine the priority of each obstacle in the current frame according to the vehicle perception information and the obstacle perception information; The first prediction module is used to predict the obstacle with the highest priority in the current frame using a rule prediction model, and to predict the remaining obstacles in the current frame except the obstacle with the highest priority using a learning prediction model to obtain a predicted trajectory corresponding to each obstacle in the current frame.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the obstacle trajectory prediction method described in any one of claims 1 to 8 when executing the program stored in the memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the obstacle trajectory prediction method according to any one of claims 1 to 8 are implemented.

12. A mobile tool, characterized in that: The electronic device comprising claim 10.

Images