Walking track prediction method based on interaction of pedestrian, environment and vehicle

Abstract

The invention discloses a walking track prediction method based on interaction of a pedestrian, an environment and a vehicle, the method is based on a physical model, pedestrian tracks can be predicted more accurately by combining interaction of the pedestrian, the environment and other intelligent agents, and the prediction method recorded in the invention is not based on a data-driven scheme, so that the prediction efficiency is greatly improved. Therefore, a large amount of labeled data is not needed, and the defects of poor model interpretability, poor interactivity and the like existing in a deep learning scheme are overcome.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and specifically relates to a walking trajectory prediction method based on the interaction between pedestrians, the environment and vehicles. Background Art

[0002] Pedestrian trajectory prediction is a key component of autonomous driving technology. It is not only related to the safety and smoothness of the vehicle, but also affects regulatory compliance and user experience. With the continuous development of autonomous driving technology, the accuracy and reliability of pedestrian trajectory prediction are becoming more and more important. Currently, the industry and academia mainly divide pedestrian prediction into two categories: prediction based on physical models and prediction based on deep learning.

[0003] The walking trajectory prediction methods based on deep learning mainly include virtual recurrent neural network (RNN) and long short-term memory network (LSTM), convolutional neural network (CNN), graph neural network (GNN), generative adversarial network (GAN), etc. Although this type of prediction method can provide highly accurate predictions, they often require a lot of data and training as support, and the model is poorly interpretable. In addition, although some deep learning-based methods consider the interaction with the environment or other intelligent agents, they are all performed through implicit methods, their interactivity cannot be guaranteed, and their interpretability is poor.

[0004] The prediction based on physical models mainly involves the following methods:

[0005] Constant velocity (CV) model: This model assumes that the target moves at a constant velocity. This is the simplest motion model and is often used to deal with situations where there is a lack of acceleration information. Its advantages are that the model is simple, easy to understand and implement, and the amount of calculation is small. Its disadvantages are that it cannot handle scenarios where acceleration or direction changes, and the prediction accuracy will decrease when the target's motion state changes.

[0006] Constant Acceleration (CA) Model: This model assumes that the target moves with constant acceleration. Compared with the CV model, the CA model introduces the acceleration parameter, which enables the model to describe more complex motion states. Its advantage is that compared with the CV model, the CA model can better handle motion prediction problems with acceleration or deceleration. Its disadvantage is that if the acceleration of the target is not constant, the prediction accuracy of the model will still be affected.

[0007] Constant velocity, constant turning rate (CVTR) model: This model is a simplified model for predicting the trajectory of an object, which combines the assumptions of constant velocity (CV) and constant turning rate (CTR). This model is usually used to deal with objects that keep a constant speed during movement and turn at a constant rate. Its advantage is that it provides better predictions than CV and CA models when dealing with scenes with more complex target motion (such as uniform circular motion). Its disadvantage is that if the acceleration or turning rate of the target is not constant, its prediction performance will decrease. The CVTR model assumes that both speed and turning rate are constant, which is rare in the real world. The movement of real objects often involves changes in speed and direction.

[0008] Social Force Model (SFM): This model considers pedestrian behavior as the result of social forces (such as attraction to the destination, repulsion to others) and physical forces (such as repulsion to obstacles). Its advantage is that the model can better simulate the interaction and avoidance behavior of pedestrians in crowded environments. Its disadvantage is that the social force model may be overly simplified when dealing with highly dynamic and complex scenes, and it is difficult to capture all the details of human behavior.

[0009] Speed ​​obstacle (VO) method: This method calculates the optimal path to avoid collision by considering the impact of other pedestrians and static obstacles on the pedestrian's planned path. Its advantage is that it is suitable for real-time obstacle avoidance and path planning. Its disadvantage is that simple obstacle avoidance cannot well describe the predicted intention of the pedestrian and is prone to the freezing robot problem, that is, two pedestrians are stuck to each other and cannot move. Summary of the invention

[0010] In response to the above problems, based on the many problems existing in the current pedestrian walking trajectory prediction, the main purpose of the present invention is to design a walking trajectory prediction method based on the interaction between pedestrians, the environment and vehicles. This method is based on a material model and introduces information on the interaction between pedestrians and the environment as well as between pedestrians and vehicles, so that the prediction of pedestrian walking trajectories is more accurate.

[0011] In order to achieve the above purpose, the present invention adopts the following technical solutions:

[0012] A walking trajectory prediction method based on the interaction between pedestrians and the environment and vehicles. The method uses a physical model to infer the trajectory of pedestrians within a set time, demarcates high-risk areas and low-risk areas in combination with environmental conditions and vehicle conditions, and calculates the final predicted walking trajectory through the interaction between the above-calculated trajectory and the high-risk areas and the low-risk areas; the environmental conditions include but are not limited to construction areas and ground elements, and the ground elements include but are not limited to road edges, ground markings, fences, zebra crossings, and lane lines.

[0013] As a further description of the present invention, the method for predicting the walking trajectory corresponding to the environmental conditions is to define boundaries according to the environmental conditions, and record the intersection of the walking trajectory and the boundary calculated according to the physical model, count the intersections entering the high-risk area, determine the deceleration of the pedestrian from the current position to the above-mentioned intersection, and then generate a new trajectory using a constant acceleration motion model and select the shortest trajectory as the final predicted trajectory; the boundaries defined according to the environmental conditions include but are not limited to road edges, ground markings, fences, zebra crossings, lane lines, and construction areas.

[0014] As a further description of the present invention, the steps of the method for predicting the walking trajectory corresponding to the environmental conditions are as follows:

[0015] S1. Generate a predicted trajectory according to the prediction method of the physical model;

[0016] S2, calculate the intersection points of the predicted trajectory in S1 and all the boundaries of the map, and add the intersection points corresponding to the trajectory entering the high-risk area from the low-risk area to the set that needs to be judged later;

[0017] S3. If there is a construction area, calculate the convex hull of all the construction areas and go to S4, otherwise go directly to S5;

[0018] S4, calculate the intersection of the trajectory in S1 and the convex hull of all construction areas in S3. If the trajectory enters the high-risk area from the low-risk area, add the corresponding intersection point to the set that needs to be judged later;

[0019] S5. For each intersection point in the above set, calculate the deceleration required to uniformly decelerate along the current speed direction and reduce the speed to 0 when reaching the intersection point;

[0020] S6. For each intersection in the above set, according to the type of the intersection, obtain its maximum allowable deceleration by looking up the table, and compare it with the deceleration obtained in S5, select the deceleration with a smaller absolute value as the final deceleration, and generate a new trajectory with the CA model;

[0021] S7. For all the trajectories in step S6, select the shortest trajectory as the final predicted trajectory.

[0022] As a further description of the present invention, in the step S2, the boundary refers to various types of edge lines on the map, including but not limited to road edges, solid lines, dotted lines, and virtual lanes. The boundary distinguishes the environmental areas, and the divided areas correspond to different risk levels. The intersection refers to the intersection of the predicted trajectory in step S1 and each edge line. The intersection corresponding to the pedestrian's predicted trajectory entering the high-risk area from the low-risk area is summarized and entered into the set that needs to be judged later.

[0023] As a further description of the present invention, the convex hull is calculated in step S3 by first clustering and then finding the convex hull. The clustering method includes but is not limited to the DBSCAN algorithm, and the algorithm for finding the convex hull includes but is not limited to Monotone Chain. The area corresponding to the convex hull is the construction area, and in the judgment of the intersection related to the construction area in step S4, the construction area, i.e., the convex hull, is regarded as a low-risk area.

[0024] As a further description of the present invention, the maximum allowable deceleration in step S6 is a pre-set value, which varies according to the corresponding intersection type; this value is compared with the value calculated in step S5, and the relatively smaller one is selected as the final deceleration.

[0025] As a further description of the present invention, the steps of the method for predicting the corresponding walking trajectory in the case of a vehicle are as follows:

[0026] S11, generating a predicted trajectory according to the prediction method of the physical model;

[0027] S21, judging whether the pedestrian can observe the vehicle according to the situation of the pedestrian and the vehicle. If it is judged that the pedestrian cannot observe the vehicle, the predicted trajectory generated in S11 is used as the final predicted trajectory. If it is judged that the pedestrian can observe the vehicle, further prediction is performed;

[0028] S31, demarcate the area formed by the trajectory of the vehicle as a high-risk area, and determine whether the predicted trajectory in S11 intersects with the high-risk area formed by the vehicle trajectory. If no intersection occurs, the predicted trajectory generated by S11 is used as the final predicted trajectory. If an intersection occurs, calculate the time it takes for the pedestrian to reach the intersection, and predict whether the pedestrian interacts with the vehicle based on the time. If no interaction occurs, use the predicted trajectory of S11 as the final trajectory. If interaction occurs, further prediction is performed.

[0029] S41. When a pedestrian interacts with a vehicle, a predicted trajectory is generated in a segmented deceleration manner.

[0030] As a further description of the present invention, the ways in which pedestrians observe vehicles in step S21 include but are not limited to the following two: the vehicle is within the angle range of ±45° of the pedestrian's walking direction; there is an intersection between the boundary of the pedestrian's predicted trajectory and the vehicle's walking trajectory, and the angle between the colliding vehicle direction and the pedestrian direction is greater than 90°.

[0031] As a further description of the present invention, in step S31, when the predicted trajectory of the pedestrian and the range of the vehicle driving trajectory have an intersection, the method for predicting whether the two interact is to calculate the intersection between the high-risk area formed by the predicted trajectory of the pedestrian and the vehicle driving trajectory, and then judge the relationship between the time when the pedestrian arrives at the intersection and the set time. The setting time is a time set to satisfy the requirement that the pedestrian can stop before reaching the intersection. If the time when the pedestrian arrives at the intersection is greater than the set time, the predicted trajectory of step S11 is the final predicted trajectory. If the time when the pedestrian arrives at the intersection is less than the set time, and the distance between the pedestrian and the intersection is greater than the set distance, further prediction calculation is performed according to step S41. If the time when the pedestrian arrives at the intersection is less than the set time, and the distance between the pedestrian and the intersection is less than the set distance, the predicted trajectory of step S11 is the final predicted trajectory.

[0032] As a further description of the present invention, the segmented deceleration method of step S41 includes several time nodes, and different deceleration upper limits are used for deceleration processing calculation in different time nodes, and several short time nodes include a reaction time node; if the walking trajectory obtained according to the segmented deceleration method causes the pedestrian to stop before reaching the intersection, then this trajectory is used as the final walking trajectory; if the walking trajectory obtained according to the segmented deceleration method prevents the pedestrian from stopping before reaching the intersection, then the final predicted trajectory is obtained by moving at a uniform speed after the segmented deceleration to the intersection.

[0033] Compared with the prior art, the technical effects of the present invention are:

[0034] The present invention provides a walking trajectory prediction method based on the interaction between pedestrians, the environment and vehicles. The method is based on a physical model and can more accurately predict pedestrian trajectories by combining the interaction between pedestrians, the environment and other intelligent entities. The prediction method recorded in the present invention is not based on a data-driven solution, so it does not require a large amount of labeled data, and eliminates the shortcomings of deep learning solutions such as poor model interpretability and poor interactivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0036] Figure 2 This is a clustering effect diagram of the present invention;

[0037] Figure 3 The effect diagram of calculating the convex hull of the present invention;

[0038] Figure 4 This is a rendering of the effect of placing cones on both sides of the road in the corner case of the present invention;

[0039] Figure 5This is a case where there is misjudgment in the identification of the construction area in the corner case of the present invention;

[0040] Figure 6 It is a schematic diagram of the situation in which the pedestrian trajectory and the construction area have two intersection points in the presence of the construction area according to the present invention;

[0041] Figure 7 This is a rendering of the space formed by the vehicle driving trajectory of the present invention. DETAILED DESCRIPTION

[0042] The present invention is described in detail below in conjunction with the accompanying drawings:

[0043] In one embodiment of the present invention, a walking trajectory prediction method based on the interaction between pedestrians, the environment and vehicles is disclosed, referring to Figure 1-7 As shown, the method estimates the trajectory of pedestrians within a certain period of time based on a physical model, delineates high-risk areas and low-risk areas in combination with environmental conditions and vehicle conditions, and calculates the final predicted walking trajectory through the interaction between the above-calculated trajectory and the high-risk areas and low-risk areas; the environmental conditions include but are not limited to construction areas, ground elements such as curbs, ground markings, fences, zebra crossings, and lane lines.

[0044] According to the actual situation of pedestrians walking on the road, the possible interactions are divided into two major categories, one is the interaction between pedestrians and the environment, and the other is the interaction between pedestrians and vehicles; the interaction between pedestrians and the environment mainly involves various road markings, curbs, fences and other road elements that may be encountered in the road environment, and also involves possible construction areas on the road; the interaction between pedestrians and vehicles mainly involves pedestrians' observation of vehicles and the relative positional relationship between the two. In the above two types of interactions, the pedestrians in the first interaction are mobile, while the environmental elements are fixed, and in the second interaction, both pedestrians and vehicles are mobile. This is the difference between the two. The goal to be achieved in this embodiment is that no matter whether the object of pedestrian interaction is fixed or mobile, the area that pedestrians may pass through is divided according to the predicted logic and its risk level is determined, and finally the possible trajectory of pedestrians in the walking process is determined according to the risk level.

[0045] 1. Interaction between pedestrians and the environment

[0046] 1. Principle of the solution

[0047] In this embodiment, the interaction between pedestrians and the environment mainly considers the following two types of elements:

[0048] (1) Construction areas temporarily constructed with cones and barrels;

[0049] (2) Map elements that have been shown on maps for a long time, typically represented by curbs, fences, and zebra crossings.

[0050] The typical performance of the interaction between pedestrians and the environment is that people will consider the cost of crossing these physical limitations to a certain extent. Specifically, when designing the scheme for the interaction between pedestrians and the environment, this embodiment mainly considers the following aspects:

[0051] (1) People are rational to a certain extent and will not easily cross the above physical areas. For example, in a construction area, people will not walk out of the construction area at will; on the road, people will not easily cross physical restrictions such as fences and run onto the road.

[0052] (2) Different elements have different restraining forces on people. For example, the restraining force of fences on people is stronger than that of curbs. The probability of people crossing the road is different in the scenario where the sidewalk and the road are separated by the curb than in the scenario where the sidewalk and the road are separated by fences. In the latter, because people need to make more effort to cross the road (need to cross the fence), the probability of crossing the road is lower than in the former scenario.

[0053] (3) Although most people are rational, there are still a small number of irrational people who will cross the above-mentioned physical areas. Therefore, we cannot assume that everyone strictly abides by the above assumptions. Those who need to break through the limitations of the physical area should still do so.

[0054] (4) The physical limits of the human body will not be exceeded. For example, a person will not move at a speed of 15 m / s, and a person will not instantly accelerate to the acceleration due to gravity g, because a person does not have that much energy.

[0055] 2. Specific implementation process

[0056] Based on the above considerations, this embodiment designs the following solutions:

[0057] S1: Use the constant velocity (CV) model to derive a trajectory of T seconds ρ (T is currently set to 10).

[0058] Use the current speed to recursively deduce the T-second trajectory using the constant velocity (CV) model. Depending on the specific situation, you can also use a CVTR model, for example.

[0059] In addition, we can also consider using the speed obtained after first-order filtering of the historical speed for recursion. The reason why the filtered speed is not used at present is that people have strong mobility and the historical trajectory has little reference value. On the other hand, people’s historical speed is not necessarily accurate.

[0060] S2: Check the intersection of ρ and all borders of the surrounding lanes in the current map environment. If the pedestrian crosses the intersection from a lower-risk area to a higher-risk area, add the intersection to the intersection set Ω. Each intersection contains the intersection position and the type of its corresponding border.

[0061] The border of a map mainly refers to the various edge lines of the map, which currently include the following attributes (including solid lines, dotted lines, virtual lane lines, curbs, etc.). The degree of response of pedestrians to borders with different attributes, that is, the maximum deceleration when passing through the borders, is different.

[0062] If a pedestrian crosses the border from the roadside to the inside of the road, that is, from a low-risk pedestrian road to a high-risk vehicle road, it is considered that the pedestrian will consider stopping before the border to a certain extent, so the intersection with the border is put into the set to be considered. The definitions of low risk and high risk are relative here.

[0063] The current risk is defined from low to high as follows:

[0064] Level 0: Pedestrian walkway (outside the road)

[0065] Level 1: Inside the construction area

[0066] Level 2: Non-vehicle lane

[0067] Level 3: Vehicle lane

[0068] S3: If a cone barrel is detected within the current sensor's detectable range, it is divided through DBSCAN (a density-based clustering algorithm) and several construction areas represented by convex polygons are calculated.

[0069] a. Clustering

[0070] This step uses the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm to cluster all cone buckets. Figure 2 For example, an ideal clustering result clusters all cone buckets (solid dots and hollow dots in the figure) into two categories: the cone buckets corresponding to the hollow dots are one category, and the cone buckets corresponding to the solid dots are another category.

[0071] b. Calculate the convex hull

[0072] This step uses the Monotone Chain algorithm to calculate the convex hull of the clustered results. Taking the above clustering results as an example, the following are generated: Figure 3 The dashed and solid convex polygons shown:

[0073] Corner case and description

[0074] When clustering, sometimes strange clustering results may occur due to improper placement of cones, missed detections, or position detection errors, for example:

[0075] Corner case 1: Figure 4 In the scenario shown, cones are placed on both sides of the road, hoping that cars will pass through carefully. If the clustering algorithm is not set up properly, the two rows of cones will be easily clustered into the same category, which will lead to the mistaken belief that the entire drivable area is in the construction area.

[0076] To address this problem, on the one hand, we need to further optimize the clustering algorithm to reduce the possibility of this problem. On the other hand, even if this problem occurs, it should not have much impact on the vehicle, because the identification of the construction area is used to make predictions for pedestrians, not for vehicle decision-making, so it will not cause vehicles to make wrong decisions such as detouring here. In addition, identifying this scene as a construction area will only make pedestrians' predictions more aggressive, making the vehicle's decision more conservative, so there will be no additional safety hazards.

[0077] When calculating the convex hull, due to the placement of cones, some non-construction areas may be mistakenly identified as construction areas, while some construction areas may be mistakenly identified as non-construction areas, for example:

[0078] Corner case 2: Figure 5 As shown, the construction area identified here is the dotted polygon area, the internal dotted polygon area should actually be the non-construction area, and the solid polygon area should be the construction area.

[0079] There are two types of problems here, and we will explain them separately:

[0080] (1) Non-construction areas are identified as construction areas. This problem will not cause much impact for the following reasons:

[0081] Such areas are generally not too large and are usually long and narrow in a single lane, which does not take up too much space for vehicles to drive. Vehicles generally do not enter this area. If a vehicle has entered this area, it is correct to brake. This area is used for prediction of pedestrians, not for vehicle decision-making, so even if the recognition is wrong, it will not directly lead to the vehicle making a wrong decision. Identifying this scene as a construction area will only make pedestrians' predictions more aggressive, making the vehicle's decision more conservative, so it will not bring additional safety hazards.

[0082] (2) The construction area is identified as a non-construction area. This problem will not cause any impact for the following reasons:

[0083] This type of scenario will only occur in areas that vehicles cannot enter. If the prediction in this area is to affect the vehicle, it must first cross the dotted area. The interaction will take effect when leaving the dotted area.

[0084] When calculating the convex hull, sometimes the cones are connected in a row. In extreme cases, there are only two cones, and the calculated convex hull degenerates into a straight line. For example:

[0085] Corner case 3: When there are only two cones, the convex hull output is a straight line.

[0086] This situation will not cause much impact. The reason is as follows: At this time, both sides of the straight line can be considered as the outside of the construction area. The process of workers going from one side to the other can be understood as entering an infinitely small construction area from the outside of the construction area, and then going out from this infinitely small construction area. At this time, the process of going out is entering a high-risk area from a low-risk area, and the interaction will be considered. In fact, as long as the straight line of the convex hull is understood as a polygon from point a to point b, and then from point b to point a, the algorithm can be directly established without any modification.

[0087] Corner case 4: When there is only one cone, the construction area cannot be distinguished and needs to be considered separately.

[0088] S4: Calculate the intersection points between ρ and the polygons of all the construction areas mentioned above. If the pedestrian crosses the intersection point from a lower-risk area to a higher-risk area, add it to the set Ω and record the type of these intersection points as construction areas.

[0089] This step is similar to S2, but it should be noted that the predicted trajectory may have two intersections with a construction area. Only the intersection points outside the construction area (i.e. the pedestrian is currently in the construction area and is walking out of the construction area) will be added to the set of considered intersection points (because it is from a low-risk to a high-risk area). Figure 6 As shown, only p2 will be added.

[0090] S5: For each intersection point p, calculate the required deceleration according to Newton's kinematics formula if the vehicle moves along the current speed direction with uniform deceleration and the speed drops to 0 when it reaches the intersection point. The formula is Where v is the pedestrian speed and s is the distance from the pedestrian to the intersection.

[0091] Currently, only the longitudinal interaction mode is considered, that is, considering the pedestrian's current forward direction, if it enters the high-risk area from the low-risk area, it will slow down to a certain extent and stop and wait before entering the high-risk area. It is currently assumed that it will move along the current speed direction with a uniform deceleration, and the speed will drop to 0 when it reaches the intersection. Based on this assumption, the required deceleration is calculated The required deceleration will be calculated for all intersection points in the above set.

[0092] S6: For each intersection point p, according to the type of intersection point p, each type of intersection point has a corresponding pre-defined maximum allowable deceleration. (The maximum here means the maximum absolute value), query to get the maximum allowable deceleration Pick And the calculation result in step 5 The smaller value of p , with deceleration as -a p The CA model launches the trajectory ρ p .

[0093] The deceleration calculated above may not always be reasonable. For example, if the deceleration exceeds the limit that the human body can withstand, then it should be considered that the pedestrian will exceed the assumption of "stopping before entering the high-risk area". In addition, different high-risk areas have different constraints on people. For example, the risk of entering a lane where a car is driving quickly towards pedestrians is obviously greater than the risk of entering a lane without cars. At this time, the former has a greater constraint on people than the latter (compared with the latter, pedestrians are more reluctant to enter the former). For another example, the risk of crossing a fence to enter a lane is greater than that of just crossing a curb to enter a lane (or the effort required is greater), so the former also has a stronger constraint on pedestrians than the latter.

[0094] In order to reflect the constraint of risk areas on people, we assume that people will decelerate within a certain range when entering the risk area. The vehicle will move at a uniform deceleration rate within the vehicle so that it can stop before entering the risk area. If the actual deceleration calculated to stop before the risk area exceeds Then let it follow Perform uniform deceleration motion. The reason why a maximum deceleration is set here is There are two main considerations:

[0095] (1) The deceleration that people can actually withstand is limited

[0096] (2) People are not completely rational, and there is no guarantee that everyone will stop before the risk area. We can only assume that people stop within a certain range of deceleration.

[0097] In order to reflect people's different resistance to different risk areas, we set different For areas with higher risk, people will restrain themselves from entering the area. The higher the setting ( The higher the absolute value of ).

[0098] For each intersection point p, query the pre-defined maximum allowable deceleration according to the type type corresponding to the intersection point p (The maximum here means the maximum absolute value), with deceleration as a p The CA model launches the trajectory ρ p , where a p =-min.

[0099] S7: For the same pedestrian, the predicted line derived from CV may have multiple intersections with the environment (multiple map element lines or construction area lines intersect with the CV line). At this time, we consider that the person will respond to the intersection that has the greatest restriction on him.

[0100] That is, for all the trajectories corresponding to the intersection points p p , take the shortest trajectory as the final trajectory generated by interaction with the environment Right now:

[0101]

[0102] 2. Interaction between pedestrians and vehicles

[0103] 1. Principle of the solution

[0104] For rational pedestrians, when they can see the vehicle, they will most likely not rush to overtake the vehicle when there is a risk of collision, but will slow down and stop in a safe position in advance. However, whether a person will stop is also affected by factors such as reaction time and field of vision, which need to be considered comprehensively.

[0105] 2. Specific implementation process

[0106] S11: Use the CV model to derive a trajectory ρ of T seconds (T is currently set to 10).

[0107] This step is the same as the S1 step of the pedestrian-environment interaction scheme.

[0108] S21: Determine whether the pedestrian can observe the vehicle. If the pedestrian cannot observe the vehicle, directly output the predicted trajectory of S11 as the final output and end; otherwise, enter S31.

[0109] Determine whether the pedestrian can observe the vehicle. Currently, the following two situations are considered. If one of the two situations is met, it is considered that the pedestrian can observe the vehicle.

[0110] (1) The judgment is made based on the current position and orientation of the pedestrian and the position of the vehicle. Specifically, the vehicle's front position is calculated based on the positioning result of the vehicle's current frame. The pedestrian's speed direction is taken as the pedestrian's orientation. When the vehicle's front point is within a certain angle range of the pedestrian's orientation, it can be considered that the pedestrian can observe the vehicle. The angle range is currently ±45°.

[0111] (2) Judge based on the collision relationship between the vehicle's future trajectory and the pedestrian's predicted trajectory. Specifically, if the pedestrian's predicted trajectory crosses the vehicle boundary inferred from the vehicle's future trajectory, that is, there is a risk of collision, and if the angle between the direction of the vehicle where the collision occurs and the direction of the pedestrian is greater than 90°, it can be considered that the pedestrian can observe the vehicle. The vehicle's future trajectory can be obtained by any prediction method or by information such as v2v. If the vehicle is an intelligent driving vehicle, the planned trajectory of the previous frame can also be used as the future trajectory.

[0112] S31: The space-time corridor formed by the future trajectory of the vehicle is a high-risk area, and other areas are low-risk areas. It is determined whether the predicted trajectory of the pedestrian is from a low-risk area to a high-risk area. If not, the trajectory output in S11 is directly used as the final trajectory output and the prediction is terminated. If yes, it is further determined whether a collision occurs when entering the high-risk area, and then it is determined whether to enter S41 based on information such as the time of the collision.

[0113] According to the future trajectory of the vehicle, the outer contour of the vehicle corresponding to each trajectory point on the current trajectory of the vehicle is calculated, where the outer contour is usually represented in the form of a box, such as Figure 7 As shown in the figure, there is a box at the time point corresponding to each trajectory point, and these boxes form a passage space in time and space. The space is a high-risk area, and other areas are low-risk areas.

[0114] The predicted trajectory of the pedestrian is verified with the spatial position of the area. If the predicted trajectory of the pedestrian does not intersect with the high-risk area throughout the whole process, or the pedestrian is in the high-risk area from the beginning, it means that the pedestrian is not entering the high-risk area from the low-risk area. In this case, the pedestrian is considered not to be affected by the vehicle, and the predicted trajectory in S11 is directly output and the prediction is terminated. Otherwise, the following steps are entered:

[0115] (1) Calculate the intersection point p (also known as the collision point) between the pedestrian’s trajectory and the high-risk area in space.

[0116] (2) Determine whether the pedestrian reaches point p later than 3 seconds later. If so, it is assumed that the pedestrian will try to stop before the collision point, and the process jumps to S41 to calculate the final trajectory. The main basis for this determination is that if it takes more than 3 seconds to reach the collision point p, the pedestrian will have enough reaction time to stop at a safe position. If the collision occurs before 3 seconds, the process proceeds to step (3) for further determination.

[0117] (3) If the pedestrian arrives at the collision point p later than the vehicle and the distance between the pedestrian and the collision point is greater than 50 cm, it is considered that the pedestrian will try to stop before the collision point, and the process jumps to S41 to calculate the final trajectory. The main basis for this judgment is that if the pedestrian is very fast and arrives at the collision boundary first, there is a possibility of overtaking, so it is necessary to ensure that the pedestrian arrives at the collision point later than the vehicle before processing the pedestrian, and it is necessary to consider that the pedestrian has a certain width and detection error. If the pedestrian is too close to the collision point (<50 cm), the pedestrian will not be processed.

[0118] S41: The predicted trajectory line is generated by using a segmented deceleration method, that is, the pedestrian trajectory of 0-3 seconds is derived by the CV uniform speed model, and the pedestrian trajectory of 3-10 seconds is derived by the CA uniform acceleration model. The specific calculation can be based on Newton's kinematics formula.

[0119] In this step, we assume that the pedestrian will follow the predicted trajectory in S11 and use the segmented deceleration method to try to stop before the collision point. Pedestrians have a certain reaction time to vehicles, so we assume that the maximum deceleration of pedestrians in interaction with vehicles at different times is different. Specifically, we assume that people need at least 1.5s to react, so within 0-1.5s, pedestrians will not make any deceleration actions; within 1.5-3s, pedestrians will use 0.25m / s 2 The upper limit of deceleration is 0.5m / s within 3-10s. 2The deceleration limit is used to decelerate; because the deceleration limit in each time period is different, we call this method segmented deceleration. Using the above segmented deceleration method, if the vehicle does not stop before the collision point, it is considered that the trajectory after the collision point will no longer continue to decelerate, but continue to move at a uniform speed to obtain the trajectory after the collision.

[0120] The above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A walking trajectory prediction method based on the interaction between pedestrians, the environment and vehicles, characterized by: The method uses a physical model to infer the trajectory of pedestrians within a set time, delineates high-risk areas and low-risk areas in combination with environmental conditions and vehicle conditions, and predicts the final walking trajectory of a person through the interaction between the above-calculated walking trajectory of the person and the high-risk areas and low-risk areas; the environmental conditions include construction areas and ground elements, and the ground elements include road edges, ground markings, fences, zebra crossings, and lane lines.

2. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 1 is characterized by: The method for predicting the walking trajectory corresponding to the environmental conditions is to define boundaries according to the environmental conditions, record the intersection of the walking trajectory and the boundary calculated according to the physical model, count the intersections entering the high-risk area, determine the deceleration of the pedestrian from the current position to the above-mentioned intersection, and then generate a new trajectory with a constant acceleration motion model and select the shortest trajectory as the final predicted trajectory; the boundaries defined according to the environmental conditions include road edges, ground markings, fences, zebra crossings, lane lines, and construction areas.

3. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 2 is characterized by: The steps of the method for predicting the walking trajectory corresponding to the environment are as follows: S1. Generate a predicted trajectory according to the prediction method of the physical model; S2, calculate the intersection points of the predicted trajectory in S1 and all the boundaries of the map, and add the intersection points corresponding to the trajectory entering the high-risk area from the low-risk area to the set that needs to be judged later; S3. If there is a construction area, calculate the convex hull of all the construction areas and go to S4, otherwise go directly to S5; S4, calculate the intersection of the trajectory in S1 and the convex hull of all construction areas in S3. If the trajectory enters the high-risk area from the low-risk area, add the corresponding intersection point to the set that needs to be judged later; S5. For each intersection point in the above set, calculate the deceleration required to uniformly decelerate along the current speed direction and reduce the speed to 0 when reaching the intersection point; S6. For each intersection in the above set, according to the type of the intersection, obtain its maximum allowable deceleration by looking up the table, and compare it with the deceleration obtained in S5, select the deceleration with a smaller absolute value as the final deceleration, and generate a new trajectory with the CA model; S7. For all the trajectories in step S6, select the shortest trajectory as the final predicted trajectory.

4. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 3 is characterized by: In the step S2, the boundary refers to various edge lines of the map, including road edges, solid lines, dotted lines, and virtual lanes. The boundary distinguishes the environmental areas, and the divided areas correspond to different risk levels. The intersection refers to the intersection of the predicted trajectory in step S1 and each edge line. The intersection corresponding to the pedestrian's predicted trajectory entering the high-risk area from the low-risk area is summarized and entered into the set that needs to be judged later.

5. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 4 is characterized by: The convex hull is calculated in step S3 by first clustering and then finding the convex hull. The clustering method includes the DBSCAN algorithm, and the algorithm for finding the convex hull includes Monotone Chain. The area corresponding to the convex hull is the construction area, and in the judgment of the intersection related to the construction area in step S4, the construction area, i.e., the convex hull, is regarded as a low-risk area.

6. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 5, characterized in that: The maximum deceleration allowed in step S6 is a pre-set value, which varies according to the corresponding intersection type; this value is compared with the value calculated in step S5, and the relatively smaller one is selected as the final deceleration.

7. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 2, characterized in that: The steps of the prediction method of the corresponding walking trajectory in the case of a vehicle are as follows: S11, generating a predicted trajectory according to the prediction method of the physical model; S21, judging whether the pedestrian can observe the vehicle according to the situation of the pedestrian and the vehicle. If it is judged that the pedestrian cannot observe the vehicle, the predicted trajectory generated in S11 is used as the final predicted trajectory. If it is judged that the pedestrian can observe the vehicle, further prediction is performed; S31, demarcate the area formed by the trajectory of the vehicle as a high-risk area, and determine whether the predicted trajectory in S11 intersects with the high-risk area formed by the vehicle trajectory. If no intersection occurs, the predicted trajectory generated by S11 is used as the final predicted trajectory. If an intersection occurs, calculate the time it takes for the pedestrian to reach the intersection, and predict whether the pedestrian interacts with the vehicle based on the time. If no interaction occurs, use the predicted trajectory of S11 as the final trajectory. If interaction occurs, further prediction is performed. S41. When a pedestrian interacts with a vehicle, a predicted trajectory is generated in a segmented deceleration manner.

8. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 7, characterized in that: In step S21, there are at least two ways in which pedestrians observe vehicles: the vehicle is within the angle range of ±45° of the pedestrian's walking direction; the boundary of the pedestrian's predicted trajectory intersects with the vehicle's walking trajectory, and the angle between the colliding vehicle's direction and the pedestrian's direction is greater than 90°.

9. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 8, characterized in that: In the step S31, when the predicted trajectory of the pedestrian intersects with the range of the vehicle's driving trajectory, the method for predicting whether the two interact is to calculate the intersection between the high-risk area formed by the predicted trajectory of the pedestrian and the vehicle's driving trajectory, and then determine the relationship between the time when the pedestrian arrives at the intersection and the set time. The time is set to satisfy the time set so that the pedestrian can stop before reaching the intersection. If the time when the pedestrian arrives at the intersection is greater than the set time, the predicted trajectory of step S11 is used as the final predicted trajectory. If the time when the pedestrian arrives at the intersection is less than the set time, and the distance between the pedestrian and the intersection is greater than the set distance, further prediction calculation is performed according to step S41. If the time when the pedestrian arrives at the intersection is less than the set time, and the distance between the pedestrian and the intersection is less than the set distance, the predicted trajectory of step S11 is used as the final predicted trajectory.

10. The walking trajectory prediction method based on the interaction between pedestrians, environment and vehicles according to claim 9, characterized in that: The segmented deceleration method of step S41 includes several time nodes, and different deceleration upper limits are used for deceleration processing calculation in different time nodes. The several short time nodes include a reaction time node. If the walking trajectory obtained by the segmented deceleration method causes the pedestrian to stop before reaching the intersection, then this trajectory is used as the final walking trajectory. If the walking trajectory obtained by the segmented deceleration method prevents the pedestrian from stopping before reaching the intersection, then the final predicted trajectory is obtained by moving at a uniform speed after the segmented deceleration to the intersection.

Images