Pedestrian risk level assessment method and device based on crossing probability prediction

Abstract

The application discloses a pedestrian danger level evaluation method and device based on a crossing probability prediction, and comprises the following steps: constructing a dangerous area in front of a self car based on a map; for a pedestrian target in front of the self car, judging whether the pedestrian target is in the dangerous area in front of the self car according to a pedestrian speed, a historical trajectory and a human body key point, and predicting a crossing probability of the pedestrian target; dividing a pedestrian danger level according to whether the pedestrian target is in the dangerous area in front of the self car and the crossing probability of the pedestrian target; and adjusting a driving plan of the self car according to the pedestrian danger level. The model is simple, has high accuracy, consumes less reasoning time and occupies less resources.

Description

Technical Field

[0001] This invention belongs to the field of intelligent autonomous driving technology, specifically relating to a method and device for assessing pedestrian hazard level based on crossing probability prediction. Background Technology

[0002] With the continuous development of deep learning, autonomous driving technology has also advanced rapidly. Existing autonomous driving technologies are generally divided into three main parts: environmental perception, planning and decision-making, and vehicle control. Environmental perception is responsible for detecting information about the vehicle's surrounding environment and obstacles, while planning and decision-making plans the vehicle's next actions based on the perceived information. Prediction serves as the bridge connecting environmental perception and planning and decision-making, predicting the next intention of obstacles based on environmental and obstacle information, thus enabling the planning to better determine the next action. Pedestrians are important targets in traffic scenarios, and predicting their behavioral intentions and assessing risks are essential components of prediction technology.

[0003] CN112487905A discloses a method and system for predicting the danger level of pedestrians around vehicles. This method collects historical pedestrian trajectories and road information, predicts future pedestrian trajectories using a pedestrian trajectory fitter, and then determines the pedestrian's danger level by whether the future pedestrian trajectory interacts with the vehicle's future trajectory. This algorithm identifies pedestrian danger by predicting future trajectories, requiring high accuracy in trajectory prediction. However, high-precision pedestrian trajectory prediction often requires large network models for learning, resulting in significant time and resource consumption for inference, while maintaining relatively low accuracy. Furthermore, this scheme primarily uses historical pedestrian trajectories and speeds when extracting pedestrian feature parameters, neglecting features such as pedestrian actions and orientation. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a method and device for assessing pedestrian hazard levels based on pedestrian probability prediction.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] This invention discloses a method for assessing pedestrian hazard level based on crossing probability prediction, comprising:

[0007] Step S1: Construct the danger zone in front of the vehicle based on the map;

[0008] Step S2: For pedestrian targets in front of the vehicle, determine whether they are in the danger zone in front of the vehicle and predict their probability of crossing based on the pedestrian's speed, historical trajectory and key points of the human body;

[0009] Step S3: Classify the pedestrian danger level based on whether the pedestrian is in the danger zone in front of the vehicle and the probability of crossing it;

[0010] Step S4: Adjust the vehicle's driving plan accordingly based on the pedestrian hazard level.

[0011] Based on the above technical solution, the following improvements can be made:

[0012] As a preferred option, step S1 specifically includes:

[0013] Step S1.1: Calculate the current position of the vehicle;

[0014] Step S1.2: Determine the lane where the vehicle is located based on its current position;

[0015] Step S1.3: Based on the lane where the vehicle is located, use the road topology relationship of the map to traverse forward and fill in the lanes up to Xm;

[0016] Step S1.4: Obtain the lane of the road Xm ahead of the vehicle, which is the danger zone ahead of the vehicle.

[0017] As a preferred option, step S2 specifically includes:

[0018] Step S2.1: Determine whether the pedestrian is moving or stationary based on their speed and key body points.

[0019] If the pedestrian is moving, proceed to step S2.2;

[0020] If the pedestrian is stationary, proceed to step S2.3;

[0021] Step S2.2: For moving pedestrians,

[0022] Input the historical trajectory and key points of the moving pedestrian into the judgment model to predict the probability of the moving pedestrian crossing;

[0023] By fitting the historical trajectory of a moving pedestrian to its forward trajectory, and based on the moving pedestrian's forward trajectory and crossing probability, it can be determined whether the pedestrian will enter or move away from the danger zone in front of the vehicle.

[0024] Step S2.3: For stationary pedestrians,

[0025] The vehicle fits the pedestrian's trajectory by matching the pedestrian's current position and orientation, and determines whether the trajectory intersects with the danger zone in front of the vehicle.

[0026] If they intersect, the probability of a pedestrian crossing is calculated based on the distance from the pedestrian's position to the intersection point.

[0027] If they do not intersect, pedestrians will not enter the danger zone in front of the vehicle.

[0028] As a preferred approach, the judgment model obtains the pedestrian crossing probability through the following steps:

[0029] Step A: Extract historical trajectory features and key point features based on the pedestrian's historical trajectory and human body key points, respectively;

[0030] Extract the features of the road edge line ahead based on the danger zone in front of the vehicle;

[0031] Step B: Fuse the pedestrian's historical trajectory features with the target features within a certain range around the pedestrian to obtain the first fused feature;

[0032] Step C: Fuse the first fused feature with the feature of the road edge line ahead to obtain the second fused feature;

[0033] Step D: Fuse the second fusion feature with the key point feature to obtain the third fusion feature;

[0034] Step E: Based on the third fusion feature, the traversal probability is obtained through regression using a multi-layer fully connected network.

[0035] As a preferred option, step S3 specifically includes the following:

[0036] By setting a crossing threshold, pedestrians are classified into different danger levels based on their current location and the relationship between the crossing probability and the crossing threshold.

[0037] Furthermore, this invention also discloses a pedestrian hazard level assessment device based on crossing probability prediction, comprising:

[0038] The danger zone construction module is used to construct the danger zone in front of the vehicle based on the map;

[0039] The judgment module is used to determine whether a pedestrian target in front of the vehicle is in the danger zone in front of the vehicle and to predict the probability of the pedestrian crossing, based on the pedestrian's speed, historical trajectory and key points of the human body.

[0040] The hazard level classification module is used to classify the pedestrian hazard level based on whether the pedestrian is in the danger zone in front of the vehicle and the probability of crossing it.

[0041] The driving plan adjustment module is used to adjust the vehicle's driving plan according to the pedestrian hazard level.

[0042] As a preferred option, the hazardous area construction module specifically includes:

[0043] The current position calculation unit is used to calculate the current position of the vehicle.

[0044] The lane determination unit is used to determine the lane in which the vehicle is located based on its current position.

[0045] The lane expansion unit is used to expand and supplement the lane to Xm by traversing forward using the road topology of the map based on the lane where the vehicle is located.

[0046] The danger zone acquisition unit is used to acquire the lane of the road Xm ahead of the vehicle, which is the danger zone ahead of the vehicle.

[0047] As a preferred solution, the judgment module specifically includes:

[0048] The judgment unit is used to determine whether a pedestrian is moving or stationary based on the pedestrian's speed and key points on the human body.

[0049] If the pedestrian is moving, the moving pedestrian calculation unit performs further operations;

[0050] If the pedestrian is stationary, the stationary pedestrian calculation unit performs further operations;

[0051] The moving pedestrian calculation unit is used to input the historical trajectory and key points of the moving pedestrian into the judgment model to predict the probability of the moving pedestrian crossing.

[0052] By fitting the historical trajectory of a moving pedestrian to its forward trajectory, and based on the moving pedestrian's forward trajectory and crossing probability, it can be determined whether the pedestrian will enter or move away from the danger zone in front of the vehicle.

[0053] The stationary pedestrian calculation unit is used to fit the forward trajectory of a stationary pedestrian by its current position and orientation, and to determine whether the forward trajectory intersects with the danger zone in front of the vehicle.

[0054] If they intersect, the probability of a pedestrian crossing is calculated based on the distance from the pedestrian's position to the intersection point.

[0055] If they do not intersect, pedestrians will not enter the danger zone in front of the vehicle.

[0056] As a preferred option, the judgment model includes:

[0057] The feature extraction unit is used to extract historical trajectory features and key point features based on the pedestrian's historical trajectory and human body key points, respectively.

[0058] Extract the features of the road edge line ahead based on the danger zone in front of the vehicle;

[0059] The first fusion unit is used to fuse the pedestrian's historical trajectory features with the target features within a certain range around the pedestrian to obtain the first fused feature;

[0060] The second fusion unit is used to fuse a fusion feature with the feature of the road edge line ahead to obtain a second fusion feature;

[0061] The third fusion unit is used to fuse the second fusion feature with the key point feature to obtain the third fusion feature;

[0062] The traversal probability calculation unit is used to obtain the traversal probability through regression of a multi-layer fully connected network based on the third fusion feature.

[0063] As a preferred option, the hazard level classification module sets a crossing threshold, and classifies pedestrians into different hazard levels based on their current location and the relationship between the crossing probability and the crossing threshold.

[0064] This invention discloses a method and device for assessing pedestrian hazard level based on crossing probability prediction, which has the following beneficial effects:

[0065] This invention combines a pedestrian's historical trajectory with key points on the human body to comprehensively calculate the pedestrian's crossing probability. It also assesses the pedestrian's danger level by considering the dangerous area in front of the vehicle and its crossing probability. Compared with high-precision pedestrian trajectory prediction, this invention has a simpler model, higher accuracy, less inference time, and less resource consumption. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 A flowchart of a pedestrian hazard level assessment method provided in an embodiment of the present invention.

[0068] Figure 2 The execution flowchart of the judgment model provided in the embodiment of the present invention is shown.

[0069] Figure 3 This is a block diagram of a multi-layer fully connected network provided in an embodiment of the present invention.

[0070] Figure 4 The sigmoid function curve provided in the embodiments of the present invention. Detailed Implementation

[0071] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Using ordinal numbers such as “first,” “second,” “third,” etc. to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, sequence, or any other way.

[0074] Furthermore, the expression "includes" is an "open-ended" expression, which means only that there is a corresponding component or step, and should not be interpreted as excluding additional components or steps.

[0075] To achieve the objectives of this invention, some embodiments of a pedestrian hazard level assessment method and device based on crossing probability prediction are provided, such as... Figure 1 As shown, the methods for assessing pedestrian risk levels include:

[0076] Step S1: Construct the danger zone in front of the vehicle based on the map;

[0077] Step S2: For pedestrian targets in front of the vehicle, determine whether they are in the danger zone in front of the vehicle and predict their probability of crossing within 0 to 2 seconds based on the pedestrian's speed, historical trajectory and key points of the human body;

[0078] Step S3: Classify the pedestrian danger level based on whether the pedestrian is in the danger zone in front of the vehicle and the probability of crossing it;

[0079] Step S4: Adjust the vehicle's driving plan accordingly based on the pedestrian hazard level.

[0080] Each step will be explained in detail below.

[0081] Step S1 specifically includes:

[0082] Step S1.1: Calculate the current position of the vehicle;

[0083] Step S1.2: Determine the lane where the vehicle is located based on its current position;

[0084] Step S1.3: Based on the lane where the vehicle is located, use the road topology relationship of the map to traverse forward and fill in the lane to 30m;

[0085] Step S1.4: Obtain the lane of the road 30m ahead of the vehicle, which is the danger zone in front of the vehicle.

[0086] In step S1.3 above, since vehicle braking takes time, calculations show that for low-speed vehicles, the area within 30m in front of the vehicle is a danger zone that requires attention. If the current road is less than 30m, it needs to be extended to a 30m range by traversing forward based on the road topology in the high-precision map. This range can be adjusted according to the actual situation.

[0087] Step S2 specifically includes:

[0088] Step S2.1: Determine whether the pedestrian is moving or stationary based on their speed and key body points.

[0089] If the pedestrian is moving, proceed to step S2.2;

[0090] If the pedestrian is stationary, proceed to step S2.3;

[0091] Step S2.2: For moving pedestrians,

[0092] First, input the historical trajectory and key points of the moving pedestrians into the judgment model to predict the probability of the moving pedestrians crossing;

[0093] Secondly, by fitting the historical trajectory of the moving pedestrian to its forward trajectory, and based on the moving pedestrian's forward trajectory and crossing probability, it is determined whether the pedestrian will enter or move away from the dangerous area in front of the vehicle.

[0094] Step S2.3: For stationary pedestrians,

[0095] The vehicle fits the pedestrian's trajectory by matching the pedestrian's current position and orientation, and determines whether the trajectory intersects with the danger zone in front of the vehicle.

[0096] If they intersect, the probability of a pedestrian crossing is calculated based on the distance from the pedestrian's position to the intersection point.

[0097] If they do not intersect, pedestrians will not enter the danger zone in front of the vehicle.

[0098] The historical trajectory of the aforementioned pedestrian is formed by several historical trajectory points, which are the locations of the pedestrian in three-dimensional space at a series of historical moments.

[0099] The aforementioned human key points refer to the skeletal key points of pedestrians. They mainly extract the positions of the nose, shoulders, elbows, wrists, hips, knees, ankles, etc. of pedestrians in the image, which contain the pedestrian's action and posture information. The extraction network can be the open-source neural network model hrnet.

[0100] "Fit the forward trajectory of a moving pedestrian by fitting its historical trajectory". Here, the forward trajectory can be fitted by using the least squares method to fit a straight line based on the historical trajectory to obtain the direction of the pedestrian's movement. The pedestrian's position and direction can be used to determine whether the pedestrian is approaching or moving away from the danger zone in front of the vehicle.

[0101] Furthermore, it is worth noting that when judging the probability of a moving pedestrian crossing, compared to first predicting the pedestrian's trajectory and then judging whether the pedestrian has crossed into the danger zone ahead based on the pedestrian's trajectory and the edge of the lane line, this invention directly predicts the probability of a moving pedestrian crossing, resulting in higher accuracy.

[0102] The following experiments are described to verify the effectiveness of the present invention.

[0103] Add 1600 positive samples and 4800 negative samples to the existing dataset, observe the data for 2 seconds, and the accuracy of predicting the probability of crossing within the next 2 seconds is shown in Table 1 below.

[0104] Positive samples start outside the danger zone in front of the vehicle and cross the edge of the danger zone in front of the vehicle within 0-2 seconds; negative samples do not cross the edge of the danger zone in front of the vehicle and remain outside the danger zone in front of the vehicle.

[0105] Table 1 Comparison of accuracy of traversal probability

[0106] Single-passage probability prediction 0.954 Single trajectory prediction 0.801

[0107] The accuracy rate of directly predicting the probability of pedestrians crossing was 0.954, while the accuracy rate of first predicting the most likely trajectory of the pedestrian and then judging whether the pedestrian will cross by whether the trajectory intersects with the danger zone in front of the vehicle was lower at 0.801. This shows that in complex scenarios, the accuracy rate of judging whether a pedestrian will enter the danger zone in front of the vehicle by predicting the trajectory is low.

[0108] Accuracy represents the proportion of samples that are correctly predicted out of all samples. The formula for calculating accuracy is:

[0109] acc = (TP + TN) / (TP + TN + FP + FN)

[0110] Where: TP represents the correct prediction of the pedestrian's trajectory as "crossing";

[0111] TN represents a pedestrian trajectory that was correctly predicted as non-crossing.

[0112] FP represents a pedestrian trajectory that is not crossing but is incorrectly predicted as crossing;

[0113] FN represents a pedestrian trajectory that was incorrectly predicted as not being a pedestrian.

[0114] Furthermore, compared to predicting only historical trajectories, incorporating human keypoints can better predict the probability of pedestrian crossing. Since some pedestrians are blurred or occluded and their keypoints cannot be detected, the HRNet detection model is used to filter out trajectories that cannot detect pedestrians from the original dataset, and trajectories with better image quality are selected for training and testing. The accuracy of the crossing probability is shown in Table 2 below.

[0115] Table 2 Comparison of accuracy of traversal probability

[0116] Single historical trajectory 0.907 Historical trajectory plus key points of the human body 0.918

[0117] Among them, the accuracy of predicting pedestrian crossing probability by incorporating human key points is 0.918, which is higher.

[0118] When a pedestrian's destination in the next 2 seconds happens to be near the edge of the lane, the fluctuation in the result may cause a pedestrian who should be crossing to be detected as not crossing. However, after adding pedestrian key points, the key points can detect the continuous forward movement of the pedestrian, which will increase the probability of the pedestrian crossing and reduce the occurrence of false judgments.

[0119] When a pedestrian is stationary, if the vehicle is traveling too fast, the historical trajectory of the pedestrian scanned by the LiDAR may fluctuate, which may lead to misjudgment of the pedestrian trajectory prediction, making it seem as if the pedestrian is walking, thus resulting in an incorrect prediction result. After adding human body key points, since human body key points can detect the stationary state of the pedestrian, the probability of the pedestrian crossing will be reduced, and the occurrence of misjudgment will be reduced.

[0120] Furthermore, the above-mentioned judgment model obtains the probability of pedestrian crossing through the following steps, such as... Figure 2 As shown, it includes:

[0121] Step A: Use a GRU gated recurrent unit to extract historical trajectory features and key point features based on the pedestrian's historical trajectory and human body key points, respectively;

[0122] Extract the features of the road edge line ahead based on the danger zone in front of the vehicle;

[0123] Step B: Use a graph convolutional neural network to fuse the pedestrian's historical trajectory features with the target features within a certain range around the pedestrian to obtain the first fused feature;

[0124] Step C: Use a graph convolutional neural network to fuse the first fused feature with the feature of the road edge line ahead to obtain the second fused feature;

[0125] Step D: Concatenate and fuse the second fusion feature with the key point feature to obtain the third fusion feature;

[0126] Step E: Based on the third fusion feature, the traversal probability is obtained through regression using a multi-layer fully connected network.

[0127] In step B, "a certain range" refers to the characteristics of vehicles and pedestrians within a certain distance (e.g., 30m) from the pedestrian at the current moment. Since the movement of other pedestrians and vehicles within a certain range around a pedestrian will affect the future movement of that pedestrian, it is necessary to fuse the characteristics of that pedestrian with the characteristics of other pedestrians and vehicles within a certain range around the pedestrian to predict whether the pedestrian will change lanes and cross the road in the future.

[0128] like Figure 3As shown, in a multilayer fully connected network, each output of the next layer is connected to all inputs of the previous layer. The output of the next layer is calculated using all inputs from the previous layer. The calculation formula is as follows:

[0129] a1 = W 11 *x1+W 12 *x2+W 13 *x3+b1

[0130] a2=W 21 *x1+W 22 *x2+W 23 x³ + b³

[0131] a3 = W 31 *x1+W 32 *x2+W 33 x³ + b³

[0132] Where x1, x2, and x3 are the inputs of the fully connected network, a1, a2, and a3 are the outputs, and W is the weight matrix of the fully connected network.

[0133] After passing through a multi-layer fully connected network, the output traversal probability value is normalized to between 0 and 1 using the sigmoid formula. The function curve is shown below. Figure 4 As shown.

[0134]

[0135] Step S3 specifically includes the following: setting a crossing threshold, and classifying pedestrians into three different danger levels based on the current location of the pedestrian and the relationship between the crossing probability and the crossing threshold;

[0136] Specifically: For pedestrians who are not in the danger zone in front of the vehicle and whose probability of crossing is less than the threshold, the danger level is 0;

[0137] For pedestrians who are not in the danger zone in front of the vehicle and whose probability of crossing is greater than the threshold, the danger level is 1;

[0138] Pedestrians in the danger zone in front of the vehicle are considered dangerous, with a danger level of 2.

[0139] It is worth noting that the above classification of hazard levels is not limited to three different hazard levels, and the number of hazard levels can be adjusted according to specific circumstances.

[0140] Step S4 specifically includes the following:

[0141] For pedestrians with a danger level of 0, the vehicle's driving plan remains unchanged, and they will continue to travel along the original road.

[0142] For pedestrians with a danger level of 1, the vehicle will adjust its driving plan as needed;

[0143] For pedestrians with a danger level of 2, vehicles should adjust their driving plans in a timely manner.

[0144] This invention can effectively identify pedestrians about to enter the danger zone in front of the vehicle by predicting the probability of pedestrian crossing, and adjust the subsequent planning and control of the vehicle in a timely manner.

[0145] This invention also discloses a pedestrian hazard level assessment device based on crossing probability prediction, comprising:

[0146] The danger zone construction module is used to construct the danger zone in front of the vehicle based on the map;

[0147] The judgment module is used to determine whether a pedestrian target in front of the vehicle is in the danger zone in front of the vehicle and to predict the probability of the pedestrian crossing, based on the pedestrian's speed, historical trajectory and key points of the human body.

[0148] The hazard level classification module is used to classify the pedestrian hazard level based on whether the pedestrian is in the danger zone in front of the vehicle and the probability of crossing it.

[0149] The driving plan adjustment module is used to adjust the vehicle's driving plan according to the pedestrian hazard level.

[0150] The hazardous area construction module specifically includes:

[0151] The current position calculation unit is used to calculate the current position of the vehicle.

[0152] The lane determination unit is used to determine the lane in which the vehicle is located based on its current position.

[0153] The lane expansion unit is used to extend the lane to 30m by traversing forward using the road topology of the map, based on the lane where the vehicle is located.

[0154] The danger zone acquisition unit is used to acquire the lane of the road 30m ahead of the vehicle, which is the danger zone in front of the vehicle.

[0155] The judgment module specifically includes:

[0156] The judgment unit is used to determine whether a pedestrian is moving or stationary based on the pedestrian's speed and key points on the human body.

[0157] If the pedestrian is moving, the moving pedestrian calculation unit performs further operations;

[0158] If the pedestrian is stationary, the stationary pedestrian calculation unit performs further operations;

[0159] The moving pedestrian calculation unit is used to input the historical trajectory and key points of the moving pedestrian into the judgment model to predict the probability of the moving pedestrian crossing.

[0160] By fitting the historical trajectory of a moving pedestrian to its forward trajectory, and based on the moving pedestrian's forward trajectory and crossing probability, it can be determined whether the pedestrian will enter or move away from the danger zone in front of the vehicle.

[0161] The stationary pedestrian calculation unit is used to fit the forward trajectory of a stationary pedestrian by its current position and orientation, and to determine whether the forward trajectory intersects with the danger zone in front of the vehicle.

[0162] If they intersect, the probability of a pedestrian crossing is calculated based on the distance from the pedestrian's position to the intersection point.

[0163] If they do not intersect, pedestrians will not enter the danger zone in front of the vehicle.

[0164] The judgment model includes:

[0165] The feature extraction unit is used to extract historical trajectory features and key point features based on the pedestrian's historical trajectory and human body key points, respectively.

[0166] Extract the features of the road edge line ahead based on the danger zone in front of the vehicle;

[0167] The first fusion unit is used to fuse the pedestrian's historical trajectory features with the target features within a certain range around the pedestrian to obtain the first fused feature;

[0168] The second fusion unit is used to fuse a fusion feature with the feature of the road edge line ahead to obtain a second fusion feature;

[0169] The third fusion unit is used to fuse the second fusion feature with the key point feature to obtain the third fusion feature;

[0170] The traversal probability calculation unit is used to obtain the traversal probability through regression of a multi-layer fully connected network based on the third fusion feature.

[0171] The danger level classification module sets a crossing threshold and classifies pedestrians into three different danger levels based on their current location and the relationship between the crossing probability and the crossing threshold.

[0172] Specifically: For pedestrians who are not in the danger zone in front of the vehicle and whose probability of crossing is less than the threshold, the danger level is 0;

[0173] For pedestrians who are not in the danger zone in front of the vehicle and whose probability of crossing is greater than the threshold, the danger level is 1;

[0174] Pedestrians in the danger zone in front of the vehicle are considered dangerous, with a danger level of 2.

[0175] The technical solution and method of the pedestrian hazard level assessment device of the present invention are similar, and will not be described in detail here.

[0176] CN115249353A discloses a method and apparatus for predicting pedestrian trajectories based on key points. This method is relatively coarse in judging stationary pedestrians. When judging that a pedestrian is stationary, it does not consider the characteristics of pedestrian movement and orientation. If the pedestrian is close to the vehicle, it cannot effectively brake, causing danger. Moreover, the accuracy of predicting pedestrian trajectories using decoders is required to be high. However, high-precision pedestrian trajectory prediction often requires a large network model for learning, which takes a lot of time and resources for inference, and the accuracy is low.

[0177] This invention combines a pedestrian's historical trajectory with key points on the human body to comprehensively calculate the pedestrian's crossing probability. It also assesses the pedestrian's danger level by considering the dangerous area in front of the vehicle and its crossing probability. Compared with high-precision pedestrian trajectory prediction, this invention has a simpler model, higher accuracy, less inference time, and less resource consumption.

[0178] It should be understood that the various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a floppy disk, CD-ROM, hard disk, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, that machine becomes an apparatus for practicing the present invention.

[0179] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.

Claims

1. A pedestrian hazard level assessment method based on crossing probability prediction, characterized in that, include: Step S1: Construct the danger zone in front of the vehicle based on the map; Step S2: For pedestrian targets in front of the vehicle, determine whether they are in the danger zone in front of the vehicle and predict their probability of crossing based on the pedestrian's speed, historical trajectory and key points of the human body; Step S3: Classify the pedestrian danger level based on whether the pedestrian is in the danger zone in front of the vehicle and the probability of crossing it; Step S4: Adjust the vehicle's driving plan accordingly based on the pedestrian hazard level; Step S2 specifically includes: Step S2.1: Determine whether the pedestrian is moving or stationary based on their speed and key body points. If the pedestrian is moving, proceed to step S2.2; If the pedestrian is stationary, proceed to step S2.3; Step S2.2: For moving pedestrians, Input the historical trajectory and key points of the moving pedestrian into the judgment model to predict the probability of the moving pedestrian crossing; By fitting the historical trajectory of a moving pedestrian to its forward trajectory, and based on the moving pedestrian's forward trajectory and crossing probability, it can be determined whether the pedestrian will enter or move away from the danger zone in front of the vehicle. Step S2.3: For stationary pedestrians, The vehicle fits the pedestrian's trajectory by matching the pedestrian's current position and orientation, and determines whether the trajectory intersects with the danger zone in front of the vehicle. If they intersect, the probability of a pedestrian crossing is calculated based on the distance from the pedestrian's position to the intersection point. If they do not intersect, pedestrians should not enter the danger zone in front of the vehicle; The judgment model obtains the probability of pedestrian crossing through the following steps: Step A: Extract historical trajectory features and key point features based on the pedestrian's historical trajectory and human body key points, respectively; Extract the features of the road edge line ahead based on the danger zone in front of the vehicle; Step B: Fuse the pedestrian's historical trajectory features with the target features within a certain range around the pedestrian to obtain the first fused feature; Step C: Fuse the first fused feature with the feature of the road edge line ahead to obtain the second fused feature; Step D: Fuse the second fusion feature with the key point feature to obtain the third fusion feature; Step E: Based on the third fusion feature, the traversal probability is obtained through regression using a multi-layer fully connected network.

2. The pedestrian hazard level assessment method according to claim 1, characterized in that, Step S1 specifically includes: Step S1.1: Calculate the current position of the vehicle; Step S1.2: Determine the lane where the vehicle is located based on its current position; Step S1.3: Based on the lane where the vehicle is located, use the road topology relationship of the map to traverse forward and fill in the lanes up to Xm; Step S1.4: Obtain the lane of the road Xm ahead of the vehicle, which is the danger zone ahead of the vehicle.

3. The pedestrian hazard level assessment method according to claim 1, characterized in that, Step S3 specifically includes the following: By setting a crossing threshold, pedestrians are classified into different danger levels based on their current location and the relationship between the crossing probability and the crossing threshold.

4. A pedestrian hazard level assessment device based on crossing probability prediction, characterized in that, include: The danger zone construction module is used to construct the danger zone in front of the vehicle based on the map; The judgment module is used to determine whether a pedestrian target in front of the vehicle is in the danger zone in front of the vehicle and to predict the probability of the pedestrian crossing, based on the pedestrian's speed, historical trajectory and key points of the human body. The hazard level classification module is used to classify the pedestrian hazard level based on whether the pedestrian is in the danger zone in front of the vehicle and the probability of crossing it. The driving plan adjustment module is used to adjust the vehicle's driving plan according to the pedestrian hazard level. The judgment module specifically includes: The judgment unit is used to determine whether a pedestrian is moving or stationary based on the pedestrian's speed and key points on the human body. If the pedestrian is moving, the moving pedestrian calculation unit performs further operations; If the pedestrian is stationary, the stationary pedestrian calculation unit performs further operations; The moving pedestrian calculation unit is used to input the historical trajectory and key points of the moving pedestrian into the judgment model to predict the probability of the moving pedestrian crossing. By fitting the historical trajectory of a moving pedestrian to its forward trajectory, and based on the moving pedestrian's forward trajectory and crossing probability, it can be determined whether the pedestrian will enter or move away from the danger zone in front of the vehicle. The stationary pedestrian calculation unit is used to fit the forward trajectory of a stationary pedestrian by its current position and orientation, and to determine whether the forward trajectory intersects with the danger zone in front of the vehicle. If they intersect, the probability of a pedestrian crossing is calculated based on the distance from the pedestrian's position to the intersection point. If they do not intersect, pedestrians should not enter the danger zone in front of the vehicle; The judgment model includes: The feature extraction unit is used to extract historical trajectory features and key point features based on the pedestrian's historical trajectory and human body key points, respectively. Extract the features of the road edge line ahead based on the danger zone in front of the vehicle; The first fusion unit is used to fuse the pedestrian's historical trajectory features with the target features within a certain range around the pedestrian to obtain the first fused feature; The second fusion unit is used to fuse a fusion feature with the feature of the road edge line ahead to obtain a second fusion feature; The third fusion unit is used to fuse the second fusion feature with the key point feature to obtain the third fusion feature; The traversal probability calculation unit is used to obtain the traversal probability through regression of a multi-layer fully connected network based on the third fusion feature.

5. The pedestrian hazard level assessment device according to claim 4, characterized in that, The hazardous area construction module specifically includes: The current position calculation unit is used to calculate the current position of the vehicle. The lane determination unit is used to determine the lane in which the vehicle is located based on its current position. The lane expansion unit is used to expand and supplement the lane to Xm by traversing forward using the road topology of the map based on the lane where the vehicle is located. The danger zone acquisition unit is used to acquire the lane of the road Xm ahead of the vehicle, which is the danger zone ahead of the vehicle.

6. The pedestrian hazard level assessment device according to claim 4, characterized in that, The danger level classification module sets a crossing threshold and classifies pedestrians into different danger levels based on their current location and the relationship between the crossing probability and the crossing threshold.

Images