Pedestrian trajectory prediction method and device

Abstract

The application relates to the technical field of automatic driving, in particular to a pedestrian trajectory prediction method and device, wherein the method comprises the following steps: acquiring a current driving state of a vehicle, determining a target prediction area according to the current driving state, and identifying all pedestrians in the target prediction area as target pedestrians; performing rule-based and data-driven pedestrian intention determination on the target pedestrians to obtain an intention prediction result; performing rule-based and data-driven end point prediction according to the intention prediction result to obtain a fusion end point value; acquiring historical trajectory information of the target pedestrians, and obtaining a predicted trajectory of the target pedestrians according to the historical trajectory information and the fusion end point value. According to the application, the target prediction area and the target pedestrians can be determined according to the current driving state, the computing power is reasonably utilized, the prediction efficiency is improved, the intention determination and the end point prediction are performed by fusing the rule-based and data-driven modes, and the accuracy of the predicted pedestrian trajectory is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a pedestrian trajectory prediction method and device. BACKGROUND

[0002] In the field of automatic driving, the state and behavior prediction of pedestrian targets is very important, which is related to whether the automatic driving system can normally and safely drive. The behavior state of pedestrian targets has great uncertainty. The traditional pedestrian trajectory prediction method generally uses the historical information of pedestrians to predict the future walking trajectory of pedestrians, but this method is based on the experience of researchers to set some prediction rules to output the future walking trajectory of pedestrians. The output trajectory result often has a large deviation from the actual situation.

[0003] In summary, the existing pedestrian trajectory prediction method only predicts based on experience setting rules, and the accuracy of the obtained trajectory result is poor.

[0004] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0005] The present application provides a pedestrian trajectory prediction method and device to solve the technical problem that the pedestrian trajectory prediction method in the related art only predicts based on experience setting rules, and the accuracy of the obtained trajectory result is poor.

[0006] To achieve the above purpose, the following technical solutions are adopted in the present application:

[0007] The first aspect embodiment of the present application provides a pedestrian trajectory prediction method, comprising the following steps:

[0008] Obtain the current driving state of the vehicle, determine the target prediction area according to the current driving state, and identify all pedestrians in the target prediction area as target pedestrians;

[0009] Perform rule and data driven pedestrian intention determination on the target pedestrians to obtain intention prediction results;

[0010] Perform rule and data driven end point prediction according to the intention prediction results to obtain a fusion end point value;

[0011] Obtain the historical trajectory information of the target pedestrians, and obtain the predicted trajectory of the target pedestrians according to the historical trajectory information and the fusion end point value.

[0012] According to the technical means, the embodiment of the present application can first determine the target prediction area, and then identify all pedestrians in the target prediction area as target pedestrians, pay more attention to the target pedestrians affecting the driving of the ego vehicle, ensure the rationality of resource utilization, and perform rule-based and data-driven pedestrian intention determination and rule-based and data-driven endpoint prediction, so as to predict the trajectory and improve the accuracy of trajectory result prediction.

[0013] Optionally, in an embodiment of the present application, the current driving state of the vehicle is obtained, the current prediction area is determined according to the current driving state, and all pedestrians in the current prediction area are identified as target pedestrians, including:

[0014] The current driving state of the vehicle is obtained, and a region state correspondence relationship between a pre-stored driving state and a prediction area is obtained;

[0015] The region state correspondence relationship is searched according to the current driving state, and a target prediction area corresponding to the current driving state is obtained;

[0016] All pedestrians in the target prediction area are identified as target pedestrians.

[0017] According to the technical means, the embodiment of the present application can select a corresponding prediction area according to the current driving state, and then only predict pedestrians in the target prediction area, reasonably utilize computing power, and improve prediction efficiency.

[0018] Optionally, in an embodiment of the present application, rule-based and data-driven pedestrian intention determination is performed on the target pedestrians to obtain an intention prediction result, including:

[0019] Pedestrian state information and current scene information of the target pedestrian are obtained;

[0020] According to the pedestrian state information, rule-based pedestrian intention determination is performed on the target pedestrian according to a preset determination rule to obtain a first intention estimation value;

[0021] A pre-trained first neural network model is obtained, the pedestrian state information and the current scene information are input into the first neural network model, and a second intention estimation value is obtained;

[0022] Current vehicle speed information is obtained;

[0023] A preset rule-based first weight table is searched to obtain a target first matrix weight corresponding to the current scene information and a target second matrix weight corresponding to the current vehicle speed information;

[0024] searching a preset data-driven based second weight table to obtain a target third matrix weight corresponding to the current scene information and a target fourth matrix weight corresponding to the current vehicle speed information;

[0025] obtain a preset rule intention weight and a data-driven intention weight;

[0026] weight and fuse the first intention estimation value and the second intention estimation value according to the target first matrix weight, the target second matrix weight, the target third matrix weight, the target fourth matrix weight, the rule intention weight, the data-driven intention weight, the current scene information and the current vehicle speed information to obtain an intention prediction value;

[0027] obtain a preset stationary intention threshold range, a no-intention threshold range and an intention threshold range;

[0028] if the intention prediction value is in the stationary intention threshold range, the intention prediction result is a stationary intention;

[0029] if the intention prediction value is in the no-intention threshold range, the intention prediction result is a no-crossing intention;

[0030] if the intention prediction value is in the intention threshold range, the intention prediction result is a crossing intention.

[0031] According to the above technical means, the embodiment of the application obtains the target first matrix weight, the target second matrix weight, the target third matrix weight and the target fourth matrix weight, and combines the current scene information and the current vehicle speed information to weight and fuse the first intention estimation value obtained by using the rule method and the second intention estimation value obtained by using the data-driven method to obtain the intention of the target pedestrian, thereby improving the accuracy of predicting the pedestrian trajectory.

[0032] Optionally, in an embodiment of the application, the pedestrian state information includes historical trajectory information, a current speed, a current position and body orientation information;

[0033] According to the pedestrian state information, the target pedestrian is determined for a rule-based pedestrian intention according to a preset determination rule to obtain a first intention estimation value, including:

[0034] According to the historical trajectory information, a moving distance in a preset time period before the current time is calculated;

[0035] if the moving distance is less than a preset first distance threshold and / or the current speed is less than a preset speed threshold, the rule-based pedestrian intention is determined as a stationary intention to obtain a first intention estimation value corresponding to the stationary intention;

[0036] determining a distance between the current position and a center line of a lane in which the vehicle is located if the moving distance is greater than or equal to the first distance threshold and the current speed is greater than or equal to the speed threshold;

[0037] when the distance between the current position and the center line of the lane in which the vehicle is located is greater than a preset second distance threshold and the body orientation information indicates that the body is facing away from the center line of the lane in which the vehicle is located, determining the rule-based pedestrian intention as no crossing intention, and obtaining a first intention estimation value corresponding to the no crossing intention;

[0038] when the distance between the current position and the center line of the lane in which the vehicle is located is less than or equal to the second distance threshold and / or the body orientation information indicates that the body is facing the center line of the lane in which the vehicle is located, determining the rule-based pedestrian intention as crossing intention, and obtaining a first intention estimation value corresponding to the crossing intention.

[0039] According to the above technical means, the embodiment of the present application sets specific intention determination rules. First, whether the moving distance and the current speed of the target user meet the conditions is determined. If the conditions are not met, whether the target pedestrian has crossing intention is determined by judging the distance between the target pedestrian and the center line of the lane in which the vehicle is located and the body orientation. The static intention, no crossing intention and crossing intention are comprehensively considered. Figure Three In this way, the trajectory prediction is effectively assisted, and the accuracy of the trajectory prediction is improved.

[0040] Optionally, in an embodiment of the present application, a rule-based and data-driven end point prediction is performed according to the intention prediction result, and a fused end point value is obtained, including:

[0041] performing rule-based end point prediction according to a preset end point prediction rule according to the intention prediction result, and obtaining a first trajectory end point;

[0042] obtaining a pre-trained second neural network model, inputting the pedestrian state information and the current scene information into the second neural network model, and obtaining a second trajectory end point;

[0043] determining a first weight value corresponding to the first trajectory end point and a second weight value corresponding to the second trajectory end point according to the Euclidean distance between the first trajectory end point and the second trajectory end point;

[0044] performing weighted fusion processing on the first trajectory end point and the second trajectory end point according to the first weight value and the second weight value, and obtaining a fused end point value;

[0045] wherein the sum of the first weight value and the second weight value is 1.

[0046] According to the technical means, the weight of the first trajectory endpoint and the weight of the second trajectory endpoint can be allocated according to the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint, so that the endpoint prediction after the weighted fusion processing is more accurate.

[0047] Optionally, in an embodiment of the present application, the pedestrian state information further includes acceleration information of the target pedestrian; and according to the intention prediction result, a rule-based endpoint prediction is performed according to a preset endpoint prediction rule to obtain a first trajectory endpoint, including:

[0048] If the intention prediction result is a stationary intention, the current position is taken as the first trajectory endpoint.

[0049] If the intention prediction result is a no-crossing intention, the body orientation information, the current speed and the acceleration information are input into a preset uniform speed model to obtain the first trajectory endpoint.

[0050] If the intention prediction result is a crossing intention, the body orientation information, the current speed and the acceleration information are input into a preset uniform speed model to obtain an initial trajectory endpoint.

[0051] A lane distance between the current position and a lane center line of the vehicle is calculated, a preset direction distance corresponding relationship is searched according to the lane distance to obtain an endpoint direction corresponding to the lane distance.

[0052] The first trajectory endpoint is obtained according to the endpoint direction and the initial trajectory endpoint.

[0053] According to the technical means, different endpoint prediction methods can be taken for target pedestrians with different intentions, the prediction accuracy of the trajectory endpoint is improved, and an accurate prediction trajectory can be generated.

[0054] Optionally, in an embodiment of the present application, the first weight value corresponding to the first trajectory endpoint and the second weight value corresponding to the second trajectory endpoint are determined according to the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint, including:

[0055] If the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint is less than a preset experience threshold, the first weight value and the second weight value are equal.

[0056] If the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint is greater than or equal to the experience threshold, a scene weight corresponding relationship between pre-stored scene information and weight values is searched to obtain the first weight value and the second weight value corresponding to the current scene information.

[0057] According to the technical means, the first terminal point and the second terminal point are selected with different weight values by determining the relationship between the Euclidean distance between the first terminal point and the second terminal point and the experience threshold, and the calculation accuracy of the fusion terminal value is improved.

[0058] The second aspect of the present application provides a pedestrian trajectory prediction device, comprising:

[0059] The target recognition module is configured to obtain a current driving state of the vehicle, determine a target prediction area according to the current driving state, and identify all pedestrians in the target prediction area as target pedestrians.

[0060] The intention determination module is configured to perform rule-based and data-driven pedestrian intention determination on the target pedestrians to obtain an intention prediction result.

[0061] The terminal prediction module is configured to perform rule-based and data-driven terminal prediction according to the intention prediction result to obtain a fusion terminal value.

[0062] The trajectory prediction module is configured to obtain historical trajectory information of the target pedestrians, and obtain a predicted trajectory of the target pedestrians according to the historical trajectory information and the fusion terminal value.

[0063] The third aspect of the present application provides a vehicle, which comprises a memory, a processor, and a pedestrian trajectory prediction program stored in the memory and executable on the processor. When the processor executes the pedestrian trajectory prediction program, the steps of the pedestrian trajectory prediction method described above are implemented.

[0064] The fourth aspect of the present application provides a computer readable storage medium, which stores a pedestrian trajectory prediction program. When the processor executes the pedestrian trajectory prediction program, the steps of the pedestrian trajectory prediction method described above are implemented.

[0065] The beneficial effects of the present application are:

[0066] (1) The embodiments of the present application can select the corresponding prediction area according to the different current driving states, and then only predict the pedestrians in the target prediction area, thereby reasonably utilizing the computing power and improving the prediction efficiency.

[0067] (2) The embodiments of the present application use rules and data-driven methods for intention determination and terminal prediction, thereby improving the accuracy of predicting pedestrian trajectories.

[0068] (3) The embodiments of the present application comprehensively consider the static intention, no-crossing intention and crossing intention Figure Three cases, thereby effectively assisting trajectory prediction and improving the accuracy of trajectory prediction.

[0069] Additional aspects and advantages of the present application will be made apparent from the following description that follows, and in part will be apparent to those of ordinary skill in the art upon examination of the following description or can be learned from practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0070] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, in which:

[0071] Figure 1 A flowchart of a pedestrian trajectory prediction method according to an embodiment of the present application;

[0072] Figure 2 A schematic diagram of a prediction area corresponding to a high-speed driving state according to an embodiment of the present application;

[0073] Figure 3 A schematic diagram of a prediction area corresponding to a low-speed driving state or a parking state according to an embodiment of the present application;

[0074] Figure 4 A schematic diagram of a pedestrian trajectory prediction device according to an embodiment of the present application;

[0075] Figure 5 An internal structure principle block diagram of a vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] Embodiments of the present application are described in detail below with reference to the attached drawings, in which like or similar elements are denoted by the same or similar reference numerals, and the embodiments described below are examples for explaining the present application and are not to be construed as limiting the present application.

[0077] A pedestrian trajectory prediction method and device according to an embodiment of the present application are described below with reference to the attached drawings. In view of the problem that the existing pedestrian trajectory prediction method mentioned in the above background technology only predicts based on a set rule and the accuracy of the trajectory result obtained is poor, the present application provides a pedestrian trajectory prediction method. In the method, the current driving state of a vehicle can be obtained, a target prediction area is determined according to the current driving state, and all pedestrians in the target prediction area are identified as target pedestrians. A rule-based and data-driven pedestrian intention determination is performed on the target pedestrians to obtain an intention prediction result. A rule-based and data-driven end point prediction is performed according to the intention prediction result to obtain a fused end point value. Historical trajectory information of the target pedestrians is obtained, and then a prediction trajectory of the target pedestrians is obtained according to the historical trajectory information and the fused end point value, thereby improving the accuracy of the trajectory result prediction. Thus, the problem that the pedestrian trajectory prediction method in the related art only predicts based on a set rule and the accuracy of the trajectory result obtained is poor is solved.

[0078] Specifically, Figure 1 A flowchart of a pedestrian trajectory prediction method provided by an embodiment of the present application is shown in FIG. 1.

[0079] As shown in FIG. 1, the pedestrian trajectory prediction method includes the following steps: Figure 1

[0080] In step S101, the current driving state of the vehicle is obtained, the target prediction area is determined according to the current driving state, and all pedestrians in the target prediction area are identified as target pedestrians.

[0081] It can be understood that the vehicle in the embodiment of the present application refers to the ego vehicle. In the automatic driving system, the CPU and memory resources are limited, and it is particularly important to maximize the algorithm benefits with limited resources. During the driving of the autonomous vehicle, the surrounding of the ego vehicle is detected, and the detection range is a large range. The pedestrian targets interacting with the ego vehicle in the detection range can be many, but not all targets can affect the driving of the ego vehicle. Therefore, it is not necessary to pay attention to all pedestrians around the ego vehicle, but only to the pedestrians that can affect the driving. The embodiment of the present application can first determine the target prediction area, and then identify all pedestrians in the target prediction area as target pedestrians. More attention is paid to the target pedestrians that affect the driving of the ego vehicle, which can ensure the rationality of resource utilization.

[0082] In one embodiment of the present application, the current driving state of the vehicle is obtained, and a region state correspondence relationship between the driving state and the prediction area is obtained in advance. The target prediction area corresponding to the current driving state is obtained by searching the region state correspondence relationship according to the current driving state. All pedestrians in the target prediction area are identified as target pedestrians.

[0083] The embodiment of the present application can screen the target prediction area according to the current driving state of the ego vehicle. The driving state can include a high-speed driving state, a low-speed driving state and a parking state, which can be determined according to the vehicle speed and scene information. That is, the screening area of the pedestrian can be divided according to the scene information and the vehicle speed, and different division methods determine the attention degree of the pedestrian trajectory prediction algorithm to the pedestrian. As shown in FIG. 2, if the current driving state is a high-speed driving state, the pedestrian target is relatively small due to the fast vehicle speed, and the pedestrian screening area can have the characteristics of "narrow and deep", that is, the faster the vehicle speed, the narrower and deeper the prediction area screened. As shown in FIG. 3, if the current driving state is a low-speed driving state, the pedestrian target is relatively large due to the slow vehicle speed, and the pedestrian screening area can have the characteristics of "wide and shallow", that is, the slower the vehicle speed, the wider and shallower the prediction area screened. As shown in FIG. 4, if the current driving state is a parking state, the pedestrian target is relatively large due to the slow vehicle speed, and the pedestrian screening area can have the characteristics of "wide and shallow", that is, the slower the vehicle speed, the wider and shallower the prediction area screened. Figure 2 Figure 3 ​​As shown, if the current driving state is a low-speed driving state or a parking state, there are more pedestrians due to the low speed, and the pedestrian screening area has the characteristics of "wide and shallow", that is, the lower the speed, the wider and shallower the predicted area. In specific implementation, the area state correspondence between the driving state and the predicted area is stored in advance, and the predicted area is an area based on the position of the ego vehicle. According to the embodiment of the application, the corresponding predicted area is selected according to the current driving state, and then only the pedestrians in the target predicted area are predicted, so that the computing power is reasonably used, and the prediction efficiency is improved.

[0084] In step S102, the target pedestrian is subjected to rule-based and data-driven pedestrian intention determination to obtain an intention prediction result.

[0085] The rule-based and data-driven pedestrian intention determination refers to fusing the intention prediction result by using the preset prediction rule and the data-driven manner. The data-driven refers to collecting massive data through the Internet or other related software as a means, organizing the data to form information, then integrating and refining the information, and forming an automatic decision model through training and fitting on the basis of data. In the related art, the historical information provided by the pedestrian is not fully utilized when predicting the pedestrian trajectory, for example, in the urban intersection scene, the target pedestrian without crossing intention is output with a predicted trajectory of crossing the street, and the pedestrian driving at low speed is predicted with a long trajectory. The accuracy of the prediction of the future walking intention of the target pedestrian determines the rationality of the predicted trajectory, and the accurate intention prediction can assist the vehicle to make correct judgments, such as speed keeping, acceleration and deceleration, and parking. The embodiment of the application combines the data-driven method to extract features from historical information, reasonably predicts the crossing state of the target pedestrian, and assists the pedestrian trajectory prediction algorithm based on the prediction rule to output a more accurate predicted trajectory.

[0086] In an embodiment of the present application, the pedestrian state information and the current scene information of the target pedestrian are acquired; according to the pedestrian state information, a rule-based pedestrian intention determination is performed on the target pedestrian according to a preset determination rule, to obtain a first intention estimation value; a first neural network model trained in advance is acquired, and the pedestrian state information and the current scene information are input into the first neural network model, to obtain a second intention estimation value; current vehicle speed information is acquired; a preset rule-based first weight table is searched to obtain a target first matrix weight corresponding to the current scene information and a target second matrix weight corresponding to the current vehicle speed information; a preset data-driven second weight table is searched to obtain a target third matrix weight corresponding to the current scene information and a target fourth matrix weight corresponding to the current vehicle speed information; a rule intention weight and a data-driven intention weight set in advance are acquired; the first intention estimation value and the second intention estimation value are weighted and fused according to the target first matrix weight, the target second matrix weight, the target third matrix weight, the target fourth matrix weight, the rule intention weight, the data-driven intention weight, the current scene information and the current vehicle speed information, to obtain an intention prediction value; a preset stationary intention threshold range, a no-intention threshold range and an intention threshold range are acquired; if the intention prediction value is within the stationary intention threshold range, the intention prediction result is a stationary intention; if the intention prediction value is within the no-intention threshold range, the intention prediction result is a no-crossing intention; if the intention prediction value is within the intention threshold range, the intention prediction result is a crossing intention.

[0087] The embodiments of the present application can obtain two target pedestrian intention estimation values through a rule and a data driven method, combine current scene information and current vehicle speed information of the ego vehicle to fuse the intention estimation values obtained by the two methods, and further obtain a final intention prediction result of the target pedestrian. The first neural network model can use a convolutional neural network (CNN), a recurrent neural network (RNN), or a graph convolutional network (GCN). During training, the training data includes images collected by a visual camera, vehicle speed information, pedestrian trajectory information, speed information, and face orientation information. The result output by the first neural network model is represented by a probability value of (0, 1). When the probability value is greater than a threshold value 0.5, it indicates that there is a crossing intention, and otherwise, it indicates that there is no crossing intention. As shown in Table 1, if the target pedestrian is determined to be in a stationary state according to a preset determination rule, the first intention estimation value obtained is represented by A, and A is set to 0. If the target pedestrian is determined to be in a state without a crossing intention according to the preset determination rule, the first intention estimation value obtained is represented by B, and B is set to 1. If the target pedestrian is determined to be in a state with a crossing intention according to the preset determination rule, the first intention estimation value obtained is represented by C, and C is set to 2. If the result output by the first neural network model determines that the target pedestrian is in a stationary state, the second intention estimation value obtained is represented by D, and D is set to 0. If the result output by the first neural network model determines that the target pedestrian is in a state without a crossing intention, the second intention estimation value obtained is represented by E, and D is set to 1. If the result output by the first neural network model determines that the target pedestrian is in a state with a crossing intention, the second intention estimation value obtained is represented by F, and F is set to 2.

[0088] Table 1: Intention prediction algorithm state matrix

[0089] Method Stationary Unintentional Intentional Context information Speed information Rules A B C S V Data-driven D E F S V

[0090] In the embodiments of the present application, the scene information can be divided into: intersection scene, on-off ramp scene, curve scene, tunnel scene, etc., and the value can be an integer between 1-10; the speed information is the current speed. The calculation formula of the intention prediction value is:

[0091] intent = (W1*I1+W2*S+W3*V) + (W4*I2+W5*S+W6*V);

[0092] Wherein, W1 is a rule intention weight, I1 is a first intention estimation value, taking values of A, B or C, W2 is a target first matrix weight, S is current scene information, W3 is a target second matrix weight, V is current vehicle speed information, W4 is a data-driven intention weight, I2 is a second intention estimation value, taking values of D, E or F, W5 is a target third matrix weight, and W6 is a target fourth matrix weight. Values of W1 to W6 are all empirical thresholds.

[0093] For example, if the current scene S is a red light intersection, taking a value of 6, and the vehicle speed V is 10 km / h, for a certain pedestrian, the rule algorithm judges that there is no crossing intention, that is, I1 takes a value of 1, and the data-driven algorithm judges that there is crossing intention, that is, I2 takes a value of 2. The calculation formula of the intention prediction value is:

[0094] intent = (W1*1+W2*6+W3*10) + (W4*2+W5*6+W6*10).

[0095] The embodiment of the application obtains the intention of the target pedestrian by pre-setting the target first matrix weight, the target second matrix weight, the target third matrix weight and the target fourth matrix weight, and combining the current scene information and the current vehicle speed information, and then performing weighted fusion processing on the first intention estimation value obtained by using the rule and the second intention estimation value obtained by using the data-driven manner, thereby improving the accuracy of predicting the pedestrian trajectory.

[0096] In an embodiment of the application, the pedestrian state information includes: historical trajectory information, current speed, current position and body orientation information. According to the pedestrian state information, the rule-based pedestrian intention judgment of the target pedestrian is performed according to a preset judgment rule, to obtain a first intention estimation value, including: calculating a moving distance in a preset time period before the current time according to the historical trajectory information; if the moving distance is less than a preset first distance threshold and / or the current speed is less than a preset speed threshold, the rule-based pedestrian intention is determined as a stationary intention, to obtain a first intention estimation value corresponding to the stationary intention; if the moving distance is greater than or equal to the first distance threshold and the current speed is greater than or equal to the speed threshold, the distance between the current position and the center line of the lane where the vehicle is located is determined; when the distance between the current position and the center line of the lane where the vehicle is located is greater than a preset second distance threshold, and the body orientation information is that the body faces away from the center line of the lane where the vehicle is located, the rule-based pedestrian intention is determined as no crossing intention, to obtain a first intention estimation value corresponding to the no crossing intention; when the distance between the current position and the center line of the lane where the vehicle is located is less than or equal to the second distance threshold and / or the body orientation information is that the body faces the center line of the lane where the vehicle is located, the rule-based pedestrian intention is determined as crossing intention, to obtain a first intention estimation value corresponding to the crossing intention.

[0097] The vehicle detected by the embodiment of the present application is in the driving process, and all pedestrians in the surrounding are detected by using a vehicle body sensor, which can be a visual sensor and a radar. The pedestrian state information of all pedestrians is detected by the embodiment of the present application, and the pedestrian state information is used for pedestrian trajectory prediction. Specifically, the vehicle body sensor can obtain a visual image, and the coordinate information of the pedestrian relative to the ego vehicle, the speed information of the pedestrian and the body orientation information of the pedestrian are obtained according to the visual image and the radar detection data, that is, the pedestrian state information of each pedestrian is obtained. When saving data, the visual images of the previous 15 frames at the current time can be stored and updated periodically.

[0098] In an embodiment, the sum L of the moving distances within a certain time T (such as 3S) is extracted from the historical trajectory information of the target pedestrian, L is compared with the first distance threshold, and at the same time, the size relationship between the current speed and the speed threshold is judged. When one of the conditions that L is less than the first distance threshold or the current speed is less than the speed threshold is met, it is determined that the intention is static. When both conditions are met, it is also determined that the intention is static. When neither condition is met, the judgment continues, that is, if the moving distance is greater than or equal to the first distance threshold and the current speed is greater than or equal to the speed threshold, the lane center line where the ego vehicle is located is selected as the reference, and the distance between the historical trajectory information of the pedestrian and the lane center line, the body orientation, the speed and the scene information are comprehensively used to determine whether the target pedestrian has the intention to cross. If the distance between the current position and the lane center line where the vehicle is located is greater than the preset second distance threshold, and the body orientation information is that the body of the pedestrian faces away from the lane center line where the vehicle is located, it is indicated that the target pedestrian is far away from the lane, and the walking direction is away from the lane. Therefore, it is determined that there is no intention to cross. On the contrary, if one of the conditions that the distance between the current position and the lane center line where the vehicle is located is greater than the preset second distance threshold and the body orientation information is that the body of the pedestrian faces away from the lane center line where the vehicle is located is not met or both conditions are not met, it is determined that the target pedestrian has the intention to cross.

[0099] The embodiment of the present application sets specific intention determination rules. First, whether the moving distance and the current speed of the target user meet the conditions is judged at the same time. If the conditions are not met, whether the target pedestrian has the intention to cross is judged by judging the distance between the target pedestrian and the lane center line where the vehicle is located and the body orientation. The static intention, no intention to cross and intention to cross are comprehensively considered. Figure Three In this way, the trajectory prediction is effectively assisted, and the accuracy of the trajectory prediction is improved.

[0100] In step S103, the end point prediction based on rules and data driving is performed according to the intention prediction result, and a fusion end point value is obtained.

[0101] In the embodiments of the present application, the trajectory endpoint is predicted according to the intention prediction result, and the predicted trajectory is generated. The rule and the data-driven method are used simultaneously when predicting the trajectory endpoint, and the accuracy of the prediction is improved.

[0102] In an embodiment of the present application, according to the intention prediction result, the rule-based endpoint prediction is performed according to a preset endpoint prediction rule, to obtain a first trajectory endpoint; a pre-trained second neural network model is obtained, the pedestrian state information and the current scene information are input into the second neural network model, to obtain a second trajectory endpoint; a first weight value corresponding to the first trajectory endpoint and a second weight value corresponding to the second trajectory endpoint are determined according to the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint; the first trajectory endpoint and the second trajectory endpoint are weighted and fused according to the first weight value and the second weight value, to obtain a fused endpoint value; and the sum of the first weight value and the second weight value is 1.

[0103] In the embodiments of the present application, the second neural network model can use a convolutional neural network (CNN), a recurrent neural network (RNN), or a graph convolutional network (GCN), etc. The pedestrian state information and the current scene information are input into the second neural network model, to infer the future trajectory endpoint of the target pedestrian. The trajectory endpoints obtained by the two methods are fused by using the complementary filtering method. The weight of the first trajectory endpoint and the weight of the second trajectory endpoint are allocated according to the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint, so that the endpoint prediction after the weighted fusion processing is more accurate.

[0104] In an embodiment of the present application, the pedestrian state information further includes: if the intention prediction result is a stationary intention, the current position is taken as the first trajectory endpoint; if the intention prediction result is a no-crossing intention, the body orientation information, the current speed and acceleration information are input into a preset uniform speed model, to obtain the first trajectory endpoint; if the intention prediction result is a crossing intention, the body orientation information, the current speed and acceleration information are input into a preset uniform speed model, to obtain an initial trajectory endpoint; the lane distance between the current position and the center line of the lane where the vehicle is located is calculated, the preset direction distance corresponding relationship is looked up according to the lane distance, to obtain an endpoint direction corresponding to the lane distance; and the first trajectory endpoint is obtained according to the endpoint direction and the initial trajectory endpoint.

[0105] The embodiments of the present application correspond to different intention prediction results with different processing. For a target pedestrian with a static intention, a particle model can be used, and the end point of the trajectory of the target pedestrian is set as the current position of the target pedestrian. For a target pedestrian without a crossing intention, a preset uniform speed model can be used, and the end point position of walking in a future certain time is calculated based on the body orientation information and the speed information. For a target pedestrian with a crossing intention, the end point direction is selected on the other side of the lane, and the end point position is obtained by the uniform speed model. The embodiments of the present application can pre-set a direction distance corresponding relationship between the lane distance and the end point direction, the lane distance refers to the distance between the pedestrian and the center line of the lane where the ego vehicle is located, and the corresponding relationship satisfies that the closer the pedestrian is to the center line of the lane, the more perpendicular the connecting line between the preset end point and the current position of the pedestrian is to the center line of the lane. The embodiments of the present application can adopt different end point prediction methods for target pedestrians with different intentions, thereby improving the prediction accuracy of the trajectory end point, and further generating an accurate predicted trajectory.

[0106] In an embodiment of the present application, the first weight value corresponding to the first trajectory end point and the second weight value corresponding to the second trajectory end point are determined according to the Euclidean distance between the first trajectory end point and the second trajectory end point, comprising: if the Euclidean distance between the first trajectory end point and the second trajectory end point is less than a preset empirical threshold, the first weight value and the second weight value are equal; if the Euclidean distance between the first trajectory end point and the second trajectory end point is greater than or equal to the empirical threshold, the scene weight corresponding relationship between the pre-stored scene information and the weight value is searched to obtain the first weight value and the second weight value corresponding to the current scene information.

[0107] In the embodiments of the present application, the first trajectory end point is taken as (x1, y1), the second trajectory end point is taken as (x2, y2), and the fusion end point value is (X, Y). When the Euclidean distance between the first trajectory end point and the second trajectory end point is less than the empirical threshold t1, the same weight value is taken for the two results respectively for weighted fusion, for example, (X, Y) = (x1*0.5+x2*0.5, y1*0.5+y2*0.5); wherein each coordinate refers to the coordinate in the ego vehicle coordinate system. When the Euclidean distance between the first trajectory end point and the second trajectory end point is greater than or equal to the empirical threshold t1, the weight value of the rule and data driven can be selected in combination with different scenes, that is, the scene weight corresponding relationship is pre-stored, and the scene weight corresponding relationship includes each scene information and the corresponding first weight value and second weight value.

[0108] In the embodiments of the present application, the relationship between the Euclidean distance between the first trajectory end point and the second trajectory end point and the empirical threshold is determined, different weight values are selected for the first trajectory end point and the second trajectory end point, and the calculation accuracy of the fusion end point value is improved.

[0109] In step S104, the historical trajectory information of the target pedestrian is obtained, and the predicted trajectory of the target pedestrian is obtained based on the historical trajectory information and the fused endpoint value.

[0110] In order to better fit the predicted trajectory of the target pedestrian, this embodiment of the application combines the historical trajectory information of the target pedestrian and the fused endpoint value, and adopts a cubic curve fitting method to fit the predicted trajectory of the target pedestrian, thereby improving the accuracy of the predicted trajectory and thus improving the reliability and stability of the autonomous driving system.

[0111] In summary, the pedestrian trajectory prediction method proposed in this application improves the accuracy of trajectory prediction by selecting target pedestrians through screening target prediction areas, determining pedestrian intentions based on preset judgment rules, predicting pedestrian intentions using a first neural network model, predicting fusion endpoint values ​​based on intention calculation results, and obtaining the predicted trajectory of the target pedestrian based on historical trajectory information and fusion endpoint values. This solves the problem in related technologies where pedestrian trajectory prediction methods rely solely on predefined rules, resulting in poor trajectory accuracy.

[0112] Next, the pedestrian trajectory prediction device according to the embodiments of this application is described with reference to the accompanying drawings.

[0113] like Figure 4 As shown, the pedestrian trajectory prediction device 10 includes: a target recognition module 100, an intent determination module 200, an endpoint prediction module 300, and a trajectory prediction module 400.

[0114] Specifically, the target recognition module 100 is used to obtain the current driving status of the vehicle, determine the target prediction area based on the current driving status, and identify all pedestrians in the target prediction area as target pedestrians.

[0115] The intent determination module 200 is used to determine the pedestrian intent of the target pedestrian based on rules and data-driven methods, and obtain the intent prediction result.

[0116] The endpoint prediction module 300 is used to determine the pedestrian's intent based on rules and data-driven methods, and obtain the intent prediction result.

[0117] The trajectory prediction module 400 is used to obtain the historical trajectory information of the target pedestrian and obtain the predicted trajectory of the target pedestrian based on the historical trajectory information and the fused endpoint value.

[0118] It should be noted that the foregoing explanation of the pedestrian trajectory prediction method embodiment also applies to the pedestrian trajectory prediction device of this embodiment, and will not be repeated here.

[0119] According to the pedestrian trajectory prediction device provided in the embodiments of the present application, the current driving state of the vehicle can be obtained, the target prediction area is determined according to the current driving state, and all pedestrians in the target prediction area are identified as target pedestrians; the rule-based and data-driven pedestrian intention determination is performed on the target pedestrians to obtain an intention prediction result; the rule-based and data-driven endpoint prediction is performed according to the intention prediction result to obtain a fusion endpoint value; the historical trajectory information of the target pedestrian is obtained, and then the prediction trajectory of the target pedestrian is obtained according to the historical trajectory information and the fusion endpoint value, thereby improving the accuracy of the trajectory result prediction. Thus, the problem that the pedestrian trajectory prediction method in the related art only performs prediction based on the set rules and the obtained trajectory result has poor accuracy is solved.

[0120] Figure 5 A structural schematic diagram of a vehicle is provided for the embodiments of the present application. The vehicle can include:

[0121] The memory 501, the processor 502 and the computer program stored in the memory 501 and executable on the processor 502.

[0122] The processor 502 implements the pedestrian trajectory prediction method provided in the above embodiments when executing the program.

[0123] Further, the vehicle further includes:

[0124] The communication interface 503 is used for communication between the memory 501 and the processor 502.

[0125] The memory 501 is used to store the computer program executable on the processor 502.

[0126] The memory 501 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0127] If the memory 501, the processor 502 and the communication interface 503 are independently implemented, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and complete the communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 5 In the figure, only one line is used to represent, but it does not mean that there is only one bus or one type of bus.

[0128] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can complete mutual communication through an internal interface.

[0129] The processor 502 can be a central processing unit (CPU) or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to perform the embodiments of the present application.

[0130] The embodiments further provide a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the pedestrian trajectory prediction method as above.

[0131] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0132] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0133] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing the specified logic functions (or steps) and / or can be implemented by one or more hardware or software components, by other physical components, by combinations thereof, and / or by means to be understood by those skilled in the art. The preferred embodiments of the present application include additional implementations, in which the order of steps can differ from that shown or discussed, including simultaneously or in reverse order, depending upon the functionality involved, as will be understood by those skilled in the art.

[0134] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a list of instructions to implement a logical function, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electronic connection having one or N wires (electronic devices), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program can be printed, because the program can be electronically captured, via the optical scan of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in the computer memory.

[0135] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented in hardware and in another embodiment, any of the following technologies known in the art or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0136] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer-readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0137] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0138] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for predicting pedestrian trajectories, characterized in that, Includes the following steps: The current driving status of the vehicle is obtained, the target prediction area is determined based on the current driving status, and all pedestrians in the target prediction area are identified as target pedestrians; The pedestrian intent is determined based on rules and data-driven methods to obtain intent prediction results; Based on the intent prediction results, rule-based and data-driven endpoint prediction is performed to obtain the fused endpoint value; Obtain the historical trajectory information of the target pedestrian, and obtain the predicted trajectory of the target pedestrian based on the historical trajectory information and the fused endpoint value; The target pedestrian is subjected to rule-based and data-driven pedestrian intent determination to obtain intent prediction results, including: Obtain the pedestrian status information and current scene information of the target pedestrian; Based on the pedestrian status information, the pedestrian intention of the target pedestrian is determined according to the preset determination rules to obtain the first intention estimation value; A pre-trained first neural network model is obtained, and the pedestrian state information and the current scene information are input into the first neural network model to obtain the second intention inference value; Get current vehicle speed information; Search the preset rule-based first weight table to obtain the target first matrix weights corresponding to the current scene information and the target second matrix weights corresponding to the current vehicle speed information; Search the preset data-driven second weight table to obtain the target third matrix weights corresponding to the current scene information and the target fourth matrix weights corresponding to the current vehicle speed information; Obtain pre-set rule intent weights and data-driven intent weights; Based on the target first matrix weight, target second matrix weight, target third matrix weight, target fourth matrix weight, rule intent weight, data-driven intent weight, current scene information, and current vehicle speed information, the first intent estimation value and the second intent estimation value are weighted and fused to obtain the intent prediction value; Obtain the preset threshold ranges for static intent, no intent, and intent; If the predicted intent value is within the static intent threshold range, then the intent prediction result is a static intent; If the intent prediction value is within the no-intent threshold range, the intent prediction result is no traversal intent; If the predicted intent value is within the intent threshold range, the intent prediction result is that there is a traversal intent.

2. The method as described in claim 1, characterized in that, The process of obtaining the vehicle's current driving status, determining the current prediction area based on the current driving status, and identifying all pedestrians within the current prediction area as target pedestrians includes: Obtain the vehicle's current driving status and retrieve the pre-stored correspondence between the driving status and the predicted area. Based on the current driving state, the corresponding relationship of the regional state is found to obtain the target prediction region corresponding to the current driving state; All pedestrians within the target prediction area are identified as target pedestrians.

3. The method as described in claim 1, characterized in that, The pedestrian status information includes: historical trajectory information, current speed, current location, and body orientation information; Based on the pedestrian status information, the target pedestrian's intention is determined according to a preset judgment rule to obtain a first intention estimation value, including: Calculate the distance traveled within a preset time period prior to the current moment based on the historical trajectory information; If the moving distance is less than a preset first distance threshold and / or the current speed is less than a preset speed threshold, then the pedestrian's intention based on the rules will be determined as a stationary intention, and the first intention calculation value corresponding to the stationary intention will be obtained. If the moving distance is greater than or equal to the first distance threshold and the current speed is greater than or equal to the speed threshold, then the distance between the current position and the center line of the lane where the vehicle is located is determined. When the distance between the current position and the center line of the lane where the vehicle is located is greater than the preset second distance threshold, and the human body orientation information is that the human body is facing away from the center line of the lane where the vehicle is located, the rule-based pedestrian intention is determined to be no intention to cross the road, and the first intention estimation value corresponding to no intention to cross the road is obtained. When the distance between the current position and the center line of the lane where the vehicle is located is less than or equal to the second distance threshold and / or the human body orientation information is that the human body is facing the center line of the lane where the vehicle is located, the rule-based pedestrian intention is determined to be a crossing intention, and the first intention calculation value corresponding to the crossing intention is obtained.

4. The method as described in claim 3, characterized in that, Based on the intent prediction results, rule-based and data-driven endpoint prediction is performed to obtain a fused endpoint value, including: Based on the intention prediction result, rule-based endpoint prediction is performed according to the preset endpoint prediction rules to obtain the endpoint of the first trajectory. A pre-trained second neural network model is obtained, and the pedestrian state information and the current scene information are input into the second neural network model to obtain the second trajectory endpoint; The first weight value corresponding to the first trajectory endpoint and the second weight value corresponding to the second trajectory endpoint are determined based on the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint. The first trajectory endpoint and the second trajectory endpoint are weighted and fused according to the first weight value and the second weight value to obtain the fused endpoint value; The sum of the first weight value and the second weight value is 1.

5. The method as described in claim 4, characterized in that, The pedestrian state information further includes: the acceleration information of the target pedestrian; based on the intent prediction result, rule-based endpoint prediction is performed according to a preset endpoint prediction rule to obtain the first trajectory endpoint, including: If the intention prediction result is a stationary intention, then the current position is taken as the endpoint of the first trajectory; If the intention prediction result is no intention to cross, then the human body orientation information, the current speed and the acceleration information are input into a preset uniform speed model to obtain the first trajectory endpoint; If the intention prediction result is that there is a cross-traffic intention, then the human body orientation information, the current speed and the acceleration information are input into a preset uniform speed model to obtain the initial trajectory endpoint; Calculate the distance between the current position and the center line of the lane where the vehicle is located, and find the preset directional distance correspondence based on the distance between the vehicle and the lane to obtain the destination direction corresponding to the distance between the vehicle and the lane. The first trajectory endpoint is obtained based on the endpoint direction and the initial trajectory endpoint.

6. The method as described in claim 4, characterized in that, Determining a first weight value corresponding to the first trajectory endpoint and a second weight value corresponding to the second trajectory endpoint based on the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint includes: If the Euclidean distance between the endpoint of the first trajectory and the endpoint of the second trajectory is less than a preset empirical threshold, then the first weight value and the second weight value are equal. If the Euclidean distance between the first trajectory endpoint and the second trajectory endpoint is greater than or equal to the empirical threshold, then the scene weight correspondence between the pre-stored scene information and the weight value is searched to obtain the first weight value and the second weight value corresponding to the current scene information.

7. A pedestrian trajectory prediction device, characterized in that, include: The target recognition module is used to obtain the current driving status of the vehicle, determine the target prediction area based on the current driving status, and identify all pedestrians in the target prediction area as target pedestrians; The intent determination module is used to determine the pedestrian's intent based on rules and data-driven methods, and to obtain the intent prediction result. The endpoint prediction module is used to perform rule-based and data-driven endpoint prediction based on the intent prediction results to obtain a fused endpoint value. The trajectory prediction module is used to acquire the historical trajectory information of the target pedestrian and obtain the predicted trajectory of the target pedestrian based on the historical trajectory information and the fused endpoint value. The target pedestrian is subjected to rule-based and data-driven pedestrian intent determination to obtain intent prediction results, including: Obtain the pedestrian status information and current scene information of the target pedestrian; Based on the pedestrian status information, the pedestrian intention of the target pedestrian is determined according to the preset determination rules to obtain the first intention estimation value; A pre-trained first neural network model is obtained, and the pedestrian state information and the current scene information are input into the first neural network model to obtain the second intention inference value; Get current vehicle speed information; Search the preset rule-based first weight table to obtain the target first matrix weights corresponding to the current scene information and the target second matrix weights corresponding to the current vehicle speed information; Search the preset data-driven second weight table to obtain the target third matrix weights corresponding to the current scene information and the target fourth matrix weights corresponding to the current vehicle speed information; Obtain pre-set rule intent weights and data-driven intent weights; Based on the target first matrix weight, target second matrix weight, target third matrix weight, target fourth matrix weight, rule intent weight, data-driven intent weight, current scene information, and current vehicle speed information, the first intent estimation value and the second intent estimation value are weighted and fused to obtain the intent prediction value; Obtain the preset threshold ranges for static intent, no intent, and intent; If the predicted intent value is within the static intent threshold range, then the intent prediction result is a static intent; If the intent prediction value is within the no-intent threshold range, the intent prediction result is no traversal intent; If the predicted intent value is within the intent threshold range, the intent prediction result is that there is a traversal intent.

8. A vehicle, characterized in that, The vehicle includes a memory, a processor, and a pedestrian trajectory prediction program stored in the memory and executable on the processor. When the processor executes the pedestrian trajectory prediction program, it implements the steps of the pedestrian trajectory prediction method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a pedestrian trajectory prediction program, which, when executed by a processor, implements the steps of the pedestrian trajectory prediction method as described in any one of claims 1-6.

Images