Pedestrian trajectory prediction method and device, and vehicle

Abstract

The invention provides a pedestrian trajectory prediction method and device, and a vehicle, and the method comprises the steps: carrying out the clustering analysis based on the obtained walking behavior data of a plurality of pedestrians in a vehicle driving process, and obtaining a plurality of pedestrian groups; based on the walking behavior data of the pedestrians in each pedestrian group, determining group walking behavior data of each pedestrian group; determining at least one piece of passable boundary information of each pedestrian group based on the obtained map information; the group walking behavior data of each pedestrian group and at least one piece of passable boundary information serve as input of a trained generative adversarial network, and a prediction result is obtained; the prediction result comprises at least one group of prediction track supporting points of each pedestrian group; and performing trajectory restoration processing based on the at least one group of prediction trajectory supporting points corresponding to each pedestrian group to obtain at least one prediction trajectory of each pedestrian group.

Description

Technical Field

[0001] This application relates to trajectory prediction technology, and more particularly to a pedestrian trajectory prediction method and device, vehicle, and storage medium. Background Technology

[0002] Currently, driver assistance systems (ADAS) are widely used on highways, significantly improving the convenience and comfort of long-distance driving. However, in urban road environments, the complex and dynamic behavior of pedestrians and other road users leads to a high degree of uncertainty in traffic scenarios, making it difficult for ADAS to operate safely and reliably, thus hindering its widespread adoption. Against this backdrop, accurately predicting pedestrian trajectories has become a key technical challenge and research focus in the current field of perception and decision-making for driver assistance systems. Summary of the Invention

[0003] This application provides a pedestrian trajectory prediction method, device, vehicle, and storage medium, which can accurately predict pedestrian trajectories on urban roads, thereby ensuring safe driving.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a method for predicting pedestrian trajectories, characterized in that it is applied to vehicles, and the method includes: During the vehicle's journey, cluster analysis is performed based on the walking behavior data of multiple pedestrians to identify multiple pedestrian groups. Based on the walking behavior data of pedestrians in each pedestrian group, determine the group walking behavior data of each pedestrian group; Based on the acquired map information, at least one passable boundary is determined for each pedestrian group; The group walking behavior data of each pedestrian group and at least one passable boundary information are used as input to a trained generative adversarial network to obtain prediction results; the prediction results include at least one set of predicted trajectory support points for each pedestrian group. Trajectory restoration is performed based on at least one set of predicted trajectory support points corresponding to each pedestrian group, resulting in at least one predicted trajectory for each pedestrian group.

[0005] This application provides a pedestrian trajectory prediction device, characterized in that it is applied to a vehicle, and the device includes: The processing unit is used to perform cluster analysis on the walking behavior data of multiple pedestrians acquired during the vehicle's operation to obtain multiple pedestrian groups. The processing unit is also used to determine the group walking behavior data of each pedestrian group based on the walking behavior data of the pedestrians in each pedestrian group; The processing unit is also used to determine at least one passable boundary information for each pedestrian group based on the acquired map information; The processing unit is further configured to take the group walking behavior data of each pedestrian group and at least one passable boundary information as input to the trained generative adversarial network to obtain a prediction result; the prediction result includes at least one set of predicted trajectory support points for each pedestrian group. The processing unit is also used to perform trajectory restoration processing based on at least one set of predicted trajectory support points corresponding to each pedestrian group, so as to obtain at least one predicted trajectory for each pedestrian group.

[0006] This application provides a vehicle, the vehicle comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the pedestrian trajectory prediction method provided in the embodiments of this application.

[0007] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the pedestrian trajectory prediction method provided in this application when executed by a processor.

[0008] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the pedestrian trajectory prediction method provided in this application.

[0009] The embodiments of this application have the following beneficial effects: By performing cluster analysis on multiple pedestrians around a vehicle, multiple pedestrian groups are obtained; the group walking behavior data and at least one corresponding passable boundary information of each pedestrian group are determined; then, a trained generative adversarial network is used to process the group walking behavior data and at least one corresponding passable boundary information of each pedestrian group to obtain at least one set of predicted trajectory support points for each pedestrian group; finally, trajectory restoration processing is performed on the at least one set of predicted trajectory support points corresponding to each pedestrian group to obtain at least one predicted trajectory for each pedestrian group. Thus, group clustering can effectively reduce the influence of the disordered characteristics of individuals on the final prediction result, while encompassing the interaction between pedestrians as the internal behavior of the group. Combined with the passable boundary information of the group for trajectory prediction, this can effectively improve the accuracy and stability of the final prediction result. Attached Figure Description

[0010] Figure 1 This is a first flowchart illustrating the pedestrian trajectory prediction method provided in this application embodiment; Figure 2This is a schematic diagram of a passable boundary for a pedestrian group, as exemplified by an embodiment of this application. Figure 3 This is a second flowchart illustrating the pedestrian trajectory prediction method provided in the embodiments of this application; Figure 4 This is a schematic diagram of the third process of the pedestrian trajectory prediction method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the fourth process of the pedestrian trajectory prediction method provided in the embodiments of this application; Figure 6 This is a schematic diagram of the predicted trajectory support points provided in the embodiments of this application; Figure 7 This is a schematic diagram of the fifth process of the pedestrian trajectory prediction method provided in the embodiments of this application; Figure 8 This is a schematic diagram of the composition structure of the pedestrian trajectory prediction device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the vehicle's structural composition provided in the embodiments of this application.

[0011] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] This application provides a method for predicting pedestrian trajectories. Figure 1 This is a first flowchart illustrating the pedestrian trajectory prediction method provided in this application embodiment, which is applied to vehicles, such as... Figure 1 As shown, the pedestrian trajectory prediction method includes the following steps: S101: During the vehicle's operation, cluster analysis is performed based on the walking behavior data of multiple pedestrians to obtain multiple pedestrian groups.

[0014] In this embodiment of the application, during the vehicle's operation, the vehicle uses its own sensors to perceive its surrounding environment in order to obtain walking behavior data of multiple pedestrians.

[0015] Furthermore, based on the walking behavior data of multiple pedestrians, cluster analysis is performed on multiple pedestrians to treat pedestrians with the same or similar walking behavior data as a group, thereby obtaining multiple pedestrian groups.

[0016] Here, walking behavior data refers to the information generated during the process of pedestrians walking.

[0017] For example, walking behavior data may include the pedestrian's current location, direction of travel, and speed.

[0018] Based on this, cluster analysis is performed on multiple pedestrians according to their current location, direction of travel, and speed. Pedestrians whose locations are close, whose directions of travel are the same or similar, and whose speeds are the same or similar are considered as a group, thus obtaining multiple pedestrian groups.

[0019] In this embodiment, pedestrian clustering can effectively reduce the impact of individual disordered features on the final trajectory prediction result, while encompassing the interaction between pedestrians as the internal behavior of the group, which can effectively improve the accuracy and stability of the final trajectory prediction result.

[0020] S102: Based on the walking behavior data of pedestrians in each pedestrian group, determine the group walking behavior data of each pedestrian group.

[0021] In this embodiment of the application, the walking behavior data of pedestrians in each pedestrian group are the same or similar, and their walking-related information is... The information can include pedestrians' current location, direction of travel, and speed. Therefore, the group location information of each pedestrian group can be determined based on the location information of all pedestrians within the group; the group's direction of travel can be determined based on the group's location information at two consecutive moments; the group's speed information can be determined based on the group's location information at two consecutive moments and the time difference between them; and the group's acceleration information can be determined based on the group's location information at the current moment and the time difference between the two moments prior to the current moment. Alternatively, the group's direction of travel can be determined based on the direction of travel of all pedestrians within the group; the group's speed information can be determined based on the speed information of all pedestrians within the group; and the group's acceleration information can be determined based on the group's speed information at two consecutive moments and the time difference between them. Based on this, the group walking behavior data of a pedestrian group includes group location information, group direction of travel information, group speed information, and group acceleration information.

[0022] S103: Based on the acquired map information, determine at least one passable boundary for each pedestrian group.

[0023] For example, such as Figure 2As shown, multiple pedestrian groups identified at an intersection include G1, G2, G3, G4, and G5. The passable boundary information for pedestrian groups G1 and G3 includes bound1, bound2, {bound3,bound4}, and {bound5,bound6}; for pedestrian group G2, it includes bound7, bound8, {bound3,bound4}, and {bound9,bound10}; for pedestrian group G4, it includes bound11, bound12, {bound5,bound6}, and {bound13,bound14}; and for pedestrian group G5, it includes bound15, bound16, {bound9,bound10}, and {bound13,bound14}. Here, {bound3,bound4} represents the area composed of bound3 and bound4. Other groups are similar and will not be elaborated further.

[0024] S104: Take the group walking behavior data of each pedestrian group and at least one passable boundary information as input to the trained generative adversarial network to obtain the prediction results; the prediction results include at least one set of predicted trajectory support points for each pedestrian group.

[0025] In this embodiment of the application, the group walking behavior data of each pedestrian group and at least one passable boundary information are used as inputs to the generator in the trained generative adversarial network to obtain at least one set of predicted trajectory support point information corresponding to at least one passable boundary information.

[0026] S105: Perform trajectory restoration processing based on at least one set of predicted trajectory support points corresponding to each pedestrian group to obtain at least one predicted trajectory for each pedestrian group.

[0027] In this embodiment of the application, the spline interpolation method can be used to perform trajectory restoration processing based on at least one set of predicted trajectory support points corresponding to each pedestrian group, so as to obtain at least one predicted trajectory for each pedestrian group.

[0028] Spline interpolation involves dividing the entire interpolation interval into several sub-intervals, fitting the data with a low-order polynomial in each sub-interval, and ensuring continuous derivatives (such as position, velocity, and acceleration) at connection points (called "nodes"), thereby obtaining a smooth curve that passes through all given points.

[0029] In some embodiments of this application, the prediction result further includes probability values ​​corresponding to at least one set of predicted trajectory support points for each pedestrian group; the probability value refers to the probability that the trajectory formed by a set of predicted trajectory support points is the actual trajectory; The method further includes displaying at least one predicted trajectory and the corresponding probability value for each pedestrian group.

[0030] In this embodiment, the trained generative adversarial network (GAN) also includes a discriminator. Based on this, at least one set of predicted trajectory support point information corresponding to each pedestrian group is used as input to the discriminator in the trained GAN, resulting in at least one probability value corresponding to at least one set of predicted trajectory support point information. Finally, at least one predicted trajectory and corresponding probability value for each pedestrian group can be displayed through the vehicle's display unit, enhancing the driver's situational awareness of surrounding traffic participants and thus supporting safe driving decisions.

[0031] In this embodiment, multiple pedestrian groups are obtained by clustering analysis of multiple pedestrians around a vehicle; the group walking behavior data of each pedestrian group and at least one corresponding passable boundary information are determined; then, a trained generative adversarial network is used to process the group walking behavior data and the corresponding at least one passable boundary information of each pedestrian group to obtain at least one set of predicted trajectory support points and corresponding probability values ​​for each pedestrian group; then, trajectory restoration processing is performed on the at least one set of predicted trajectory support points corresponding to each pedestrian group to obtain at least one predicted trajectory for each pedestrian group; finally, at least one predicted trajectory and corresponding probability value for each pedestrian group are displayed. Thus, group clustering can effectively reduce the influence of the disordered characteristics of individuals on the final prediction result, while encompassing the interaction between pedestrians as the internal behavior of the group. Combined with the passable boundary information of the group for trajectory prediction, this effectively improves the accuracy and stability of the final prediction result.

[0032] In some embodiments of this application, before performing the pedestrian trajectory prediction method, the method further includes the following steps: S301: Acquire group walking behavior data of multiple pedestrian groups to be trained and at least one passable boundary information.

[0033] S302: Based on the group walking behavior data of multiple pedestrian groups to be trained and at least one passable boundary information, the initial generative adversarial network is iteratively trained. By optimizing the adversarial loss between the generator and the discriminator in the initial generative adversarial network, the convergence condition is met, and the trained generative adversarial network is obtained. The generator takes as input group walking behavior data and at least one passable boundary information for each pedestrian group to be trained, and outputs at least one set of predicted trajectory support points for each pedestrian group to be trained. The discriminator takes as input at least one set of predicted trajectory support points and the actual trajectory for each pedestrian group to be trained, and outputs the probability values ​​corresponding to at least one set of predicted trajectory support points for each pedestrian group to be trained. The loss function of the discriminator includes an evaluation index on whether the generated predicted trajectory support points are within the corresponding passable boundary, and the loss function of the generator includes an evaluation index on whether the generated predicted trajectory support points are within the corresponding passable boundary.

[0034] It's important to note that Generative Adversarial Networks (GANs) consist of a Generator Net (GN) and a Discriminator Net (DN). The generator can be understood as a "faker," taking as input the group walking behavior data of each pedestrian group to be trained and at least one passable boundary information, and outputting at least one set of predicted trajectory support points for each group. The generator's goal is to generate fake data from random noise, attempting to fool the discriminator. The discriminator can be understood as a "fake detector," taking as input the generator's output—at least one set of predicted trajectory support points for each pedestrian group—and outputting the probability values ​​corresponding to those support points. The discriminator's goal is to determine whether the input data is real or fabricated by the generator. Its ultimate goal is for the generator to learn to generate samples that are almost indistinguishable from the real data distribution.

[0035] It should be noted that incorporating an evaluation metric into the discriminator's loss function regarding whether the generated predicted trajectory support points are within the corresponding passable boundaries can enhance its ability to identify trajectories that violate traffic rules. Similarly, incorporating this metric into the generator's loss function can guide it to generate reasonable trajectories that conform to scene semantics and road topology constraints. Ultimately, this can effectively improve the accuracy of the generated predicted trajectories.

[0036] The loss function of the discriminator can be expressed by formula (1): (1) The loss function of the generator can be expressed by formula (2): (2) in,

[0037] This represents the generator network output data. Represents the output of the discriminator network. This represents the input data to the generator network. This represents the input data to the discriminator network. Representing the k The coordinates of the points and These represent the x and y coordinates of the point, respectively. Representing the i Information on the passable boundaries of each group.

[0038] The training process for generative adversarial networks is described below: for i = 1 : m do for j = 1 : n do From the prior distribution Take k samples

[0039] From data distribution Take k training samples

[0040] The discriminator network parameters are updated using the stochastic gradient ascent method.

[0041]

[0042] end for From the prior distribution Take k samples

[0043] The generator network parameters are updated using stochastic gradient descent.

[0044]

[0045] end for Note: m is the total number of training iterations, and n is the number of times the discriminator network is trained in each iteration.

[0046] In some embodiments of this application, before performing cluster analysis based on the acquired walking behavior data of multiple pedestrians to obtain multiple pedestrian groups, the method further includes: Obtain the location information of each pedestrian at the current moment and at the previous moment; Location information can be represented by latitude and longitude.

[0047] Based on each pedestrian's current location information and previous location information, the direction of travel information for each pedestrian is obtained; In this embodiment of the application, the direction indicated by connecting the current position information of each pedestrian with the position information of the previous position is the direction of travel information.

[0048] Based on each pedestrian's current location information, previous location information, and the time difference between adjacent times, the speed information of each pedestrian is obtained; In this embodiment of the application, the speed information of each pedestrian can be obtained by dividing the distance between the current location information and the previous location information of each pedestrian by the difference between adjacent time points.

[0049] Based on each pedestrian's current location, direction of travel, and speed, walking behavior data for each pedestrian is obtained.

[0050] In some embodiments of this application, determining the group walking behavior data of each pedestrian group based on the walking behavior data of pedestrians in each pedestrian group includes: Based on the current and previous position information of each pedestrian group, the center position of the smallest bounding rectangle including all pedestrians is determined, thus obtaining the group position information of each pedestrian group at the current and previous times. In this embodiment, based on the current position information of pedestrians in each pedestrian group, the center position of the smallest bounding rectangle including all pedestrians is determined, and this center position is used as the group position information of each pedestrian group at the current time. Similarly, based on the previous position information of pedestrians in each pedestrian group, the center position of the smallest bounding rectangle including all pedestrians is determined, and this center position is used as the group position information of each pedestrian group at the previous time.

[0051] Based on the group location information of each pedestrian group at the current moment and the group location information at the previous moment, the group movement direction information of each pedestrian group is obtained; In this embodiment of the application, the direction indicated by connecting the group location information of each pedestrian group at the current moment with the group location information at the previous moment is the group movement direction information.

[0052] Based on the group location information of each pedestrian group at the current moment and the group location information at the previous moment, as well as the time difference between adjacent moments, the group speed information of each pedestrian group is obtained; In this embodiment of the application, the group speed information of each pedestrian group can be obtained by dividing the moving distance between the group position information at the current time and the group position information at the previous time by the difference between adjacent time periods.

[0053] Based on the group position information of each pedestrian group at the current moment and the group position information of the two moments before the current moment, as well as the adjacent time difference, the group acceleration information of each pedestrian group is obtained; In this embodiment, the movement distance between the group position information of each pedestrian group at the time two moments before the current time and the group position information at the time two moments before the current time is divided by the difference between adjacent time points to obtain the group acceleration information of each pedestrian group at the previous time point. Similarly, the movement distance between the group position information of each pedestrian group at the time two moments before the current time and the group position information at the current time is divided by the difference between adjacent time points to obtain the group acceleration information of each pedestrian group at the current time. Finally, the difference between the group acceleration information of each pedestrian group at the previous time point and the group acceleration information at the current time is divided by the difference between adjacent time points to obtain the group acceleration information of each pedestrian group.

[0054] Based on the group location information, group direction of movement information, group speed information, and group acceleration information of each pedestrian group at the current moment, the group walking behavior data of each pedestrian group is obtained.

[0055] In some embodiments of this application, the walking behavior data includes the pedestrian's current location information, direction of travel information, and speed information; Based on this, cluster analysis is performed on the acquired walking behavior data of multiple pedestrians to obtain multiple pedestrian groups, including: Based on the location, direction of travel, and degree information of each pedestrian at the current moment, a hierarchical clustering algorithm is used to perform cluster analysis to obtain the multiple pedestrian groups.

[0056] In this embodiment of the application, the clustering analysis process using the hierarchical clustering algorithm is described as follows: Input: Pedestrian information input = {P1, P2, ... ,Pn} Output: Population partition output = {G1, G2, ... ,Gk} Clustering process: 1. Initially, each pedestrian is an independent group, at which point k=n 2. Calculate the distance matrix, travel direction deviation matrix, and velocity deviation matrix between groups. 3. Set the disThrethold, velocityThrethold, and headingThrethold thresholds. 4. Begin the group merging process, with an initial group size of k=n: (1) Find the two closest groups Gi and Gj from the distance matrix. <j (2) Determine whether Gi and Gj satisfy the disThrethold, velocityThrethold, and headingThrethold threshold conditions. If the conditions are met, merge the two populations Gi = Gi∪Gj. (3) Renumber the clustered groups. (4) Delete the j-th row and j-th column of the distance matrix, the direction deviation matrix, and the velocity deviation matrix. (5) Calculate the distance, direction of travel deviation, and speed deviation between the merged population Gi and other populations. (6) Update the current number of groups k = k - 1 (7) Exit the loop when the number of groups k no longer decreases. 5. Return to group segmentation The relevant conditions include the following: (3) (4) (5) Where Gi and Gj represent different groups, and m and n represent pedestrian information in each group.

[0057] For example, it can be set , , .

[0058] In some embodiments of this application, the group walking behavior data includes group location information; determining at least one passable boundary information for each pedestrian group based on the acquired map information includes the following steps: S401: Obtain environmental information around the vehicle.

[0059] In this embodiment, the vehicle is equipped with sensors for collecting information about the vehicle's surrounding environment, such as one or more of a camera, vision sensor, lidar, millimeter-wave radar, and ultrasonic radar, but not limited to these. During vehicle operation, the aforementioned sensors acquire real-time information about the vehicle's surrounding environment.

[0060] S402: Based on environmental information and the group location information of each pedestrian group, obtain obstacle information that affects the trajectory of each pedestrian group.

[0061] In this embodiment, the vehicle-mounted system analyzes and processes the environmental information around the vehicle collected by sensors installed on the vehicle, and identifies obstacles around the vehicle, such as roadblocks like traffic cones and signs distributed at intervals on the road. Furthermore, by combining the group location information of each pedestrian group, it obtains the obstacle information corresponding to that pedestrian group.

[0062] The obstacle information may include the obstacle's location information and a unique identifier for each obstacle.

[0063] S403: Based on map information and the group location information of each pedestrian group, obtain traffic light information that affects the trajectory of each pedestrian group.

[0064] In this embodiment of the application, if a group of pedestrians is in a corner of an intersection, traffic lights in two directions will affect the trajectory of the group of pedestrians. Therefore, it is necessary to accurately obtain the traffic light information that affects the trajectory of each group of pedestrians based on map information and the group location information of each group of pedestrians.

[0065] S404: Based on map information, obstacle information and traffic light information affecting the trajectory of each pedestrian group, determine at least one passable boundary for each pedestrian group.

[0066] In this embodiment of the application, based on map information and obstacle information affecting the trajectory of each pedestrian group, at least one candidate passable boundary information for each pedestrian group is determined; further, based on traffic light information affecting the trajectory of each pedestrian group, impassable boundary information is filtered out from the at least one candidate passable boundary information, thereby obtaining at least one final passable boundary information for each pedestrian group.

[0067] For example, if based on map information and obstacle information affecting the trajectory of each pedestrian group, the candidate passable boundary information for each pedestrian group includes candidate passable boundary information in the west, south, and north directions; further, if the traffic light information affecting the trajectory of the pedestrian group indicates that the traffic light in the west direction is red and the traffic light in the north direction is green, then the passable boundary information for the pedestrian group includes passable boundary information in the south and north directions.

[0068] In this embodiment of the application, by combining the passable boundary information of each pedestrian group, the accessibility of the final predicted trajectory can be effectively guaranteed, thereby improving the rationality of the prediction results.

[0069] In some embodiments of this application, the process of performing trajectory restoration based on at least one set of predicted trajectory support points corresponding to each pedestrian group to obtain at least one predicted trajectory for each pedestrian group includes the following steps: S501: Based on the fifth-order spline difference method, perform trajectory restoration processing on at least one set of predicted trajectory support points corresponding to each pedestrian group to obtain multiple intermediate predicted points between each set of predicted trajectory support points.

[0070] S502: Based on multiple intermediate prediction points between at least one set of prediction trajectory support points corresponding to each pedestrian group, obtain at least one prediction trajectory for each pedestrian group.

[0071] In this embodiment, the concept of predicted trajectory support points is introduced to simplify the infinite-dimensional continuous problem of predicting the trajectory into a finite-dimensional solvable problem. Here, the initial position time is used as the initial point; for example, four discrete points are taken at 2-second intervals. These, along with the start and end points of the predicted trajectory, form a total of six discrete points, which serve as the trajectory support points for the predicted trajectory. Figure 6 As shown.

[0072] In the preceding section, six trajectory support points were obtained through training a generative adversarial network (GAN). The trajectory reconstruction technique primarily relies on these support points, employing quintic spline interpolation to reconstruct the predicted trajectory. Figure 6 As can be seen, the support points for the predicted trajectory are as follows: (6) Based on trajectory support points, the predicted trajectory can be divided into the following five parts: (7) The fifth-degree polynomial on the i-th subinterval can be represented by the following equation: (8) In the formula: and These are the fourth and second derivatives of the curve at the i-th interpolation point, respectively; and The definition is as follows: , , Taking the first and third derivatives of equation (8) respectively, we get: (9) (10) Combining equations (9) and (10), we can obtain the following relationship:

[0073] in: , , , , , Considering the starting point constraint information of the predicted trajectory, the following four constraint equations can be obtained:

[0074] Finally, using the fifth-order spline interpolation method, without considering longitudinal acceleration attenuation, the complete and continuous predicted trajectory equation is obtained as follows:

[0075] Based on this, multiple intermediate prediction points are obtained between at least one set of prediction trajectory support points corresponding to each pedestrian group, thereby obtaining at least one prediction trajectory for each pedestrian group.

[0076] Based on the above embodiments, this application specifically provides a method for predicting pedestrian trajectories. Figure 7 This is a schematic diagram of the fifth process of the pedestrian trajectory prediction method provided in the embodiments of this application, as shown below. Figure 7 As shown, the pedestrian trajectory prediction method includes the following steps: S701: Acquire environmental information about the vehicle's surroundings while the vehicle is in motion.

[0077] S702: Construct triplet information for multiple pedestrians based on environmental information, including location information, direction of travel information, and speed information.

[0078] S703: Based on the location, direction of travel, and speed information of multiple pedestrians, a hierarchical clustering algorithm is used to perform cluster analysis to obtain multiple pedestrian groups.

[0079] The specific steps of cluster analysis have been described in the previous examples and will not be repeated here.

[0080] S704: Determine the group position information, group movement direction information, group speed information, and group acceleration information for each pedestrian group.

[0081] S705: Determine at least one passable boundary for each pedestrian group.

[0082] S706: Take the group position information, group movement direction information, group speed information, and group acceleration information of each pedestrian group, as well as the corresponding at least one passable boundary information, as input to the trained generative adversarial network to obtain at least one set of predicted trajectory support points and corresponding probability values ​​for each pedestrian group; the probability value refers to the probability that the trajectory formed by a set of predicted trajectory support points is the true trajectory.

[0083] The training of generative adversarial networks has been described in the previous embodiments and will not be repeated here.

[0084] S707: Based on the quintic spline difference method, perform trajectory restoration processing on at least one set of predicted trajectory support points corresponding to each pedestrian group to obtain at least one predicted trajectory for each pedestrian group.

[0085] S708: Displays at least one predicted trajectory and the corresponding probability value for each pedestrian group.

[0086] In this embodiment, multiple pedestrian groups are obtained by clustering analysis of multiple pedestrians around the vehicle; the group walking behavior data and at least one corresponding passable boundary information of each pedestrian group are determined; then, a trained generative adversarial network is used to process the group walking behavior data and at least one corresponding passable boundary information of each pedestrian group to obtain at least one set of predicted trajectory support points and corresponding probability values ​​for each pedestrian group; then, trajectory restoration processing is performed on the at least one set of predicted trajectory support points for each pedestrian group to obtain at least one predicted trajectory for each pedestrian group; finally, at least one predicted trajectory and corresponding probability value for each pedestrian group are displayed. Thus, group clustering can effectively reduce the influence of the disordered characteristics of individuals on the final prediction result, while encompassing the interaction between pedestrians as the internal behavior of the group. Combined with the passable boundary information of the group for trajectory prediction, this effectively improves the accuracy and stability of the final prediction result.

[0087] Based on the above embodiments, this application also provides a pedestrian trajectory prediction device. Figure 8 This is a schematic diagram of the composition structure of the pedestrian trajectory prediction device provided in the embodiments of this application, as shown below. Figure 8 As shown, the pedestrian trajectory prediction device 80 includes: The processing unit 801 is used to perform cluster analysis based on the walking behavior data of multiple pedestrians acquired during the vehicle's operation to obtain multiple pedestrian groups. The processing unit 801 is also used to determine the group walking behavior data of each pedestrian group based on the walking behavior data of the pedestrians in each pedestrian group; The processing unit 801 is also used to determine at least one passable boundary information for each pedestrian group based on the acquired map information. The processing unit 801 is further configured to take the group walking behavior data of each pedestrian group and at least one passable boundary information as input to the trained generative adversarial network to obtain a prediction result; the prediction result includes at least one set of predicted trajectory support points for each pedestrian group. The processing unit 801 is further configured to perform trajectory restoration processing based on at least one set of predicted trajectory support points corresponding to each pedestrian group, so as to obtain at least one predicted trajectory for each pedestrian group.

[0088] In this embodiment, multiple pedestrian groups are obtained by clustering analysis of multiple pedestrians around the vehicle; the group walking behavior data and at least one corresponding passable boundary information of each pedestrian group are determined; then, a trained generative adversarial network is used to process the group walking behavior data and at least one corresponding passable boundary information of each pedestrian group to obtain at least one set of predicted trajectory support points for each pedestrian group; finally, trajectory restoration processing is performed on the at least one set of predicted trajectory support points for each pedestrian group to obtain at least one predicted trajectory for each pedestrian group. Thus, group clustering can effectively reduce the influence of the disordered characteristics of individuals on the final prediction result, while encompassing the interaction between pedestrians as the internal behavior of the group. Combined with the passable boundary information of the group for trajectory prediction, this effectively improves the accuracy and stability of the final prediction result.

[0089] In some embodiments of this application, the prediction result further includes the probability value corresponding to at least one set of predicted trajectory support points for each pedestrian group; the probability value refers to the probability that the trajectory formed by a set of predicted trajectory support points is the actual trajectory; it also includes a display unit 802 for displaying at least one predicted trajectory and the corresponding probability value for each pedestrian group.

[0090] In some embodiments of this application, the processing unit 801 is further configured to acquire group walking behavior data of multiple pedestrian groups to be trained and at least one passable boundary information; based on the group walking behavior data of the multiple pedestrian groups to be trained and at least one passable boundary information, iteratively train an initial generative adversarial network, optimizing the adversarial loss between the generator and the discriminator in the initial generative adversarial network until the convergence condition is met, thereby obtaining the trained generative adversarial network; wherein, the input of the generator is the group walking behavior data of each pedestrian group to be trained and at least one passable boundary information, and the output is at least one set of predicted trajectory support points for each pedestrian group to be trained; the input of the discriminator is at least one set of predicted trajectory support points and the actual trajectory for each pedestrian group to be trained, and the output is the probability value corresponding to at least one set of predicted trajectory support points for each pedestrian group to be trained; the loss function of the discriminator includes an evaluation index on whether the generated predicted trajectory support points are within the corresponding passable boundary, and the loss function of the generator includes an evaluation index on whether the generated predicted trajectory support points are within the corresponding passable boundary.

[0091] In some embodiments of this application, the processing unit 801 is further configured to acquire the location information of each pedestrian at the current time and the location information at the previous time; obtain the walking direction information of each pedestrian based on the location information of each pedestrian at the current time and the location information at the previous time; obtain the speed information of each pedestrian based on the location information of each pedestrian at the current time and the location information at the previous time, as well as the time difference between adjacent times; and obtain the walking behavior data of each pedestrian based on the location information, walking direction information, and speed information of each pedestrian at the current time.

[0092] In some embodiments of this application, the processing unit 801 is further configured to: determine the center position of the smallest bounding rectangle including all pedestrians based on the position information of pedestrians in each pedestrian group at the current time and the position information at the previous time, thereby obtaining the group position information of each pedestrian group at the current time and the group position information at the previous time; obtain the group movement direction information of each pedestrian group based on the group position information of each pedestrian group at the current time and the group position information at the previous time; obtain the group speed information of each pedestrian group based on the group position information of each pedestrian group at the current time and the group position information at the previous time, as well as the adjacent time difference; obtain the group acceleration information of each pedestrian group based on the group position information of each pedestrian group at the current time and the group position information at the two times before the current time, as well as the adjacent time difference; and obtain the group walking behavior data of each pedestrian group based on the group position information, group movement direction information, group speed information, and group acceleration information of each pedestrian group at the current time.

[0093] In some embodiments of this application, the processing unit 801 is further configured to perform clustering analysis using a hierarchical clustering algorithm based on the location information, direction of travel information, and degree information of each of the plurality of pedestrians at the current time, to obtain the plurality of pedestrian groups.

[0094] In some embodiments of this application, the processing unit 801 is further configured to acquire environmental information around the vehicle; acquire obstacle information affecting the trajectory of each pedestrian group based on the environmental information and the group location information of each pedestrian group; acquire traffic light information affecting the trajectory of each pedestrian group based on the map information and the group location information of each pedestrian group; and determine at least one passable boundary information for each pedestrian group based on the map information, the obstacle information affecting the trajectory of each pedestrian group, and the traffic light information.

[0095] In some embodiments of this application, the processing unit 801 is further configured to perform trajectory restoration processing on at least one set of predicted trajectory support points corresponding to each pedestrian group based on the quintic spline difference method, to obtain multiple intermediate predicted points between each set of predicted trajectory support points; and to obtain at least one predicted trajectory for each pedestrian group based on the multiple intermediate predicted points between at least one set of predicted trajectory support points corresponding to each pedestrian group.

[0096] This application also provides another vehicle. Figure 9 This is a schematic diagram of the vehicle's structural composition provided in the embodiments of this application, such as... Figure 9 As shown, the vehicle 90 includes: a processor 901 and a memory 902 configured to store computer programs capable of running on a central controller; The processor 901 is configured to execute the method steps in the foregoing embodiments when running a computer program.

[0097] Of course, in practical applications, such as Figure 9 As shown, the various components in the vehicle 90 are coupled together via a bus system 903. It can be understood that the bus system 903 is used to enable communication between these components. In addition to a data bus, the bus system 903 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 9 The general designated all buses as Bus System 903.

[0098] In practical applications, the aforementioned processor can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic devices used to implement the functions of the aforementioned processor can also be other types, and this embodiment of the invention does not impose specific limitations.

[0099] The aforementioned memory can be volatile memory, such as random access memory (RAM). Access memory); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and providing instructions and data to the processor.

[0100] In an exemplary embodiment, the present invention also provides a computer-readable storage medium for storing a computer program.

[0101] Optionally, the computer-readable storage medium can be applied to any of the methods in the embodiments of the present invention, and the computer program causes the computer to execute the corresponding processes implemented by the processor in the various methods of the embodiments of the present invention. For the sake of brevity, these will not be described in detail here.

[0102] In the several embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0103] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, in the various embodiments of the present invention, all functional units can be integrated into one processing module, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0106] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0107] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0108] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for predicting pedestrian trajectories, characterized in that, Applied to vehicles, the method includes: During the vehicle's journey, cluster analysis is performed based on the walking behavior data of multiple pedestrians to identify multiple pedestrian groups. Based on the walking behavior data of pedestrians in each pedestrian group, determine the group walking behavior data of each pedestrian group; Based on the acquired map information, at least one passable boundary is determined for each pedestrian group; The group walking behavior data of each pedestrian group and at least one passable boundary information are used as input to a trained generative adversarial network to obtain prediction results; the prediction results include at least one set of predicted trajectory support points for each pedestrian group. Trajectory restoration is performed based on at least one set of predicted trajectory support points corresponding to each pedestrian group, resulting in at least one predicted trajectory for each pedestrian group.

2. The method according to claim 1, characterized in that, The prediction results also include probability values ​​corresponding to at least one set of predicted trajectory support points for each pedestrian group; the probability value refers to the probability that the trajectory formed by a set of predicted trajectory support points is the actual trajectory. The method further includes displaying at least one predicted trajectory and the corresponding probability value for each pedestrian group.

3. The method according to claim 2, characterized in that, Before executing the pedestrian trajectory prediction method, the method further includes: Acquire group walking behavior data of multiple pedestrian groups to be trained and at least one passable boundary information; Based on the group walking behavior data of the multiple pedestrian groups to be trained and at least one passable boundary information, the initial generative adversarial network is iteratively trained. By optimizing the adversarial loss between the generator and the discriminator in the initial generative adversarial network, the convergence condition is met, and the trained generative adversarial network is obtained. The generator takes as input group walking behavior data and at least one passable boundary information for each pedestrian group to be trained, and outputs at least one set of predicted trajectory support points for each pedestrian group to be trained. The discriminator takes as input at least one set of predicted trajectory support points and the actual trajectory for each pedestrian group to be trained, and outputs probability values ​​corresponding to at least one set of predicted trajectory support points for each pedestrian group to be trained. The loss function of the discriminator includes an evaluation index on whether the generated predicted trajectory support points are within the corresponding passable boundary, and the loss function of the generator includes an evaluation index on whether the generated predicted trajectory support points are within the corresponding passable boundary.

4. The method according to any one of claims 1 to 3, characterized in that, Before performing cluster analysis based on the acquired walking behavior data of multiple pedestrians to obtain multiple pedestrian groups, the method further includes: Obtain the location information of each pedestrian at the current moment and at the previous moment; Based on each pedestrian's current location information and previous location information, the direction of travel information for each pedestrian is obtained; Based on each pedestrian's current location information, previous location information, and the time difference between adjacent times, the speed information of each pedestrian is obtained; Based on each pedestrian's current location, direction of travel, and speed, walking behavior data for each pedestrian is obtained.

5. The method according to claim 4, characterized in that, The determination of group walking behavior data for each pedestrian group, based on the walking behavior data of individuals within each group, includes: Based on the current and previous position information of each pedestrian group, the center position of the smallest bounding rectangle including all pedestrians is determined, thus obtaining the group position information of each pedestrian group at the current and previous times. Based on the group location information of each pedestrian group at the current moment and the group location information at the previous moment, the group movement direction information of each pedestrian group is obtained; Based on the group location information of each pedestrian group at the current moment and the group location information at the previous moment, as well as the time difference between adjacent moments, the group speed information of each pedestrian group is obtained; Based on the group position information of each pedestrian group at the current moment and the group position information of the two moments before the current moment, as well as the adjacent time difference, the group acceleration information of each pedestrian group is obtained; Based on the group location information, group direction of movement information, group speed information, and group acceleration information of each pedestrian group at the current moment, the group walking behavior data of each pedestrian group is obtained.

6. The method according to any one of claims 1 to 3, characterized in that, The walking behavior data includes the pedestrian's current location, direction of travel, and speed. The clustering analysis based on the acquired walking behavior data of multiple pedestrians yields multiple pedestrian groups, including: Based on the location, direction of travel, and speed information of each pedestrian at the current moment, a hierarchical clustering algorithm is used to perform cluster analysis to obtain the multiple pedestrian groups.

7. The method according to any one of claims 1 to 3, characterized in that, The group walking behavior data includes group location information; the determination of at least one passable boundary information for each pedestrian group based on the acquired map information includes: Obtain environmental information surrounding the vehicle; Based on the environmental information and the group location information of each pedestrian group, obstacle information affecting the trajectory of each pedestrian group is obtained; Based on the map information and the group location information of each pedestrian group, obtain the traffic light information that affects the trajectory of each pedestrian group; Based on the map information, obstacle information and traffic light information that affect the trajectory of each pedestrian group, at least one passable boundary information for each pedestrian group is determined.

8. The method according to any one of claims 1 to 3, characterized in that, The trajectory restoration process, based on at least one set of predicted trajectory support points corresponding to each pedestrian group, yields at least one predicted trajectory for each pedestrian group, including: Based on the fifth-order spline interpolation method, trajectory restoration processing is performed on at least one set of predicted trajectory support points corresponding to each pedestrian group to obtain multiple intermediate predicted points between each set of predicted trajectory support points. Based on multiple intermediate prediction points between at least one set of prediction trajectory support points corresponding to each pedestrian group, at least one prediction trajectory for each pedestrian group is obtained.

9. A pedestrian trajectory prediction device, characterized in that, Applied to vehicles, the device includes: The processing unit is used to perform cluster analysis on the walking behavior data of multiple pedestrians acquired during the vehicle's operation to obtain multiple pedestrian groups. The processing unit is also used to determine the group walking behavior data of each pedestrian group based on the walking behavior data of the pedestrians in each pedestrian group; The processing unit is also used to determine at least one passable boundary information for each pedestrian group based on the acquired map information; The processing unit is further configured to take the group walking behavior data of each pedestrian group and at least one passable boundary information as input to the trained generative adversarial network to obtain a prediction result; the prediction result includes at least one set of predicted trajectory support points for each pedestrian group. The processing unit is also used to perform trajectory restoration processing based on at least one set of predicted trajectory support points corresponding to each pedestrian group, so as to obtain at least one predicted trajectory for each pedestrian group.

10. A vehicle, characterized in that, The vehicles include: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 8.

Images