Pedestrian trajectory prediction method, device and equipment and storage medium

Abstract

Embodiments of the present application provide a pedestrian trajectory prediction method, device and equipment and storage medium, belonging to the computer technical field. The pedestrian trajectory prediction method specifically comprises: obtaining pedestrian motion video data; determining the trajectory position data and key point sequence data of the pedestrian according to the pedestrian motion video data; inputting the trajectory position data and key point sequence data into a convolution model of a space-time attention mechanism to obtain prediction data of the pedestrian trajectory. The present application predicts the future trajectory of the pedestrian through various information such as the historical trajectory of the pedestrian, the interaction relationship between pedestrians and the motion state of the pedestrian itself, improves the referenceability of the prediction, and adopts a convolution model of a space-time attention mechanism with a double-flow structure, simultaneously predicts the pedestrian trajectory according to the trajectory position data and key point sequence data, and improves the precision and accuracy of the pedestrian trajectory prediction.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a pedestrian trajectory prediction method, a pedestrian trajectory prediction device, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In recent years, pedestrian trajectory prediction has received increasing attention in various fields, such as autonomous vehicles, intelligent security, and smart cities. In the field of autonomous vehicles, pedestrian trajectory prediction is one of the most important and challenging research directions. Enabling cars to make predictions in advance, much like humans, is crucial to reducing traffic accidents. However, in traffic scenarios, pedestrian trajectories are influenced not only by their own intentions but also by surrounding pedestrians, vehicles, and their own actions, making pedestrian trajectory prediction an extremely challenging task.

[0003] With the advent of the deep learning era, related methods have emerged in the field of pedestrian trajectory prediction. For example, pedestrian trajectory prediction is performed by using spatiotemporal attention through historical pedestrian trajectories and the interaction between pedestrians in captured images. However, this method does not consider the pedestrian's own motion behavior, and the accuracy of the predicted pedestrian trajectory is not high. Summary of the Invention

[0004] One objective of this invention is to provide a pedestrian trajectory prediction method to solve the technical problem that the prediction accuracy of pedestrian trajectories in the prior art is not high; another objective is to provide a pedestrian trajectory prediction device; a third objective is to provide an electronic device; and a fourth objective is to provide a computer storage medium.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for predicting pedestrian trajectories, the method comprising:

[0007] Acquire pedestrian movement video data;

[0008] Based on the pedestrian movement video data, determine the pedestrian's trajectory location data and key point sequence data; wherein, the key point sequence data consists of data on all key parts of the pedestrian's body.

[0009] The trajectory location data and the key point sequence data are input into a convolutional model with a spatiotemporal attention mechanism to obtain predicted data of pedestrian trajectory.

[0010] Based on the aforementioned technical features, by setting up a convolutional model with a spatiotemporal attention mechanism featuring a dual-stream structure, and simultaneously predicting pedestrian trajectories based on trajectory location data and key point sequence data, the accuracy and precision of pedestrian trajectory prediction are improved.

[0011] Furthermore, determining the pedestrian's trajectory location data and key point sequence data based on the pedestrian motion video data includes:

[0012] Multi-target tracking and localization are performed on pedestrians in the pedestrian motion video data to determine the trajectory and location data of the pedestrians;

[0013] Key points in the pedestrian movement video are identified to determine the key point sequence data.

[0014] Based on the above technical features, the accuracy of the collected data is improved by utilizing multi-target tracking technology and key point recognition technology.

[0015] Furthermore, the trajectory location data and the key point sequence data are input into a convolutional model with a spatiotemporal attention mechanism to obtain predicted pedestrian trajectory data, including:

[0016] The trajectory position data and the key point sequence data are saved according to a preset format to obtain formatted trajectory position data and formatted key point sequence data;

[0017] The formatted trajectory location data and the formatted keypoint sequence data are input into a convolutional model with a spatiotemporal attention mechanism to obtain the predicted data of the pedestrian trajectory.

[0018] Based on the above technical features, storing data in a formatted manner is beneficial for subsequent data analysis and processing.

[0019] Furthermore, the step of saving the trajectory position data and the key point sequence data according to a preset format to obtain formatted trajectory position data and formatted key point sequence data includes:

[0020] Based on the trajectory position data, the formatted trajectory position data is determined using equation (1):

[0021]

[0022] In equation (1), T o The input information for the convolutional model with spatiotemporal attention mechanism is: In1 is the formatted trajectory position data under the preset format; x and y are the x and y coordinates of the midpoint of the human body contacting the ground; n is the number of pedestrians; and t is the time.

[0023] Based on the keypoint sequence data, the formatted keypoint sequence data is determined using equation (2):

[0024]

[0025] In equation (2), T o The input information is the convolutional model with spatiotemporal attention mechanism, k is the key point sequence of pedestrians, In2 is the formatted key point sequence data under the preset format, n is the number of pedestrians, and t is the time.

[0026] Furthermore, the convolutional model of the spatiotemporal attention mechanism includes a graph convolution module, a temporal convolution module, an attention mechanism module, a fusion module, and a temporal transformation module;

[0027] The step of inputting the formatted trajectory location data and the formatted keypoint sequence data into a convolutional model with a spatiotemporal attention mechanism to obtain the predicted data of the pedestrian trajectory includes:

[0028] The formatted trajectory position data and the formatted key point sequence data are input into the graph convolution module to obtain trajectory position data and key point sequence data after graph convolution.

[0029] The trajectory position data and key point sequence data after graph convolution are input into the temporal convolution module to obtain trajectory position data and key point sequence data after spatiotemporal convolution.

[0030] The trajectory position data and key point sequence data after graph convolution and the trajectory position data and key point sequence data after spatiotemporal convolution are input into the attention mechanism module to obtain spatiotemporally fused trajectory position data and key point sequence data.

[0031] The spatiotemporal fusion trajectory location data and key point sequence data are input into the fusion module to obtain pedestrian trajectory data fused with multimodal features;

[0032] The pedestrian trajectory data fused from the multimodal features is input into the time series conversion module for time series conversion to obtain the predicted data of the pedestrian trajectory.

[0033] Based on the aforementioned technical features, the spatiotemporal attention mechanism convolutional model includes a graph convolution module, a temporal convolution module, an attention mechanism module, a fusion module, and a temporal transformation module. The graph convolution module performs graph convolution on the data, the temporal convolution module performs temporal convolution on the graph-convolutioned data, the attention mechanism module performs spatiotemporal fusion of the graph-convolutioned data and the temporal-convolutioned data, the fusion module fuses keypoint sequence data into trajectory location data to obtain multimodal feature fused pedestrian trajectory data, and the temporal transformation module performs temporal transformation on the multimodal feature fused pedestrian trajectory data to obtain pedestrian trajectory prediction data. Based on these modules, the accuracy and reference value of pedestrian trajectory prediction data are improved.

[0034] Furthermore, the method also includes:

[0035] Acquire historical pedestrian movement video data;

[0036] Based on the historical pedestrian movement data, the trajectory location data and historical key point sequence data of the historical pedestrians are determined;

[0037] Based on the historical pedestrian trajectory location data and historical key point sequence data, a convolutional model for spatiotemporal attention mechanism is trained.

[0038] Determine whether the prediction accuracy of the convolutional model of the spatiotemporal attention mechanism reaches the preset accuracy.

[0039] If the prediction accuracy of the convolutional model with the spatiotemporal attention mechanism does not reach the preset accuracy, training continues until the prediction accuracy of the convolutional model with the spatiotemporal attention mechanism reaches the preset accuracy.

[0040] Based on the aforementioned technical characteristics, a convolutional model with a spatiotemporal attention mechanism is trained using historical pedestrian movement data until the model's prediction accuracy reaches a preset accuracy, thereby improving the accuracy of the model's predictions and the applicability of the prediction data.

[0041] Furthermore, the key body parts include the head, shoulders, elbows, wrists, hips, knees, and feet.

[0042] A pedestrian trajectory prediction device, comprising:

[0043] The data acquisition module is used to acquire pedestrian movement video data;

[0044] The data processing module is used to determine the pedestrian's trajectory location data and key point sequence data based on the pedestrian movement video data;

[0045] The trajectory prediction module is used to input the trajectory location data and the key point sequence data into a convolutional model with a spatiotemporal attention mechanism to obtain predicted data of pedestrian trajectory.

[0046] An electronic device is configured to perform the pedestrian trajectory prediction method described above.

[0047] A computer-readable storage medium storing instructions for causing a machine to perform the steps described in any possible implementation of the pedestrian trajectory prediction method.

[0048] The beneficial effects of this invention are:

[0049] (1) This invention combines multiple information such as pedestrian historical trajectory, interaction relationship between pedestrians and pedestrian's own movement state to predict the future trajectory of pedestrians, thereby improving the reference value of the prediction;

[0050] (2) This invention uses a convolutional model with a spatiotemporal attention mechanism and a dual-stream structure to predict pedestrian trajectories based on trajectory location data and key point sequence data, thereby improving the accuracy and precision of pedestrian trajectory prediction. Attached Figure Description

[0051] Figure 1 This is a schematic diagram illustrating the process of a pedestrian trajectory prediction method according to an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of the convolutional model structure for the spatiotemporal attention mechanism.

[0053] Figure 3 A diagram of the attention mechanism structure;

[0054] Figure 4 A flowchart for multimodal information fusion;

[0055] Figure 5 This is a schematic diagram illustrating the functional modules of a pedestrian trajectory prediction device according to an embodiment of the present invention. Detailed Implementation

[0056] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0057] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0058] This embodiment proposes a method for predicting pedestrian trajectories, such as... Figure 1 The diagram shows a flowchart of a pedestrian trajectory prediction method provided in this embodiment.

[0059] Step S100: Acquire pedestrian motion video data.

[0060] Pedestrian movement video data can be collected by the vehicle's front-view camera and panoramic camera.

[0061] Step S200: Based on the pedestrian movement video data, determine the pedestrian's trajectory location data and key point sequence data; wherein, the key point sequence data is the data of all key parts of the pedestrian's body.

[0062] This embodiment utilizes multi-target tracking and positioning technology to locate the trajectory position data of each pedestrian from pedestrian movement video data, and uses key point recognition technology to identify the key point sequence of each pedestrian from pedestrian movement video data. The key points include each pedestrian's head, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left foot, and right foot.

[0063] In this embodiment, the trajectory data of the human body is defined as T = {T o ,T p}, T o For the input information of the convolutional model of the spatiotemporal attention mechanism, this embodiment uses formula (1) to format and save the trajectory position data. Related methods include UpdateNet, SiamRPN++, etc.; for the key point sequence data, this embodiment uses formula (2) to format and save the data. Related methods include Coordinate NET, HRNet, etc.; T p The label information of the convolutional model with spatiotemporal attention mechanism can be represented by formula (3).

[0064]

[0065]

[0066]

[0067] T oFor the input information of the convolutional model of spatiotemporal attention mechanism, In1 is the formatted trajectory position data under the preset format, In2 is the formatted key point sequence data under the preset format, x and y are the x and y coordinates of the midpoint of the human body contacting the ground, n is the number of pedestrians, t is the time, and k is the key point sequence of pedestrians.

[0068] It is worth mentioning that after saving the trajectory location data and key point sequence data according to the preset format, this embodiment also stores the trajectory location data and key point sequence data of each pedestrian into two array matrices respectively. The format of the array matrix is ​​(C,T,V), where C refers to the dimension information, T refers to the temporal information, V refers to the number of pedestrians in the trajectory location data array matrix, and V refers to the number of key points in the key point sequence data array matrix. Then, the two array matrices are input into the convolutional model of the spatiotemporal attention mechanism.

[0069] Step S300: Input the trajectory location data and key point sequence data into the spatiotemporal attention mechanism convolutional model to obtain the predicted data of pedestrian trajectory.

[0070] The convolutional model of the spatiotemporal attention mechanism in this embodiment is as follows: Figure 2 As shown, the model includes a GCN graph convolution module, a TCN temporal convolution module, an attention mechanism module, and a fusion module. Since the number of pedestrians in a given frame at any given time is uncertain, and the weighted relationships of their interactions are also uncertain, ordinary convolution cannot solve these problems, while graph convolution can. Therefore, this embodiment adds a GCN graph convolution module to the model. Furthermore, since pedestrian trajectory prediction involves temporal information, we incorporate a TCN temporal convolution module, which can handle temporal information, on top of the graph convolution. Because graph convolution and temporal convolution process pedestrian trajectory location data separately, a spatiotemporal attention mechanism is incorporated. Figure 3 As shown, attention operations are performed on both spatial and temporal channels simultaneously, and then the information from both is fused. Convolution is used to learn key information, and the output is then processed according to the input information dimensions. This operation effectively extracts key location information and keyframe sequence information. Finally, a fusion module integrates the pedestrian's keypoint sequence data into the trajectory position data, resulting in trajectory position data containing the keypoint sequence data. This trajectory position data then undergoes a temporal transformation. First, the temporal dimension is transformed from the third dimension to the second dimension. Then, a temporal expansion convolution changes the acquired trajectory temporal sequence to the predicted temporal sequence. Finally, the temporal sequence is transformed from the second dimension back to the third dimension for output. The update formula for the graph convolution is f. out =A * XW,f out A is the output of the graph convolution module. *The adjacency matrix is ​​used for adaptation, W is the weight parameter for graph convolution update, X is the input pedestrian trajectory location data or keypoint sequence data, and the update formula for temporal convolution is T. out =XW,T out The output of the temporal convolution is W, the weight parameters updated by the graph convolution are W, and the input pedestrian trajectory location data or key point sequence data is X. The adaptive adjacency matrix is ​​learned autonomously during the model training process.

[0071] The specific operation of the above-mentioned fusion module is as follows: Figure 4 As shown, the pedestrian trajectory location data directly learns the trajectories of V individuals in the scene, while the pedestrian keypoint sequence data learns the action features of each pedestrian. Therefore, there are V spatiotemporal attention map convolutions that process the pedestrian keypoint sequences. Pooling is performed on the last dimension of the output of the V spatiotemporal attention map convolutions of pedestrian keypoint sequences, transforming the output dimension from (B,C,T,V) to (B,C,T,1), where B is Batch_Size (the number of data samples passed to the program for training in a single batch). Then, the V results are concatenated on the 4th dimension to generate a result consistent with the output of the spatiotemporal attention map convolution of the pedestrian trajectory. Next, the two results are concatenated on the 2nd dimension to generate a feature of (B,2C,T,V). Finally, a convolution is performed to transform it into (B,C,T,V) and output it.

[0072] In this embodiment, the trajectory data of the human body is defined as T = {T o ,T p}, used for training convolutional models with spatiotemporal attention mechanisms, T o The input information for the convolutional model with a spatiotemporal attention mechanism can contain a sequence length of 12 frames, where T... o As input data for model training, T p The label information for the spatiotemporal attention mechanism convolutional model is used as the input ground truth label. Through iterative updates of network parameters, the model is ultimately trained. Specifically, this implementation obtains historical pedestrian trajectory location data and historical keypoint sequence data as training samples for the spatiotemporal attention mechanism convolutional model. After each training session, the accuracy is assessed. When the model's prediction accuracy reaches a preset accuracy, model training is stopped, and the model at this point is saved.

[0073] This embodiment combines various information such as pedestrian historical trajectory, interaction relationships between pedestrians, and pedestrian's own movement state to predict the future trajectory of pedestrians, which improves the reliability of the prediction. In addition, this embodiment adopts a convolutional model with a spatiotemporal attention mechanism with a dual-stream structure, and predicts pedestrian trajectory based on trajectory location data and key point sequence data, thereby improving the accuracy and precision of pedestrian trajectory prediction.

[0074] This embodiment proposes a pedestrian trajectory prediction device, such as... Figure 5 The diagram shows a functional module schematic of a pedestrian trajectory prediction device 200 provided in this embodiment.

[0075] Data acquisition module 210 is used to acquire pedestrian motion video data;

[0076] The data processing module 220 is used to determine the trajectory location data and key point sequence data of pedestrians based on pedestrian motion video data;

[0077] The trajectory prediction module 230 is used to input trajectory location data and key point sequence data into a convolutional model with a spatiotemporal attention mechanism to obtain predicted data of pedestrian trajectory.

[0078] This embodiment also provides an electronic device in which, in a typical configuration, the computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0079] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0080] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0081] This embodiment also provides a computer-readable storage medium storing instructions that, when executed by a processor, are adapted to perform a program containing steps of a pedestrian trajectory prediction method.

[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A method for predicting pedestrian trajectories, characterized in that, The method includes: Acquire pedestrian movement video data; Based on the pedestrian movement video data, determine the pedestrian's trajectory location data and key point sequence data; wherein, the key point sequence data consists of data on all key parts of the pedestrian's body. The trajectory location data and the key point sequence data are input into the spatiotemporal attention mechanism convolutional model to obtain the predicted data of the pedestrian trajectory; the spatiotemporal attention mechanism convolutional model includes a graph convolution module, a temporal convolution module, an attention mechanism module, a fusion module and a temporal transformation module; Formatted trajectory location data and formatted keypoint sequence data are input into a convolutional model with a spatiotemporal attention mechanism to obtain predicted data of the pedestrian trajectory, including: The formatted trajectory position data and the formatted key point sequence data are input into the graph convolution module to obtain trajectory position data and key point sequence data after graph convolution. The trajectory position data and key point sequence data after graph convolution are input into the temporal convolution module to obtain trajectory position data and key point sequence data after spatiotemporal convolution. The trajectory position data and key point sequence data after graph convolution and the trajectory position data and key point sequence data after spatiotemporal convolution are input into the attention mechanism module. Attention operations are performed on the spatial channel and the temporal channel to obtain spatiotemporally fused trajectory position data and key point sequence data. The spatiotemporal fusion trajectory location data and key point sequence data are input into the fusion module to obtain pedestrian trajectory data fused with multimodal features; The fusion module pools the last dimension of the output of the spatiotemporal attention map convolution of V pedestrian keypoint sequences, transforming the output dimension from (B,C,T,V) to (B,C,T,1), where B is the number of data samples passed to the program for training in a single pass. Then, the V results are concatenated in the fourth dimension to generate a result consistent with the output of the spatiotemporal attention map convolution of the pedestrian trajectory. Next, the two results are concatenated in the second dimension to generate a feature of (B,2C,T,V). Finally, it is transformed into (B,C,T,V) through a convolution and output. The pedestrian trajectory data fused from the multimodal features is input into the time series conversion module for time series conversion to obtain the predicted data of the pedestrian trajectory.

2. The pedestrian trajectory prediction method according to claim 1, characterized in that, The step of determining the pedestrian's trajectory location data and key point sequence data based on the pedestrian motion video data includes: Multi-target tracking and positioning of pedestrians in the pedestrian motion video data is performed to determine the trajectory and location data of the pedestrians. Key points in the pedestrian movement video are identified to determine the key point sequence data.

3. The pedestrian trajectory prediction method according to claim 1, characterized in that, The trajectory location data and the keypoint sequence data are input into a convolutional model with a spatiotemporal attention mechanism to obtain predicted pedestrian trajectory data, including: The trajectory position data and the key point sequence data are saved according to a preset format to obtain formatted trajectory position data and formatted key point sequence data; The formatted trajectory location data and the formatted keypoint sequence data are input into a convolutional model with a spatiotemporal attention mechanism to obtain the predicted data of the pedestrian trajectory.

4. The pedestrian trajectory prediction method according to claim 3, characterized in that, The step of saving the trajectory position data and the key point sequence data according to a preset format to obtain formatted trajectory position data and formatted key point sequence data includes: Based on the trajectory position data, the formatted trajectory position data is determined using equation (1): （1）； In equation (1), The input information for the convolutional model with spatiotemporal attention mechanism. This is formatted trajectory location data in a preset format. and Let x and y be the x-coordinates of the midpoint where the human body touches the ground. The number of pedestrians For time; Based on the keypoint sequence data, the formatted keypoint sequence data is determined using equation (2): （2）； In equation (2), The input information for the convolutional model with spatiotemporal attention mechanism. For the key point sequence of pedestrians, This is formatted keypoint sequence data under a preset format. The number of pedestrians For time.

5. The pedestrian trajectory prediction method according to claim 1, characterized in that, The method further includes: Acquire historical pedestrian movement video data; Based on the historical pedestrian movement data, the trajectory location data and historical key point sequence data of the historical pedestrians are determined; Based on the historical pedestrian trajectory location data and historical key point sequence data, a convolutional model for spatiotemporal attention mechanism is trained. Determine whether the prediction accuracy of the convolutional model of the spatiotemporal attention mechanism reaches the preset accuracy. If the prediction accuracy of the convolutional model with the spatiotemporal attention mechanism does not reach the preset accuracy, training continues until the prediction accuracy of the convolutional model with the spatiotemporal attention mechanism reaches the preset accuracy.

6. The pedestrian trajectory prediction method according to claim 5, characterized in that, The key body parts include the head, shoulders, elbows, wrists, hips, knees, and feet.

7. A pedestrian trajectory prediction device, characterized in that, include: The data acquisition module is used to acquire pedestrian movement video data; The data processing module is used to determine the pedestrian's trajectory location data and key point sequence data based on the pedestrian movement video data; The trajectory prediction module is used to input the trajectory location data and the key point sequence data into the spatiotemporal attention mechanism convolutional model to obtain the predicted data of the pedestrian trajectory; the spatiotemporal attention mechanism convolutional model includes a graph convolution module, a temporal convolution module, an attention mechanism module, a fusion module and a temporal transformation module; Formatted trajectory location data and formatted keypoint sequence data are input into a convolutional model with a spatiotemporal attention mechanism to obtain predicted data of the pedestrian trajectory, including: The formatted trajectory position data and the formatted key point sequence data are input into the graph convolution module to obtain trajectory position data and key point sequence data after graph convolution. The trajectory position data and key point sequence data after graph convolution are input into the temporal convolution module to obtain trajectory position data and key point sequence data after spatiotemporal convolution. The trajectory position data and key point sequence data after graph convolution and the trajectory position data and key point sequence data after spatiotemporal convolution are input into the attention mechanism module. Attention operations are performed on the spatial channel and the temporal channel to obtain spatiotemporally fused trajectory position data and key point sequence data. The spatiotemporal fusion trajectory location data and key point sequence data are input into the fusion module to obtain pedestrian trajectory data fused with multimodal features; The fusion module pools the last dimension of the output of the spatiotemporal attention map convolution of V pedestrian keypoint sequences, transforming the output dimension from (B,C,T,V) to (B,C,T,1), where B is the number of data samples passed to the program for training in a single pass. Then, the V results are concatenated on the fourth dimension to generate a result consistent with the spatiotemporal attention map convolution output of the pedestrian trajectory. Next, the two results are concatenated on the second dimension to generate a feature of (B,2C,T,V). Finally, it is transformed into (B,C,T,V) through a convolution and output. The pedestrian trajectory data fused from the multimodal features is input into the time series conversion module for time series conversion to obtain the predicted data of the pedestrian trajectory.

8. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the pedestrian trajectory prediction method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions for causing a machine to perform the pedestrian trajectory prediction method as described in any one of claims 1-6.

Images