Pedestrian trajectory prediction method based on social intention interaction

Abstract

The embodiment of the invention discloses a pedestrian trajectory prediction method based on social intention interaction, relates to the technical field of intelligent manufacturing, and can improve the accuracy and success rate of pedestrian trajectory prediction. The method comprises the following steps: extracting pedestrian position information in a scene in continuous frames, and creating a data list; designing a motion state analysis module through the data list, carrying out kinematics analysis on pedestrians, and respectively constructing state tensors of a subject and a neighbor group; establishing a network to encode the state tensor, extracting intention information, and realizing intention interaction; decoding the interaction result to obtain the trajectory representation of the main body in the scene; and optimizing the trajectory representation by using a designed trajectory optimizer to realize complete and accurate pedestrian trajectory prediction. The method is suitable for pedestrian trajectory prediction in a dense pedestrian scene.

Description

Technical Field

[0001] The present invention relates to a prediction technology branch in general image data processing or generation technology, and is mainly used in intelligent manufacturing scenarios involving new generation information technology, and in particular to a pedestrian trajectory prediction method based on social intention interaction. Background Art

[0002] Pedestrian trajectory prediction refers to predicting the possible movement trajectory of pedestrians in a certain period of time in the future given the historical data of pedestrian positions within a period of time. This technology has been widely used in strong human-computer interaction fields such as autonomous driving and intelligent manufacturing. However, the complex situations in actual scenes pose great challenges to the task of pedestrian trajectory prediction. On the one hand, pedestrian movement is highly autonomous and flexible, especially in social environments, where pedestrians' routes are affected by various complex factors. On the other hand, the number of neighbors of the subject pedestrian in the spatial scene is constantly changing. How to process them and analyze the interaction of their social intentions with the subject is crucial to the final trajectory prediction result. Therefore, there is an urgent need for a pedestrian trajectory prediction method based on social intention interaction to achieve effective and accurate pedestrian trajectory prediction in the spatial field.

[0003] Early methods had difficulty capturing complex environments and potential state information, and they were replaced by data-driven deep learning methods. Existing methods have solved some challenges. The LSTM-based method analyzes the social interaction between the subject and the neighbors, treats the interaction information equally, and cannot mine potential key information at future moments. Although the attention mechanism-based method no longer treats the interaction information between the subject and the neighbors equally, it is limited to the external interaction between the attention mechanism, and cannot capture the intrinsic connection between the subject and the neighbors. At the same time, it wastes a lot of computing resources on the processing of long-distance pedestrians. The VAE-based method uses latent variables generated over time to carry out trajectory measurement, but the social analysis of neighborhood relations is not deep enough. There is a common problem in the existing methods. The error gradually increases with the increase of the prediction period. Therefore, how to improve the accuracy of pedestrian trajectory prediction in complex scenes in order to achieve reliable pedestrian trajectory prediction has become a problem that needs to be studied and solved. Summary of the invention

[0004] The embodiments of the present invention provide a pedestrian trajectory prediction method based on social intention interaction, which can improve the accuracy of pedestrian trajectory prediction.

[0005] To achieve the above object, the embodiments of the present invention adopt the following technical solutions: A pedestrian trajectory prediction method based on social intention interaction, comprising: S1, extracting pedestrian position information in a scene from consecutive frames of captured video data, and creating a data list according to the pedestrian position information; S2. Using the data list, perform kinematic analysis on the pedestrian and obtain state tensors of the subject and the neighbor group; S3. Use the state tensor to perform intention interaction and obtain an interaction result; S4, decoding the interaction result and obtaining a trajectory representation of the subject in the scene; S5. After optimizing the trajectory representation, the finally obtained trajectory is subjected to DBSCAN clustering processing.

[0006] Among them, in the created data list, the first column is the selected observation time, the second column is the pedestrian ID selected as the subject, and the third and fourth columns are the horizontal and vertical coordinate information of the pedestrian as the subject in the two-dimensional plane. For example: for each frame of the image at the observation time, extract the current position of all pedestrians in the image and construct a data list. The first column is the selected observation time, the second column is the pedestrian ID selected as the subject, and the third and fourth columns are the position of the subject in the two-dimensional plane at the current moment. You can select one target for inspection in turn, and other pedestrians are temporarily set as global neighbors.

[0007] Furthermore, we can focus on the motion state of all pedestrians under global observation and design a motion state analysis module; specifically, introduce the perception information of pedestrians to design a motion state module, and model it according to the relationship between the selected subject and the neighbor group. Select subject i and neighbor j for motion state decomposition, and incorporate perception information to prepare for further construction of the state tensor of the subject and the neighbor group. S2 specifically includes: performing motion state analysis on the current subject to obtain the state tensor s of the subject; performing motion state analysis on the neighbor group of the current subject to obtain the state tensor o of the neighbor group. Among them, the motion state analysis of the current subject to obtain the state tensor S of the subject includes: establishing the entire observation period of the continuous frame T ob The state tensor of the inner subject is , N represents the main batch that needs to be processed, represents the real number domain; the motion state of subject i at time t is ( x t i , y t i , v t i , θ t i ), x t i , y t iThey represent the horizontal and vertical coordinates of the position of subject i in the scene at time t, v t i , θ t i They represent the tangential velocity variable and normal velocity variable of subject i in the scene at time t respectively; , , the tangential velocity of body i is expressed as v t ix , the normal velocity of subject i is expressed as v t iy .

[0008] After constructing the state tensor of subject i, the motion state of each individual in the neighbor group is decomposed in the same way as the state tensor of the subject, and the state tensor of each individual in the neighbor group can be obtained, thereby further describing the interaction information between the subject and the neighbor group. The motion state analysis of the neighbor group of the current subject to obtain the state tensor O of the neighbor group includes: establishing the entire observation period of the continuous frame T ob The state tensor of the inner neighbor group is , N n represents the number of neighbors, and the motion perception state of neighbor j at time t is expressed as , represents the distance between subject i and neighbor j at time t, θ t ij represents the velocity angle between subject i and neighbor j, , , x t j , y t j They represent the horizontal and vertical coordinates of the position of neighbor j in the scene at time t, v t j represents the tangential velocity variable of neighbor j in the scene at time t, → is a vector symbol, ob t ij Indicates the corresponding T ob The final distance of , , v t ij represents the velocity of subject i relative to neighbor j at time t,v t jx represents the tangential velocity of neighbor j, t represents the current moment, ξ Indicates the time period from the previous moment to the end of the observation period ( T ob - t ) is the time required for the subject i to meet its neighbor j at the current speed.

[0009] S3 of this embodiment includes: preprocessing the obtained state tensor, and then using the intention information of the neighbor group to obtain the attention weight; using the attention weight to determine the result of the intention interaction.

[0010] The preprocessing of the obtained state tensor includes: performing linear transformation processing and position encoding on the state tensor of the neighbor group, and obtaining a preprocessing result for the neighbor group. ,in, d model Indicates O pe The characteristic dimension, O pe The vector projected to Query is Q h o , the vector projected to Key is K h o , the vector projected to Value is V h o ,in , h =1,…, ε , d ε For passing ε The feature dimension after the second projection; the state tensor of the subject is linearly transformed and positionally encoded, and the preprocessing result for the subject is obtained , where the query vector Q is obtained by performing three linear transformations on Spe s , Key vector K s and Value vector V s ,and .

[0011] The method of using the intention information of the neighbor group to obtain the attention weight includes: obtaining the attention weight A of each head in the neighbor group h o , , M o represents the perceptual mask, when Greater than r i When it is determined to be a non-neighbor, r i The input of the interactive result decoder is MultiHead(Qh o ,K h o ,V h o ), the input of the subject intention extraction module is .

[0012] Among them, the entire T ob The distance between period subject i and each global neighbor j With observation radius r i The size relationship is set to be greater than r i As non-neighbors, they are set to 0 in the perception mask, otherwise they are regarded as neighbors and set to 1, and then a sparse perception mask can be obtained. As shown in formulas (8) and (9), the designed neighbor-aware intention extraction module has two outputs, namely MultiHead(Q h o ,K h o ,V h o )and The former is used as the input of the interaction result decoder, and the latter is used as the input of the subject intention extraction module.

[0013] The attention weight , perception mask , attention score , the projection matrix ,'.' is used to convert Att h Indexed ε The results of the attention functions of the heads are concatenated together.

[0014] The using the attention weight to determine the result of the intention interaction includes: The number of neighbors N n The dimensions are averaged, where , T ob represents the last moment of the observation period; then V S Expand the dimension and get V S The expansion result of , and, for A S Expand the dimension to get A S The expansion result of , A S is the attention weight in the subject intention extraction module, ; Then adjust the influence of the neighbor group attention weight through the proportional factor λ to get the final attention score , W S is the learned linear transformation matrix.

[0015] S4 of this embodiment includes: decoding the interaction results between the subject and the neighbor group to obtain the trajectory representation of the subject; decoding the attention score, and using the decoding result as the input of the neighbor-perceived intention extraction module in the next layer of the network. Specifically, the interaction result is decoded to obtain the trajectory representation of the subject in the scene, which includes: decoding the interaction results between the subject and the neighbor group to obtain the trajectory representation of the subject; decoding the attention score result as the input of the neighbor-perceived intention extraction module in the next layer of the network. The interaction results between the subject and the neighbor group are decoded to obtain the trajectory representation of the subject; decoding the attention score result as the input of the neighbor-perceived intention extraction module in the next layer of the network, which includes: decoding the interaction results and the attention score results, and inputting them into the corresponding decoders for processing. In order to alleviate the problem of gradient disappearance and explosion and to obtain a smoother gradient, the first part of the encoder is designed as a residual connection. It consists of a droupout function, an add function and layer normalization. After processing the input, two output results C are obtained respectively. s and C o , as shown in formulas (12), (13).

[0016] In order to improve the embedding quality, C s and C o It is sent to the multi-layer perceptron for processing, as shown in equations (14) and (15). ’ o , C ’ s C o , C s The processing result after the multi-layer perceptron. In order to slow down the model degradation, a layer of residual connection is superimposed after the multi-layer perceptron. The final processing result C is obtained ” o , C ” s , as shown in equations (16) and (17). In the last layer of the decoder, the branch where s is located outputs C ” s It will then be processed through a full connection to obtain the trajectory representation ,in l Indicates the number of predicted trajectories, T pred Represents the number of periods of the predicted trajectory.

[0017] Optionally, the trajectory representation can be optimized using the designed trajectory optimizer, which includes: supervising the prediction results with the loss function of multi-dimensional information fusion; performing DBSCAN clustering on the multiple trajectories finally obtained to automatically determine the number of categories. This method is more flexible than manually setting the number of categories and is more in line with the pedestrian autonomous navigation strategy. The loss function of multi-dimensional information fusion supervises the prediction results, which includes: selecting the predicted position of each period for comparison with the actual position. Calculating their Euclidean distance to obtain the distance prediction error of each period L dis , as shown in formula (18). p t igt is the real position of subject i at time t, T pred represents the number of periods to be predicted, p t i Represents the 2D spatial coordinates of subject i at time step t.

[0018] For angle processing, the last moment of the observation period is introduced T ob . Each predicted moment is connected with the previous moment to construct a direction vector, and then the angle between the predicted moment and the real trajectory direction vector is examined, as shown in formula (19). The target loss function of the model is L As shown in formula (20), the distance error and angle error between each predicted result and the true result are back-propagated to the network to minimize the target loss function. L result.

[0019] The pedestrian trajectory prediction method based on social intention interaction provided by the embodiment of the present invention combines the trajectory prediction needs of pedestrians in the spatial field with the functions in the field of computer deep learning, more accurately captures the social norms between pedestrians in the spatial scene, and realizes more realistic and accurate trajectory prediction. The present invention includes: extracting the position information of pedestrians in the scene in continuous frames and creating a data list; designing a motion state analysis module through the data list, performing kinematic analysis on pedestrians, and constructing state tensors of the subject and the neighbor group respectively; building a network to encode the state tensor and extract intent information to realize intent interaction; decoding the interaction results to obtain the trajectory representation of the subject in the scene; optimizing the trajectory representation using the designed trajectory optimizer, thereby improving the accuracy of pedestrian trajectory prediction.

[0020] Formula (1) to Formula (20) in this embodiment include:

[0021] BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of the overall framework flow provided for an embodiment of the present invention; Figure 2 A schematic diagram of a data extraction module provided in an embodiment of the present invention; Figure 3 A schematic diagram of trajectory analysis provided by an embodiment of the present invention; Figure 4 A schematic diagram of a network structure provided by an embodiment of the present invention; Figure 5 A schematic diagram of perceptual mask attention provided by an embodiment of the present invention; Figure 6 A heat map of social intention interaction attention weights provided by an embodiment of the present invention; Figure 7 A schematic diagram of a loss function for multi-dimensional information fusion provided by an embodiment of the present invention; Figure 8 A schematic diagram of a simple scene trajectory visualization provided by an embodiment of the present invention; Fig. 9 A schematic diagram of complex scene trajectory visualization provided by an embodiment of the present invention; Fig.10 A schematic diagram of an attention map within an observation range provided by an embodiment of the present invention; Fig.11 The present invention provides a flow chart of the method. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention. It can be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "one", "said" and "the" used herein may also include plural forms. It should be further understood that the term "including" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used here may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items. It can be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the field to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0025] The embodiment of the present invention provides a pedestrian trajectory prediction method based on social intention interaction, such as Fig.11 As shown, including: S1, extracting pedestrian position information in a scene from consecutive frames of captured video data, and creating a data list according to the pedestrian position information; S2. Using the data list, perform kinematic analysis on the pedestrian and obtain state tensors of the subject and the neighbor group; S3. Use the state tensor to perform intention interaction and obtain an interaction result; S4, decoding the interaction result and obtaining a trajectory representation of the subject in the scene; S5. After optimizing the trajectory representation, the finally obtained trajectory is subjected to DBSCAN clustering processing.

[0026] In this embodiment, in S1, the pedestrian position information in the scene is extracted in consecutive frames, and a data list is created, which includes: like Figure 2As shown in the figure, for each image at the observation time, the current positions of all pedestrians in the image are extracted to construct a data list. The first column is the selected observation time, the second column is the ID of the pedestrian selected as the subject, and the third and fourth columns are the positions of the subject in the two-dimensional plane at the current moment. One target is selected for inspection in turn, and other pedestrians are temporarily defined as global neighbors.

[0027] In this embodiment, in S2, the motion state analysis module is designed through the data list, the kinematic analysis of the pedestrians is performed, and the state tensors of the subject and the neighbor group are constructed respectively, including: focusing on the motion state of all pedestrians under global observation, designing a motion state analysis module; performing motion state analysis on the current subject, and constructing the state tensor s of the subject; performing motion state analysis on the neighbor group of the current subject, and constructing the state tensor o of the neighbor group.

[0028] Specifically, the motion state module is designed by introducing the perception information of pedestrians, and the model is built based on the relationship between the selected subject and the neighbor group. Figure 3 As shown in the figure, subject i and neighbor j are selected for motion state decomposition, and the perception information is integrated to prepare for further construction of the state tensor of the subject and neighbor group.

[0029] Define the position of a subject i in the scene at time t as ( x t i , y t i ), the tangential velocity is expressed as v t ix , the normal velocity is expressed as v t iy The motion state of the subject is described based on the subject's position and velocity vector, as shown in equations (1) and (2). Finally, the motion state of the currently selected subject i at time t is expressed as ( x t i , y t i , v t i , θ t i ), then the entire observation period T ob The state tensor of the inner subject is ,in N Represents the batch of principals to be processed.

[0030] After constructing the state tensor of subject i, the motion state of each individual in the neighbor group is decomposed in the same way. This further describes the interaction information between the subject and the neighbor group. The distance between subject i and neighbor j at time t is It is the key factor to characterize the current intention interaction between them, as shown in formula (3). In order to enhance the understanding of social behavior and better characterize the social motion state, their speed angle is introduced θ t ij , as shown in formula (4).

[0031] Incorporating the perception of potential impacts at future moments throughout the entire time domain, assuming that pedestrians are not affected by sudden factors during their travel, and examining the time period of the current moment relative to the end of the observation period ( T ob - t ) is the time required for the subject i to meet its neighbor j at the current speed ξ The size of the time period is selected as the time for them to maintain the tangential speed and continue to travel along the tangential direction, which is associated with the distance at the current moment to obtain the final distance under the observation period. ob t ij , as shown in equations (5) and (6). Finally, the motion perception state of neighbor j at time t is expressed as , then the entire observation period T ob The state tensor of the neighbor group in is ,in N n Represents the number of neighbors, the size of the tensor o and the number of neighbors N n The numerical value is related.

[0032] In this embodiment, in S3, a network is built to encode the state tensor and extract intent information to realize intention interaction, including: designing a neighbor-aware intention extraction module to extract the intent information of the neighbor group to obtain the attention score result and attention weight; designing a subject intention extraction module to receive the attention weight output from the neighbor-aware intention extraction module to realize intention interaction.

[0033] Specifically, Figure 4 As shown, the neighbor group state tensor o is obtained through linear transformation and position encoding. Different, learned linear projections ε Projected to Query vector Q h o , Key vector K h o and Value vector V ho ,in , h =1,…, ε , d ε For passing ε The feature dimension after projection. According to formula (7), the attention weight A of each head in the neighbor group is obtained h o . Investigate the entire T ob The distance between period subject i and each global neighbor j With observation radius r i The size relationship is set to be greater than r i is regarded as a non-neighbor and set to 0 in the perception mask, otherwise it is regarded as a neighbor and set to 1, and then a sparse perception mask can be obtained, such as Figure 5 As shown in equations (8) and (9), the designed neighbor-aware intention extraction module has two outputs, namely MultiHead(Q h o ,K h o ,V h o )and The former is used as the input of the interaction result decoder, and the latter is used as the input of the subject intention extraction module.

[0034] The attention weight , perception mask , attention score , the projection matrix ,'.' is used to convert Att h Indexed ε The results of the attention functions of the heads are concatenated together.

[0035] The subject state tensor s is obtained by linear transformation and position encoding. , after three linear transformations, we get the Query vector Q S , Key vector K s and Value vector V S ,in In order to eliminate the influence of random errors and outliers in the neighbor group, the attention weight The number of neighbors N The dimensions are averaged to obtain . Considering the Matched to V S Expand the dimension and get At the same time, as in formula (10), after obtaining the attention score, A S Expand the dimension to get The scaling factor λ is introduced (this parameter is ultimately obtained through training) to adjust the influence of the introduced neighbor group attention weights, and the final interaction result is obtained through full connection, as shown in formula (11). Figure 6 As shown in the figure, the interaction results are visualized, and both the horizontal and vertical axes represent historical moments. The (a) row represents the weight heat map of the influence of the subject state at each historical moment, the (b) row represents the weight heat map of the neighbor group state at each historical moment after perceptual mask processing, and the (c) row represents the interaction influence weight heat map after (b) is superimposed on (a).

[0036] In the formula is the attention weight in the subject intention extraction module, is the learned linear transformation matrix.

[0037] In this embodiment, in S4, the interaction result is decoded to obtain the trajectory representation of the subject in the scene, including: decoding the interaction result between the subject and the neighbor group to obtain the trajectory representation of the subject; decoding the attention score result as the input of the neighbor perception intention extraction module in the next layer of the network.

[0038] Specifically, Figure 4 As shown, the interaction result Attention(Q s ,K s ,V s ) and the attention score result MultiHead(Q h o ,K h o ,V h o ) are decoded and input into the corresponding decoder for processing. In order to alleviate the problem of gradient disappearance and explosion and obtain smoother gradients, the first part of the encoder is designed as a residual connection. It consists of a dropout function, an add function and a layer normalization. After processing the input, C s and C o , as shown in formulas (12), (13).

[0039] In order to improve the embedding quality, C s and C o It is sent to the multi-layer perceptron for processing, as shown in equations (14) and (15). ’ o , C ’ s C o , Cs The processing result after the multi-layer perceptron. In order to slow down the model degradation, a layer of residual connection is superimposed after the multi-layer perceptron. The final processing result C is obtained ” o , C ” s , as shown in equations (16) and (17). In the last layer of the decoder, the branch where s is located outputs C ” s It will then be processed through a full connection to obtain the trajectory representation ,in l Indicates the number of predicted trajectories, T pred Represents the number of periods of the predicted trajectory.

[0040] In this embodiment, in S5, the trajectory representation is optimized using the designed trajectory optimizer to achieve complete and accurate pedestrian trajectory prediction, including designing a loss function of multi-dimensional information fusion to supervise the prediction results; DBSCAN clustering is performed on the multiple trajectories finally obtained to automatically determine the number of classifications.

[0041] Specifically, Figure 7 As shown, the predicted position of each period is selected for comparison with the actual position. Their Euclidean distance is calculated to obtain the distance prediction error of each period, as shown in formula (18). p t igt is the real position of subject i at time t.

[0042] For angle processing, the last moment of the observation period is introduced T ob . Each predicted moment is connected with the previous moment to construct a direction vector, and then the angle between the predicted moment and the true trajectory direction vector is examined, as shown in formula (19). The target loss function of the model is shown in formula (20). The distance error and angle error between each predicted result and the true result are back-propagated to the network to minimize the target loss function. L The DBSCAN clustering process is performed on the multiple trajectories finally obtained to automatically determine the number of categories, which is more flexible than manually setting the number of categories and is more in line with the pedestrian autonomous navigation strategy.

[0043] The method proposed in the present invention is tested on the public datasets ETH, UCY and SDD. Figure 8 As shown in Figure 1, the trajectory prediction results of the agent itself and its neighbors increase from 1 to 4 in a simple scenario are shown. The method proposed in this paper makes better predictions than the SOTA model SocialVAE. Fig. 9As shown in Figure 1, a scene with richer social information and a larger number of pedestrians is shown. The neighbors are represented by gray tracks, and the attention score is characterized by its transparency. The lower the transparency, the higher the attention score, and the higher the transparency, the lower the attention score. Fig.10 As shown in the figure, it is an attention map within the observation range, where the first row is HOTEL, the second row is UNIV, and the third row is ZARA. The yellow circle is the agent's observation area, the red dot is the agent's position in the current scene, the red line segment represents the agent's historical trajectory in the current scene, the blue line segment represents the real trajectory, and the orange line segment represents the predicted trajectory.

[0044] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The above is only a specific implementation method of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or replacements that can be easily thought of by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A pedestrian trajectory prediction method based on social intention interaction, characterized in that: include: S1, extracting pedestrian position information in a scene from consecutive frames of captured video data, and creating a data list according to the pedestrian position information; S2. Using the data list, perform kinematic analysis on the pedestrian and obtain state tensors of the subject and the neighbor group; S3. Use the state tensor to perform intention interaction and obtain an interaction result; S4, decoding the interaction result and obtaining a trajectory representation of the subject in the scene; S5. After optimizing the trajectory representation, the finally obtained trajectory is subjected to DBSCAN clustering processing.

2. The method according to claim 1, characterized in that In the created data list, the first column is the selected observation time, the second column is the pedestrian ID selected as the subject, and the third and fourth columns are the horizontal coordinate information and the vertical coordinate information of the pedestrian as the subject in the two-dimensional plane.

3. The method according to claim 2, characterized in that S2 include: Analyze the motion state of the current subject and obtain the state tensor S of the subject; Perform motion state analysis on the neighbor group of the current subject to obtain the state tensor O of the neighbor group.

4. The method according to claim 3, characterized in that The motion state analysis of the current subject is performed to obtain the subject's state tensor S, including: obtaining the entire observation period of the continuous frame T ob The state tensor of the inner body , N represents the main batch that needs to be processed, represents the field of real numbers; Among them, the motion state of subject i at time t is ( x t i , y t i , v t i , θ t i ), x t i， y t i They represent the horizontal and vertical coordinates of the position of subject i in the scene at time t, v t i， θ t i They represent the tangential velocity variable and normal velocity variable of subject i in the scene at time t respectively; , , the tangential velocity of body i is expressed as v t ix , the normal velocity of body i is expressed as v t iy .

5. The method according to claim 4, characterized in that The motion state analysis of the neighbor group of the current subject is performed to obtain the state tensor O of the neighbor group, including: The entire observation period of the consecutive frames is established T ob The state tensor of the inner neighbor group is , N n represents the number of neighbors, and the motion perception state of neighbor j at time t is expressed as , represents the distance between subject i and neighbor j at time t, θ t ij represents the velocity angle between subject i and neighbor j, , , x t j , y t j They represent the horizontal and vertical coordinates of the position of neighbor j in the scene at time t, v t j represents the tangential velocity variable of neighbor j in the scene at time t, → is a vector symbol, ob t ij Indicates the corresponding T ob The final distance in, , , v t ij represents the velocity of subject i relative to neighbor j at time t, v t jx represents the tangential velocity of neighbor j, t represents the current moment, ξ Represents the time period from the previous moment to the end of the observation period ( T ob -t ) is the time required for the subject i to meet its neighbor j at the current speed.

6. The method according to claim 1, characterized in that S3 include: The obtained state tensor is preprocessed, and then the attention weight is obtained using the intention information of the neighbor group; The attention weights are used to determine the outcome of the intended interaction.

7. The method according to claim 6, characterized in that The preprocessing of the obtained state tensor includes: performing linear transformation processing and position encoding on the state tensor of the neighbor group, and obtaining a preprocessing result for the neighbor group. ,in, d model Indicates O pe The characteristic dimension, O pe The vector projected to Query is Q h o , the vector projected to Key is K h o , the vector projected to Value is V h o , ,h=1,…, ε , d ε For passing ε Feature dimension after projection; Perform linear transformation and position encoding on the subject's state tensor, and obtain the preprocessing results for the subject , where S pe After three linear transformations, we get the Query vector Q s , Key vector K s and Value vector V s ,and .

8. The method according to claim 7, characterized in that The method of using the intention information of the neighbor group to obtain the attention weight includes: obtaining the attention weight A of each head in the neighbor group h o , , M o represents the perceptual mask, when Greater than r i When it is determined to be a non-neighbor, r i is the observation radius, and the input of the interaction result decoder is , the input of the subject intention extraction module is .

9. The method according to claim 8, characterized in that The using the attention weight to determine the result of the intention interaction includes: The number of neighbors N n The average processing result is ,in , T ob represents the last moment of the observation period; Then V S Expand the dimension and get V S The expansion result of , and, for A S Expand the dimension to get A S The expansion result of , A S is the attention weight in the subject intention extraction module, ; Then the influence of the neighbor group attention weight is adjusted by the proportional factor λ to obtain the final attention score , W S is the learned linear transformation matrix.

10. The method according to claim 1, characterized in that S4 include: Decode the interaction results between the subject and the neighbor group to obtain the trajectory representation of the subject; The attention scores are decoded and used as input to the neighbor-aware intent extraction module in the next layer of the network.

Images