Crowd motion tracking method in pedestrian and road scenes based on attention feature optimization

Abstract

A method for tracking crowd motion in pedestrian and road scenes based on attention feature optimization includes the following steps: S1. Basic model selection, wherein the basic model includes a feature template, a Gaussian mixture model, and a motion trend prediction algorithm; S2. Building an attention mechanism-based module: adding the attention mechanism module to the feature extraction part of the basic model can not only extract high-level features such as semantic information, but also restore the target individual detail information lost in the feature extraction process through feature fusion; S3. Improving the basic model through the attention mechanism module. By adding the attention mechanism-based module to the feature extraction part of the basic model, the feature map extracted in the feature extraction part can be further optimized and a high-quality feature map can be generated. The method for tracking crowd motion in pedestrian and road scenes based on attention feature optimization of the present invention can improve the accuracy of identifying the position of the target individual, thereby better predicting the target's motion trajectory.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular, to a dense crowd motion tracking method for pedestrian and road scenes based on feature optimization using an attention module. Background Art

[0002] Crowd tracking is a significant challenge in computer vision. Tracking pedestrian movements in pedestrian crossing scenarios can prevent road accidents and is therefore of great significance to the industry. The key algorithmic process for crowd tracking involves first extracting features from video frames containing dense crowds. Based on these extracted features, each target individual is detected. By identifying the position of each target individual within a continuous video sequence, their movement trends can be predicted.

[0003] Currently, there are many methods for tracking the movement of dense crowds. On the one hand, some methods have been proposed to identify the location of target individuals in dense crowds by extracting low-level features such as color and outline from video frames. On the other hand, some methods extract high-level features such as semantic information and temporal information to accurately detect the location of target individuals in video sequences.

[0004] However, in real-world scenarios, crowd density continues to increase, and models that extract these traditional features are insufficient to meet the challenge of detecting the location of individual targets in high-density crowds. Therefore, deep learning models, which have powerful feature learning capabilities, are used to extract the features of each target individual in dense crowds. However, deep learning models require devices with strong computing power to run, and deep learning models take a long time to process data, and cannot provide real-time feedback on the received visual data. In real-world scenarios, such as pedestrian crossings, detection algorithms are required to process and provide real-time feedback on the visual data received on site. For this reason, even though deep learning models achieve high accuracy in many detection tasks, they are not suitable for target detection in real-world scenarios such as pedestrian crossings.

[0005] The attention mechanism is used to extract the features of targets in visual data due to its powerful feature extraction capabilities. In order to extract the features of each target individual in a pedestrian scene, the present invention proposes a new attention mechanism-based module to obtain the target features of dense crowds in pedestrian scenes. In addition, during the feature extraction process, part of the detail information of the original video frame may be lost. In order to preserve these detail information well, the attention mechanism-based module proposed in the present invention also includes a feature optimization process to preserve these important detail information. Therefore, a dense crowd motion tracking framework including a feature template, a Gaussian mixture model and a motion trend prediction algorithm is selected as the basic model, and the attention mechanism module proposed in the present invention that can optimize features is added to the basic model. This can effectively identify the exact position of the target individual in a dense crowd, thereby accurately predicting its motion trend.

[0006] In summary, the significance of solving the above problems lies in providing a feature optimization method based on the attention module for dense crowd motion tracking tasks in scenes such as pedestrian roads. This method can not only extract more reliable high-level features, but also prevent the loss of detailed information in the original video frames. Summary of the Invention

[0007] The present invention provides a method for tracking crowd motion in pedestrian road scenes based on attention feature optimization. This method optimizes the feature maps of video frames based on an attention mechanism module. While extracting high-quality features, it can also effectively preserve important detail information of the video frames that is easily lost during the feature extraction process. It effectively extracts more reliable features and accurately identifies the position of each target in dense crowd motion tracking tasks.

[0008] The technical solutions of the present invention are as follows:

[0009] A method for tracking crowd motion in pedestrian road scenes based on attention feature optimization includes the following steps: S1. Basic model selection: The basic model includes a feature template, a Gaussian mixture model and a motion trend prediction algorithm; S2. Building an attention mechanism-based module: Adding the attention mechanism module to the feature extraction part of the basic model; S3. Improving the basic model through the attention mechanism module.

[0010] Preferably, in the above-mentioned method for tracking crowd motion in pedestrian road scenes based on attention feature optimization, in step S2, two attention mechanism operations are used to obtain high-quality features, and after each attention mechanism operation is completed, an effective feature fusion method is used to help the output result of the attention mechanism retain the detailed information of the original video frame.

[0011] Preferably, in the above-mentioned method for tracking crowd motion in pedestrian road scenes based on attention feature optimization, in step S2, a module based on an attention mechanism that can extract high-level features and achieve feature optimization is added to the feature extraction part of the basic model.

[0012] Preferably, in the above-mentioned method for tracking crowd motion in pedestrian road scenes based on attention feature optimization, in step S3, the video frame is first input into a feature extraction part including an attention mechanism-based module; then the feature extraction part transmits a high-quality feature map that can accurately identify the position of the target individual to the crowd motion trajectory prediction part; after algorithm processing, the crowd motion trajectory prediction part will output a result map including the motion trajectory of the target individual.

[0013] Preferably, in the above-mentioned method for tracking crowd motion in pedestrian road scenes based on attention feature optimization, in the feature extraction part, the feature template extracts features from the input video frame and outputs a feature map of the video frame; then the feature map is input into the above-mentioned attention mechanism-based module to extract high-quality features and perform feature optimization; after the optimized feature map is output from the attention mechanism-based module, it will be input into the Gaussian mixture model for processing, thereby outputting a Gaussian distribution feature map containing the target individual position, and the Gaussian distribution feature map will be output as the result of the feature extraction part, and then input into the crowd motion trajectory prediction part.

[0014] According to the technical solution of the present invention, the beneficial effects produced are:

[0015] Compared to existing traditional algorithms that primarily extract low-level or high-level features, the method of the present invention, due to its attention-based module, can not only extract high-quality high-level features, but also prevent the loss of detailed information in the original video frame through effective feature fusion. Because the attention-based module can generate high-quality feature maps, these feature maps enable the Gaussian mixture model to more accurately identify the location of the target individual, thereby allowing the motion trend prediction algorithm to more accurately predict the target's motion trajectory. In the test experiment, by testing on a dataset of pedestrian crossing scenes, the accuracy achieved by the model containing the attention-based module was higher than that of the model without this module.

[0016] On the other hand, compared with deep learning-based models, the present invention primarily processes and outputs data in real time after acquiring video frames. However, deep learning models, due to their relatively high algorithmic complexity, cannot process video data in real time. Therefore, considering response time, the present invention is more suitable than deep learning models for crowd tracking tasks in pedestrian crossing scenarios.

[0017] In order to better understand and illustrate the concept, working principle and effect of the present invention, the present invention is described in detail below with reference to the accompanying drawings and through specific embodiments: BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the specific implementation or the description of the prior art.

[0019] Figure 1 is a flow chart of a method for tracking dense crowd motion based on an attention module according to the present invention;

[0020] Figure 2 is a flow chart of an algorithm involved in the dense crowd motion tracking model based on the attention module of the present invention;

[0021] Figure 3 is a flow chart of the feature extraction part of the present invention; and

[0022] Figure 4 It is a schematic diagram of the attention mechanism module of the present invention. DETAILED DESCRIPTION

[0023] To make the purpose, technical methods and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and specific examples. These examples are merely illustrative and not intended to limit the present invention.

[0024] In the model for predicting the motion trajectory of a dense crowd, the video frame is first input into the feature extraction part to extract the features of the target individual, and then the generated feature map is input into the Gaussian mixture model for calculation, thereby identifying the location of the target individual. When the positions of all target individuals in a video sequence are identified, the motion trend or motion trajectory of the target individuals in this video sequence can be predicted by the motion trend prediction algorithm. In this process, the quality of the acquired features affects the accuracy of the target individual position identification. Therefore, extracting high-quality feature maps is conducive to accurately identifying the position of the target individual and predicting its motion trajectory. In order to achieve this purpose, the present invention proposes a new feature extraction and optimization module based on the attention mechanism, and adds the module to the dense crowd motion tracking framework including feature templates, Gaussian mixture models and motion trend prediction algorithms. While obtaining high-level features, it can also prevent the features from losing important detail information of the original video frame.

[0025] like Figure 1 As shown, the dense crowd motion tracking method based on the attention module of the present invention includes the following steps:

[0026] S1. Basic model selection, where the basic model includes a feature template, a Gaussian mixture model, and a motion trend prediction algorithm. Because we need to track the motion trends of dense crowds in pedestrian crossing scenarios, we selected a dense crowd motion tracking model that includes a feature template, a Gaussian mixture model, and a motion trend prediction algorithm as the basic model. This basic model requires minimal computation, enabling real-time data processing in pedestrian crossing scenarios.

[0027] S2. Build an attention mechanism-based module: Add an attention mechanism-based module that can extract high-level features and achieve feature optimization to the feature extraction part of the basic model. In this step, two attention mechanism operations are used to obtain high-quality features, and after each attention mechanism operation is completed, an effective feature fusion method is used to help the output of the attention mechanism retain the detailed information of the original video frame. In this attention mechanism module, not only can high-level features such as semantic information be extracted, but the target individual details lost in the feature extraction process can also be restored through feature fusion. Therefore, the high-quality feature map generated by the attention mechanism module can effectively help the model framework to detect the position of the target individual in the pedestrian crossing scene.

[0028] S3. Improving the basic model using the attention mechanism module. By adding the attention mechanism module to the feature extraction portion of the basic model, the feature maps extracted in the feature extraction portion can be further optimized and high-quality feature maps can be generated. The high-quality feature maps generated by the attention mechanism module can be effectively used by the Gaussian mixture model to identify the location of individual objects.

[0029] In this step, the video frame is first input into the feature extraction part containing the attention mechanism-based module; the feature extraction part then transmits the high-quality feature map that can accurately identify the target individual's position to the crowd motion trajectory prediction part; after algorithm processing, the crowd motion trajectory prediction part will output the result map containing the target individual's motion trajectory. In the feature extraction part, the feature template of the basic model extracts features from the input video frame and outputs the feature map of the video frame; the feature map is then input into the attention mechanism-based module to extract high-quality features and perform feature optimization; after the optimized feature map is output from the attention mechanism-based module, it will be input into the Gaussian mixture model for processing, thereby outputting a Gaussian distribution feature map containing the target individual's position. The Gaussian distribution feature map is output as the result of the feature extraction part and then input into the crowd motion trajectory prediction part.

[0030] The implementation process of the model of the present invention is as follows Figure 2As shown in the figure, a video frame containing a pedestrian crossing the road is first received by the feature extraction component (T1), which generates a high-quality feature map. Then, the motion trend prediction algorithm (T2) receives the feature map from the feature extraction component and predicts the target individual's motion trajectory based on the target individual's features in the feature map. Finally, a result map showing the target individual's motion trajectory is output as the final result.

[0031] In order to accurately predict the motion trajectory of the target individual, the framework used to complete the motion tracking task needs to obtain high-quality dense crowd feature maps, so as to accurately detect the position of the target individual in each frame. Therefore, in the basic model of the present invention, a module based on the attention mechanism that can extract high-quality feature maps is proposed and added to the feature extraction part of the basic model of the present invention. Figure 3 As shown, in the improved feature extraction part, the pre-set feature template will first extract relevant features from the input video frame to generate a feature map. Among them, the feature template of this part is a part of the area obtained in the input video frame, and this area will be used as a convolution kernel to perform convolution operation with the video frame, and then generate a feature map. Then, the feature map extracted by the feature template will be input into the new attention mechanism-based module proposed in the present invention to achieve further feature extraction and optimization. After the module generates a high-quality feature map, the output feature map will be input into the Gaussian mixture model to identify the position of each target individual in the video frame. Finally, the Gaussian distribution feature map containing the target individual position information output from the Gaussian mixture model will be input into the motion trend prediction algorithm (T2) to track the moving target.

[0032] In the attention mechanism module of the present invention, the algorithm flow is as follows Figure 4 First, the feature map F1 generated by the feature template and the specific area features selected in F1 (i.e. Figure 4 The selected region features in R1 will be convolved in the first attention mechanism processing (i.e., one layer of convolution operation, Conv), and then the result of the convolution operation will be normalized. After the normalization process, the feature map f1 will be output as the result of the first attention mechanism operation. Since the feature map F1 contains a lot of detailed information of the original video frame, in order to prevent the loss of this detailed information, the present invention uses an effective feature fusion method to add this detailed information to f1. The implementation formula is as follows:

[0033] F2=(F1*α1+f1*β1)*F1

[0034] Among them, the feature map F2 represents the feature map obtained after feature fusion; α1 and β1 represent the coefficients in the weighted average calculation formula. Figure 4 middle, Represents weighted average operation; Represents a multiplication operation.

[0035] After obtaining the feature map F2, the feature map will be processed by the second attention mechanism operation. Similar to the first attention mechanism operation, in the second attention mechanism operation, the feature map F2 will follow the specific area features selected in F2 (i.e. Figure 4 A convolution operation is performed on the selected region features (R2) in the image. The result of the convolution operation is normalized. After normalization, the feature map f2 is output from the second attention mechanism operation.

[0036] Finally, in order to use the detailed information of the target individual in the original video frame to further optimize the feature map f2, the following formula is used:

[0037] F3=F1*α2+f2*β2

[0038] Among them, F3 represents the final output feature map of the attention mechanism module, which contains high-quality semantic features and detailed information of the target individual. α2 and β2 represent the coefficients in the weighted average operation formula. Finally, F3 will be received by the Gaussian mixture model and used to identify the position of the target individual.

[0039] This paper uses an improved attention-based module for dense crowd motion tracking tasks. This improved attention module can extract high-level features such as semantic information while also effectively optimizing the extracted features, thereby preventing the loss of important information in the original input data during feature extraction. By incorporating this attention module into the basic model for dense crowd motion tracking, higher-quality features of target individuals can be generated, helping the model to more accurately identify the location of each target individual and thus better predict the target individual's motion trends in a video sequence.

[0040] In order to verify the performance of the present invention, the present invention was tested on a dataset in a pedestrian road scene. The accuracy of the results obtained by the model with the attention mechanism module (the model of the present invention) was higher than that of the model without the attention mechanism module (as shown in Table 1).

[0041] Table 1: Ablation experiment data: Data obtained from ablation experiments on a model with an attention mechanism module (model of the present invention) and a model without an attention mechanism module.

[0042] Accuracy (%) Model without attention mechanism module 57.63 Model with attention mechanism module (model of the present invention) 59.98

[0043] The above description is the best embodiment based on the concept and working principle of the invention. The above embodiment should not be understood as limiting the scope of protection of the present claims. Other embodiments and combinations of implementations of the present invention are also within the scope of protection of the present invention.

Claims

1. A method for tracking crowd movement in pedestrian scenes based on attention feature optimization, characterized in that: A video frame containing a pedestrian crossing the road is received by the feature extraction part, and the pre-set feature The feature template first extracts relevant features from the input video frame and generates a feature map; The feature map extracted by the feature template is input into the attention mechanism-based module to achieve further feature extraction and optimization; In this attention-based module, the feature map generated by the feature template ,exist Select specific regional characteristics , and In the first attention mechanism processing, a convolution operation is performed, that is, a layer of convolution operation, and the result of the convolution operation is normalized; After normalization, as the result of the first attention mechanism operation, the feature map is output, because the feature map contains a lot of detailed information of the original video frame, so in order to prevent the loss of these detailed information, the attention mechanism-based module adds these detailed information to the The implementation formula is as follows: Among them, the feature map Represents the feature map obtained after feature fusion; and Represents the coefficient in the weighted average calculation formula; When obtaining the feature map After that, the feature map is processed by the second attention mechanism operation. Similar to the first attention mechanism operation, in the second attention mechanism operation, the feature map Follow Specific regional characteristics selected Perform a convolution operation and normalize the result of the convolution operation. After normalization, the feature map Output from the second attention mechanism operation; In order to use the detailed information of the target individual in the original video frame to further optimize the feature map , which is achieved through the following formula: in, Represents the final output feature map of the attention mechanism module, which contains high-quality semantic features and detailed information of the target individual. and represents the coefficient in the weighted average calculation formula, Received by the Gaussian mixture model and used to identify the location of the target individual; Finally, the Gaussian distribution feature map containing the target individual position information output from the Gaussian mixture model is input into the operation Motion trend prediction algorithm is used to track moving targets.

2. The method for tracking crowd movement in pedestrian scenes based on attention feature optimization according to claim 1 is characterized in that: While extracting semantic feature information, the extracted features are optimized to prevent the loss of important information in the original input data. The attention mechanism-based module is added to the basic model for dense crowd motion tracking to produce higher-quality features of target individuals, helping the model to more accurately identify the position of each target individual and predict the target individual's motion trend in a video sequence.

Images