A method for dense crowd motion tracking based on multiple attention modules

Abstract

A method for tracking the movement of dense crowds based on multiple attention modules includes the following steps: S1. Selecting a basic model: A dense crowd tracking framework that completes feature extraction tasks using feature templates and Gaussian mixture models is selected as the basic model; S2. Building a multiple attention module within the feature extraction portion of the basic model (i.e., the feature template) to optimize the features extracted by the feature template, thereby generating high-quality attention features; and S3. Improving the basic model using the multiple attention modules. The method for tracking the movement of dense crowds based on multiple attention modules of the present invention can obtain the precise location of a target individual, thereby improving the accuracy of detecting the target individual's location.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and more specifically relates to a method for tracking dense crowd motion based on multiple attention modules. Background Art

[0002] As a key research area in computer vision, object motion tracking in dense crowds poses a significant challenge not only in academia but also holds significant application value in industry. The primary approach to addressing this problem is to first feed a video frame containing a dense crowd into a feature extraction algorithm. The algorithm then detects the position of each individual based on the extracted pedestrian features. Finally, the motion trajectory of each individual is identified within a continuous video sequence.

[0003] In the past, many different approaches have been used to address the problem of dense crowd tracking. Some models extract low-level features such as color and boundaries from video frames to determine the location of the target individual. Other models have been developed to extract high-level features such as semantic and temporal information from the target individual within the video frame, thereby better tracking the target individual's motion trajectory within the video. However, with the increasing density of crowds in many real-life settings, traditional feature extraction models are no longer sufficient to accurately track the target individual in dense crowds. In this context, deep learning models, due to their powerful feature extraction capabilities, have been adopted to address the challenge of dense crowd tracking. However, deep learning models not only require high-performance hardware to run, but also have long processing times. In practical applications, data processing models must process data in real time. Therefore, while deep learning models can achieve high accuracy in tasks such as object detection, they are not suitable for practical applications such as dense crowd tracking.

[0004] At present, the attention mechanism is widely used in the field of computer vision due to its effective feature extraction capability to obtain target features.

[0005] In the original model used for dense crowd motion tracking, the process first uses a feature template as a convolution kernel and performs a convolution operation on the input video frame to generate a feature map. Next, a specific area in the feature map is extracted and calculated with the Gaussian kernel in the Gaussian mixture model to calculate the location of the target individual. Once the positions of all target individuals are predicted, the motion trend algorithm predicts their motion trajectory based on the target individual's position in consecutive frames.

[0006] However, the current major problem and the difficulty in solving it is that, during the feature extraction step, simply convolving the feature template with the input video frame is insufficient to obtain high-quality features. This approach can easily lead to misidentification, such as missing the features of a target individual and mistaking them for background.

[0007] Therefore, a method with high detection accuracy suitable for dense crowd motion tracking is needed. Summary of the Invention

[0008] The present invention provides a method for tracking the movement of dense crowds based on multiple attention modules, which can obtain the precise position of the target individual and improve the accuracy of detecting the target individual's position.

[0009] The technical solutions of the present invention are as follows:

[0010] A method for tracking the movement of dense crowds based on a multi-attention module includes the following steps: S1. selecting a basic model: the basic model includes a feature template and a Gaussian mixture model; S2. building a multi-attention module; S3. using the multi-attention module to improve the basic model.

[0011] Preferably, in the above-mentioned method for dense crowd motion tracking based on multiple attention modules, in step S2, two attention mechanism operations are set in the multiple attention module: the first attention mechanism operation is used to further extract the features of the target individual; the second attention mechanism operation is used to fuse the feature map input to the multiple attention module with the feature map of the optimized first attention mechanism operation.

[0012] Preferably, in the above-mentioned method for dense crowd motion tracking based on multiple attention modules, step S3 further includes adding the constructed multiple attention modules to the feature extraction part of the basic model.

[0013] Preferably, in the above-mentioned method for dense crowd motion tracking based on multiple attention modules, in step S3, the feature template obtains features from the input video frame and generates a feature map, and the feature map is input into the above-mentioned multiple attention module to further complete the feature extraction and optimization tasks to generate an optimized feature map; the optimized feature map output by the multiple attention module is input into the crowd motion trajectory prediction part for algorithm processing, and the result map containing the motion trajectory of the target individual is output.

[0014] Preferably, in the above-mentioned method for dense crowd motion tracking based on multiple attention modules, the optimized feature map output by the multiple attention modules is input into a Gaussian mixture model to obtain a Gaussian distribution feature map containing the target individual position. The feature map is output from the feature extraction part as an optimized output result and transmitted to the crowd motion trajectory prediction part.

[0015] According to the technical solution of the present invention, the beneficial effects produced are:

[0016] Compared to existing algorithms that rely on extracting low-level features such as color or outline, this invention incorporates multiple attention modules to optimize the features extracted by the feature template, thereby generating high-quality attention features. These attention features enable the Gaussian mixture model to better detect the location of the target individual, allowing the motion trend prediction algorithm to more accurately predict the target individual's trajectory.

[0017] Compared with existing frameworks built by deep learning models, the present invention can obtain video frame features and output results at a higher speed. Due to the large amount of computation required by deep learning, although it can output highly accurate results in many recognition and detection tasks, it takes a long time to process data, making it unsuitable for application in real-world scenarios. However, the present invention takes a shorter time to process data, making it more suitable for application in real-world scenarios.

[0018] In order to better understand and illustrate the concept, working principle and effect of the present invention, the present invention is described in detail below with reference to the accompanying drawings and through specific embodiments: BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the specific implementation or the description of the prior art.

[0020] Figure 1 is a flow chart of a method for dense crowd motion tracking based on multiple attention modules of the present invention;

[0021] Figure 2 It is a flow chart of the algorithm involved in the method of the present invention;

[0022] Figure 3 This is a flowchart of the feature extraction part including the multi-attention module;

[0023] Figure 4 It is a schematic diagram of the two-attention mechanism operation of the multi-attention module involved in the method of the present invention. DETAILED DESCRIPTION

[0024] To make the purpose, technical methods and advantages of the present invention more clear, the following is a further detailed description with reference to the accompanying drawings and specific examples. These examples are merely illustrative and not intended to limit the present invention.

[0025] In order to apply the multi-attention module to the task of motion tracking in dense crowds, the present invention adds a multi-attention module to the feature extraction part on the basis of a framework model that combines feature templates, Gaussian mixture models and motion trend prediction algorithms to improve the quality of features extracted by feature templates, thereby generating high-quality feature maps. By inputting these high-quality feature maps into the Gaussian mixture model, the accurate position of the target individual in the dense crowd can be obtained, and these accurate positions help to improve the accuracy of motion trend prediction. By using a module containing multiple attention mechanisms, more reliable pedestrian features are extracted in the video frame. Through this strategy, the shortcoming that traditional methods such as feature templates and Gaussian mixture models can only extract low-quality features can be improved, thereby better identifying the position of each target in the dense crowd, and then completing the motion tracking task.

[0026] Figure 1 is a flow chart of the method for dense crowd motion tracking based on multiple attention modules of the present invention, such as Figure 1 As shown, the following steps are included:

[0027] S1. Selecting a Basic Model: The basic model includes a feature template and a Gaussian mixture model. To enable the application of this invention in real-world scenarios involving dense crowds, the present invention selected a dense crowd tracking framework that uses feature templates and a Gaussian mixture model to perform feature extraction as the basic model. This model was also chosen because it has a low computational load, resulting in high data processing efficiency and low latency in real-world scenarios.

[0028] S2. Build a multi-attention module. Because the attention mechanism can extract high-level features such as semantic information when processing visual data, building a multi-attention module that utilizes the attention mechanism can more efficiently extract more high-quality attention features from visual data. Furthermore, by combining the high-quality attention features extracted by the attention mechanism with the feature maps extracted by the feature template, this multi-attention module can effectively preserve important details in the visual data, making it more convenient to detect the location of target individuals in dense crowds.

[0029] Specifically, in the multi-attention module, two attention mechanism operations are set up to optimize features: the first attention mechanism operation is used to further extract the features of the target individual; the second attention mechanism operation is to merge the feature map input to the multi-attention module with the optimized feature map of the first attention mechanism operation, thereby preventing the loss of important information in the original feature map.

[0030] S3. Improve the basic model using a multi-attention module. To extract high-quality features, the established multi-attention module is added to the feature extraction portion of the basic model. Once the feature template extracts features from the input video frame and generates a feature map, this feature map is fed into the multi-attention module to further perform feature extraction and optimization. The optimized feature map output by the multi-attention module is then fed into a Gaussian mixture model to identify the location of the target individual.

[0031] The implementation steps of the method of the present invention are as follows Figure 2 As shown. First, a video frame containing a dense crowd scene will be input into the feature extraction part (T1) in the framework of the present invention and an optimized feature map will be generated; then, the feature map will be input into the crowd motion trajectory prediction part (T2) to track the motion trajectory of the target individual. In the crowd motion trajectory prediction part, the neighboring motion concurrent algorithm is adopted. After being processed by the algorithm, the result map containing the motion trajectory of the target individual will be output from the framework. In the task of predicting the motion trajectory of the target individual, it is very critical whether the position of the target individual detected in each frame is accurate; and whether the position detection of the target individual is accurate or not is still determined by the quality of the extracted features. Therefore, in order to obtain a more reliable dense crowd feature map, the present invention adds a multi-attention module in the feature extraction part that is specifically used to obtain high-quality feature maps.

[0032] Figure 3 This is a flowchart of the feature extraction part including the multi-attention module.

[0033] In the feature extraction section, a pre-set feature template is first used to perform feature extraction on the input video frame. The feature template in this section is a region extracted from the input video frame, which is then used as a convolution kernel to perform a convolution operation with the video frame to extract the features of the video frame. Once the features of the video frame are extracted, the feature map containing the video frame features is input into the multi-attention module for further feature extraction. After processing by the multi-attention module, the feature map containing high-quality features is input into the Gaussian mixture model to obtain a Gaussian distribution feature map containing the target individual's location. This feature map is output from the feature extraction section as an optimized output and transmitted to the crowd motion trajectory prediction section.

[0034] In order to obtain an optimized high-quality feature map, the multi-attention module of the present invention uses two attention mechanism operations, the specific details of which are as follows: Figure 4As shown in . First, the feature map extracted by the feature template will be input into the first attention mechanism operation in the multi-attention module, and the input feature map is defined as F1. In F1, the feature of a specific area (selected area feature) R1 is selected; then R1 will be used as a convolution kernel to perform a convolution operation (conv) with F1 to obtain more semantic information and common information of dense crowds. The output of the convolution operation will be normalized to generate an attention feature map. The attention feature map will undergo a feature fusion operation with the feature map F1 to add the detailed features in F1 to the attention feature map. Among them, Represents the weighted average operation. The weighted average operation formula for feature fusion operation after the first attention mechanism is completed can be expressed as:

[0035] F2=F1*α1+F a1 *β1

[0036] Among them F a1 represents the attention feature map output by the first attention mechanism operation, F2 represents the feature map output by the feature fusion operation, and α1 and β1 represent the coefficients in the weighted average operation formula.

[0037] The feature map F2 obtained after feature fusion is concatenated with the feature map F1 in the channel dimension. The original feature maps F1 and F2 each have 1 channel, but after the concatenation operation, the number of channels in the concatenated feature map is 2. The resulting feature map is then fed into the second attention mechanism operation for further feature extraction.

[0038] In the second attention mechanism operation, the feature of a specific region (selected region feature) R2 is first selected from the feature map F2. The length and width of R2 are the same as those of R1, and the position of R2 in F2 is the same as that of R1 in F1. Then, R1 and R2 are used as convolution kernels to perform a convolution operation on the feature map obtained by concatenating F1 and F2. The convolution operation formula of F1 and F2 with R1 and R2 is as follows:

[0039] F a2 =F1⊙R1+F2⊙R2

[0040] Among them, F a2 represents the feature map output from the convolution layer, and ⊙ represents the convolution operation. a2 It will be normalized to get F a2 ′. F a2 ′ will be output as the attention feature map generated by the second attention mechanism operation. Finally, in order to effectively combine the high-quality features of the attention feature map with the detailed information in the video frame features, F a2′ and the feature map F1 are fused again, and the formula is as follows:

[0041] F3=F1*α2+F a2 ′*β2

[0042] Where F3 represents the feature map after feature fusion, and α2 and β2 represent the coefficients in the calculation formula. Finally, F3 is output from the multi-attention module as the optimized feature map and transmitted to the Gaussian mixture model for target individual location recognition.

[0043] To efficiently extract target individual features using an attention mechanism for dense crowd motion tracking, the present invention utilizes a multi-attention module to enhance feature extraction and optimize the feature maps obtained from video frames. This allows the model to extract more semantic information and high-level features such as common information about the target individual from the feature maps, thereby improving the accuracy of detecting the target individual's position. This in turn allows for the acquisition of high-quality target individual features, enabling better detection of the target individual's position and more accurate and efficient prediction of the target's motion trajectory. In addition to extracting high-level features, the multi-attention module effectively integrates the feature maps containing these high-level features with the feature maps initially extracted using the feature template through the attention mechanism. This strategy ensures that the feature maps contain high-level features while preserving the detailed information in the original feature maps. The optimized, high-quality feature maps output from the multi-attention module, after computational processing using a Gaussian mixture model, can more accurately identify the target individual's position within the video frame. The multi-attention module employed in the present invention demonstrates excellent performance in tests on dense crowd datasets, achieving higher accuracy than a framework without the multi-attention module.

[0044] In order to verify the performance of the present invention, the present invention was tested on a dataset in a dense crowd scene in a square. The accuracy of the results obtained by adding a multi-attention module (the framework of the present invention) was higher than that of the results obtained by not adding a multi-attention module (as shown in Table 1).

[0045] Table 1: Ablation experiment data: Data obtained by ablation experiments on the framework with multiple attention modules (the framework of the present invention) and the framework without multiple attention modules

[0046] Accuracy (%) Model without multi-attention module 77.66 Model with multiple attention modules (model of the present invention) 79.69

[0047] The above description is the best embodiment based on the concept and working principle of the invention. The above embodiment should not be understood as limiting the scope of protection of the present claims. Other embodiments and combinations of implementations of the present invention are also within the scope of protection of the present invention.

Claims

1. A method for tracking dense crowd motion based on multiple attention modules, characterized in that: A video frame containing a dense crowd scene is input to the feature extraction part; In the feature extraction part, a pre-set feature template is first used to extract features from the input video frame. After the features of the video frame are extracted, the feature map containing the video frame features is input into the multi-attention module for further feature extraction. The multi-attention module adopts two attention mechanisms. The feature map extracted by the feature template is first input into the first attention mechanism operation in the multi-attention module, and the input feature map is defined as ,exist Select the features of a specific area ;Should As the convolution kernel and A convolution operation is performed to obtain more semantic information and common information of dense crowds; the output of the convolution operation is normalized to generate an attention feature map, which is consistent with the feature map Perform a feature fusion operation to The detailed features in are added to the attention feature map. After the first attention mechanism ends, the weighted average operation formula for feature fusion operation can be expressed as: in represents the attention feature map output by the first attention mechanism operation, Represents the feature map output by the feature fusion operation, and Represents the coefficient in the weighted average calculation formula; Feature map obtained after feature fusion With feature map The concatenation operation is performed on the channel dimension, wherein the feature map With the feature map The number of channels of each is 1. After the splicing operation, the number of channels of the spliced ​​feature map is 2. The feature map obtained after the splicing operation is input into the second attention mechanism operation for further feature extraction operation; In the second attention mechanism operation, from the feature map Select the features of a specific area , Length and width of The same, and exist The position in exist The position in is the same; and As the convolution kernel, and The concatenated feature map is subjected to convolution operation, where and and and The convolution operation formula is as follows: in, represents the feature map output from the convolutional layer, Represents the convolution operation; After normalization, we get , The attention feature map generated as the second attention mechanism operation is output; in order to effectively combine the high-quality features of the attention feature map with the detailed information in the video frame features, With feature map Perform feature fusion again, and the formula is as follows: in Represents the feature map after feature fusion, and Indicates the coefficient in the operation formula; at last, The optimized feature map is output from the multi-attention module and transmitted to the Gaussian mixture model for position recognition of the target individual; After being processed by the multi-attention module, the feature map containing high-quality features is input into the Gaussian mixture model to obtain a Gaussian distribution feature map containing the target individual position. The Gaussian distribution feature map is output from the feature extraction part as an optimized output result and transmitted to the crowd motion trajectory prediction part.

Images