Lightweight Behavior Recognition Method and System Based on Feature Compression

Abstract

This invention provides a lightweight behavior recognition method and system based on feature compression, belonging to the field of computer recognition technology. The method includes: acquiring a video sequence to be recognized; processing the acquired video sequence using a pre-trained recognition model to obtain behavior recognition results in the video; wherein the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network including at least one bottleneck structure, and the bottleneck structure including two sequentially connected feature compression modules. This invention can effectively reduce the computational and parameter count of convolutional neural networks while efficiently extracting information from feature maps; it can activate spatiotemporal features through an attention mechanism, making the network focus more on valuable feature information, and can easily model motion information between video frames; it has a low parameter and computational cost.

Description

Technical Field

[0001] This invention relates to the field of computer recognition technology, specifically to a lightweight behavior recognition method and system based on feature compression. Background Technology

[0002] Human behavior recognition primarily addresses the challenge of analyzing and processing video sequences captured by cameras to enable computers to identify ongoing human actions and behaviors within the video. The goal is to identify ongoing activities as early and accurately as possible and infer their category. Behavior recognition methods are particularly important for surveillance systems that need to prevent crime and dangerous incidents. The difficulty of behavior recognition lies in the fact that, because it processes video, it involves an additional temporal dimension compared to image processing. Therefore, efficiently processing this extra temporal information in video becomes crucial for behavior recognition. Furthermore, behavior recognition is susceptible to external interference, such as variations in the posture, appearance, and size of the observed object; occlusion and complex backgrounds can obstruct the detection of the object; and environmental changes (such as lighting) can further complicate recognition. Moreover, current research in behavior recognition mainly focuses on benchmark datasets, and robust solutions for real-world applications are still lacking.

[0003] Since the emergence of deep learning, more and more convolutional neural network models have been applied to behavior recognition. Based on different methods, they can be divided into three main categories: two-stream network models, for example, the main idea of ​​which is to use two CNN branches to capture motion information and spatial information respectively, and then fuse them. However, the disadvantage is that the computational cost of the two-branch network and optical flow is too high; 3D convolutional neural networks, the main idea of ​​which is to use 3D convolutional kernels to process video to capture spatiotemporal information in the video. The significant disadvantage of 3D convolutional neural networks is that their parameter count and computational cost are tens of times higher than those of equivalent 2D networks; in addition, there is a method that uses 2D convolutional neural networks to capture spatial information in video, and then uses other methods such as 1D convolution to capture temporal information, and finally fuses the two information. However, this method provides relatively limited improvement in accuracy.

[0004] Most existing solutions achieve lightweighting by modifying the network structure, such as the EfficientNet and X3D series, which use neural network architecture search techniques to find the optimal network structure for a given dataset. While this can bring significant performance improvements, the search process itself consumes substantial computing resources, and the method lacks portability, potentially requiring a re-search when faced with different datasets. Other methods abandon the 3D convolutional neural network approach, using a 2D convolutional neural network with an additional temporal modeling module for action recognition. A similar approach is TSM, which uses a 2D convolutional neural network as the feature extraction backbone and incorporates an additional time-shifting module. This module supplements the 2D network with temporal information by shifting the feature map channels along the time direction. The specific structure is as follows: Figure 1 As shown.

[0005] Existing action recognition methods struggle to meet real-time processing requirements, commonly exhibiting issues such as: SlowFast uses two 3DCNN branches to process fast and slow video streams at the input level, fusing the outputs of the two branches through lateral connections. While it effectively extracts spatiotemporal context information, its execution time is far from satisfactory. Furthermore, X3D utilizes neural network architecture search techniques to progressively expand the 2D image classification network across multiple network dimensions, resulting in a series of efficient network families. Nevertheless, 3DCNN-based methods still involve a large number of parameters. Driven by the development of 2DCNNs, most existing methods handle sparsely sampled frame segments in videos. However, traditional 2D frameworks lack the ability to capture temporal information. To model long-term temporal information in videos, recent research has incorporated different temporal modules into 2DCNNs using channel attention. For example, STM uses a spatiotemporal module to capture spatiotemporal information in video clips and uses a channel-associated motion module to extract motion information between adjacent frames. Additionally, TEINet introduces a TEI module to learn temporal and motion features by processing channel-related and temporal interaction information. Although these 2D methods are more lightweight than 3D networks, they still fall short in modeling temporal information. Summary of the Invention

[0006] The purpose of this invention is to provide a lightweight behavior recognition method and system based on feature compression, so as to solve at least one of the technical problems existing in the background art.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] On one hand, the present invention provides a lightweight behavior recognition method based on feature compression, comprising:

[0009] Obtain the video sequence to be identified;

[0010] The acquired video sequence to be identified is processed using a pre-trained recognition model to obtain the behavior recognition result in the video; wherein, the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network is used to extract the spatiotemporal features of the video frame image, and the classification network is used to identify the behavior in the video frame image based on the extracted spatiotemporal features; the feature extraction network includes at least one bottleneck structure, the bottleneck structure includes two feature compression modules connected in sequence; wherein, the feature compression module is used to: divide the channels of the feature map into high frequency and low frequency, and compress the spatial resolution of the low frequency channel.

[0011] Optionally, to enhance the ability to capture features, a CBAM attention mechanism is used between the two feature compression modules to activate spatial and channel features; the CBAM attention mechanism is supplemented with a spatiotemporal information activation module to activate and model temporal features of spatiotemporal information in the video; the CBAM attention mechanism is supplemented with a motion information modeling module to represent motion information features between video frames.

[0012] Optionally, in the feature compression module, the original input feature map is divided into a high-frequency part and a low-frequency part by a first convolution; in order to exchange information between the high-frequency part and the low-frequency part, there are four paths in the convolution: high-frequency to high-frequency path, low-frequency to low-frequency path, high-frequency to low-frequency path that reduces the spatial resolution of the feature map by using average pooling, and low-frequency to high-frequency path that increases the spatial resolution of the feature map by using upsampling.

[0013] Optionally, the first convolution is followed by a second convolution. The second convolution uses depthwise convolution to process high-frequency and low-frequency features separately, retaining only high-frequency to high-frequency paths and low-frequency to low-frequency paths. After the second convolution, a feature map similar to the original features is generated. After the second convolution, the original features and the generated features are concatenated along the channel dimension and then subjected to channel shuffling to enhance the information interaction between the feature maps, thus obtaining the final output features.

[0014] Optionally, in the information activation module, attention feature maps related to channels and spatial attention feature maps are obtained, and the attention feature maps related to channels and spatial attention feature maps are fused to output a spatiotemporal attention feature map. Then, the original features and the spatiotemporal attention feature map are added together through residual connection.

[0015] Optionally, in the motion information modeling module, the dimensionality of the channels is reduced by a point convolution, and then the difference between two adjacent frames is obtained by subtracting them. The differences are connected in the time dimension. Since the differences are subtracted pairwise, there will be gaps at the end. Zeros are filled into the last gap. Then, spatial pooling is used to obtain spatially correlated feature maps. A point convolution is used to expand the channels to keep the number of channels consistent with the input feature channels. Attention weights are generated for the channels through a sigmoid layer. Then, the attention weights are used to activate the motion-sensitive channels to obtain motion attention feature maps. Finally, the original features and motion attention feature maps are added together through residual connections.

[0016] Secondly, the present invention provides a lightweight behavior recognition system based on feature compression, comprising:

[0017] The acquisition module is used to acquire the video sequence to be identified;

[0018] A recognition module is used to process the acquired video sequence to be recognized using a pre-trained recognition model to obtain the behavior recognition result in the video; wherein, the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network is used to extract the spatiotemporal features of the video frame image; the classification network is used to recognize the behavior in the video frame image based on the extracted spatiotemporal features; the feature extraction network includes at least one bottleneck structure, the bottleneck structure includes two feature compression modules connected in sequence; wherein, the feature compression module is used to: divide the channels of the feature map into high frequency and low frequency, and compress the spatial resolution of the low frequency channel.

[0019] Thirdly, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the lightweight behavior recognition method based on feature compression as described above.

[0020] Fourthly, the present invention provides a computer program product, including a computer program that, when run on one or more processors, is used to implement the lightweight behavior recognition method based on feature compression as described above.

[0021] Fifthly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the lightweight behavior recognition method based on feature compression as described above.

[0022] The beneficial effects of this invention are: it can effectively reduce the computational and parameter count of convolutional neural networks, while efficiently extracting information from feature maps; it can activate spatiotemporal features through an attention mechanism, making the network focus more on valuable feature information, and can easily model motion information between video frames; it has a low parameter count and computational load.

[0023] The advantages of additional aspects of the invention will be set forth more clearly in the following description or will be learned by practice of the invention. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a diagram of the time-off network structure in the prior art.

[0026] Figure 2 This is a diagram of the lightweight behavior recognition network structure based on feature compression as described in an embodiment of the present invention.

[0027] Figure 3 This is a structural diagram of the feature compression module according to an embodiment of the present invention.

[0028] Figure 4 This is a bottleneck structure diagram according to an embodiment of the present invention. Wherein, Figure 4 (a) represents convolutions without additional depthwise convolutions. Figure 4 (b) is for convolution with additional depth.

[0029] Figure 5 This is a structural diagram of the information activation module according to an embodiment of the present invention.

[0030] Figure 6 This is a structural diagram of the motion information modeling module described in an embodiment of the present invention.

[0031] Figure 7 This is a flowchart of a lightweight behavior recognition method based on feature compression as described in an embodiment of the present invention. Detailed Implementation

[0032] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0033] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0034] It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as here.

[0035] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0036] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0037] To facilitate understanding of the present invention, the present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0038] Those skilled in the art should understand that the accompanying drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily essential for implementing the present invention.

[0039] Example 1

[0040] In this embodiment 1, a lightweight behavior recognition system based on feature compression is first provided, including:

[0041] The acquisition module is used to acquire the video sequence to be identified;

[0042] A recognition module is used to process the acquired video sequence to be recognized using a pre-trained recognition model to obtain the behavior recognition result in the video; wherein, the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network is used to extract the spatiotemporal features of the video frame image; the classification network is used to recognize the behavior in the video frame image based on the extracted spatiotemporal features; the feature extraction network includes at least one bottleneck structure, the bottleneck structure includes two feature compression modules connected in sequence; wherein, the feature compression module is used to: divide the channels of the feature map into high frequency and low frequency, and compress the spatial resolution of the low frequency channel.

[0043] In this embodiment 1, the above-described system is used to implement a lightweight behavior recognition method based on feature compression, including:

[0044] The acquisition module is used to obtain the video sequence to be identified;

[0045] Using a recognition module, the acquired video sequence to be recognized is processed based on a pre-trained recognition model to obtain the behavior recognition result in the video; wherein, the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network is used to extract the spatiotemporal features of the video frame image, and the classification network is used to recognize the behavior in the video frame image based on the extracted spatiotemporal features; the feature extraction network includes at least one bottleneck structure, the bottleneck structure includes two feature compression modules connected in sequence; wherein, the feature compression module is used to: divide the channels of the feature map into high frequency and low frequency, and compress the spatial resolution of the low frequency channel.

[0046] In this embodiment 1, in order to enhance the ability to capture features, a CBAM attention mechanism is used between the two feature compression modules to activate spatial and channel features; the CBAM attention mechanism is equipped with a spatiotemporal information activation module to activate and model the spatiotemporal information in the video as temporal features; the CBAM attention mechanism is also equipped with a motion information modeling module to represent the motion information features between video frames.

[0047] In this embodiment 1, in the feature compression module, the original input feature map is divided into high-frequency and low-frequency parts by a first convolution. To exchange information between the high-frequency and low-frequency parts, the convolution has four paths: a high-frequency to high-frequency path, a low-frequency to low-frequency path, a high-frequency to low-frequency path that reduces the spatial resolution of the feature map by using average pooling, and a low-frequency to high-frequency path that increases the spatial resolution of the feature map by upsampling. The first convolution is followed by a second convolution, which uses depthwise convolution to process the high-frequency and low-frequency features separately, retaining only the high-frequency to high-frequency and low-frequency to low-frequency paths. After the second convolution, a feature map similar to the original features is generated. After the second convolution, the original features and the generated features are concatenated along the channel dimension, and then subjected to channel shuffling to enhance the information interaction between the feature maps, resulting in the final output features.

[0048] In the information activation module, attention feature maps related to channels and spatial attention feature maps are obtained. The attention feature maps related to channels and spatial attention feature maps are fused to output a spatiotemporal attention feature map. Then, the original features and the spatiotemporal attention feature map are added together through residual connection.

[0049] In the motion information modeling module, a point convolution is used to reduce the dimensionality of the channels, and then the difference between two adjacent frames is obtained by subtracting them. The differences are then connected in the time dimension. Since the differences are subtracted pairwise, gaps will appear at the end. Zeros are filled into the last gap. Then, spatial pooling is used to obtain spatially correlated feature maps. A point convolution is used to expand the channels to keep the number of channels consistent with the input feature channels. Attention weights are generated for the channels through a sigmoid layer. Then, the attention weights are used to activate the motion-sensitive channels to obtain motion attention feature maps. Finally, the original features and motion attention feature maps are added together through residual connections.

[0050] Example 2

[0051] like Figure 2 As shown in this embodiment 2, a lightweight behavior recognition network based on a feature compression module is provided. The network mainly includes the following modules: feature compression module, spatiotemporal information activation module, and motion information modeling module.

[0052] like Figure 2 As shown, the lightweight behavior recognition network based on the feature compression module has a workflow consisting of several stages, each composed of stacked basic bottleneck structures. Each bottleneck structure comprises a basic feature compression module, a spatiotemporal activation module, and a motion modeling module. The input video sequence is first converted into feature maps by a convolutional layer, then spatiotemporal features are gradually extracted through a series of bottleneck structures, and finally, the correct classification result is obtained after passing through a classifier.

[0053] Typically, CNNs extract redundant feature maps containing rich semantic information to improve the performance of downstream tasks. However, for resource-constrained devices, it is necessary to balance model performance with the richness of redundant information. Therefore, in this embodiment 2, a feature compression module is designed to reduce spatial and channel redundancy in the feature maps. Specifically, the feature map can be divided into two parts: high-frequency and low-frequency. The high-frequency part represents more dramatic changes in the image, such as edges, while the low-frequency part describes monotonously changing parts of the image, such as the background. Since low-frequency features represent monotonically changing parts of the image, spatial information can be shared between adjacent locations while compressing the spatial resolution of the image, without sacrificing model accuracy, thereby reducing model overhead.

[0054] In this embodiment 2, the feature compression module first divides the feature map channels into high-frequency and low-frequency channels with a high-to-low frequency ratio of 7:1. Then, it compresses the spatial resolution of the low-frequency channels by half to reduce spatial redundancy. Furthermore, the feature maps of images generated by CNNs contain many similar pairs. Therefore, depthwise convolution can be used to generate similar feature maps. The generated feature maps can be viewed as replacing similar pairs in a cost-effective way, thereby reducing channel redundancy. The specific structure of the feature compression module is as follows: Figure 3 As shown.

[0055] Figure 3 In the given input feature map X∈R c×h×w Where c represents the number of channels in the feature map, and h×w represents the spatial resolution of the feature map. Convolution C p1 (i.e., the first convolution) divides X into high-frequency components X0. H and low-frequency part X L In order to exchange information between the high-frequency and low-frequency components, C... p1 There are four paths: H2H and L2L, representing high-frequency to high-frequency and low-frequency to low-frequency paths, respectively. Specifically, H2L uses average pooling to reduce the spatial resolution of the feature map to achieve the high-frequency to low-frequency path, while L2H uses upsampling to increase the spatial resolution of the feature map to achieve the low-frequency to high-frequency path. Convolution C d1 (That is, the second convolution) uses depthwise convolution to process high-frequency and low-frequency features separately, retaining only two paths: high-frequency to high-frequency and low-frequency to low-frequency. After convolution C d1 Then, the model will generate features Y similar to the original features. H1 Y L1 Similar feature maps Y H2 Y L2 Finally, the original features and the generated features are concatenated along the channel dimension to obtain the final output Y of each module. H Y LThen, a channel shuffling operation is performed to enhance the information exchange between feature maps.

[0056] Then, a bottleneck structure is built using feature compression modules. Each bottleneck structure consists of two basic feature compression modules, with an additional depthwise convolution used to process the feature map when stride=2. Notably, to enhance feature capture, a CBAM attention mechanism is used between the two feature compression modules. This attention mechanism effectively activates spatial and channel features. To reduce computational cost, this operation is only applied to a portion of the bottleneck structure. The specific settings of the bottleneck structure are as follows... Figure 4 As shown.

[0057] As mentioned earlier, traditional temporal feature modeling methods, such as 3D convolution, typically consume a large amount of resources, which is inconsistent with the requirements of real-time performance. The main purpose of the spatiotemporal information activation module is to activate the spatiotemporal information in the video in a simple way, thereby modeling temporal features without introducing excessive computational costs.

[0058] Mathematically, the overall input and output of the model are 4D: (N×T, C, H, W), where N represents the batch size, T represents the segment length, C represents the channel number, and H and W represent the height and width of the feature map, respectively. The original CBAM module uses spatial pooling and channel pooling to obtain channel awareness in both spatial and channel dimensions, but it lacks the ability to process temporal information. To address this issue, the information activation module designed in Example 2 uses an additional one-dimensional convolution to process temporal information. The result of the information activation module is as follows... Figure 5 As shown, the input dimensions are first expanded from three dimensions (C, H, W) to four dimensions (NT, C, H, W). First, average pooling and max pooling are used to eliminate spatial information. Then, a 1×1 convolution is used to control the number of channels, where r represents the channel reduction rate. An activation function is then used to obtain spatially relevant features. Third, the features are transformed into N, C, T, and a 1D convolution is used to process the temporal dimension T to capture temporal information. Next, the features are transformed back to their original dimensions, and a sigmoid layer is used to obtain attention weights, which are then weighted and added to the original feature map. After these operations, a channel-related attention feature map is obtained. Then, channel pooling and a 7×7 convolution are used in a similar manner to obtain a spatially relevant attention feature map. Finally, the output attention feature map not only relates to channels and space but also contains some temporal information. Finally, a residual connection is used to add the original features and the attention feature map together.

[0059] While the information activation module can stimulate spatiotemporal features, the network is still insufficient to capture short-term correlated motion features between frames. To alleviate this problem, previous researchers have often used optical flow to add additional motion information. Although optical flow always brings significant improvements to deep learning models, the extraction and storage of optical flow information consumes substantial computing resources. Therefore, in this embodiment 2, the purpose of using the inter-frame motion information modeling module is to represent inter-frame motion information using a simple method.

[0060] like Figure 6 As shown, the input sequence X v First, a pointwise convolution is used to reduce the channel dimension. Then, the difference between two adjacent frames is obtained by subtracting them. Next, these differences are concatenated along the time dimension. Since the differences are subtracted pairwise, a peacock pattern appears at the end. For ease of processing, zeros are simply filled into the last empty position.

[0061]

[0062] X represents v In the t-th frame, a spatiotemporal activation module is used, employing spatial pooling to obtain spatially relevant feature maps. Then, a point convolution is used to expand the channels, maintaining the same number as the input feature channels. Finally, a sigmoid layer generates attention weights for the channels, which can then be used to activate motion-sensitive channels. The final output also needs to be summed through residual connections.

[0063] Y m =X v Θσ(f(F m ))

[0064] To construct a lightweight action recognition network based on feature compression, spatiotemporal activation modules and motion modeling modules are inserted into the first three convolutions of each bottleneck, respectively. The entire model's video processing flow is as follows: Figure 7 As shown, the specific steps are as follows: First, sparse sampling is used to acquire video frame segments. The input video frame segments first pass through the first convolutional layer, which is a regular convolutional layer. Then, they pass through a series of bottleneck structures, where the number of channels gradually increases while the spatial resolution of the feature map gradually decreases. Specifically, in the first bottleneck of the first stage, the feature map is divided into high-frequency and low-frequency components. The last convolutional layer merges the low-frequency branch into the high-frequency branch to facilitate classification. Finally, the output feature map passes through a fully connected layer and a classifier to obtain the final output result.

[0065] In summary, in this embodiment 2, a feature compression module was designed to address the spatial redundancy of feature maps. This module focuses on compressing the model from the perspective of feature maps, effectively removing spatial and channel redundancy and reducing the number of parameters and computational load in the deep behavior recognition model. An efficient spatiotemporal information activation module was designed to encode information in the horizontal and vertical directions of the feature maps, capturing temporal features using 1D convolution and generating attention weights. This method is simpler and more efficient than other methods of capturing temporal information. An efficient motion information modeling module was also designed, using the differences between video frames to represent short-term motion information. This complements the spatiotemporal information activation module, significantly increasing model accuracy without incurring additional computational overhead. A reasonable combination of the feature compression module, spatiotemporal information activation module, and motion information modeling module constitutes a 2D CNN network for behavior recognition. This 2D network achieves recognition accuracy comparable to other 2D networks while significantly reducing computational cost and parameter count.

[0066] Example 3

[0067] Embodiment 3 of the present invention provides an electronic device, including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute a lightweight behavior recognition method based on feature compression, the method including the following steps:

[0068] Obtain the video sequence to be identified;

[0069] The acquired video sequence to be identified is processed using a pre-trained recognition model to obtain the behavior recognition result in the video; wherein, the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network is used to extract the spatiotemporal features of the video frame image, and the classification network is used to identify the behavior in the video frame image based on the extracted spatiotemporal features; the feature extraction network includes at least one bottleneck structure, the bottleneck structure includes two feature compression modules connected in sequence; wherein, the feature compression module is used to: divide the channels of the feature map into high frequency and low frequency, and compress the spatial resolution of the low frequency channel.

[0070] Example 4

[0071] Embodiment 4 of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements a lightweight behavior recognition method based on feature compression. The method includes the following steps:

[0072] Obtain the video sequence to be identified;

[0073] The acquired video sequence to be identified is processed using a pre-trained recognition model to obtain the behavior recognition result in the video; wherein, the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network is used to extract the spatiotemporal features of the video frame image, and the classification network is used to identify the behavior in the video frame image based on the extracted spatiotemporal features; the feature extraction network includes at least one bottleneck structure, the bottleneck structure includes two feature compression modules connected in sequence; wherein, the feature compression module is used to: divide the channels of the feature map into high frequency and low frequency, and compress the spatial resolution of the low frequency channel.

[0074] Example 5

[0075] Embodiment 5 of the present invention provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute a lightweight behavior recognition method based on feature compression, the method comprising the following steps:

[0076] Obtain the video sequence to be identified;

[0077] The acquired video sequence to be identified is processed using a pre-trained recognition model to obtain the behavior recognition result in the video; wherein, the pre-trained recognition model includes a feature extraction network and a classification network, the feature extraction network is used to extract the spatiotemporal features of the video frame image, and the classification network is used to identify the behavior in the video frame image based on the extracted spatiotemporal features; the feature extraction network includes at least one bottleneck structure, the bottleneck structure includes two feature compression modules connected in sequence; wherein, the feature compression module is used to: divide the channels of the feature map into high frequency and low frequency, and compress the spatial resolution of the low frequency channel.

[0078] In summary, the lightweight behavior recognition method and system based on feature compression described in this invention employs a feature compression module to address the spatial redundancy of feature maps. This module focuses on compressing the model from the perspective of the feature map, effectively reducing spatial and channel redundancy and minimizing the number of parameters and computational load in the deep behavior recognition model. An efficient spatiotemporal activation module is designed to encode information in the horizontal and vertical directions of the feature map, capturing temporal features using 1D convolution and generating attention weights, which is simpler and more efficient than other methods of capturing temporal information. An efficient motion information modeling module is also designed, using the differences between video frames to represent short-term motion information. This complements the spatiotemporal activation module, significantly increasing model accuracy without incurring additional computational overhead. A reasonable combination of the feature compression module, spatiotemporal activation module, and motion information modeling module constitutes a 2D CNN network for behavior recognition. This 2D network achieves recognition accuracy comparable to other 2D networks while significantly reducing computational cost and parameter count.

[0079] The method described in this invention, targeting complex urban scenarios, combines deep learning technology to design a lightweight human behavior recognition algorithm model. This model consumes fewer computing resources, processes video faster, and can be applied to monitoring, security, and other fields in smart city construction. It enhances the model's deployability and application scope, meets the needs of real-time video processing, and provides timely early warning and protection against various dangerous situations.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.

Claims

1. A lightweight behavior recognition method based on feature compression, characterized in that, include: Obtain the video sequence to be identified; The acquired video sequence to be identified is processed using a pre-trained recognition model to obtain the behavior recognition result in the video. The pre-trained recognition model includes a feature extraction network and a classification network. The feature extraction network is used to extract spatiotemporal features of video frame images. The classification network is used to identify behaviors in video frame images based on the extracted spatiotemporal features. The feature extraction network includes at least one bottleneck structure, which includes two sequentially connected feature compression modules. The feature compression module is used to: divide the channels of the feature map into high-frequency and low-frequency channels, and compress the spatial resolution of the low-frequency channels. To enhance the feature capture capability, a CBAM attention mechanism is used between the two feature compression modules to activate spatial and channel features. The CBAM attention mechanism includes a spatiotemporal information activation module for activating and modeling temporal features of the spatiotemporal information in the video. The M-attention mechanism incorporates a motion information modeling module to represent the motion information features between video frames. In the feature compression module, the original input feature map is divided into high-frequency and low-frequency components using a first convolution. To exchange information between the high-frequency and low-frequency components, this convolution has four paths: a high-frequency to high-frequency path, a low-frequency to low-frequency path, a high-frequency to low-frequency path that reduces the spatial resolution of the feature map using average pooling, and a low-frequency to high-frequency path that increases the spatial resolution of the feature map using upsampling. The first convolution is followed by a second convolution, which uses depthwise convolution to process the high-frequency and low-frequency features separately, retaining only the high-frequency to high-frequency and low-frequency to low-frequency paths. After the second convolution, a feature map similar to the original features is generated. After the second convolution, the original and generated features are concatenated along the channel dimension, and then subjected to channel shuffling to enhance the information interaction between the feature maps, resulting in the final output features.

2. The lightweight behavior recognition method based on feature compression according to claim 1, characterized in that, In the information activation module, attention feature maps related to channels and spatial attention feature maps are obtained. The attention feature maps related to channels and spatial attention feature maps are fused to output a spatiotemporal attention feature map. Then, the original features and the spatiotemporal attention feature map are added together through residual connection.

3. The lightweight behavior recognition method based on feature compression according to claim 2, characterized in that, In the motion information modeling module, a point convolution is used to reduce the dimensionality of the channels, and then the difference between two adjacent frames is obtained by subtracting them. The differences are then connected in the time dimension. Since the differences are subtracted pairwise, gaps will appear at the end. Zeros are filled into the last gap. Then, spatial pooling is used to obtain spatially correlated feature maps. A point convolution is used to expand the channels to keep the number of channels consistent with the input feature channels. Attention weights are generated for the channels through a sigmoid layer. Then, the attention weights are used to activate the motion-sensitive channels to obtain motion attention feature maps. Finally, the original features and motion attention feature maps are added together through residual connections.

4. A lightweight behavior recognition system based on feature compression, characterized in that, include: The acquisition module is used to acquire the video sequence to be identified; A recognition module is used to process the acquired video sequence to be recognized using a pre-trained recognition model to obtain the behavior recognition result in the video. The pre-trained recognition model includes a feature extraction network and a classification network. The feature extraction network is used to extract spatiotemporal features of video frame images. The classification network is used to recognize behaviors in video frame images based on the extracted spatiotemporal features. The feature extraction network includes at least one bottleneck structure, which includes two feature compression modules connected in sequence. The feature compression module is used to: divide the channels of the feature map into high-frequency and low-frequency channels, and compress the spatial resolution of the low-frequency channels. To enhance the feature capture capability, a CBAM attention mechanism is used between the two feature compression modules to activate spatial and channel features. The CBAM attention mechanism includes a spatiotemporal information activation module for activating and modeling temporal features of the spatiotemporal information in the video. The CBAM attention mechanism incorporates a motion information modeling module to represent the motion information features between video frames. In the feature compression module, the original input feature map is divided into high-frequency and low-frequency components using a first convolution. To exchange information between the high-frequency and low-frequency components, this convolution has four paths: a high-frequency to high-frequency path, a low-frequency to low-frequency path, a high-frequency to low-frequency path that reduces the spatial resolution of the feature map using average pooling, and a low-frequency to high-frequency path that increases the spatial resolution of the feature map using upsampling. The first convolution is followed by a second convolution, which uses depthwise convolution to process the high-frequency and low-frequency features separately, retaining only the high-frequency to high-frequency and low-frequency to low-frequency paths. After the second convolution, a feature map similar to the original features is generated. After the second convolution, the original and generated features are concatenated along the channel dimension, and then subjected to channel shuffling to enhance the information interaction between the feature maps, resulting in the final output features.

5. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the lightweight behavior recognition method based on feature compression as described in any one of claims 1-3.

6. A computer program product, characterized in that, Includes a computer program, which, when run on one or more processors, is used to implement the lightweight behavior recognition method based on feature compression as described in any one of claims 1-3.

7. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions that implement the lightweight behavior recognition method based on feature compression as described in any one of claims 1-3.

Images