Campus monitoring behavior identification method and system based on reasoning enhancement

Abstract

The invention provides a campus monitoring behavior identification method and system based on reasoning enhancement, and belongs to the technical field of campus security, and the method comprises the steps: obtaining a real-time video stream of campus monitoring; preprocessing the obtained video stream to obtain a space-time embedding sequence; recognizing action behaviors of the space-time embedded sequence by adopting a context sensing encoder based on reasoning enhancement; wherein an attention caching mechanism and a length extrapolation mechanism are introduced into the context-aware encoder. According to the method, the attention caching mechanism is utilized to optimize the calculation efficiency, and the context information of the long sequence is fully utilized through mixed relative position coding in the length extrapolation mechanism, so that the accuracy of action recognition is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of campus safety technology, and in particular relates to a campus monitoring behavior recognition method and system based on reasoning enhancement. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In response to the increasingly complex needs of campus security monitoring, real-time video analysis technology has become a core research direction for ensuring campus safety. This technology requires models to perform instant analysis based only on current and historical information, which is essential for quickly identifying and responding to potential threats. Its application range is not limited to conventional security monitoring, but also plays a key role in risk warning and rapid response to emergencies. By processing and analyzing video data in real time, this technology can promptly detect abnormal behavior, predict potential risks, and provide rapid decision support in emergency situations, thereby comprehensively improving the efficiency and accuracy of campus security management. This advanced analysis method provides a smarter and more proactive solution for campus security management, effectively enhancing the school's ability to respond to various security challenges.

[0004] In order to accurately understand the current behavior, the system needs to process and analyze contextual information over a longer time span, which inevitably increases the computational complexity. At the same time, in order to meet the needs of real-time applications, the system must be able to process continuously input video streams with extremely low latency. These two requirements make the implementation of real-time behavior recognition technology particularly complex.

[0005] Among the existing methods, there are methods based on RNN architecture models, whose sequential processing characteristics are suitable for streaming data, but gradients are prone to vanishing or exploding in long sequences, and it is difficult to capture long-distance dependencies. There are also models based on the Transformer architecture, which can effectively capture long-distance dependencies through the self-attention mechanism, and the performance is significantly improved. However, Transformer shows inefficiency and redundancy when processing video streams. First, Transformer lacks a memory mechanism, and the interaction between elements in the window needs to be recalculated at each time step. Since most of the elements in the window at adjacent moments are the same, this leads to a lot of redundant calculations. Secondly, the computational complexity of self-attention is O(n 2 ) As the sequence length increases, the amount of computation increases quadratically, making it difficult to handle very long contexts. In addition, the inconsistency between the sequence length of the training data and the sequence length during inference will affect the accuracy of behavior recognition during inference, and if long sequences are used for training, the training cost will increase. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention proposes a campus monitoring behavior recognition method and system based on reasoning enhancement. First, through the attention cache mechanism, the repeated calculation of repeated elements is avoided, and the calculation complexity is reduced to O(n) Secondly, a length extrapolation strategy is proposed to enable the model to process longer sequences during testing than during training, improving performance and reducing the cost of training long sequences.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, a method for identifying campus monitoring behaviors based on reasoning enhancement is disclosed, comprising: Get real-time video stream of campus surveillance; Preprocess the acquired video stream to obtain a spatiotemporal embedding sequence; Adopting context-aware encoder based on inference enhancement to recognize action behaviors in spatiotemporal embedding sequences; Among them, the context-aware encoder introduces an attention cache mechanism and a length extrapolation mechanism.

[0008] Furthermore, the inference-enhanced context-aware encoder includes a spatiotemporal causal self-attention module, a global average pooling, and an action recognition classifier.

[0009] Furthermore, in the spatiotemporal causal self-attention module, an attention cache mechanism and a length extrapolation mechanism are introduced in each spatiotemporal causal self-attention layer.

[0010] Furthermore, the attention cache mechanism introduces a key cache and a value cache to save the keys and values ​​of previously calculated video frames. When it arrives, the attention cache mechanism updates the cache expression as follows:

[0011]

[0012] in, Represents a key cache; Represents a value cache; A key cache representing the previous frame; Represents the value cache of the previous frame; represents the key vector, and ; represents a value vector, and ; represents the feature vector of the current frame, and Linear mapping matrices representing keys and values ​​respectively; Represents a concatenation operation.

[0013] Furthermore, the length extrapolation mechanism adopts hybrid relative position coding, using learnable position coding at close intervals to capture the dynamic information of the video sequence; and using linearly decaying position bias at long distance intervals to give the model extrapolation capabilities in the inference stage.

[0014] Furthermore, the spatiotemporal embedding sequence is input into the spatiotemporal causal self-attention module to obtain the enhanced spatiotemporal sequence , take the feature map of the last frame in the enhanced spatiotemporal sequence [-1], after global average pooling, is sent to the classifier to obtain the action category of the current frame.

[0015] In the second aspect, a campus monitoring behavior recognition system based on reasoning enhancement is disclosed, comprising: The acquisition module is configured to: obtain a real-time video stream of campus monitoring; A preprocessing module is configured to: preprocess the acquired video stream to obtain a spatiotemporal embedding sequence; A recognition module, which is configured to: recognize action behaviors on spatiotemporal embedding sequences using a context-aware encoder based on inference enhancement; Among them, the context-aware encoder introduces an attention cache mechanism and a length extrapolation mechanism.

[0016] In a third aspect, an electronic device is disclosed, including a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps of the above-mentioned campus monitoring behavior recognition method based on reasoning enhancement are completed.

[0017] In a fourth aspect, a computer-readable storage medium is disclosed for storing computer instructions, which, when executed by a processor, complete the steps of the above-mentioned campus monitoring behavior recognition method based on reasoning enhancement.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a campus surveillance behavior recognition method based on reasoning enhancement, which utilizes the architectural advantages of Transformer to effectively handle long-distance dependencies, improves the accuracy of prediction and the robustness of recognition, thereby more efficiently ensuring campus safety.

[0019] This paper proposes an inference enhancement technology that improves the inference speed and accuracy of the model without increasing the extra training cost, including a cache attention mechanism and a length extrapolation strategy. The cache attention mechanism saves the previously calculated keys and values, avoiding redundant attention calculations and reducing the computational complexity to O(n), improving the speed of real-time recognition; the length extrapolation strategy enables the model to process longer sequences than the training stage in the inference stage of behavior recognition through hybrid positional encoding (HRPE), thereby improving the monitoring performance and reducing the cost of long sequence training, and improving the recognition accuracy.

[0020] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0022] Figure 1 This is a schematic diagram of the campus monitoring behavior recognition method based on reasoning enhancement described in Example 1 of the present invention. DETAILED DESCRIPTION

[0023] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0024] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0025] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0026] Embodiment 1 In one or more embodiments, a method for identifying campus monitoring behaviors based on reasoning enhancement is disclosed, comprising the following steps: Step 1: Get the real-time video stream of campus surveillance.

[0027] Step 2: Preprocess the acquired video stream to obtain a spatiotemporal embedding sequence.

[0028] Step 2-1: For a given video stream F ,The goal of online action detection is to identify the latest video frame in real time The actions that take place in is the number of observed video frames. Note that any future information It is not available.

[0029] Step 2-2: Use Patch Embedding to divide each frame image in the video stream into multiple small blocks, and map each small block into a vector of fixed dimension through linear transformation, thereby converting a single frame image into an embedding sequence that can be processed by the Transformer model.

[0030] Step 2-3: Stack multiple frames to get a The spatiotemporal embedding sequence ,in Indicates the length of the stream buffer, Indicates the number of embeddings per frame.

[0031] The spatiotemporal embedding sequence through block embedding operation can efficiently extract local features, reduce computational complexity, and adapt to dynamic and multi-scene and multi-view tasks.

[0032] In the specific application scenario of campus monitoring, it not only improves the efficiency of detecting abnormal behaviors, but also helps the model flexibly adapt to changing environments, thereby greatly improving the overall performance and practicality.

[0033] Step 3: Use the context-aware encoder (CAE) based on inference enhancement to capture context information of the video frames in the cache and accurately identify the current action, including: The context-aware encoder maintains a fixed-length queue as a streaming buffer to cache the video stream and dynamically updates it as new video frames arrive. At each update, CAE applies a spatiotemporal attention mechanism to the video frames in the streaming buffer to capture contextual information and recognize the current action.

[0034] Step 3-1: Construct a spatiotemporal causal self-attention module, in which an attention cache mechanism and a length extrapolation mechanism are introduced in each spatiotemporal causal self-attention layer.

[0035] Step 3-1-1: Stack multiple layers of spatiotemporal causal self-attention layers and add a causal mask to the attention matrix of each layer to ensure that the embedding at each moment only focuses on the current and previous time steps when modeling the sequence context.

[0036] Step 3-1-2: Introduce the Cache Attention Mechanism (CAM) in each spatiotemporal causal self-attention layer.

[0037] The context-aware encoder captures the spatiotemporal dependencies of video frames in the buffer through the spatiotemporal causal self-attention layer. However, due to the existence of There are many overlapping frames, which leads to a lot of repeated calculations in the interactions between elements, resulting in computational redundancy.

[0038] To improve the reasoning efficiency, an attention cache mechanism is introduced in each spatiotemporal causal self-attention layer of CAE. Sum value cache , save the key (Key) and value (Value) of the previously calculated video frame. When it arrives, CAM updates the cache according to the following formula: (1) (2) in, Represents a key cache; Represents a value cache; A key cache representing the previous frame; Represents the value cache of the previous frame; represents the key vector, and ; represents a value vector, and ; represents the feature vector of the current frame, and Linear mapping matrices representing keys and values ​​respectively; Represents a concatenation operation, which is used to concatenate the key / value cache at the previous moment in formula (1) and formula (2) with the key / value at the current moment to update the key / value cache.

[0039] In this way, each layer only needs to calculate relevant features for the current fragment, as shown in the following formula: (3) in, represents the attention matrix; represents the query vector, and ; The linear mapping matrix representing the query; represents the parameter normalized according to the feature dimension. Formula (3) represents the calculation of the attention score between all elements.

[0040] Through the key-value cache mechanism, the original spatiotemporal causal self-attention is simplified to the elements and Each time a new video frame arrives, we only need to calculate the query of the new frame and the key value in the cache to perform attention calculation, thus reducing the computational complexity from Reduce to , effectively reducing redundant calculations and significantly improving the efficiency of spatiotemporal reasoning.

[0041] Step 3-1-3: Design a length extrapolation mechanism (LEM) to overcome the difference in length distribution between the training and testing phases through positional encoding, so that the model trained on short sequences can use longer contexts in the testing phase, thereby improving recognition accuracy.

[0042] Position in the video frame Elements and positions at The attention score between the elements at can be expressed as: (4) in, Indicates location Query and location The dot product between the keys at is used to calculate the attention score; is the position offset term, Represents the parameter normalized according to the feature dimension.

[0043] Then, a hybrid relative position encoding (HRPE) is designed. ) using a learnable positional encoding ( ), to capture the dynamic information of the video sequence; at long distance intervals ( ) uses a linearly decaying position bias to give the model extrapolation capabilities during the inference phase. The formula is as follows: (5) in, Represents the position bias of the mixed position encoding at positions i and j; Indicates location Inquiries from Indicates location Positional encoding; represents the linear attenuation slope; Represents a linear attenuation bias, which is used to ensure that at the junction The smooth transition at .

[0044] During the training phase, CAE converts the length of T (T>n) The stream cache is taken as input. In the test phase, by adjusting the cache size in the cache attention mechanism (i.e. and ), extending the time span of the stream cache to At this time, the linear attenuation interval of HRPE generates a position offset for the extended time segment, ranging from arrive This extended historical information provides a wider range of action contexts, thus improving the robustness of predictions.

[0045] Step 3-2: Input the spatiotemporal embedding sequence into the spatiotemporal causal self-attention module to obtain the enhanced spatiotemporal sequence ,Pick The feature map of the last frame in , after global average pooling, it is sent to the classifier to obtain the action category of the current frame; In this embodiment, the classifier is a neural network including a fully connected layer and a Softmax activation function, which improves the detection efficiency of abnormal behaviors under campus monitoring.

[0046] Embodiment 2 In one or more embodiments, a campus monitoring behavior recognition system based on reasoning enhancement is disclosed, including: The acquisition module is configured to: obtain a real-time video stream of campus monitoring; A preprocessing module is configured to: preprocess the acquired video stream to obtain a spatiotemporal embedding sequence; A recognition module, which is configured to: recognize action behaviors on spatiotemporal embedding sequences using a context-aware encoder based on inference enhancement; Among them, the context-aware encoder introduces an attention cache mechanism and a length extrapolation mechanism into the spatiotemporal causal self-attention layer.

[0047] Embodiment 3 This embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the steps of the above-mentioned campus monitoring behavior recognition method based on reasoning enhancement are completed.

[0048] Embodiment 4 This embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the above-mentioned campus monitoring behavior recognition method based on reasoning enhancement are completed.

[0049] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0050] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0051] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0052] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0053] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A campus monitoring behavior recognition method based on reasoning enhancement, characterized in that: include: Get real-time video stream of campus surveillance; Preprocess the acquired video stream to obtain a spatiotemporal embedding sequence; Adopting context-aware encoder based on inference enhancement to recognize action behaviors in spatiotemporal embedding sequences; Among them, the context-aware encoder introduces an attention cache mechanism and a length extrapolation mechanism.

2. A campus monitoring behavior recognition method based on reasoning enhancement as claimed in claim 1, characterized in that: The preprocessing obtains a spatiotemporal embedding sequence, specifically: Block embedding is used to divide each frame of the video stream into multiple small blocks, and each small block is mapped to a vector of fixed dimension through linear transformation; Stack multiple frames of images to get The spatiotemporal embedding sequence ,in, Indicates the length of the stream buffer, Indicates the number of embeddings per frame.

3. A campus monitoring behavior recognition method based on reasoning enhancement as claimed in claim 1, characterized in that: The inference-enhanced context-aware encoder includes a spatiotemporal causal self-attention module, a global average pooling, and an action recognition classifier.

4. A campus monitoring behavior recognition method based on reasoning enhancement as described in claim 3, characterized in that: In the spatiotemporal causal self-attention module, an attention cache mechanism and a length extrapolation mechanism are introduced in each spatiotemporal causal self-attention layer.

5. A campus monitoring behavior recognition method based on reasoning enhancement as claimed in claim 4, characterized in that: The attention cache mechanism introduces a key cache and a value cache to save the keys and values ​​of previously calculated video frames. When it arrives, the attention cache mechanism updates the cache expression as follows: in, Represents a key cache; Represents a value cache; A key cache representing the previous frame; Represents the value cache of the previous frame; represents the key vector, and ; represents a value vector, and ; represents the feature vector of the current frame, and Linear mapping matrices representing keys and values ​​respectively; Represents a concatenation operation.

6. A campus monitoring behavior recognition method based on reasoning enhancement as claimed in claim 4, characterized in that: The length extrapolation mechanism adopts hybrid relative position coding, using learnable position coding at close intervals to capture the dynamic information of the video sequence; and using linearly decaying position bias at long distances to give the model extrapolation capabilities in the inference phase.

7. The campus monitoring behavior recognition method based on reasoning enhancement as claimed in claim 1 is characterized in that: Input the spatiotemporal embedding sequence into the spatiotemporal causal self-attention module to obtain the enhanced spatiotemporal sequence , take the feature map of the last frame in the enhanced spatiotemporal sequence , after global average pooling, it is sent to the classifier to obtain the action category of the current frame.

8. A campus monitoring behavior recognition system based on reasoning enhancement, characterized in that: include: The acquisition module is configured to: obtain a real-time video stream of campus monitoring; A preprocessing module is configured to: preprocess the acquired video stream to obtain a spatiotemporal embedding sequence; A recognition module, which is configured to: recognize action behaviors on spatiotemporal embedding sequences using a context-aware encoder based on inference enhancement; Among them, the context-aware encoder introduces an attention cache mechanism and a length extrapolation mechanism.

9. An electronic device, characterized in that: It includes a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the campus monitoring behavior recognition method based on reasoning enhancement as described in any one of claims 1 to 7 is completed.

10. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the campus monitoring behavior recognition method based on reasoning enhancement as described in any one of claims 1-7.

Images