A violence recognition method and system based on multi-feature channel time sequence shift

Abstract

The application discloses a violent behavior recognition method and system based on multi-feature channel time sequence shift, belongs to the image recognition technical field, and can solve the problem of low recognition accuracy of violent behavior in the prior art. The method comprises the following steps: S1, continuously N frames of images in a target video are cached in time sequence by using a data buffer; N is greater than or equal to 2; S2, motion feature channels of each frame of image in the current cache are generated according to the N frames of images in the current cache; S3, a time sequence shift module is introduced into a pre-trained two-dimensional convolutional neural network, and the two-dimensional convolutional neural network is improved based on the motion feature channels, so that a behavior detection model is obtained; and S4, whether violent behavior exists in the N frames of images in the current cache is determined by using the behavior detection model according to the motion feature channels and color feature channels of each frame of image in the current cache. The application is used for recognizing violent behavior.

Description

Technical Field

[0001] This invention relates to a method and system for identifying violent behavior based on multi-feature channel temporal shifting, belonging to the field of image recognition technology. Background Technology

[0002] With the widespread adoption of public safety surveillance cameras, the amount of surveillance video data has exploded. Relying solely on manual monitoring is no longer sufficient to identify violent behavior in videos in real time and efficiently. Machine learning models are increasingly being applied to the identification of violent behavior.

[0003] However, existing machine learning models can usually only extract spatial features from a single frame of an image when identifying violent behavior, but cannot capture the temporal features of the image. This leads to the neglect of the temporal continuity of the action, making it easy for machine learning models to confuse similar actions such as "running" with "chasing", "hugging" with "fighting", resulting in low accuracy in identifying violent behavior. Summary of the Invention

[0004] This invention provides a method and system for identifying violent behavior based on multi-feature channel temporal shifting, which can solve the problem of low accuracy in identifying violent behavior in existing technologies.

[0005] On one hand, the present invention provides a method for identifying violent behavior based on multi-feature channel temporal shifting, the method comprising: S1. Use a data buffer to cache N consecutive frames of images in the target video in time sequence; N≥2; S2. Based on the N frames of images currently cached, generate the motion feature channels for each frame of the currently cached image. S3. Introduce a temporal shift module into the pre-trained two-dimensional convolutional neural network, and improve the two-dimensional convolutional neural network based on the motion feature channel to obtain a behavior detection model; S4. Based on the motion feature channel and color feature channel of each currently cached frame image, use the behavior detection model to determine whether there is violent behavior in the currently cached N frames image.

[0006] Optionally, S2 specifically includes: S21. In each frame of the currently cached image, delineate local regions containing motion features to obtain N local regions; S22. Generate motion feature channels for each frame of the currently cached image based on the N local regions.

[0007] Optionally, in S21, a local region containing motion features is delineated in each currently cached frame image, specifically including: Determine the centroid of motion for each frame currently cached; In each currently cached frame, a local region is defined centered on its center of motion and within a preset range.

[0008] Optionally, S22 specifically includes: Denoising is performed on each local region to remove motion features smaller than a preset amplitude; Motion feature channels for each frame of the currently cached image are generated based on the N local regions after denoising.

[0009] Optionally, motion feature channels for each frame of the currently cached image are generated based on the N local regions after denoising, specifically including; The motion mask of each local region after denoising is generated using the inter-frame difference method, and the motion mask is used as the motion feature channel of the corresponding image.

[0010] Optionally, in S3, the two-dimensional convolutional neural network is improved based on the motion feature channels, specifically including: An input channel is added to the two-dimensional convolutional neural network so that the number of input channels of the two-dimensional convolutional neural network matches the sum of the number of motion feature channels and color feature channels of each frame of the image; The channel weights of the added input channels are determined based on the channel weights of the original input channels in the two-dimensional convolutional neural network.

[0011] Optionally, in S4, the behavior detection model is used to determine whether violent behavior exists in the currently cached N frames of images, specifically including: S41. Utilize the behavior detection model to determine the motion entropy of the local region corresponding to each currently cached frame image, and reduce the attention weight of the behavior detection model for local regions with motion entropy lower than a preset entropy value. S42. Use the behavior detection model to detect human targets in local areas that are higher than or equal to the preset entropy value, and adjust the attention weight of the behavior detection model for local areas that are higher than or equal to the preset entropy value according to the detection results. S43. Determine whether there is violent behavior in the currently cached N frames of images based on the local regions and attention weights of the M consecutive frames of images; 2≤M≤N.

[0012] Optionally, in S42, the attention weights of the behavior detection model for local regions with entropy values ​​higher than or equal to the preset entropy value are adjusted according to the detection results, specifically including: Reduce the attention weight of the behavior detection model on local areas where the detection results match the preset results; the preset results are no human targets detected or the minimum distance between multiple human targets exceeds the preset distance.

[0013] Optionally, S43 specifically includes: The probability of violent behavior in the M consecutive frames of images is determined based on the local regions and their attention weights. The presence probability and the motion entropy corresponding to each frame of the image are used to determine whether violent behavior exists in each currently cached frame.

[0014] On the other hand, the present invention provides a violent behavior recognition system based on multi-feature channel temporal shifting, the system comprising: A buffer unit is used to cache N consecutive frames of images in the target video in time sequence using a data buffer; N≥2; The channel generation unit is used to generate motion feature channels for each frame of the currently cached image based on the N currently cached frames. A modeling unit is used to introduce a temporal shift module into a pre-trained two-dimensional convolutional neural network and improve the two-dimensional convolutional neural network based on the motion feature channel to obtain a behavior detection model. The recognition unit is used to determine whether violent behavior exists in the currently cached N frames of images based on the motion feature channels and color feature channels of each currently cached frame of images, using the behavior detection model.

[0015] The beneficial effects that this invention can produce include: This invention utilizes a data buffer to sequentially cache N consecutive frames of images from a target video. This ensures the integrity of the temporal information of the images, facilitating subsequent steps to capture continuous motion features, while also reducing memory usage and avoiding data redundancy. By defining local regions containing motion features within the images, moving targets with small pixel proportions are locally magnified, improving the recognition rate of moving targets and effectively solving the problem of missed detection of distant moving targets. By generating motion feature channels from multiple consecutive frames and making corresponding improvements to the input channels of the two-dimensional convolutional neural network (2D convolutional neural network), and introducing a temporal shift module, the 2D convolutional neural network can capture the spatiotemporal and motion features of the images with zero computation and zero parameters. This enhances the sensitivity of the 2D convolutional neural network for motion capture, enabling it to focus on the temporal continuity of actions and distinguish similar actions, thereby improving the accuracy of violent behavior recognition. Furthermore, this invention employs a three-level anti-interference smoothing decision algorithm to eliminate interference from similar actions and background interference, significantly reducing the false alarm rate. This method boasts advantages such as high sensitivity, low false alarm rate, lightweight design, strong compatibility, and low implementation cost, making it widely applicable to various security monitoring scenarios. Attached Figure Description

[0016] Figure 1 The flowchart shows a violent behavior recognition method based on multi-feature channel temporal shift provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will now be described in detail with reference to the embodiments, but the present invention is not limited to these embodiments.

[0018] This invention provides a method for identifying violent behavior based on multi-feature channel temporal shifting, such as... Figure 1 As shown, the method includes: S1. Use a data buffer to cache N consecutive frames of images in the target video in time sequence; N≥2.

[0019] The data buffer can be a circular buffer, a dual buffer, a single buffer, or a buffer pool, etc., preferably a circular buffer, and N is preferably 8. This embodiment utilizes the data buffer to cache N consecutive frames of the target video in real time at the current moment, and performs subsequent steps on the currently cached N frames. Then, it continues to cache the corresponding N consecutive frames in real time at the next moment and performs subsequent steps on the N frames cached at the next moment. This cycle repeats, effectively constructing a time-series sliding window of fixed length N frames on the target video stream. This ensures the integrity of the temporal information of the images, facilitates the capture of continuous motion features in subsequent steps, and reduces memory usage while avoiding data redundancy.

[0020] S2. Based on the currently cached N frames of images, generate the motion feature channels for each frame of the currently cached image, specifically including: S21. In each frame of the currently cached image, delineate local regions containing motion features to obtain N local regions.

[0021] Specifically, in this embodiment, for each currently cached frame of image, the high-frequency motion region in the image is highlighted by calculating its cumulative difference intensity, thus obtaining the image moments of each frame of image. Then, the motion centroid of each currently cached frame of image is determined based on the image moments, and a local region containing motion features is delineated with the motion centroid as the center, thereby achieving automatic locking of the motion occurrence region. The specific steps are as follows: 1) Calculate the image moments for each frame currently cached: (1) In formula (1), For image moments, For pixels in the image The cumulative difference intensity at that point for The order of the moment in the direction (a non-negative integer). for The order of the moment in the direction (a non-negative integer).

[0022] 2) Determine the motion centroid of each frame in the current cache: (2) In formula (2), Let these be the coordinates of the moving centroid of the image. First order Directional moment, First order Directional moment, It is the zeroth moment.

[0023] 3) Delineate the local region containing motion characteristics; In each frame of the currently cached image, with its center of motion centroid Centered on the target area, a Region of Interest (ROI) is defined according to a preset range. The ROI is a local region that contains motion features.

[0024] The shape of the ROI region can be square, circular, elliptical, or irregular polygonal. This embodiment adopts... a square, The side length of the square ROI region is in pixels, preferably 224.

[0025] Furthermore, this embodiment uses a boundary protection formula to prevent the boundaries of the ROI region from exceeding the original image range, thus ensuring that the ROI region is always a valid image region. The boundary protection formula is: (3) (4) In formulas (3) and (4), To determine the starting point coordinates of the ROI region, The coordinates of the centroid of motion of the original image. The width of the original image. The height of the original image. The side length in pixels of the square ROI region.

[0026] This embodiment improves the recognition rate of moving targets by defining local regions containing motion features in the original image and magnifying the moving targets that have a small pixel proportion in the image. Experiments have verified that after defining the local regions, the recognition rate of moving targets increased from 65% to over 92%, effectively solving the problem of missed detection of long-distance moving targets.

[0027] S22. Generate motion feature channels for each frame of the currently cached image based on N local regions, specifically including: 1) Perform noise reduction processing on each local region to eliminate motion features smaller than a preset amplitude; Denoising can be achieved using either hard thresholding or soft thresholding, with hard thresholding being preferred. Hard thresholding is a common method for signal denoising in the wavelet transform domain. Its core idea is to distinguish and suppress noise components by setting a threshold. The preset amplitude can be flexibly determined based on information such as the frequency of light flicker or sensor background noise. Denoising can eliminate subtle light flicker and sensor background noise, retaining only significant human target motion features, which helps improve the accuracy of subsequent violent behavior recognition.

[0028] 2) Generate motion feature channels for each frame of the currently cached image based on the N local regions after denoising; This embodiment utilizes inter-frame differencing to generate a motion mask for each local region after denoising, and then uses the motion mask as the corresponding motion feature channel of the image. Inter-frame differencing can be a two-frame differencing method, a three-frame differencing method, or a four-frame differencing method. This embodiment preferably uses a three-frame differencing method. The motion feature channels of a frame image can be represented as: (5) In formula (5), For the first The motion mask of a local region of a frame image, i.e., the first... Pixel values ​​of the motion feature channel of a frame image; For the first Pixel values ​​of a local region in a frame image. For the first Pixel values ​​of a local region in a frame image. For the first The pixel values ​​of a local region in a frame image.

[0029] 3) Feature channel splicing; The motion feature channels are concatenated with the red (R), green (G), and blue (B) color feature channels of the RGB image along the channel dimension to generate a four-channel tensor. This preserves the spatial semantic information of environmental and human body details while supplementing the temporal features of the action.

[0030] This embodiment generates a motion feature channel for the image, supplementing the original image features with motion vector information indicating the speed and dynamism of the movement. This enables preliminary identification of the temporal characteristics of motion in the image, improving the sensitivity of subsequent steps in recognizing instantaneous explosive movements. Experiments have verified that capturing a "momentary punch" based on the motion feature channel is more than 150ms faster than without considering the motion feature channel. Furthermore, generating a motion feature channel helps eliminate interference from lighting changes on RGB image recognition, thereby improving recognition accuracy.

[0031] S3. A temporal shift module is introduced into the pre-trained two-dimensional convolutional neural network, and the two-dimensional convolutional neural network is improved based on the motion feature channel to obtain the behavior detection model, specifically including: 1) Introduce a temporal shifting module into a pre-trained two-dimensional convolutional neural network; This embodiment inserts a Temporal Shift Module (TSM) between the residual blocks of a pre-trained 2D Convolutional Neural Network (2D CNN). The TSM can shift a portion of the 2D CNN's input channels along the time dimension, thereby promoting information exchange between feature maps of adjacent frames when the 2D CNN processes image data. This improves the temporal modeling capability of the 2D CNN without increasing its computational cost or parameter count, achieving efficient temporal modeling in the 2D CNN with zero computation and zero parameters.

[0032] Specifically, the temporal shifting process of TSM is as follows: before each residual block, 1 / 8 of the feature map's feature channels are shifted forward along the time axis (looking back), 1 / 8 of the feature map's feature channels are shifted backward along the time axis (looking forward), and the remaining 6 / 8 of the feature channels are kept unchanged (keeping the current state), thereby achieving cross-frame information exchange. Let the feature map be... The total number of feature channels is Then the timing shift process of TSM can be represented as: (6) In formula (6), For feature maps, This represents the total number of feature channels. For channel index (channel number). For the input feature map At any moment ,aisle The feature representation of the location, The feature map after TSM timing shift at time... ,aisle The output feature value (or feature vector) at that location.

[0033] Since the temporal shift operation of TSM only involves memory address movement, the floating-point operation cost (FLOPs) is 0. Therefore, without increasing computing power or the number of parameters, 2D CNNs can possess the temporal modeling capabilities of 3D Convolutional Neural Networks (3D CNNs). This balances the computational speed of 2D CNNs with the computational accuracy of 3D CNNs, which is beneficial for improving the efficiency and accuracy of violent behavior recognition. Experiments have verified that the real-time performance of 2D CNNs with the introduction of TSM is enhanced. While keeping the number of parameters unchanged, its single-frame inference time is only about 40ms (CPU environment, including differential calculation), and the processing frame rate can reach 15 FPS~20 FPS, which can meet the real-time processing requirements of 1080P video streams and can be deployed on ordinary edge computing devices.

[0034] 2) Improvements to the two-dimensional convolutional neural network based on motion feature channels; This embodiment adds input channels to the 2D CNN so that the number of input channels of the two-dimensional convolutional neural network matches the sum of the number of motion feature channels and color feature channels of each frame of image.

[0035] For example, ResNet-50, pre-trained on the ImageNet database, is used as a 2D CNN for illustration. ResNet-50 originally has three input channels, which correspond to the red (R), green (G), and blue (B) color feature channels of an RGB image, respectively. Since a motion feature channel is generated for each frame of the image in step S2 of this embodiment, an additional input channel needs to be added to ResNet-50 to make the number of input channels of ResNet-50 match the number of feature channels of the image.

[0036] Then, the channel weights of the added input channels are determined based on the channel weights of the original input channels in ResNet-50, thus obtaining the improved ResNet-50, i.e., the behavior detection model. Specifically, a mean initialization strategy can be used, where the channel weights of the original three input channels are used as the channel weights of the added input channels. The channel weights of the added input channels can be expressed as: (7) (8) In formulas (7) and (8), The channel weights of the original input channels. To increase the channel weights of the input channels, For channel index (channel number).

[0037] The mean initialization strategy is used to determine the channel weights of the added input channels. The original channel weights can be reused, ensuring that the original feature response distribution of the improved ResNet-50 is not destroyed in the initial stage of training. This reduces the model training cost and the dependence of the training process on the dataset, allowing the model to be quickly adapted to the existing monitoring system without replacing the monitoring system's hardware. It has advantages such as good compatibility and low deployment cost.

[0038] S4. Based on the motion feature channels and color feature channels of each currently cached frame image, use the behavior detection model to determine whether there is violent behavior in the currently cached N frames image.

[0039] This embodiment inputs a four-channel tensor, composed of the motion feature channels and color feature channels of each frame of image, into the behavior detection model. The behavior monitoring model can then identify whether violent behavior exists in the currently cached N frames of images. In practice, since videos may contain motion features such as birds flying, fast-moving vehicles passing by, and changes in lighting, these irrelevant motion features can easily interfere with the identification of violent behavior, affecting the accuracy of identification and leading to a high false alarm rate. Therefore, this embodiment introduces a three-level anti-interference smoothing decision algorithm into the behavior detection model. Through motion entropy filtering, spatial clustering constraints, and temporal consistency smoothing, the influence of interference factors is eliminated step by step, improving the anti-interference ability of the behavior detection model.

[0040] Specifically, the implementation process of the three-level anti-interference smoothing decision algorithm is as follows: 1) Motion entropy filtering; Based on the motion feature channel, the motion entropy of the local region corresponding to each frame of the currently cached image is determined by the behavior detection model. The motion entropy represents the disorder of the motion direction in the local region. The motion entropy can be used to distinguish between "vehicle motion / running motion in a single direction" and "disordered limb collision motion".

[0041] Specifically, if the motion entropy of a frame is lower than a preset entropy value, it can be determined as unidirectional motion (such as vehicle movement or running). In this case, the attention weight of the behavior detection model for the local region of that frame should be reduced to eliminate interference from similar movements. If the motion entropy of a frame is higher than or equal to the preset entropy value, it is determined as disordered motion, indicating that there may be violent behavior in that frame, and the frame proceeds to the next level. The preset entropy value is an empirical threshold, preferably 0.3.

[0042] 2) Spatial aggregation restrictions; The behavior detection model is used to detect human targets in local regions with an entropy value higher than or equal to the preset value, and the attention weight of the behavior detection model for local regions with an entropy value higher than or equal to the preset value is adjusted according to the detection results.

[0043] Specifically, if the detection result corresponding to a certain frame of image meets the preset result (the preset result is that no human target is detected or the minimum distance between multiple detected human targets exceeds the preset distance), it means that there are no people or people are far apart in the local area corresponding to the frame of image. Then it can be determined that there is no violent behavior in the local area, and its motion features mainly come from background interference. At this time, the attention weight of the behavior detection model to the local area of ​​the frame of image should be reduced to eliminate background interference such as changes in light and shadow. If the detection result corresponding to a certain frame of image does not meet the preset result, it means that there are dense people in the local area corresponding to the frame of image, and the motion features mainly come from the area where people gather, indicating that there may be violent behavior in the frame of image.

[0044] 3) Time-series consistency and smoothness; After applying two levels of adjustments—motion entropy filtering and spatial clustering constraint—to the currently cached N frames, this embodiment constructs a sliding window of length M (2≤M≤N). The sliding window stores local regions of M consecutive frames within the adjusted N frames. Then, based on the local regions of the M consecutive frames and their attention weights, it is determined whether violent behavior exists in the currently cached N frames. M is preferably 5.

[0045] Specifically, for local regions of M-frame images, this embodiment uses a behavior detection model to calculate the original score of the presence of violent behavior in the local region of each frame image ( Then calculate the average of the original scores corresponding to the M frames, and then use... The activation function transforms the average value into a probability distribution, yielding the probability of violent behavior in M ​​frames of images. This helps avoid instantaneous false alarms in a single frame. The formula for calculating the probability of existence is: (9) In formula (9), The channel weights of the original input channels. For activation function, For the first The original score corresponding to the frame image.

[0046] Subsequently, this embodiment determines whether violent behavior exists in each currently cached frame image based on the existence probability and the motion entropy corresponding to each frame image. Specifically, when the existence probability of violent behavior in M ​​frames is greater than a preset probability and the motion entropy corresponding to a certain frame image in M ​​frames is greater than a preset entropy value, it is determined that violent behavior exists in that frame image. The preset probability is preferably 0.85, and the preset entropy value is preferably 0.3.

[0047] In practical application of the method described in this embodiment, the improved ResNet-50 can be trained and validated using datasets such as RWF-2000. The dataset should contain a large number of videos of violent and normal behavior in real surveillance scenarios. Before training, the video segments are divided into multiple video units according to time sequence. Each video unit contains N consecutive frames. For each video unit, the motion feature channels of each frame are generated according to step S2. Then, the motion feature channels of each frame are concatenated with its three color feature channels to generate a four-channel tensor for each frame, and training and validation sets are created. Then, the improved ResNet-50 is trained using the training set and validated using the validation set. During training, the initial learning rate can be set to 0.001, and the Stochastic Gradient Descent (SGD) optimization algorithm is used. The training run consists of 50 epochs, with the learning rate decreasing to 1 / 10 of its original value every 10 epochs. The accuracy and false positive rate on the validation set are monitored. Training stops once the validation set accuracy stabilizes, thus obtaining the behavior detection model. This behavior detection model is then integrated into a real-time monitoring system. The model can read video streams in real time and identify violent behaviors in the video according to the method described in this embodiment, transmitting the identification results to the monitoring system's alarm system.

[0048] Another embodiment of the present invention provides a violent behavior recognition system based on multi-feature channel temporal shifting, the system comprising: A buffer unit is used to cache N consecutive frames of images in the target video in time sequence using a data buffer; N≥2; The channel generation unit is used to generate motion feature channels for each frame of the currently cached image based on the N currently cached frames. The modeling unit is used to introduce a temporal shift module into a pre-trained two-dimensional convolutional neural network and improve the two-dimensional convolutional neural network based on the motion feature channel to obtain a behavior detection model. The recognition unit is used to determine whether violent behavior exists in the currently cached N frames of images based on the motion feature channels and color feature channels of each frame of the currently cached image, using a behavior detection model.

[0049] This embodiment utilizes a data buffer to cache N consecutive frames of images from the target video in a time sequence. This ensures the integrity of the temporal information of the images, facilitating subsequent steps to capture continuous motion features, while also reducing memory usage and avoiding data redundancy. By delineating local regions containing motion features in the images and locally magnifying moving targets with small pixel proportions, the recognition rate of moving targets is improved, effectively solving the problem of missed detection of long-distance moving targets. By generating motion feature channels from multiple consecutive frames and making corresponding improvements to the input channel of the 2D convolutional neural network, while introducing a temporal shift module, the 2D convolutional neural network can capture the spatiotemporal and motion features of the images with zero computation and zero parameters. This enhances the sensitivity of the 2D convolutional neural network for motion capture, enabling it to focus on the temporal continuity of actions and distinguish similar actions, thereby improving the accuracy of violent behavior recognition. Furthermore, this embodiment uses a three-level anti-interference smoothing decision algorithm to eliminate interference from similar actions and background interference, significantly reducing the false alarm rate. This method boasts advantages such as high sensitivity, low false alarm rate, lightweight design, strong compatibility, and low implementation cost, making it widely applicable to various security monitoring scenarios.

[0050] The above description is merely a few embodiments of this application and is not intended to limit this application in any way. Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any changes or modifications made by those skilled in the art without departing from the scope of the technical solution of this application using the disclosed technical content are equivalent to equivalent implementation cases and fall within the scope of the technical solution.

Claims

1. A method for identifying violent behavior based on multi-feature channel temporal shifting, characterized in that, The method includes: S1. Use a data buffer to cache N consecutive frames of images in the target video in time sequence; N≥2; S2. Based on the N frames of images currently cached, generate the motion feature channels for each frame of the currently cached image. S3. Introduce a temporal shift module into the pre-trained two-dimensional convolutional neural network, and improve the two-dimensional convolutional neural network based on the motion feature channel to obtain a behavior detection model; S4. Based on the motion feature channel and color feature channel of each currently cached frame image, use the behavior detection model to determine whether there is violent behavior in each currently cached frame image.

2. The method according to claim 1, characterized in that, S2 specifically includes: S21. In each frame of the currently cached image, delineate local regions containing motion features to obtain N local regions; S22. Generate motion feature channels for each frame of the currently cached image based on the N local regions.

3. The method according to claim 2, characterized in that, In S21, local regions containing motion features are defined in each currently cached frame image, specifically including: Determine the centroid of motion for each frame currently cached; In each currently cached frame, a local region is defined centered on its center of motion and within a preset range.

4. The method according to claim 2, characterized in that, S22 specifically includes: Denoising is performed on each local region to remove motion features smaller than a preset amplitude; Motion feature channels for each frame of the currently cached image are generated based on the N local regions after denoising.

5. The method according to claim 4, characterized in that, Motion feature channels for each frame of the currently cached image are generated based on N local regions after denoising, specifically including; The motion mask of each local region after denoising is generated using the inter-frame difference method, and the motion mask is used as the motion feature channel of the corresponding image.

6. The method according to claim 1, characterized in that, S3 improves the two-dimensional convolutional neural network based on the motion feature channels, specifically including: An input channel is added to the two-dimensional convolutional neural network so that the number of input channels of the two-dimensional convolutional neural network matches the sum of the number of motion feature channels and color feature channels of each frame image; The channel weights of the added input channels are determined based on the channel weights of the original input channels in the two-dimensional convolutional neural network.

7. The method according to claim 2, characterized in that, S4 utilizes the behavior detection model to determine whether violent behavior exists in each currently cached frame of image, specifically including: S41. Utilize the behavior detection model to determine the motion entropy of the local region corresponding to each currently cached frame image, and reduce the attention weight of the behavior detection model for local regions with motion entropy lower than a preset entropy value. S42. Use the behavior detection model to detect human targets in local areas that are higher than or equal to the preset entropy value, and adjust the attention weight of the behavior detection model for local areas that are higher than or equal to the preset entropy value according to the detection results. S43. Determine whether there is violent behavior in each currently cached frame image based on the local regions and attention weights of M consecutive frames of images; 2≤M≤N.

8. The method according to claim 7, characterized in that, S42 adjusts the attention weights of the behavior detection model for local regions with entropy values ​​higher than or equal to the preset entropy value based on the detection results, specifically including: Reduce the attention weight of the behavior detection model on local areas where the detection results match the preset results; the preset results are no human targets detected or the minimum distance between multiple human targets exceeds the preset distance.

9. The method according to claim 7, characterized in that, S43 specifically includes: The probability of violent behavior in the M consecutive frames of images is determined based on the local regions and their attention weights. The presence probability and the motion entropy corresponding to each frame of the image are used to determine whether violent behavior exists in each currently cached frame.

10. A violent behavior recognition system based on multi-feature channel temporal shifting, characterized in that, The system includes: A buffer unit is used to cache N consecutive frames of images in the target video in time sequence using a data buffer; N≥2; The channel generation unit is used to generate motion feature channels for each frame of the currently cached image based on the N currently cached frames. A modeling unit is used to introduce a temporal shift module into a pre-trained two-dimensional convolutional neural network and improve the two-dimensional convolutional neural network based on the motion feature channel to obtain a behavior detection model. The recognition unit is used to determine whether violent behavior exists in each currently cached frame image based on the motion feature channel and color feature channel of each currently cached frame image using the behavior detection model.

Images