A fighting behavior recognition method, device, equipment and storage medium
By combining 3D-CNN and TDN models, spatiotemporal feature information is extracted from video streams, solving the problem of accurate identification of fighting behavior in public places and realizing timely fighting alarms and security monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHANGSHA HISENSE INTELLIGENT SYST RES INST CO LTD
- Filing Date
- 2022-12-19
- Publication Date
- 2026-05-15
AI Technical Summary
Existing security monitoring systems cannot accurately identify fighting behavior in public places, especially in rail transit scenarios, where timely alarms are not possible, leading to increased risks of crowd congestion and stampedes. Furthermore, existing technologies are easily affected by the environment.
A method combining 3D-CNN and TDN models is used to extract image frame features from the video stream. Motion information in multiple consecutive frames is captured by 3D convolution kernels and temporal difference classification, and a fighting score is calculated to determine whether to issue an alarm.
It improves the accuracy and timeliness of fighting behavior identification, reduces the irrationality of manually set parameters, and enhances the effectiveness of security monitoring in public places.
Smart Images
Figure CN115719517B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and specifically to a method, apparatus, device, and storage medium for recognizing fighting behavior. Background Technology
[0002] Existing security monitoring systems typically report anomalies through real-time feedback from monitoring personnel. However, this manual monitoring method suffers from drawbacks such as unreliability and high manpower consumption. In rail transit scenarios, where passenger flow is high, the failure to promptly alert authorities to fights can lead to congestion and even stampedes. Therefore, a timely alarm method for detecting fights can provide immediate response and crucial evidence for maintaining public order.
[0003] Chinese patent (CN113191278A) discloses a fighting behavior detection method based on the fusion of video and audio data. It primarily selects a deep network structure to process audio and video segments, and determines fighting behavior by collecting pedestrian actions in frequency bands and audio from audio segments. Chinese patent (CN113468975A) discloses a fighting behavior detection method and apparatus, which mainly involves acquiring a sequence of images to be detected and determining the corresponding feature heatmap; determining alarm areas based on the feature heatmap; and detecting the alarm areas to determine whether fighting behavior exists within those areas.
[0004] While the aforementioned existing technologies can all identify and determine fighting behavior to some extent, the audio-based method is easily affected by the surrounding environment. While feature heatmaps can locate individuals, they cannot accurately identify fighting behavior if the person is carrying an object. Summary of the Invention
[0005] In view of this, this application provides a method, apparatus, device and storage medium for identifying fighting behavior, in order to solve the problem that the prior art cannot accurately identify fighting behavior in public places.
[0006] In a first aspect, embodiments of this application provide a method for identifying fighting behavior, including:
[0007] Image frames are acquired from the video stream and stored in a preset queue in chronological order.
[0008] The 3D-CNN model for recognizing fighting actions identifies human behavior in the image frames in the queue, extracting features from spatial and temporal dimensions to capture motion information in multiple consecutive frames.
[0009] The preset temporal difference classification TDN (Temporal Difference Network) model is used to extract features based on the differences in consecutive image frames, construct wavelets for local motion information, and model global motion information across multiple sizes and slices.
[0010] The fighting score for each pedestrian is obtained based on the output of the 3D-CNN model and the TDN model. The fighting score is used to determine whether to trigger an alarm.
[0011] One possible implementation involves using a preset 3D-CNN model for fighting action recognition to identify human behavior in image frames within the queue, extracting features from spatial and temporal dimensions to capture motion information across multiple consecutive frames, including:
[0012] Select kd consecutive frames from the queue to form a 3D image queue, where kd is the number of consecutive frames processed by each 3D convolution kernel;
[0013] The 3D convolution kernel performs convolution calculations in the image sequence to obtain output feature values, which are used to determine the probability of whether pedestrians are fighting in the image frame.
[0014] One possible implementation involves the 3D convolution kernel performing convolution calculations in an image sequence, including: the 3D convolution kernel sliding across multiple consecutive frames, with each sliding kd consecutive frames performing multiplication and addition calculations on the elements at corresponding positions and the parameters in the convolution kernel.
[0015] One possible implementation is that the 3D-CNN model includes 3 convolutional layers, 1 fully connected layer and 2 pooling layers, wherein the first two convolutional layers are 3D convolutional layers with kernel sizes of 7×7×3 and 7×6×3, and the two convolutional kernels process regions of size 7×7 and 7×6 in 3 consecutive frames, respectively.
[0016] One possible implementation involves using a pre-defined temporal difference classification (TDN) model to extract features based on the differences between consecutive image frames, constructing wavefronts for local motion information, and modeling global motion information across multiple sizes and slices, including:
[0017] From each video segment, a graphic frame is randomly selected to obtain T frames I = [T1, T2, ..., Tn], where the shape of I is [T, C, H, W].
[0018] A video-level mobile information modeling framework TDN model is established by applying time difference maps and sparse sampling strategies. The TDN consists of S-TDM and L-TDM.
[0019] S-TDM is used to model local motion information in RGB images with overlaid time difference information, while L-TDM is used to model global motion information across multiple sizes and segments.
[0020] One possible implementation involves obtaining a first error rate and a second error rate for the 3D-CNN model and the TDN model on the test set, respectively. The first error rate is used to determine a first weight of the 3D-CNN model, and the second error rate is used to determine a second weight of the TDN model. The first weight and the second weight are used to calculate the overall fighting score.
[0021] One possible implementation involves determining whether a fight has occurred based on the fight score, while simultaneously determining whether the duration of the fight meets the warning time period. The cumulative fight value is then determined based on the fight score and the duration of the fight, and the set threshold is used to determine whether to trigger an alarm.
[0022] Secondly, embodiments of this application provide a fighting behavior recognition device, including:
[0023] The video processing module is used to acquire image frames from the video stream and store the image frames into a preset queue in chronological order.
[0024] The first feature acquisition module is used to identify human behavior in the image frames in the queue by using a preset 3D-CNN model for fighting action recognition, and to extract features from the spatial and temporal dimensions to capture motion information in multiple consecutive frames.
[0025] The second feature acquisition module is used to extract features based on the differences between consecutive image frames using a preset temporal difference classification (TDN) model, construct wavelets for local motion information and model global motion information across multiple sizes and segments.
[0026] The judgment module is used to obtain the fighting score of each pedestrian based on the output of the 3D-CNN model and the TDN model. The fighting score is used to determine whether to trigger an alarm.
[0027] Thirdly, embodiments of this application provide an electronic device, including:
[0028] processor;
[0029] Memory;
[0030] And a computer program, wherein the computer program is stored in the memory, the computer program including instructions that, when executed by the processor, cause the electronic device to perform the method described in any possible implementation of the first aspect.
[0031] Fourthly, embodiments of this application provide a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method described in any possible implementation of the first aspect.
[0032] In this embodiment, the 3D-CNN model directly operates on the input through a three-dimensional deep neural network, effectively extracting the spatiotemporal features of fighting behavior for detection. The TDN model captures short-term and long-term temporal information from the video. By adaptively combining the 3D convolutional recognition results and the TDN results, the unreasonableness caused by manually set parameters is reduced, effectively improving the recognition effect of fighting behavior. Attached Figure Description
[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 A flowchart illustrating a method for recognizing fighting behavior provided in an embodiment of this application;
[0035] Figure 2 A schematic diagram illustrating the process of retaining image frames after splitting a video stream, as provided in an embodiment of this application;
[0036] Figure 3 A schematic diagram of a CNN framework provided for an embodiment of this application;
[0037] Figure 4 A schematic diagram of a 3D-CNN framework provided in an embodiment of this application;
[0038] Figure 5 A schematic diagram of the structure of a 3D-CNN model provided in an embodiment of this application;
[0039] Figure 6 This is a schematic diagram of the structure of a TDN model provided in an embodiment of this application;
[0040] Figure 7 Example images of fighting behavior recognition provided in the embodiments of this application;
[0041] Figure 8 Example images of fighting behavior recognition provided in the embodiments of this application;
[0042] Figure 9 A schematic diagram of the frame of a fighting behavior recognition device provided in an embodiment of this application;
[0043] Figure 10 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0044] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0045] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0046] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.
[0047] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0048] Every year, tens of thousands of fights and other illegal and criminal acts occur in many public places, seriously endangering people's mental and property safety, and even threatening their lives. How to accurately identify the behavior of fighters has become an increasingly important focus and a crucial technology in video surveillance. With the rapid development of intelligent surveillance systems in China, it is of great significance for public place security monitoring and intelligent urban management. Currently, most public place surveillance systems only store video footage and do not include intelligent recognition. Security personnel need to rely on visual observation and manual operation to investigate suspicious suspects and abnormal behavior. Due to the limited quantity and quality of surveillance video, the efficiency of human observation is limited, making it impossible to monitor every video segment 24 hours a day. This results in some fights in public places not being detected and stopped in time, causing serious impact and harm. Based on the above reasons, this application provides a method for identifying fighting behavior.
[0049] See Figure 1 The above is a flowchart illustrating a method for recognizing fighting behavior provided in an embodiment of this application. See also... Figure 1 The fighting behavior recognition method in this embodiment includes:
[0050] S101, Obtain image frames from the video stream and store the image frames into a preset queue in chronological order.
[0051] This application primarily addresses the handling of fights in public places by using surveillance video to monitor crowds in real time. A video stream is obtained according to a preset time period, and the video stream is split into consecutive image frames. These image frames are arranged chronologically, and a pre-defined queue is used to store the image frames. In this embodiment, the queue follows a first-in, first-out (FIFO) principle.
[0052] like Figure 2 As shown, if a video stream is split into 10 image frames AJ, but the pedestrian's actions in image frames C and D, H and I remain essentially unchanged, this embodiment will select any one of image frames C and D, H and I. Thus, of the 10 image frames split from the video stream, only 8 need to be retained. These 8 image frames are sequentially stored in a queue, which not only does not affect the accuracy of pedestrian behavior recognition but also improves the speed of image frame processing.
[0053] The 10 image frames mentioned above are just illustrative. Generally, a video stream will contain more image frames and more similar image frames. Following the method described in this application can greatly improve the processing speed of image frames.
[0054] S102, using a preset 3D-CNN model for fighting action recognition, the system identifies human behavior in the image frames in the queue, extracting features from spatial and temporal dimensions to capture motion information in multiple consecutive frames.
[0055] Generally, a simple way to apply CNNs to videos is to use a CNN to recognize each frame of the image, such as... Figure 3 As shown in Figure 4, traditional 2D-CNN structures do not consider temporal features. Therefore, the 3D-CNN model proposed in this embodiment is used for action recognition. By performing 3D convolutions in the convolutional layers of the CNN, useful features can be learned in both spatial and temporal dimensions. As shown in Figure 4, 3D-CNN treats consecutive frames in the video as a spatiotemporal cube, using it as input to the CNN network. 3D convolutional kernels are used to operate on the spatiotemporal cube to extract spatial and temporal features. By selecting different convolutional kernels to convolve the cube, various spatiotemporal features can be obtained.
[0056] See Figure 5The 3D-CNN model provided in this embodiment includes three convolutional layers, one fully connected layer, and two pooling layers. The first two convolutional layers are 3D convolutional layers with kernel sizes of 7×7×3 and 7×6×3, meaning each kernel processes regions of 7×7 and 7×6 sizes across three consecutive frames. The action recognition model uses 3D convolution, enabling it to extract features from both spatial and temporal dimensions, thereby capturing motion information from multiple consecutive frames.
[0057] Specifically, in this embodiment, the image is normalized to a fixed size. It is input into a 3D-CNN model, and the model's output is the probability P of whether a fight has occurred. In the fight recognition task, kd represents the time dimension, which is the number of consecutive frames processed by each 3D convolutional kernel. In the 3D convolution calculation, kd consecutive frames are first assembled into a 3D image sequence, and then convolution calculation is performed on the image sequence. The 3D convolutional kernel slides across kd consecutive frames, and each time it slides, the element at the corresponding position in kd consecutive frames is multiplied and added with the parameters in the convolutional kernel, finally obtaining a value in the output feature map.
[0058] S103 uses a preset temporal difference classification (TDN) model to extract features based on the differences between consecutive image frames, constructs wavelets for local motion information, and models global motion information across multiple sizes and segments.
[0059] Transient Node Digest (TDN) is a 2D convolutional network-based method. Compared to 3D convolutional methods, TDN networks have fewer parameters, lower computational overhead, and are simpler to train. TDN proposes a sparse and global sampling strategy for classifying and recognizing each video. TDN has two modules: Short-term TDM and Long-term TDM. TDN investigates two specific forms to perform motion supplementation within local windows and motion enhancement across different segments, respectively. These two modules are integrated into the main network via residual connections.
[0060] Fighting is a continuous action within a time period, and fighting action recognition needs to fully consider the changes in action within that time period. Each video V is divided into T segments of equal length with no overlap. A frame is randomly extracted from each segment to obtain T frames I = [T1, T2, ..., Tn], where the shape of I is [T, C, H, W]. By applying time-difference mapping and sparse sampling strategies, a video-level motion information modeling framework, TDN, is proposed.
[0061] The TDN model in this embodiment mainly consists of two parts: S-TDM and L-TDM, which can be ported to any existing CNN with very low computational cost. S-TDM models local motion information for RGB images with superimposed time difference information, while L-TDM models global motion information across multiple sizes and slices. The model structure is as follows: Figure 6 As shown, the temporal model, because it utilizes features extracted from the differences between consecutive frames, achieves higher accuracy than models that directly utilize the original image features.
[0062] S104, Based on the outputs of the 3D-CNN model and the TDN model, obtain the fighting score for each pedestrian. The fighting score is used to determine whether to trigger an alarm.
[0063] In order to obtain the fighting score of pedestrians, this application embodiment obtains the first error rate and the second error rate of the 3D-CNN model and the TDN model on the test set, respectively. The first error rate is used to determine the first weight of the 3D-CNN model, and the second error rate is used to determine the second weight of the TDN model. The first weight and the second weight are used to calculate the overall fighting score.
[0064] In this embodiment, specifically, the first error rate e1 of the 3D-CNN model on the test set is first obtained. Based on the first error rate of the 3D-CNN model, the first weights of the 3D-CNN model are calculated. , log Obtain the second error rate e² of the temporal difference model on the test set, and calculate the second weights of the model based on the second error rate of the temporal difference model. , log .
[0065] Obtain the predicted probability values Gm(x) from the two models, based on the first weight. Second weight The obtained values are used to calculate the overall fighting score: .
[0066] In this embodiment, when determining whether a fight has occurred based on the fight score, it is also determined whether the duration of the fight action meets the warning time period. The cumulative fight value is determined based on the fight score and the duration of the fight action to determine whether a set threshold is reached. The set threshold is used to determine whether to issue an alarm.
[0067] For example, analysis of multiple image frames in a video stream might reveal pedestrians fighting, accumulating a certain score for the fight. However, if these image frames only last a few seconds, and subsequent monitored frames show normal pedestrian behavior, there might be a misjudgment, and no alarm should be triggered. Conversely, if a fight in a public place is determined, the fighting score reaches the threshold requiring an alarm, and the monitoring time meets the preset duration, then an alarm should be triggered. See also... Figure 7 and Figure 8 The image shown is a scene image of pedestrians fighting, identified by the method in this embodiment. It can be seen that the fighting behavior recognition method provided in this embodiment can effectively identify pedestrian fighting behavior.
[0068] As can be seen from the above embodiments, this application provides a method for recognizing fighting behavior. The 3D-CNN model directly operates on the input through a three-dimensional deep neural network, which can effectively extract the spatiotemporal feature information of fighting behavior for detection. The TDN model captures short-term and long-term temporal information in the video. By adaptively combining the 3D convolutional recognition results and the TDN results, the unreasonableness caused by manually set parameters is reduced, effectively improving the effect of fighting behavior recognition.
[0069] In addition to the fighting behavior recognition method provided in the above embodiments, this application also provides an embodiment of a fighting behavior recognition device.
[0070] See Figure 9 The fighting behavior recognition device 200 includes:
[0071] The video processing module 201 is used to acquire image frames from the video stream and store the image frames into a preset queue in chronological order.
[0072] The first feature acquisition module 202 is used to identify human behavior in the image frames in the queue by using a preset fighting action recognition 3D-CNN model, and extract features from the spatial and temporal dimensions to capture motion information in multiple consecutive frames.
[0073] The second feature acquisition module 203 is used to extract features based on the differences between consecutive image frames using a preset temporal difference classification (TDN) model, construct wavelets for local motion information, and model global motion information across multiple sizes and segments.
[0074] The judgment module 204 is used to obtain the fighting score of each pedestrian based on the output of the 3D-CNN model and the TDN model. The fighting score is used to determine whether to trigger an alarm.
[0075] Corresponding to the above embodiments, this application also provides an electronic device.
[0076] See Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device 300 may include a processor 301, a memory 302, and a communication unit 303. These components communicate via one or more buses. Those skilled in the art will understand that the electronic device structure shown in the figures does not constitute a limitation on the embodiments of this application. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0077] The communication unit 303 is used to establish a communication channel, thereby enabling the electronic device to communicate with other devices.
[0078] The processor 301 serves as the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 302, and calls data stored in the memory to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 301 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.
[0079] Memory 302 is used to store the execution instructions of processor 301. Memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0080] When the execution instructions in memory 302 are executed by processor 301, the electronic device 300 is able to perform some or all of the steps in the above method embodiments.
[0081] Corresponding to the above embodiments, this application also provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, wherein when the program runs, it can control the device where the computer-readable storage medium is located to execute some or all of the steps in the above method embodiments. In specific implementation, the computer-readable storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0082] Corresponding to the above embodiments, this application also provides a computer program product containing executable instructions that, when executed on a computer, cause the computer to perform some or all of the steps in the above method embodiments.
[0083] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.
[0084] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0085] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0086] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0087] The above description is merely a specific embodiment of this application. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application. The protection scope of this application should be determined by the protection scope of the claims.
Claims
1. A method for recognizing fighting behavior, characterized in that, include: Image frames are acquired from the video stream and stored in a preset queue in chronological order. The 3D-CNN model for recognizing fighting actions identifies human behavior in the image frames in the queue, extracting features from spatial and temporal dimensions to capture motion information in multiple consecutive frames. The process includes: selecting kd consecutive frames from the queue to form a 3D image queue, where kd is the number of consecutive frames processed by each 3D convolution kernel; the 3D convolution kernel performing convolution calculations in the image sequence to obtain output feature values, which are used to determine the probability of pedestrians fighting in the image frames; the 3D convolution kernel performing convolution calculations in the image sequence includes: the 3D convolution kernel sliding across multiple consecutive frames, and each time sliding through kd consecutive frames, multiplying and adding the elements at corresponding positions with the parameters in the convolution kernel. A pre-defined temporal difference classification (TDN) model is used to extract features based on the differences between consecutive image frames, construct wavelets for local motion information, and model global motion information across multiple sizes and segments. The fighting score for each pedestrian is obtained based on the output of the 3D-CNN model and the TDN model. The fighting score is used to determine whether to trigger an alarm.
2. The method for identifying fighting behavior according to claim 1, characterized in that, The 3D-CNN model includes 3 convolutional layers, 1 fully connected layer and 2 pooling layers. The first two convolutional layers are 3D convolutional layers with kernel sizes of 7×7×3 and 7×6×3, respectively. The two kernels process regions of size 7×7 and 7×6 in 3 consecutive frames.
3. The method for recognizing fighting behavior according to claim 2, characterized in that, The method employs a preset temporal difference classification (TDN) model to extract features based on the differences between consecutive image frames, constructs wavefronts for local motion information, and models global motion information across multiple sizes and slices, including: One graphic frame is randomly extracted from each video segment to obtain T frames I=[T1,T2,...,Tn], where the shape of I is [T,C,H,W]. A video-level mobile information modeling framework TDN model is established by applying time difference maps and sparse sampling strategies. The TDN consists of S-TDM and L-TDM. S-TDM is used to model local motion information in RGB images with overlaid time difference information, while L-TDM is used to model global motion information across multiple sizes and segments.
4. The method for identifying fighting behavior according to any one of claims 1-3, characterized in that, The first error rate and the second error rate of the 3D-CNN model and the TDN model on the test set are obtained respectively. The first error rate is used to determine the first weight of the 3D-CNN model, and the second error rate is used to determine the second weight of the TDN model. The first weight and the second weight are used to calculate the overall fighting score.
5. The method for identifying fighting behavior according to claim 4, characterized in that, When determining whether a fight has occurred based on the fight score, it is also determined whether the duration of the fight meets the warning time period. The cumulative fight value is determined based on the fight score and the duration of the fight, and the set threshold is used to determine whether to issue an alarm.
6. A fighting behavior recognition device, characterized in that, include: The video processing module is used to acquire image frames from the video stream and store the image frames into a preset queue in chronological order. The first feature acquisition module is used to identify human behavior in the image frames in the queue by using a preset 3D-CNN model for fighting action recognition, and to extract features from the spatial and temporal dimensions to capture motion information in multiple consecutive frames. The second feature acquisition module is used to extract features based on the differences between consecutive image frames using a preset temporal difference classification (TDN) model, construct wavelets for local motion information and model global motion information across multiple sizes and segments. The judgment module is used to obtain the fighting score of each pedestrian based on the output of the 3D-CNN model and the TDN model. The fighting score is used to determine whether to trigger an alarm.
7. An electronic device, characterized in that, include: processor; Memory; And a computer program, wherein the computer program is stored in the memory, the computer program including instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 5.