A fighting behavior recognition method based on an implicit social model

Abstract

The application discloses a fighting behavior recognition method based on an implicit social model and relates to the technical field of behavior recognition detection, and comprises the following steps: step S01, continuously image frames are extracted from a monitoring video stream; step S02, a fighting image dataset with labeled action candidate box coordinates and category information is used to train an action target detector; step S03, image enhancement preprocessing is performed on the image frames, and then the image frames are sent to a human target key point detection model adopting a YOLOPose human key point extraction model; and step S04, preprocessing operation is performed on the extracted human key point information. The application extracts the interaction information between people in the form of features by embedding a social information feature extraction network in the analysis of interframe time sequence and single-frame spatial information based on a YOLOPose and ActionNet video understanding framework, so that accurate recognition of fighting behavior and the occurrence position of the fighting behavior under monitoring video is realized.

Description

Technical Field

[0001] This invention relates to the field of behavior recognition and detection technology, and in particular to a method for recognizing fighting behavior based on an implicit social model. Background Technology

[0002] With social development and technological advancements, public safety has become a crucial factor in urban management. To maintain good social order, prevent violent incidents, and respond promptly and effectively to emergencies, an increasing number of cities are adopting advanced video surveillance technology and artificial intelligence systems to improve the safety management of public areas. Taking effective measures to prevent and respond quickly to such incidents in public places is particularly important. Fighting behavior recognition technology, as an emerging technological approach, has demonstrated enormous potential in this field.

[0003] Fighting behavior recognition technology is primarily based on computer vision and deep learning principles. By analyzing video streams captured by cameras, it automatically detects fighting behavior in the footage. When a suspected fighting pattern is identified, the system immediately triggers an alarm, notifying security personnel or the police to intervene promptly, thereby minimizing the harm caused by violent incidents. Deploying fighting behavior recognition algorithms in public places is not only a manifestation of technological progress but also an important measure to ensure public safety and maintain social stability.

[0004] Existing methods for identifying fighting behavior mainly fall into two categories: The first category is based on multi-frame video. This involves designing video understanding models to analyze and identify fighting behaviors appearing in the video, or designing skeleton extraction and skeleton behavior recognition models to identify target actions. This type of method can simultaneously analyze inter-frame temporal information and single-frame spatial information, but the network can only output the category of the fighting behavior and human skeleton information, unable to pinpoint the specific location of the fighting behavior in the video. Furthermore, because this type of method requires video analysis, it consumes significant resources, has a slow processing speed, and cannot meet real-time requirements. The second category is based on single-frame images. This involves target detection to classify and pinpoint the location of fighting behaviors appearing in the image. This type of method does not consider inter-frame relationships and simply detects fighting behavior from an image perspective. Compared to the first type, it is faster and has fewer parameters, but it has a higher rate of false positives, identifying many actions similar to the target fighting behavior. Secondly, fighting often occurs in two-person or multi-person scenarios, and existing methods cannot extract interaction information between people, leading to both false positives and false negatives when identifying fighting behavior. Summary of the Invention

[0005] In view of the above-mentioned shortcomings and deficiencies of the existing technology, the present invention provides a fighting behavior recognition method based on an implicit social model. Based on the YOLOPose and ActionNet video understanding framework, while analyzing the inter-frame temporal information and single-frame spatial information, a social information feature extraction network is embedded in it to extract the interaction information between people in the form of features, thereby realizing the accurate recognition of fighting behavior and its location in surveillance video.

[0006] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0007] A method for identifying fighting behavior based on an implicit social model includes the following steps:

[0008] Step S01: Extract consecutive image frames from the surveillance video stream. F t ( t =1,2,..., f );in, f The number of consecutive image frames;

[0009] Step S02: Using a fighting image dataset with action candidate box coordinates and category information labeled "fighting" and "non-fighting", train an action target detector with YOLOv7 as the target detection model;

[0010] Step S03: Image frame F t Image enhancement preprocessing is performed, followed by input into a human target keypoint detection model using the YOLOPose human keypoint extraction model to obtain the coordinates of human skeleton points; simultaneously, the image frames are... F t The image frame is fed into the motion target detector in step S02 to obtain the image frame. F t The target detection results of fighting behavior include the image location information of the fighting behavior marked by the target detection box and the fighting behavior category, which includes two categories: fighting and normal.

[0011] Step S04: Combining the bounding box coordinates of the target detection boxes for the fighting category actions in step S03, preprocess the extracted human key point information, and set a fixed calculation frame length. F From visible key points K n ( n =1,2,...,17), calculate the fixed frame length according to formula (1). M The speed of movement of an individual v i ( i =1,2,..., M ),

[0012] (1)

[0013] in, for N Intra-frame n A set of pixel displacements of key points in an individual human body. Define the displacement length of the keypoint with the largest displacement in the set; define the displacement velocities of human keypoints at three speed levels: low speed, medium speed, and high speed. M The movement speeds of individual behaviors are divided into three clusters of different speed magnitudes, resulting in... N Intra-frame M The movement information of an individual behavior is recorded as m , m =( m 1, m 2, m 3), m This indicates that M Individual behavior in N The set of visible key points within a frame; during the calculation process, the fixed calculation frame length F, representing the length of the time window, is the same as the frame number N.

[0014] Step S05: Construct a social information feature extraction network, which consists of a spatial convolution module and a temporal convolution module. The spatial features output by the spatial convolution module are... V spatial The temporal features output by the temporal convolution module are V temporal By V spatial and V temporal Convolutional fusion yields spatiotemporal features V st ;

[0015] Step S06: Set up a social information quantification module to quantify the spatiotemporal features from step S05 into scores; the social information quantification module uses a fully connected network based on logistic regression algorithm to train the weights of the fully connected network, and... V st Mapped to scores , Social information scores for sets of different velocity magnitudes;

[0016] Step S07: Use a pseudo-3D video understanding module based on the ActionNet network to extract the action categories of continuous image frames. The pseudo-3D video understanding module consists of a spatiotemporal excitation module, a channel excitation module, and a motion excitation module. The three modules are connected in parallel to form the backbone network of the video understanding network. After training the video understanding, the video-level behavior recognition task is completed to obtain the predicted behavior category.

[0017] Step S08: Embed the social information quantification module in step S06 into the pseudo 3D video understanding module in step S07 to form a fighting behavior recognition model based on the implicit social model, and train the fighting behavior recognition model based on the implicit social model in the form of multi-loss backpropagation.

[0018] Step S09: Use the fighting behavior recognition model based on the implicit social model trained in step S08 to determine whether there is fighting behavior in the video frame to be detected.

[0019] Furthermore, in step S02, the fighting image dataset consists of RGB images and skeletal images. The fighting image dataset includes fighting behavior images captured from the public datasets UBI-Fights and Hockey-Fights, as well as collected indoor fighting behavior images.

[0020] Furthermore, in step S03, the image frame F t Image enhancement preprocessing includes random cropping, flipping, and random color perturbation of the image.

[0021] Further, in step S04, the current velocity threshold set [0, 0.01, 0.1, 1.2] is used to... N Intra-frame M The movement speed of each individual was divided into three clusters of different speed magnitudes, among which the human body... N Intra-frame motion velocity is obtained by calibrating the camera to establish a mapping relationship between image pixel coordinates and corresponding world coordinates in space, thus acquiring the target human body's motion velocity. N The actual displacement within a frame time is used to calculate the human movement speed, and the human target is classified according to the speed value interval. The human key points of different speed levels after classification are given by formula (2):

[0022] (2)

[0023] in, C x , C y These are the top-left and bottom-right coordinates of the image location bounding box where the fighting occurred, as marked by the target detection box. W , HThese are the width and height of the image location bounding box, respectively. box conf , class conf These are the confidence values ​​for the target detection bounding box and the detection category, respectively. , The first n The horizontal and vertical coordinates of key points on an individual's body. Indicates the first n Confidence values ​​of key points in an individual's body.

[0024] Furthermore, in step S05, the spatiotemporal features V st Calculate according to formulas (3)-(5),

[0025] (3)

[0026] (4)

[0027] (5)

[0028] in, V spatial The spatial features output by the spatial convolution module. V temporal This represents the temporal features output by the temporal convolution module. V st for V spatial and V temporal Spatiotemporal features after convolutional fusion, Conv2D t Conv2D is a 2D convolution function used for temporal convolution. s is a 2D convolution function used for spatial convolution, and ReLU is an activation function in artificial neural networks.

[0029] Furthermore, in step S06, the loss function of the social information quantification module is formula (6).

[0030] (6)

[0031] in, S p Indicates the predicted score, S t Represents the true score. Indicates the number of samples.

[0032] Furthermore, in step S07, the pseudo-3D video understanding module is inserted into the ResNet-50 backbone network to form the basic architecture of the video understanding module.

[0033] Furthermore, in step S08, the video classification loss function of the pseudo-3D video understanding module uses FocalLoss, i.e., formula (7).

[0034] (7)

[0035] in, It is the loss value of the video classification loss function. The weight parameters are for easily classified samples. For hyperparameters, for positive samples, equal For negative samples, equal . Calculate according to formula (8),

[0036] (8)

[0037] in, To predict the probability that a sample is classified as "fighting" for the model, To predict the true category value.

[0038] Further, in step S09, the method for determining whether there is fighting behavior in the video frame to be detected is as follows: the video classification module outputs the "fighting" category, and at the same time, a target box of the "fighting" category appears in a single frame of video, and the predicted probability value of the target box, i.e. the category, is greater than the threshold set by the model inference. During inference, the thresholds are set to 0.5 and 0.65 respectively. Video frames that meet the above conditions are determined to have "fighting" behavior.

[0039] Furthermore, the training loss function of the fighting behavior recognition model based on the implicit social model is Equation (9).

[0040] (9)

[0041] in, , These are the weighted loss values ​​for classification and regression tasks, respectively. , These are the loss functions for video classification and social information score prediction, respectively.

[0042] The beneficial effects of this invention are:

[0043] 1. This invention uses the changes in human body key points in consecutive frames to model velocity information, reflecting the inter-frame velocity changes when fighting occurs, thus enhancing the network's ability to recognize fighting behavior.

[0044] 2. This invention combines the structure of human skeletal point movement speed information with social relationship feature extraction network, and extracts the social relationships of human bodies with similar movement speeds within multiple frames based on convolutional neural network, so as to achieve effective modeling of fighting behavior with rapid speed changes.

[0045] 3. This invention employs a lightweight and real-time pseudo-3D video understanding module, which enhances the network's ability to extract temporal information.

[0046] 4. This invention employs the Focal Loss function to address the model training problem caused by imbalanced samples. Furthermore, it quantifies social information into scores, which are then jointly learned with the video understanding module, combining classification and regression tasks to improve the accuracy of fighting behavior recognition. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the social information feature extraction network of the present invention;

[0048] Figure 2 This is a schematic diagram of the pseudo-3D video understanding module structure of the present invention;

[0049] Figure 3 This is a schematic diagram of the basic architecture of the video understanding module of the present invention;

[0050] Figure 4 This is a schematic diagram of the basic architecture of the fighting behavior recognition method based on the implicit social model of the present invention. Detailed Implementation

[0051] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] This invention provides a method for identifying fighting behavior based on an implicit social model, comprising the following steps:

[0053] Step S01: Extract consecutive image frames from the surveillance video stream. F t ( t =1,2,..., f, );in, f The number of consecutive image frames;

[0054] Step S02: Using a dataset of fighting images labeled with action candidate bounding box coordinates ("fighting" and "non-fighting") and category information, a YOLOv7-based action target detector is trained to predict fighting behavior at the image level. Specifically, the action target detector uses the YOLOv7 target detection model as its basic architecture and pre-trains the YOLOv7 model using fighting images. The trained model is used to obtain the target detection results of fighting behavior in image frames, including the image location information of the fighting behavior marked by the target detection box and the fighting behavior category. These two types of information are used to preprocess human motion information and assist in constructing a social information feature extraction network.

[0055] The fighting image dataset comprises images collected from publicly available datasets and a collection of images from video stream frames. It consists of RGB images and skeletal images. Specifically, the dataset includes fighting behavior images captured from the publicly available UBI-Fights and Hockey-Fights datasets, as well as collected indoor fighting behavior images. More specifically, the dataset consists of 3486 RGB images and 1754 skeletal images, with a training set of 4200 samples and a test set of 1040 samples. The data annotations include the coordinates and category information of action candidate boxes for "fighting" and "non-fighting".

[0056] Step S03: Image frame F t Perform image enhancement preprocessing, specifically, image frames F t Image enhancement preprocessing includes random cropping, flipping, and random color perturbation of the image. The image is then fed into a human target keypoint detection model using the YOLOPose human keypoint extraction model to obtain the coordinates of human skeleton points. Specifically, the image size input to the human keypoint detection model is 640 pixels, the human keypoint confidence threshold is set to 0.5, and the candidate box intersection-over-union threshold is set to 0.65. Simultaneously, an action target detector is used to process the image frames. F t Perform target action recognition, that is, image frames F t The image frame is fed into the motion target detector in step S02 to obtain the image frame. F t The target detection results for fighting behavior include image location information of the fighting behavior marked by the target detection box and the fighting behavior category, which includes two categories: fighting and normal. Specifically, the image size input to the action target detector is 640, the action category confidence threshold is set to 0.55, and the candidate box intersection-union threshold is set to 0.55.

[0057] After the implicit social model-based fighting behavior recognition model of the present invention is trained, the monitoring video stream used for inference and use is a real-time acquired video stream captured by remote monitoring. The target detector and the human target key point detection model are respectively the YOLOv7 target detection model and the YOLOPose human key point extraction model. The two are responsible for processing the collected video monitoring data. The former is used to output the detection result of the fighting category, and the latter is used to output the human key points.

[0058] Step S04: Combining the bounding box coordinates of the target detection boxes for the fighting category actions in step S03, preprocess the extracted human key point information, and set a fixed calculation frame length. F Specifically, the fixed calculation frame length is set to 16, based on visible keypoints. K n ( n =1,2,...,17), calculate the fixed frame length according to formula (1). M The speed of movement of an individual v i ( i =1,2,..., M ),

[0059] (1)

[0060] in, for N Intra-frame n A set of pixel displacements of key points in an individual human body. The set defines the displacement length of the keypoint with the largest displacement in the set; it also defines the displacement velocities of human keypoints at three speed levels: low, medium, and high. Specifically, the speed threshold set is set to [0, 0.01, 0.1, 1.2], meaning if... v i If the value is greater than 0 and less than or equal to 0.01, it is considered low speed; if v i If the value is greater than 0.01 and less than or equal to 0.1, it is considered medium speed; if v i A value greater than 0.1 and less than or equal to 1.2 indicates a high speed. M The movement speeds of individual behaviors are divided into three clusters of different speed magnitudes, resulting in... N Intra-frame M The motion information of each individual behavior is obtained, that is, the motion information of M individuals behavior within 16 frames, denoted as... m , m =( m 1, m 2, m 3), m This indicates thatM Individual behavior in N The set of visible key points within a frame, where the human body's movement speed within N frames is determined by calibrating the camera, establishing a mapping relationship between image pixel coordinates and the world coordinates of corresponding points in space, obtaining the actual displacement of the target human body within N frames, calculating the human body's movement speed, and classifying the human target according to the speed value interval. The human body key points of different speed levels after classification are given by formula (2):

[0061] (2)

[0062] in, C x , C y These are the top-left and bottom-right coordinates of the image location bounding box where the fighting occurred, as marked by the target detection box. W , H These are the width and height of the image location bounding box, respectively. box conf , class conf These are the confidence values ​​for the target detection bounding box and the detection category, respectively. , The first n The horizontal and vertical coordinates of key points on an individual's body. Indicates the first n The confidence score of each human body keypoint. The YOLOPose human body keypoint extraction model is used to detect 17 keypoints of the human body and output the coordinate position and confidence score of each keypoint.

[0063] Step S05: Construct a social information feature extraction network to extract spatiotemporal features of changes in the movement speed of key human body points and individual variations from the obtained skeletal motion information. For example... Figure 1 As shown, the social information feature extraction network consists of a spatial convolution module and a temporal convolution module. The spatial features output by the spatial convolution module are... V spatial , representing the spatial features output by the spatial convolution module; the temporal features output by the temporal convolution module are . V temporal , representing the temporal features output by the temporal convolution module, by... V spatial and V temporal Convolutional fusion yields spatiotemporal features V st .

[0064] Specifically, in step S05, spatiotemporal features V st Calculate according to formulas (3)-(5),

[0065] (3)

[0066] (4)

[0067] (5)

[0068] in, V spatial The spatial features output by the spatial convolution module. V temporal This represents the temporal features output by the temporal convolution module. V st for V spatial and V temporal Spatiotemporal features after convolutional fusion, Conv2D t Conv2D is a 2D convolution function used for temporal convolution. s This is a 2D convolution function used for spatial convolution, with a kernel size of 3, a padding size of 1, and ReLU is the activation function in artificial neural networks.

[0069] Step S06: Set up a social information quantification module to quantify the spatiotemporal features from step S05 into scores; the social information quantification module uses a fully connected network based on logistic regression algorithm to train the weights of the fully connected network, and... V st Mapped to scores , For social information scores of different velocity scales, this step trains the weights of a fully connected network to achieve inference and prediction of social scores, where the dimension of the fully connected network is set to (2048, 1).

[0070] Specifically, in step S06, the loss function of the social information quantification module is formula (6).

[0071] (6)

[0072] The loss is calculated using the mean squared error loss function, which measures the average difference between the model's predicted values ​​and the actual values. S p Indicates the predicted score, S t Represents the true score. Indicates the number of samples.

[0073] Step S07: Extract action categories from consecutive image frames using a pseudo-3D video understanding module based on the ActionNet network architecture. Specifically, in step S07, the pseudo-3D video understanding module is inserted into the ResNet-50 backbone network, forming the basic architecture of the video understanding module. A schematic diagram of the inserted pseudo-3D video understanding module structure is shown below. Figure 2 As shown, the basic architecture of the video understanding module is as follows: Figure 3 As shown.

[0074] The pseudo-3D video understanding module consists of a spatiotemporal excitation module, a channel excitation module, and a motion excitation module. These three modules are connected in parallel to form the backbone network of the video understanding network. After training, the video understanding module completes the video-level behavior recognition task and obtains the predicted behavior category.

[0075] Step S08: Embed the social information quantification module in step S06 into the pseudo 3D video understanding module in step S07 to form a fighting behavior recognition model based on an implicit social model, and train the fighting behavior recognition model based on an implicit social model in the form of multi-loss backpropagation; the network training process is realized by backpropagation of the loss function and updating the model parameters.

[0076] Specifically, in step S08, the video classification loss function of the pseudo-3D video understanding module uses FocalLoss, i.e., formula (7).

[0077] (7)

[0078] in, It is the loss value of the video classification loss function. The weight parameters are for easily classified samples. For hyperparameters, for positive samples, equal For negative samples, equal ,in, and The values ​​of the two hyperparameter variables were set to 0.25 and 2.0, respectively; Calculate according to formula (8),

[0079] (8)

[0080] in, The model predicts the probability that a sample will be classified as "fighting," with a value ranging from 0 to 1. It is the probability that the sample category is "normal"; To predict the true category value.

[0081] Step S09: Use the fighting behavior recognition model based on the implicit social model trained in step S08 to determine whether there is fighting behavior in the video frame to be detected.

[0082] Specifically, in step S09, the method for determining whether there is fighting behavior in the video frame to be detected is as follows: the video classification module outputs the "fighting" category, and at the same time, a target box of the "fighting" category appears in a single frame of video, and the predicted probability value of the target box, i.e. the category, is greater than the threshold set by the model inference. During inference, the thresholds are set to 0.5 and 0.65 respectively. Video frames that meet the above conditions are determined to have "fighting" behavior.

[0083] Specifically, the training loss function of the fighting behavior recognition model based on the implicit social model is formula (9).

[0084] (9)

[0085] in, , These are the loss weights for classification and regression tasks, set to 0.6 and 0.4 respectively. These values ​​can be adjusted flexibly based on the training performance of the two models. , These are the loss functions for video classification and social information score prediction, respectively. A schematic diagram of the basic architecture of the fighting behavior recognition method based on an implicit social model of this invention is shown below. Figure 4 As shown.

[0086] The method for determining whether a fighting behavior exists in a video frame to be detected is as follows: the video classification module outputs the "fighting" category, and at the same time, a target box of the "fighting" category appears in a single frame of the video, and the predicted probability value of the target box, i.e. the category, is greater than the threshold set by the model inference. During inference, the thresholds are set to 0.5 and 0.65 respectively. Video frames that meet the above conditions are determined to have "fighting" behavior.

[0087] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Any modifications, alterations, substitutions, and variations made by those skilled in the art to the above embodiments are within the scope of the present invention.

Claims

1. A method for recognizing fighting behavior based on an implicit social model, characterized in that: Includes the following steps: Step S01: Extract consecutive image frames from the surveillance video stream. F t ，t =1,2,..., f ;in, f The number of consecutive image frames; Step S02: Using a fighting image dataset with action candidate box coordinates and category information labeled "fighting" and "non-fighting", train an action target detector with YOLOv7 as the target detection model; Step S03: Image frame F t Image enhancement preprocessing is performed, followed by input into a human target keypoint detection model using the YOLOPose human keypoint extraction model to obtain the coordinates of human skeleton points; simultaneously, the image frames are... F t The image frame is fed into the motion target detector in step S02 to obtain the image frame. F t The target detection results of fighting behavior include the image location information of the fighting behavior marked by the target detection box and the fighting behavior category, which includes two categories: fighting and normal. Step S04: Combining the bounding box coordinates of the target detection boxes for the fighting behavior category actions in Step S03, preprocess the extracted human keypoint information, and set a fixed calculation frame length. F From visible key points K n , n =1,2,...,17, calculate the fixed calculation frame length according to formula (1). M The speed of movement of an individual v i , i =1,2,..., M , （1） in, for N Intra-frame n A set of pixel displacements of key points in an individual human body. Define the displacement length of the keypoint with the largest displacement in the set; define the displacement velocities of human keypoints at three speed levels: low speed, medium speed, and high speed. M The movement speeds of individual behaviors are divided into three clusters of different speed magnitudes, resulting in... N Intra-frame M The movement information of an individual behavior is recorded as m , m =( m 1, m 2, m 3), m This indicates that M Individual behavior in N The set of visible key points within a frame; Step S05: Construct a social information feature extraction network to extract spatiotemporal features of human key point movement speed changes and individual variations from the obtained skeletal motion information. The social information feature extraction network consists of a spatial convolution module and a temporal convolution module. The spatial features output by the spatial convolution module are... V spatial The temporal features output by the temporal convolution module are V temporal By V spatial and V temporal Convolutional fusion yields spatiotemporal features V st ; Step S06: Set up a social information quantification module to quantify the spatiotemporal features from step S05 into scores; the social information quantification module uses a fully connected network based on logistic regression algorithm to train the weights of the fully connected network, and... V st Mapped to scores , Social information scores for sets of different velocity magnitudes; Step S07: Use a pseudo-3D video understanding module based on the ActionNet network to extract the action categories of continuous image frames. The pseudo-3D video understanding module consists of a spatiotemporal excitation module, a channel excitation module, and a motion excitation module. The three modules are connected in parallel to form the backbone network of the video understanding network. After training the video understanding, the video-level behavior recognition task is completed to obtain the predicted behavior category. Step S08: Embed the social information quantification module in step S06 into the pseudo 3D video understanding module in step S07 to form a fighting behavior recognition model based on the implicit social model, and train the fighting behavior recognition model based on the implicit social model in the form of multi-loss backpropagation. Step S09: Use the fighting behavior recognition model based on the implicit social model trained in step S08 to determine whether there is fighting behavior in the video frame to be detected.

2. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S02, the fighting image dataset consists of RGB images and skeletal images. The fighting image dataset includes fighting behavior images captured from the public datasets UBI-Fights and Hockey-Fights, as well as collected indoor fighting behavior images.

3. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S03, the image frame F t Image enhancement preprocessing includes random cropping, flipping, and random color perturbation of the image.

4. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S04, the current velocity threshold set [0, 0.01, 0.1, 1.2] is used to... N Intra-frame M The movement speed of each individual was divided into three clusters of different speed magnitudes, among which the human body... N Intra-frame motion speed is obtained by calibrating the camera to establish a mapping relationship between image pixel coordinates and corresponding world coordinates in space, thus acquiring the target human body's motion speed. N The actual displacement within a frame time is used to calculate the human movement speed, and the human target is classified according to the speed value interval. The human key points of different speed levels after classification are given by formula (2): （2） in, C x , C y These are the top-left and bottom-right coordinates of the image location bounding box where the fighting occurred, as marked by the target detection box. W , H These are the width and height of the image location bounding box, respectively. box conf , class conf These are the confidence values ​​for the target detection bounding box and the detection category, respectively. , The first n The horizontal and vertical coordinates of key points on an individual's body. Indicates the first n Confidence values ​​of key points in an individual's body.

5. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S05, spatiotemporal features V st Calculate according to formulas (3)-(5), （3） （4） （5） in, V spatial The spatial features output by the spatial convolution module. V temporal This represents the temporal features output by the temporal convolution module. V st for V spatial and V temporal The spatiotemporal features after convolutional fusion, where ReLU is the activation function in artificial neural networks.

6. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S06, the loss function of the social information quantification module is formula (6). （6） in, S p Indicates the predicted score, S t Represents the true score. Indicates the number of samples.

7. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S07, the pseudo-3D video understanding module is inserted into the ResNet-50 backbone network, forming the basic architecture of the video understanding module.

8. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S08, the video classification loss function of the pseudo 3D video understanding module uses Focal Loss, i.e., formula (7). （7） in, It is the loss value of the video classification loss function. The weight parameters are for easily classified samples. For hyperparameters, for positive samples, equal For negative samples, equal , Calculate according to formula (8), （8） in, To predict the probability that a sample is classified as "fighting" for the model. To predict the true category value.

9. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: In step S09, the method for determining whether there is fighting behavior in the video frame to be detected is as follows: the video classification module outputs the "fighting" category, and at the same time, a target box of the "fighting" category appears in a single frame of video, and the predicted probability value of the target box, i.e. the category, is greater than the threshold set by the model inference. During inference, the thresholds are set to 0.5 and 0.65 respectively. Video frames that meet the above conditions are determined to have "fighting" behavior.

10. The method for identifying fighting behavior based on an implicit social model according to claim 1, characterized in that: The training loss function of the fighting behavior recognition model based on the implicit social model is Equation (9). （9） in, , These are the weighted loss values ​​for classification and regression tasks, respectively. , These are the loss functions for video classification and social information score prediction, respectively.

Images