Pedestrian abnormal behavior recognition method and device based on attitude tracking
By adopting a pedestrian abnormal behavior recognition method based on posture tracking, abnormal behavior recognition is decoupled into posture tracking and action recognition, which solves the problems of high accuracy and generalization of monitoring image models in different scenarios and achieves high-precision recognition of pedestrian behavior.
Patent Information
- Application Number
- CN202411042568.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-15
AI Technical Summary
Existing abnormal behavior recognition models for surveillance images require a large amount of data and are difficult to achieve high-precision and strong generalization recognition in various monitoring scenarios.
A pedestrian abnormal behavior recognition method based on posture tracking is adopted. By using feature extraction, feature fusion and pedestrian posture tracking modules, abnormal behavior recognition is decoupled into posture tracking and action recognition problems, and behavior recognition is performed using an abnormal posture classification module.
It achieves strong generalization and high accuracy in pedestrian abnormal behavior recognition, and improves the model's recognition ability in different scenarios.
Smart Images

Figure CN118968622A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the fields of computer vision and motion recognition, and relates to a method for identifying abnormal behavior of pedestrians, and specifically to a method and device for identifying abnormal behavior of pedestrians based on posture tracking. Background Art
[0002] Abnormal behavior of pedestrians in surveillance images is an important part of the video analysis algorithm. This task provides alarm information to the intelligent security platform by analyzing the real-time posture of pedestrians in real-time surveillance video streams. Abnormal behavior recognition of pedestrians in surveillance images can detect different actions of pedestrians in specific scenes, such as standing still, walking, running, fighting, and falling. Effective monitoring of these behaviors can accurately detect abnormal behavior events in the monitored area, and can also issue corresponding alarms in a timely manner to minimize the loss of life and property caused by sudden abnormal events or accidents. Real-time and efficient abnormal behavior detection can also greatly improve the intelligence level of the monitoring system, which can greatly save human and material resources, and play an important role in protecting the safety of people's lives and property and maintaining social security and stability.
[0003] In recent years, with the continuous development of deep learning, especially the excellent performance of convolutional neural networks in target recognition, more and more behavior recognition tasks can be efficiently automated. Some studies have applied convolutional neural networks to the task of abnormal behavior recognition in surveillance images and achieved good recognition results.
[0004] However, the current surveillance image abnormal behavior recognition model has high requirements for the amount of data. Whether there is enough abnormal behavior image data will directly affect the model effect. In reality, due to different shooting scenes, technologies, lighting, angles, and pedestrian appearances, there will not be enough pedestrians performing enough abnormal behaviors in the same surveillance scene. Therefore, it is difficult for general surveillance image abnormal behavior recognition models to have strong generalization and high-precision recognition performance in various surveillance scenes.
[0005] Therefore, the existing abnormal behavior recognition methods for surveillance images still have problems such as the need to improve recognition accuracy and poor generalization. Summary of the invention
[0006] The present invention is made to solve the above-mentioned problem, and its purpose is to provide a method and device for identifying abnormal behavior of pedestrians based on posture tracking.
[0007] The present invention provides a method for identifying abnormal pedestrian behavior based on posture tracking, which is used to obtain abnormal pedestrian behavior identification results based on multiple continuous frames of a monitoring video stream, and has the following characteristics, including the following steps: step S1, preprocessing each continuous frame to obtain a corresponding preprocessed image; step S2, inputting all preprocessed images into a pedestrian abnormal behavior identification model to obtain abnormal pedestrian behavior identification results, wherein the pedestrian abnormal behavior identification model includes: a feature extraction module, which is used to extract three levels of features from each preprocessed image; a feature fusion module, which is used to fuse the features of the three levels to obtain three fused features; a pedestrian posture tracking module, which is used to obtain pedestrian posture estimation data based on the three fused features; an abnormal posture classification module, which is used to obtain the corresponding pedestrian abnormal behavior identification results of the pedestrians based on the posture estimation graphs of each pedestrian in the pedestrian posture estimation data corresponding to all continuous frames, and the pedestrian abnormal behavior identification results are normal behavior or abnormal behavior.
[0008] The method for identifying abnormal pedestrian behavior based on posture tracking provided by the present invention may also have the following characteristics: wherein, the construction process of the pedestrian abnormal behavior recognition model includes the following steps: step T1, using the feature extraction module, the feature fusion module and the pedestrian posture tracking module as the pedestrian posture tracking model, and using the abnormal posture classification module as the action recognition model; step T2, using multiple labeled images as the first training set; step T3, training the pedestrian posture tracking model according to the first training set to obtain a trained pedestrian posture tracking model; step T4, inputting each image in the first training set into the trained pedestrian posture tracking model in sequence to obtain the corresponding posture estimation map; step T5, using all posture estimation maps and corresponding action categories as the second training set; step T6, training the action recognition model according to the second training set to obtain a trained action recognition model; step T7, constructing a pedestrian abnormal behavior recognition model based on the trained pedestrian posture tracking model and the trained action recognition model.
[0009] The method for identifying abnormal pedestrian behavior based on posture tracking provided by the present invention may also have the following features: wherein the pedestrian posture estimation data includes a pedestrian heat map, a posture point heat map, a pedestrian detection frame and a pedestrian identification number. In step T3, the pedestrian posture estimation data generated by the image input pedestrian posture tracking model and the pedestrian posture corresponding to the image are used to calculate the loss error, and the parameters of the pedestrian posture tracking model are updated through back propagation of the loss error.
[0010] The method for identifying abnormal pedestrian behavior based on posture tracking provided by the present invention may also have the following characteristics: wherein the loss error includes cross entropy loss, posture key point distance loss and pedestrian feature vector distance loss.
[0011] The method for identifying abnormal pedestrian behavior based on posture tracking provided by the present invention may also have the following characteristics: wherein the action categories include normal behavior and abnormal behavior, and in step T6, the loss error is calculated based on the abnormal pedestrian behavior recognition result generated by the posture estimation graph input into the action recognition model and the action category corresponding to the posture estimation graph, and the action recognition model is updated by back propagation of the loss error.
[0012] The method for identifying abnormal pedestrian behavior based on posture tracking provided by the present invention may also have the following feature: in step T6, when the classification accuracy of all images obtained by the action recognition model reaches a preset value, a trained action recognition model is obtained, and the loss error is a cross entropy loss error.
[0013] The method for identifying abnormal pedestrian behavior based on posture tracking provided by the present invention may also have the following characteristics: wherein the posture estimation graph is composed of black and white colors, the skeleton of the pedestrian in the posture estimation graph is white, and the other parts of the posture estimation graph are set to black.
[0014] The method for identifying abnormal pedestrian behavior based on posture tracking provided by the present invention may also have the following features: wherein the preprocessing includes scaling the long sides of continuous frames to a specific size and adjusting the short sides accordingly in proportion, filling the short sides, and performing standardization operations.
[0015] The present invention also provides a pedestrian abnormal behavior identification device based on posture tracking, which is used to obtain pedestrian abnormal behavior identification results based on multiple consecutive frames of a monitoring video stream, and has the following characteristics, including: a preprocessing unit, which is used to preprocess each consecutive frame to obtain a corresponding preprocessed image; a behavior identification unit, including a pedestrian abnormal behavior identification model, which is used to input all preprocessed images into the pedestrian abnormal behavior identification model to obtain pedestrian abnormal behavior identification results, wherein the pedestrian abnormal behavior identification model includes: a feature extraction module, which is used to extract three levels of features from each preprocessed image; a feature fusion module, which is used to fuse the three levels of features to obtain three fused features; a pedestrian posture tracking module, which is used to obtain pedestrian posture estimation data based on the three fused features; an abnormal posture classification module, which is used to obtain the corresponding pedestrian abnormal behavior identification result of the pedestrian based on the posture estimation map of each pedestrian in the pedestrian posture estimation data corresponding to all consecutive frames, and the pedestrian abnormal behavior identification result is normal behavior or abnormal behavior.
[0016] Functions and Effects of the Invention
[0017] According to the method and device for identifying abnormal behavior of pedestrians based on posture tracking involved in the present invention, the pedestrian posture trajectories of consecutive different frames are obtained through the feature extraction module, the feature fusion module and the pedestrian posture tracking module, and these posture trajectories are used to further cut out continuous pictures containing only a single pedestrian posture in the original image as posture estimation graphs, so as to perform behavior recognition through the abnormal posture classification module, decouple the abnormal behavior recognition problem into two problems: posture tracking and action recognition using posture graphs, and achieve strong generalization and high precision of pedestrian behavior recognition. Therefore, the method and device for identifying abnormal behavior of pedestrians based on posture tracking of the present invention have good generalization and accuracy of abnormal behavior recognition of pedestrians. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 is a block diagram of a device for identifying abnormal behavior of pedestrians in an embodiment of the present invention;
[0019] Figure 2 is a block diagram of a pedestrian abnormal behavior recognition model in an embodiment of the present invention;
[0020] Figure 3 is a schematic diagram of the structure of a feature extraction module, a feature fusion module, and a pedestrian posture tracking module in an embodiment of the present invention;
[0021] Figure 4 is a schematic diagram of a process for constructing a pedestrian abnormal behavior recognition model in an embodiment of the present invention;
[0022] Figure 5 It is a flow chart of a method for identifying abnormal pedestrian behavior based on posture tracking in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate the method and device for identifying abnormal pedestrian behavior based on posture tracking of the present invention.
[0024] The present embodiment provides a pedestrian abnormal behavior recognition device based on posture tracking, hereinafter referred to as the pedestrian abnormal behavior recognition device, which is used to obtain pedestrian abnormal behavior recognition results based on multiple continuous frames of a monitoring video stream.
[0025] In this embodiment, the real-time streaming protocol of the monitoring video stream selects the RTSP stream of the camera end, the video encoding and decoding method adopts H.264, the video frame rate is 24FPS, and the video resolution is 720P.
[0026] Figure 1 It is a block diagram of a device for identifying abnormal behavior of pedestrians in an embodiment of the present invention.
[0027] like Figure 1As shown, the pedestrian abnormal behavior recognition device 100 includes a preprocessing unit 11 and a behavior recognition unit 12 .
[0028] The preprocessing unit 11 is used to preprocess each continuous frame to obtain a corresponding preprocessed image.
[0029] The preprocessing includes scaling the long sides of consecutive frames to a specific size and adjusting the short sides accordingly, padding the short sides, and performing a standardization operation.
[0030] The behavior recognition unit 12 includes a pedestrian abnormal behavior recognition model, which is used to input all pre-processed images into the pedestrian abnormal behavior recognition model to obtain pedestrian abnormal behavior recognition results.
[0031] Figure 2 It is a block diagram of a pedestrian abnormal behavior recognition model in an embodiment of the present invention.
[0032] like Figure 2 As shown, the pedestrian abnormal behavior recognition model 200 includes a feature extraction module 21, a feature fusion module 22, a pedestrian posture tracking module 23 and an abnormal posture classification module 24.
[0033] The feature extraction module 21 is used to extract three levels of features from each pre-processed image.
[0034] The feature fusion module 22 is used to fuse the features of the three levels to obtain three fused features.
[0035] The pedestrian posture tracking module 23 is used to obtain pedestrian posture estimation data according to the three fusion features.
[0036] Among them, pedestrian posture estimation data includes pedestrian heat map, posture point heat map, pedestrian detection box and pedestrian identification number.
[0037] In this embodiment, the pedestrian posture estimation data specifically includes the pedestrian probability heat map at each position on the preprocessed image, namely the pedestrian heat map, 17 posture key point heat maps, namely the posture point heat map, the pedestrian prediction box coordinates, namely the pedestrian detection box, and the pedestrian re-identification features, namely the pedestrian identification number.
[0038] In this embodiment, the pedestrian probability heat map is used to take the points with larger values in the feature map as the most likely suspicious locations of pedestrians; the width and height information around the suspicious locations of pedestrians are predicted through the coordinates of the pedestrian prediction box to obtain the detection box; the 17 posture key point heat map is used to obtain the posture information, i.e. the possible locations of pedestrian joints, from the points with larger values in the feature map, i.e. the 17 posture key points; the same pedestrian target between different frames is located through the pedestrian re-identification features, and the above data are comprehensively used to obtain the pedestrian posture trajectories of consecutive frames, thereby extracting the posture estimation map.
[0039] Figure 3 It is a structural diagram of a feature extraction module, a feature fusion module, and a pedestrian posture tracking module in an embodiment of the present invention.
[0040] like Figure 3 As shown, the feature extraction module 21 includes a convolution layer C0, a maximum pooling layer, 3 residual modules C1, 4 residual modules C2, 6 residual modules C3, and 3 residual modules C4.
[0041] The convolution kernel size of the convolution layer C0 is 7×7, the sliding step is 2, and the output is 320×320×64.
[0042] The pooling size of the maximum pooling layer is 3×3, the sliding step is 2, and the output is 160×160×64.
[0043] The convolution kernel size of the residual module C1 is 3×3, the sliding step is 2, and the output is 160×160×64.
[0044] The convolution kernel size of the residual module C2 is 3×3, the sliding step is 2, and the output is 80×80×128.
[0045] The convolution kernel size of the residual module C3 is 3×3, the sliding step is 2, and the output is 40×40×256.
[0046] The convolution kernel size of the residual module C4 is 3×3, the sliding step is 2, and the output is 20×20×512.
[0047] The feature fusion module 22 includes a convolution layer C2-0, a convolution layer C3-0, a convolution layer C4-0, an upsampling layer U0, an upsampling layer U1, a convolution layer C2-1, a convolution layer C3-1 and a convolution layer C4-1.
[0048] The convolution kernel size of convolution layer C2-0 is 3×3, the sliding step is 2, and the output is 80×80×128.
[0049] The convolution kernel size of the convolution layer C3-0 is 3×3, the sliding step is 2, and the output is 40×40×256.
[0050] The convolution kernel size of the convolution layer C4-0 is 3×3, the sliding step is 2, and the output is 20×20×512.
[0051] The upsampling layer U0 uses bilinear interpolation and the output is 40×40×512.
[0052] The upsampling layer U1 uses bilinear interpolation and the output is 80×80×256.
[0053] The convolution kernel size of convolution layer C2-1 is 3×3, the sliding step is 2, and the output is 80×80×256.
[0054] The convolution kernel size of convolution layer C3-1 is 3×3, the sliding step is 2, and the output is 40×40×256.
[0055] The convolution kernel size of convolution layer C4-1 is 3×3, the sliding step is 2, and the output is 20×20×256.
[0056] Convolutional layer C2-1, convolutional layer C3-1, and convolutional layer C4-1 correspond to features of one level respectively, totaling to features of three levels.
[0057] The pedestrian posture tracking module 23 includes convolution layer C5, convolution layer C6, convolution layer C7-0, convolution layer C7-1, convolution layer C7-2 and convolution layer C7-3.
[0058] The convolution kernel size of the convolution layer C5 is 3×3, the sliding step is 2, and the output of the features of the three levels are 80×80×256, 40×40×256, and 20×20×256 respectively.
[0059] The convolution kernel size of convolution layer C6 is 3×3, the sliding step is 2, and the outputs of the three layers are 80×80×256, 40×40×256, and 20×20×256 respectively.
[0060] The convolution kernel size of the convolution layer C7-0 is 3×3, the sliding step is 2, and the outputs of the three layers are 80×80×2, 40×40×2, and 20×20×2, which are the pedestrian probability heat maps.
[0061] The convolution kernel size of the convolution layer C7-1 is 3×3, the sliding step is 2, and the outputs of the three layers are 80×80×4, 40×40×4, and 20×20×4, respectively, which are the coordinates of the pedestrian prediction box.
[0062] The convolution kernel size of the convolution layer C7-2 is 3×3, the sliding step size is 2, and the outputs of the three layers are 80×80×17, 40×40×17, and 20×20×17, respectively, that is, 17 posture key point heat maps.
[0063] The convolution kernel size of the convolution layer C7-3 is 3×3, the sliding step is 2, and the outputs of the three layers are 80×80×128, 40×40×128, and 20×20×128, respectively, which are the pedestrian re-identification features.
[0064] The abnormal posture classification module 24 is used to obtain the corresponding pedestrian abnormal behavior recognition result of the pedestrian according to the posture estimation graph of each pedestrian in the pedestrian posture estimation data corresponding to all continuous frames. Among them, the pedestrian abnormal behavior recognition result is normal behavior or abnormal behavior. In this embodiment, both normal behavior and abnormal behavior are composed of multiple preset action categories. For example, the action categories included in abnormal behavior include fighting, falling, etc.
[0065] The posture estimation graph is composed of black and white, the skeleton of the pedestrian in the posture estimation graph is white, and the other parts of the posture estimation graph are set to black. In this embodiment, the scene information and the appearance information of the pedestrian can be well filtered out by black and white, thereby solving the problem of small amount of abnormal posture data.
[0066] In this embodiment, for each pedestrian, behavior recognition is performed through posture estimation images of 5 consecutive frames, and the behavior category with the highest predicted probability corresponding to the behavior category is selected from the 5 frames as the predicted action, and the pedestrian abnormal behavior recognition result of the pedestrian is further obtained according to whether the predicted action belongs to the normal behavior range or the abnormal behavior range. For example, if the behavior category with the highest predicted probability in the 5 frames is fighting, and fighting belongs to the abnormal behavior range, the pedestrian abnormal behavior recognition result of the pedestrian is abnormal behavior.
[0067] In this embodiment, the abnormal posture classification module 24 includes a convolution layer C0, a maximum pooling layer, 3 residual modules C1, 4 residual modules C2, 6 residual modules C3, 3 residual modules C4, an average pooling layer and a fully connected layer connected in sequence.
[0068] Figure 4 It is a flow chart of constructing a pedestrian abnormal behavior recognition model in an embodiment of the present invention.
[0069] like Figure 4 As shown in Figure 1, the construction process of the pedestrian abnormal behavior recognition model includes the following steps:
[0070] Step T1, the feature extraction module 21, the feature fusion module 22 and the pedestrian posture tracking module 23 are used as the pedestrian posture tracking model, and the abnormal posture classification module 24 is used as the action recognition model. In this embodiment, the parameters of each module in step T1 are randomly set and gradually optimized and updated in subsequent training.
[0071] Step T2: multiple labeled images are used as the first training set. In this embodiment, the images are surveillance images, and are labeled with pedestrian identification numbers, detection frames, and posture estimation information.
[0072] Step T3, training the pedestrian posture tracking model according to the first training set to obtain a trained pedestrian posture tracking model. Wherein, in step T3, the pedestrian posture estimation data generated by the pedestrian posture tracking model according to the image input and the pedestrian posture corresponding to the image are used to calculate the loss error, and the parameters of the pedestrian posture tracking model are updated by back propagation of the loss error. In this embodiment, the above-mentioned loss error includes the cross entropy loss calculated between the pedestrian position point in the annotated posture estimation information and the pedestrian heat map in the pedestrian posture estimation data, the box regression loss calculated between the annotated detection box and the pedestrian detection box in the pedestrian posture estimation data, the posture key point distance loss calculated between the pedestrian joint point in the annotated posture estimation information and the posture point heat map in the pedestrian posture estimation data, and the pedestrian feature vector distance loss calculated between the annotated identification number and the pedestrian identification number in the pedestrian posture estimation data.
[0073] In this embodiment, when training the pedestrian posture tracking model, the image batch size entering the model each time is 16, and the adaptive moment estimation optimization algorithm is used for parameter optimization. The learning rate is 0.0001, and the termination condition of the model training is the convergence of the model parameters of each layer.
[0074] Step T4, input each image in the first training set into the trained pedestrian posture tracking model in sequence to obtain the corresponding posture estimation map.
[0075] Step T5: taking all posture estimation graphs and corresponding action categories as the second training set, wherein the action categories include normal behavior and abnormal behavior.
[0076] Step T6, training the action recognition model according to the second training set to obtain a trained action recognition model.
[0077] Among them, in step T6, the loss error is calculated based on the abnormal behavior recognition result of the pedestrian generated by the posture estimation graph input action recognition model and the action category corresponding to the posture estimation graph, and the action recognition model is updated by back propagation of the loss error. When the classification accuracy obtained by the action recognition model for classifying all images reaches a preset value, a trained action recognition model is obtained. In this embodiment, the above-mentioned loss error is a cross entropy loss error.
[0078] In this embodiment, the trained pedestrian posture tracking model and action recognition model are tested on a validation set consisting of 5,000 surveillance scene images, and the accuracy of the trained pedestrian posture tracking model is 95%, and the classification accuracy of the trained action recognition model is 97%. Therefore, the pedestrian abnormal behavior recognition model 200 can recognize abnormal behavior in images with high accuracy.
[0079] Step T7, constructing a pedestrian abnormal behavior recognition model 200 based on the trained pedestrian posture tracking model and the trained action recognition model.
[0080] The following describes the process of using the pedestrian abnormal behavior recognition device 100 to perform a pedestrian abnormal behavior recognition method based on posture tracking in conjunction with the accompanying drawings.
[0081] Figure 5 It is a flow chart of a method for identifying abnormal pedestrian behavior based on posture tracking in an embodiment of the present invention.
[0082] like Figure 5 As shown, the pedestrian abnormal behavior recognition method based on posture tracking includes the following steps:
[0083] Step S1, using the preprocessing unit 11 to preprocess each continuous frame to obtain a corresponding preprocessed image;
[0084] Step S2: using the behavior recognition unit 12 to input all pre-processed images into the pedestrian abnormal behavior recognition model 200 to obtain the pedestrian abnormal behavior recognition result.
[0085] Functions and Effects of the Embodiments
[0086] According to the method and device for pedestrian abnormal behavior recognition based on posture tracking involved in this embodiment, the pedestrian posture trajectories of consecutive different frames are obtained through the feature extraction module, the feature fusion module and the pedestrian posture tracking module, and these posture trajectories are used to further cut out continuous pictures containing only a single pedestrian posture in the original picture as the posture estimation map, so as to perform behavior recognition through the abnormal posture classification module, decouple the abnormal behavior recognition problem into two problems: posture tracking and action recognition using the posture map, and achieve strong generalization and high precision of pedestrian behavior recognition. In short, this method has good generalization and accuracy of pedestrian abnormal behavior recognition.
[0087] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A method for identifying abnormal pedestrian behavior based on posture tracking, which is used to obtain abnormal pedestrian behavior identification results based on multiple consecutive frames of a surveillance video stream, characterized in that: The following steps are involved: Step S1, preprocessing each of the continuous frames to obtain a corresponding preprocessed image; Step S2, inputting all the pre-processed images into a pedestrian abnormal behavior recognition model to obtain the pedestrian abnormal behavior recognition result. The pedestrian abnormal behavior recognition model includes: A feature extraction module, for extracting three levels of features from each of the preprocessed images; A feature fusion module, used to fuse the features at three levels to obtain three fused features; A pedestrian posture tracking module, used to obtain pedestrian posture estimation data according to the three fusion features; The abnormal posture classification module is used to obtain the abnormal behavior recognition result of the corresponding pedestrian according to the posture estimation graph of each pedestrian in the pedestrian posture estimation data corresponding to all the continuous frames, The pedestrian abnormal behavior recognition result is normal behavior or abnormal behavior.
2. According to the method for identifying abnormal pedestrian behavior based on posture tracking in claim 1, Features: in, The construction process of the pedestrian abnormal behavior recognition model includes the following steps: Step T1, using the feature extraction module, the feature fusion module and the pedestrian posture tracking module as a pedestrian posture tracking model, and using the abnormal posture classification module as an action recognition model; Step T2, using multiple labeled images as the first training set; Step T3, training the pedestrian posture tracking model according to the first training set to obtain a trained pedestrian posture tracking model; Step T4, inputting each image in the first training set into the trained pedestrian posture tracking model in sequence to obtain the corresponding posture estimation graph; Step T5, taking all the posture estimation graphs and corresponding action categories as a second training set; Step T6, training the action recognition model according to the second training set to obtain a trained action recognition model; Step T7, constructing the pedestrian abnormal behavior recognition model based on the trained pedestrian posture tracking model and the trained action recognition model.
3. The method for identifying abnormal pedestrian behavior based on posture tracking according to claim 2 is characterized in that: in, The pedestrian posture estimation data includes a pedestrian heat map, a posture point heat map, a pedestrian detection frame and a pedestrian identification number. In step T3, the pedestrian posture estimation data generated by the pedestrian posture tracking model according to the image input and the pedestrian posture corresponding to the image are used to calculate the loss error, and the parameters of the pedestrian posture tracking model are updated by back propagation of the loss error.
4. The method for identifying abnormal pedestrian behavior based on posture tracking according to claim 3 is characterized in that: in, The loss error includes cross entropy loss, posture key point distance loss and pedestrian feature vector distance loss.
5. The method for identifying abnormal pedestrian behavior based on posture tracking according to claim 2 is characterized in that: in, The action category includes the normal behavior and the abnormal behavior, In step T6, the loss error is calculated based on the abnormal pedestrian behavior recognition result generated by the action recognition model inputted into the posture estimation graph and the action category corresponding to the posture estimation graph, and the action recognition model is updated by back propagation of the loss error.
6. The method for identifying abnormal pedestrian behavior based on posture tracking according to claim 5 is characterized in that: in, In step T6, when the classification accuracy rate obtained by the action recognition model for classifying all images reaches a preset value, the trained action recognition model is obtained. The loss error is a cross entropy loss error.
7. The method for identifying abnormal pedestrian behavior based on posture tracking according to claim 1, characterized in that: in, The posture estimation graph is composed of black and white. The skeleton of the pedestrian in the pose estimation image is white. The other parts of the posture estimation graph are set to black.
8. The method for identifying abnormal pedestrian behavior based on posture tracking according to claim 1, characterized in that: in, The preprocessing includes scaling the long sides of the continuous frames to a specific size and adjusting the short sides accordingly in proportion, padding the short sides, and performing a standardization operation.
9. A pedestrian abnormal behavior recognition device based on posture tracking, used to obtain pedestrian abnormal behavior recognition results based on multiple consecutive frames of a monitoring video stream, characterized in that: include: A preprocessing unit, used for preprocessing each of the continuous frames to obtain a corresponding preprocessed image; The behavior recognition unit includes a pedestrian abnormal behavior recognition model, which is used to input all the pre-processed images into the pedestrian abnormal behavior recognition model to obtain the pedestrian abnormal behavior recognition result. The pedestrian abnormal behavior recognition model includes: A feature extraction module, for extracting three levels of features from each of the preprocessed images; A feature fusion module, used to fuse the features at three levels to obtain three fused features; A pedestrian posture tracking module, used to obtain pedestrian posture estimation data according to the three fusion features; The abnormal posture classification module is used to obtain the abnormal behavior recognition result of the corresponding pedestrian according to the posture estimation graph of each pedestrian in the pedestrian posture estimation data corresponding to all the continuous frames, The pedestrian abnormal behavior recognition result is normal behavior or abnormal behavior.
Citation Information
Patent Citations
Transferable campus violent behavior video identification method based on attitude estimation
CN110826459A
Park abnormal behavior identification method and device based on attitude estimation
CN111753665A
Customer abnormal behavior identification method based on human skeleton
CN115862128A
Real-time multi-person abnormal behavior identification method and system based on lightweight network
CN117437691A
Pedestrian abnormal behavior video identification method based on attitude estimation
CN117789255A
