Deep learning-based business hall abnormal behavior identification method and system
Through deep learning-based methods, the data set and the behavior recognition model are constructed, the key point features and comprehensive features of abnormal behavior are extracted, and the similarity thresholds are automatically identified, which solves the problem of high cost of manual labeling and intervention in the prior art. , achieving efficient abnormal behavior identification and supervision efficiency improvement.
Patent Information
- Application Number
- CN202411303474.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2025-01-03
AI Technical Summary
The existing business hall abnormal behavior supervision system requires a lot of manual marking and intervention, resulting in high labor and time costs, and it is difficult to achieve automated identification and rapid response.
Using deep learning-based methods, the data set is constructed and the behavior recognition model is trained. By extracting the key points and comprehensive features of abnormal behavior, and combining the similarity thresholds to automatically identify and re-label, reducing manual intervention, improving recognition accuracy and efficiency.
It improves the accuracy and supervision efficiency of abnormal behavior recognition, significantly reduces the workload of manual calibration, realizes automatic analysis and rapid identification of violations, and enhances the generalization ability of the model and the ability to respond to complex situations.
Smart Images
Figure CN119251904A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of abnormal behavior recognition, and in particular to a method for recognizing abnormal behavior in a business hall based on deep learning. Background Art
[0002] The power business hall is an important window for power companies to directly contact customers. Compliance supervision of the behavior of business hall staff is very important for improving service quality, ensuring safe and compliant operations, and improving work efficiency. It not only affects the normal operation of the business hall, but also directly affects the image of the power company and user satisfaction.
[0003] With the continuous advancement of technology, video monitoring has become an important means of monitoring the compliance of power business halls. By installing high-definition network surveillance cameras and combining video and image intelligent learning algorithms to mark and learn illegal behaviors, real-time monitoring and intelligent analysis of employee behavior in business halls can be achieved.
[0004] However, in the existing supervision system for abnormal behavior of business halls, manual labeling of a large amount of sample data is still required. Not only is manual participation required during the initial sample labeling, but also timely manual intervention is required when abnormal violations occur to label the abnormal violations. Manual sample labeling runs through the entire system operation process, resulting in a large amount of manpower and time costs in the labeling process. Summary of the invention
[0005] The present invention provides a method for identifying abnormal behavior in a business hall based on deep learning.
[0006] The technical solution of the present invention is as follows:
[0007] A method for identifying abnormal behavior in a business hall based on deep learning, comprising the following steps:
[0008] S1. Dataset construction: Obtain image data containing different abnormal behaviors, pre-process the image data, and further classify and mark the abnormal behaviors in the images, including abnormal behaviors of personnel, black screen behaviors of devices, and normal behaviors. Abnormal behaviors of personnel include playing with mobile phones, sleeping on the job, and smoking. Mark abnormal behaviors of personnel, abnormal behaviors of devices, and normal behaviors to obtain the initial abnormal behavior datasets of personnel, abnormal behavior datasets of devices, and normal behavior datasets;
[0009] S2. Model training: The personnel abnormal behavior data set, the equipment abnormal behavior data set and the normal behavior data set are divided into a training set and a test set according to a preset ratio, and the training set is input into the behavior recognition model for convolution and pooling processing to extract multiple key point features corresponding to each abnormal behavior, and further extract the descriptor of each key point feature, and all the descriptors of each abnormal behavior are weighted summed based on their preset weights to obtain the comprehensive features of each abnormal behavior. The behavior recognition model is trained based on the comprehensive features to obtain a trained behavior recognition model;
[0010] S3, abnormal behavior recognition: the acquired surveillance video of the business hall staff is divided into equal proportions to obtain a picture frame sequence, and the picture frame sequence is input into the trained behavior recognition model. Based on the set comparison similarity threshold, the recognized abnormal behavior type and the comparison similarity corresponding to each abnormal behavior are output;
[0011] S4. Dataset update: All abnormal behavior images obtained are added to the personnel abnormal behavior dataset and equipment abnormality dataset according to their types.
[0012] Specifically, each frame of the acquired image data contains one or more abnormal behaviors, a rectangular frame is set for each abnormal behavior according to its position in the image frame, and manual marking is performed in the rectangular frame area.
[0013] Specifically, if, within a preset time period, the proportion of newly added counts in the updated abnormal personnel behavior data set and the abnormal equipment data set exceeds a threshold, the updated abnormal personnel behavior data set and the abnormal equipment data set are used to retrain the behavior recognition model to obtain a retrained behavior recognition model, and the retrained behavior recognition model is used to perform abnormal behavior recognition.
[0014] Specifically, the classification mark of the abnormal behavior of playing with a mobile phone is: the picture includes both the device screen and the person, and the distance between the person and the device screen is within a specific distance range. If the key point features of the abnormal behavior of playing with a mobile phone are identified in the hand area of the person, and the key point features of the device black screen behavior are identified on the device screen, then the device screen area is marked with the abnormal behavior of playing with a mobile phone, and the hand area is marked with the abnormal behavior of playing with a mobile phone, and the key point features of the device black screen behavior are added to the key point features of the abnormal behavior of playing with a mobile phone.
[0015] If key features of abnormal mobile phone playing behavior are identified in the hand area of the person, and the device screen does not have key features of black screen behavior, the abnormal mobile phone playing behavior is marked only in the hand area.
[0016] Furthermore, the classification mark of the abnormal behavior of sleeping on the job is: if the picture includes both the device screen and the person, and the distance between the person and the device screen is within a specific distance range, if the key point features of the abnormal behavior of sleeping on the job are identified in the head area of the person, and the key point features of the device black screen behavior are identified in the device screen, then the device black screen abnormal behavior is marked in the device screen area and the sleeping on the job abnormal behavior is marked in the head area, and the key point features of the device black screen behavior are added to the key point features of the abnormal behavior of sleeping on the job.
[0017] If the key features of abnormal behavior of sleeping on the job are identified in the head area, and the key features of abnormal behavior of black screen are not present on the device screen, the abnormal behavior of sleeping on the job is marked only in the head area.
[0018] Furthermore, the classification and marking of abnormal smoking behavior is as follows: if key point features of abnormal smoking behavior are identified in both the mouth area and the hand area of a person, the abnormal smoking behavior is marked in the mouth area, wherein the key point features of abnormal smoking behavior are specifically: the degree of mouth opening, mouth smoke, and the angle between the index finger and the middle finger.
[0019] Specifically, the classification mark of the device black screen behavior is: if the device screen has key point features of the device black screen behavior, then the device black screen abnormal behavior is marked in the device screen area, wherein the key point features of the device black screen behavior are: color features of the display screen area.
[0020] Specifically, the preprocessing in S1 includes data cleaning, format conversion, normalization and data enhancement.
[0021] The present invention also provides a business hall abnormal behavior recognition system based on deep learning, comprising:
[0022] Dataset construction module: used to obtain image data containing different abnormal behaviors, and further classify and mark the abnormal behaviors in the images after preprocessing the image data, including abnormal behaviors of personnel, black screen behaviors of devices, and normal behaviors. Abnormal behaviors of personnel include playing with mobile phones, sleeping on the job, and smoking. Abnormal behaviors of personnel, abnormal behaviors of devices, and normal behaviors are marked to obtain the initial abnormal behavior datasets of personnel, abnormal equipment, and normal behaviors.
[0023] Model training module: used to divide the personnel abnormal behavior data set, equipment abnormal behavior data set and normal behavior data set into training set and test set respectively according to preset ratios, input the training set into the behavior recognition model for convolution and pooling processing to extract multiple key point features corresponding to each abnormal behavior, further extract the descriptor of each key point feature, perform weighted summation on all descriptors of each abnormal behavior based on its preset weight, obtain the comprehensive feature of each abnormal behavior, train the behavior recognition model based on the comprehensive feature, and obtain a trained behavior recognition model;
[0024] Abnormal behavior recognition module: used to divide the acquired monitoring video of the business hall staff into equal proportions to obtain a picture frame sequence, input the picture frame sequence into the trained behavior recognition model, and output the recognized abnormal behavior type and the comparison similarity corresponding to each abnormal behavior based on the set comparison similarity threshold;
[0025] Dataset update module: used to add all abnormal behavior images obtained to the personnel abnormal behavior dataset and equipment abnormality dataset according to their types.
[0026] The beneficial effects of the present invention are:
[0027] 1. The method and system for identifying abnormal behavior in business halls based on deep learning provided by the present invention divides data sets into different categories, extracts key point features of each abnormal behavior, sets different weights accordingly, and conducts model training by comprehensively considering multiple identification features, thereby improving the accuracy of abnormal behavior identification and realizing automatic analysis and rapid identification of illegal behaviors.
[0028] 2. The present invention sets a comparison similarity threshold to identify abnormal behaviors and automatically re-mark them, thereby improving the recognition accuracy and generalization ability of the model. At the same time, the automatic recalibration of abnormal behaviors can significantly reduce the workload of manual calibration and improve the accuracy of abnormal behavior identification and supervision efficiency.
[0029] 3. The present invention only needs to manually extract a small number of pictures for review and verification. After the review is passed, the marked qualified abnormal behavior pictures are added to the corresponding data sets of different categories, which enhances the diversity of data used for model training, enables the model to continuously learn new features and patterns, and continuously improves the ability to deal with complex and changeable abnormal violations. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In the attached picture:
[0031] Figure 1 The figure is a flowchart of a method for identifying abnormal behavior in a business hall based on deep learning in an embodiment. DETAILED DESCRIPTION
[0032] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.
[0033] Example
[0034] This embodiment provides a method for identifying abnormal behavior in a business hall based on deep learning. Figure 1 , including the following steps:
[0035] Step 1: Dataset construction: Obtain image data containing different abnormal behaviors, pre-process the image data, and further classify and mark the abnormal behaviors in the images, including abnormal personnel behavior, black screen behavior of equipment, and normal behavior. Abnormal personnel behavior includes playing with mobile phones, sleeping on the job, and smoking. Mark abnormal personnel behavior, abnormal equipment behavior, and normal behavior to obtain the initial abnormal personnel behavior dataset, abnormal equipment dataset, and normal behavior dataset.
[0036] Specifically, the acquired image data of different abnormal behaviors are preprocessed, and the preprocessing includes data cleaning, format conversion, normalization and data enhancement.
[0037] Each frame of acquired image data contains one or more abnormal behaviors. A rectangular frame is set for each abnormal behavior according to its position in the image frame, and manual marking is performed in the rectangular frame area.
[0038] Specifically, the classification mark of abnormal mobile phone playing behavior is: the picture includes both the device screen and the person, and the distance between the person and the device screen is within a specific distance range. If the key point features of abnormal mobile phone playing behavior are identified in the person's hand area, and the key point features of device black screen behavior are identified on the device screen, then the device screen area is marked as abnormal device black screen behavior, and the hand area is marked as abnormal mobile phone playing behavior, and the key point features of device black screen behavior are added to the key point features of abnormal mobile phone playing behavior.
[0039] If key features of abnormal mobile phone playing behavior are identified in the hand area of the person, and the device screen does not have key features of black screen behavior, the abnormal mobile phone playing behavior is marked only in the hand area.
[0040] The classification mark of abnormal behavior of sleeping on the job is as follows: if the image includes both the device screen and the person, and the distance between the person and the device screen is within a specific distance range, if the key point features of abnormal behavior of sleeping on the job are identified in the head area of the person, and the key point features of abnormal behavior of device black screen are identified on the device screen, then the device screen area is marked as abnormal behavior of device black screen, and the head area is marked as abnormal behavior of sleeping on the job, and the key point features of device black screen behavior are added to the key point features of abnormal behavior of sleeping on the job.
[0041] If the key features of abnormal behavior of sleeping on the job are identified in the head area, and the key features of abnormal behavior of black screen are not present on the device screen, the abnormal behavior of sleeping on the job is marked only in the head area.
[0042] The classification and marking of abnormal smoking behavior is as follows: if key point features of abnormal smoking behavior are identified in both the mouth and hand areas of a person, the abnormal smoking behavior is marked in the mouth area, where the key point features of abnormal smoking behavior are specifically: the degree of mouth opening, mouth smoke, and the angle between the index and middle fingers.
[0043] The classification mark of the device black screen behavior is: if the device screen has key point features of the device black screen behavior, then the device black screen abnormal behavior is marked in the device screen area, where the key point features of the device black screen behavior are: color features of the display screen area.
[0044] Step 2: Model training: The personnel abnormal behavior dataset, equipment abnormal behavior dataset and normal behavior dataset are divided into training set and test set according to preset ratios respectively. The training set is input into the behavior recognition model for convolution and pooling processing to extract multiple key point features corresponding to each abnormal behavior. The descriptor of each key point feature is further extracted. All descriptors of each abnormal behavior are weighted summed based on their preset weights to obtain the comprehensive features of each abnormal behavior. The behavior recognition model is trained based on the comprehensive features to obtain a trained behavior recognition model.
[0045] Specifically, the personnel abnormal behavior data set, equipment abnormal behavior data set and normal behavior data set are divided into 70% training set and 30% validation set respectively.
[0046] The training set is input into the recognition model, and the convolution layer is used to perform convolution processing on the image data in the input training set to extract the key point features in each image frame, and at the same time, the descriptors of each key point feature are extracted. These descriptors contain the local information of the key point features; then the pooling layer is used to pool the image data after convolution processing, thereby reducing the amount of calculation while retaining the key point feature information.
[0047] Assign a weight to each key point feature extracted according to its importance, then multiply the descriptor of each key point feature by its corresponding weight to obtain a weighted feature vector, add up all the weighted feature vectors to obtain a comprehensive feature vector corresponding to each abnormal behavior, i.e., the comprehensive feature, and train the behavior recognition model based on the comprehensive feature of each abnormal behavior to obtain a trained behavior recognition model.
[0048] The model training step also includes using the test set to verify the performance of the behavior recognition model during training, and adjusting the parameters and structure of the model based on the evaluation results, so that the accuracy of the finally trained behavior recognition model on the test set reaches the set standard.
[0049] Furthermore, the trained behavior recognition model is deployed to the monitoring system of the actual business hall to monitor the behavior of the staff in the business hall in real time.
[0050] Step 3: Abnormal behavior identification: The acquired surveillance video of the business hall staff is divided into equal proportions to obtain a picture frame sequence, which is input into the trained behavior recognition model. Based on the set comparison similarity threshold, the identified abnormal behavior type and the comparison similarity corresponding to each abnormal behavior are output.
[0051] Specifically, monitoring videos are collected from high-definition cameras installed at the counters and self-service areas of the business hall to ensure that the quality of the monitoring videos is high enough to clearly identify the behavioral details of personnel and facilitate analysis in subsequent steps.
[0052] Image frames are extracted from a monitoring video at an equal proportion and time period, and the people and equipment in the extracted image frames are preliminarily judged. Specifically, if at least one complete head and one shoulder are recognized in the image frame, the corresponding area in the image frame is judged to be a person; if a certain area in the image frame is recognized to have a rectangular border that meets the aspect ratio of the display screen border, the area in the image frame is judged to be a device. In this embodiment, the device is specifically a display screen.
[0053] In this embodiment, the comparison similarity threshold is set to 98%. By automatically marking the generated abnormal behavior pictures, the generalization ability of the model can be improved, and the accuracy of abnormal behavior recognition can be further improved.
[0054] Step 4: Dataset update: All abnormal behavior images obtained are added to the personnel abnormal behavior dataset and equipment abnormality dataset according to their types.
[0055] Furthermore, in the updated personnel abnormal behavior data set and equipment abnormal behavior data set, if within a preset time period, the proportion of newly added numbers in the personnel abnormal behavior data set and equipment abnormal behavior data set exceeds a threshold, the updated personnel abnormal behavior data set and equipment abnormal behavior data set are used to retrain the behavior recognition model to obtain a retrained behavior recognition model, and the retrained behavior recognition model is used to perform abnormal behavior recognition.
[0056] A certain number of pictures are randomly selected from all abnormal behavior pictures for manual review, and it should be ensured that each type of picture close to abnormal behavior is involved. In this embodiment, 40 sample pictures of each type of behavior close to abnormal behavior are used for manual review. While ensuring the quality of the generated new library of sample pictures, the efficiency and timeliness of manual review can be improved. The number of sample pictures used for manual review is not fixed and can be adjusted according to actual needs.
[0057] Through iterative cycles, the business hall abnormal behavior identification system based on deep learning can continue to learn and continuously improve its ability to deal with complex and changeable abnormal and illegal behaviors.
[0058] As an optimization solution of the above solution,
[0059] After preprocessing the image data in step 1, classify and mark the abnormal behaviors in some images to obtain the initial personnel abnormal behavior data set, equipment abnormality data set, normal behavior data set, and unlabeled data set;
[0060] The unlabeled data set is identified using the trained behavior recognition model to obtain the recognition results, and the pictures in which abnormal behaviors of people and black screen behaviors of devices are identified are constructed into temporary identification data sets of abnormal behaviors of people and temporary identification data sets of black screen behaviors of devices.
[0061] The behavior recognition model is trained using the temporary identification personnel abnormal behavior dataset and the temporary identification device black screen behavior dataset to obtain the final behavior recognition model, and the final behavior recognition model is used to perform abnormal behavior recognition in step three.
[0062] The present invention also provides a business hall abnormal behavior recognition system based on deep learning, comprising:
[0063] Dataset construction module: used to obtain image data containing different abnormal behaviors, and further classify and mark the abnormal behaviors in the images after preprocessing the image data, including abnormal behaviors of personnel, black screen behaviors of devices, and normal behaviors. Abnormal behaviors of personnel include playing with mobile phones, sleeping on the job, and smoking. Abnormal behaviors of personnel, abnormal behaviors of devices, and normal behaviors are marked to obtain the initial abnormal behavior datasets of personnel, abnormal equipment, and normal behaviors.
[0064] Model training module: used to divide the personnel abnormal behavior data set, equipment abnormal behavior data set and normal behavior data set into training set and test set respectively according to preset ratios, input the training set into the behavior recognition model for convolution and pooling processing to extract multiple key point features corresponding to each abnormal behavior, further extract the descriptor of each key point feature, perform weighted summation on all descriptors of each abnormal behavior based on its preset weight, obtain the comprehensive feature of each abnormal behavior, train the behavior recognition model based on the comprehensive feature, and obtain a trained behavior recognition model;
[0065] Abnormal behavior recognition module: used to divide the acquired monitoring video of the business hall staff into equal proportions to obtain a picture frame sequence, input the picture frame sequence into the trained behavior recognition model, and output the recognized abnormal behavior type and the comparison similarity corresponding to each abnormal behavior based on the set comparison similarity threshold;
[0066] Dataset update module: used to add all abnormal behavior images obtained to the personnel abnormal behavior dataset and equipment abnormality dataset according to their types.
Claims
1. A method for identifying abnormal behavior in business halls based on deep learning, characterized in that: The following steps are involved: S1. Dataset construction: Obtain image data containing different abnormal behaviors, pre-process the image data, and further classify and mark the abnormal behaviors in the images, including abnormal behaviors of personnel, black screen behaviors of devices, and normal behaviors. Abnormal behaviors of personnel include playing with mobile phones, sleeping on the job, and smoking. Mark abnormal behaviors of personnel, abnormal behaviors of devices, and normal behaviors to obtain the initial abnormal behavior datasets of personnel, abnormal behavior datasets of devices, and normal behavior datasets; S2. Model training: The personnel abnormal behavior data set, the equipment abnormal behavior data set and the normal behavior data set are divided into a training set and a test set according to a preset ratio, and the training set is input into the behavior recognition model for convolution and pooling processing to extract multiple key point features corresponding to each abnormal behavior, and further extract the descriptor of each key point feature, and all the descriptors of each abnormal behavior are weighted summed based on their preset weights to obtain the comprehensive features of each abnormal behavior. The behavior recognition model is trained based on the comprehensive features to obtain a trained behavior recognition model; S3, abnormal behavior recognition: the acquired surveillance video of the business hall staff is divided into equal proportions to obtain a picture frame sequence, and the picture frame sequence is input into the trained behavior recognition model. Based on the set comparison similarity threshold, the recognized abnormal behavior type and the comparison similarity corresponding to each abnormal behavior are output; S4. Dataset update: All abnormal behavior images obtained are added to the personnel abnormal behavior dataset and equipment abnormality dataset according to their types.
2. The method for identifying abnormal behavior in a business hall based on deep learning according to claim 1, characterized in that: Each frame of image data acquired in S1 contains one or more abnormal behaviors. A rectangular frame is set for each abnormal behavior according to its position in the image frame, and manual marking is performed in the rectangular frame area.
3. The method for identifying abnormal behavior in a business hall based on deep learning according to claim 1 is characterized in that: In the updated abnormal personnel behavior data set and abnormal equipment data set in S4, if the proportion of newly added numbers in the abnormal personnel behavior data set and abnormal equipment data set exceeds a threshold within a preset time period, the updated abnormal personnel behavior data set and abnormal equipment data set are used to retrain the behavior recognition model to obtain a retrained behavior recognition model, and the retrained behavior recognition model is used to perform abnormal behavior recognition.
4. The method for identifying abnormal behavior in a business hall based on deep learning according to claim 1, characterized in that: The classification mark of abnormal mobile phone playing behavior is: the picture includes both the device screen and the person, and the distance between the person and the device screen is within a specific distance range. If the key point features of abnormal mobile phone playing behavior are identified in the person's hand area, and the key point features of device black screen behavior are identified on the device screen, then the device screen area is marked with abnormal device black screen behavior, and the hand area is marked with abnormal mobile phone playing behavior, and the key point features of device black screen behavior are added to the key point features of abnormal mobile phone playing behavior. If key features of abnormal mobile phone playing behavior are identified in the hand area of the person, and the device screen does not have key features of black screen behavior, the abnormal mobile phone playing behavior is marked only in the hand area.
5. At the same time, according to the method for identifying abnormal behavior of a business hall based on deep learning according to claim 1, it is characterized in that: The classification mark of the abnormal behavior of sleeping on the job is: if the image includes both the device screen and the person, and the distance between the person and the device screen is within a specific distance range, if the key point features of the abnormal behavior of sleeping on the job are identified in the head area of the person, and the key point features of the device black screen behavior are identified in the device screen, then the device black screen abnormal behavior mark is performed in the device screen area and the head area of the abnormal behavior of sleeping on the job is performed respectively, and the key point features of the device black screen behavior are added to the key point features of the abnormal behavior of sleeping on the job. If the key features of abnormal behavior of sleeping on the job are identified in the head area, and the key features of abnormal behavior of black screen are not present on the device screen, the abnormal behavior of sleeping on the job is marked only in the head area.
6. The method for identifying abnormal behavior in business halls based on deep learning according to claim 1, characterized in that: The classification marking of the abnormal smoking behavior is: if key point features of abnormal smoking behavior are identified in both the mouth area and the hand area of the person, the abnormal smoking behavior is marked in the mouth area, wherein the key point features of the abnormal smoking behavior are specifically: the degree of mouth opening, mouth smoke, and the angle between the index finger and the middle finger.
7. The method for identifying abnormal behavior in a business hall based on deep learning according to claim 1, characterized in that: The classification mark of the device black screen behavior is: if the device screen has key point features of the device black screen behavior, then the device black screen abnormal behavior is marked in the device screen area, where the key point features of the device black screen behavior are: color features of the display screen area.
8. The method for identifying abnormal behavior in business halls based on deep learning according to claim 1, characterized in that: The preprocessing in S1 includes data cleaning, format conversion, normalization and data enhancement.
9. A business hall abnormal behavior recognition system based on deep learning, characterized in that: include: Dataset construction module: used to obtain image data containing different abnormal behaviors, and further classify and mark the abnormal behaviors in the images after preprocessing the image data, including abnormal behaviors of personnel, black screen behaviors of devices, and normal behaviors. Abnormal behaviors of personnel include playing with mobile phones, sleeping on the job, and smoking. Abnormal behaviors of personnel, abnormal behaviors of devices, and normal behaviors are marked to obtain the initial abnormal behavior datasets of personnel, abnormal equipment, and normal behaviors. Model training module: used to divide the personnel abnormal behavior data set, equipment abnormal behavior data set and normal behavior data set into training set and test set respectively according to preset ratios, input the training set into the behavior recognition model for convolution and pooling processing to extract multiple key point features corresponding to each abnormal behavior, further extract the descriptor of each key point feature, perform weighted summation on all descriptors of each abnormal behavior based on its preset weight, obtain the comprehensive feature of each abnormal behavior, train the behavior recognition model based on the comprehensive feature, and obtain a trained behavior recognition model; Abnormal behavior recognition module: used to divide the acquired monitoring video of the business hall staff into equal proportions to obtain a picture frame sequence, input the picture frame sequence into the trained behavior recognition model, and output the recognized abnormal behavior type and the comparison similarity corresponding to each abnormal behavior based on the set comparison similarity threshold; Dataset update module: used to add all abnormal behavior images obtained to the personnel abnormal behavior dataset and equipment abnormality dataset according to their types.