An off-duty monitoring system and identification method thereof

Abstract

The present application relates to the technical field of off-duty recognition, in particular to an off-duty monitoring system and a recognition method thereof, comprising: a target monitoring camera for acquiring image data collected by the target monitoring camera in a post area; a storage for storing image data of the post area collected by the target monitoring camera in a certain time period; a camera interface of the target monitoring camera for providing a data transmission interface; a processor for analyzing the image or picture collected by the target monitoring camera through an AI service and outputting an image detection result; a loading algorithm module for starting an algorithm analysis subroutine for the algorithm run by the off-duty monitoring system; and a communication module for remotely transmitting the image data collected by the target monitoring camera. The present application proposes an off-duty intelligent recognition method based on an attention mechanism and deep learning for the problem of off-duty of personnel in the safe production of enterprises, introduces a BOT structure, and improves the attention degree of the neural network to the target area.

Description

Technical Field

[0001] This invention relates to the field of absenteeism detection technology, specifically to an absenteeism monitoring system and its detection method. Background Technology

[0002] Personnel being on duty is a fundamental requirement of enterprise safety operation principles. Early detection and elimination of various accident hazards are crucial measures to prevent major accidents. In enterprise production settings, unauthorized absence from duty often leaves safety hazards, leading to accidents without timely warning and resulting in significant losses. For example, in a dye chemical plant in Tianjin, an operator's violation of operating procedures and unauthorized absence from duty caused a serious accident resulting in multiple injuries and fatalities. Therefore, intelligent personnel absence detection technology has significant research value and broad application prospects. Through intelligent identification, it enables dynamic risk monitoring of enterprises, timely detection of personnel absence, reduces the rate of employee absence, and decreases the probability of production accidents caused by absence.

[0003] To address the issue of staff absenteeism, early traditional methods relied heavily on manual monitoring, involving manual recording and inspection of staff attendance. This approach was resource-intensive, unable to guarantee continuous 24 / 7 monitoring, and its accuracy and efficiency were questionable. A more costly solution emerged: wearable hardware detection. This involves using electronic trigger devices to monitor window positions in real time using infrared light. If a staff member is present, they will block the infrared beam, indicating their presence. However, this method requires installing a hardware monitoring device at each window, which is cumbersome. Furthermore, prolonged exposure to electronic interference can pose health risks.

[0004] With the development of computer vision technology, some manufacturers have proposed methods for detecting absenteeism based on frame difference and background subtraction. However, these methods are highly dependent on the frame sequence of the camera. If one or several consecutive frames have problems, the recognition results may be affected. In addition, there are also methods that use detectors obtained by cascading AdaBoost sub-classifiers and detectors that use multimodal information fusion to determine absenteeism. However, none of these methods can fully meet the needs of enterprises for intelligent absenteeism recognition in actual production scenarios.

[0005] In response to the problems mentioned above, we propose an absence monitoring system and its identification method. Summary of the Invention

[0006] The purpose of this invention is to provide an absence monitoring system and its identification method to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution.

[0008] An absence monitoring system, comprising:

[0009] The target surveillance camera is used to acquire image data collected by the target surveillance camera in the area of ​​the work post. The target surveillance camera is one or more of several target surveillance cameras involved in the absence detection model scenario in the risk monitoring and early warning system.

[0010] The storage device is used to store image data of the work area collected by the target surveillance camera during a certain period of time;

[0011] The camera interface of the target surveillance camera is used to provide a data transmission interface;

[0012] The processor is used to analyze images or pictures captured by the target surveillance camera through AI services and output image detection results;

[0013] The algorithm loading module is used to start the algorithm analysis subroutine for the algorithms running in the off-duty monitoring system;

[0014] The communication module is used to remotely transmit image data captured by the target surveillance camera.

[0015] A method for identifying absenteeism in a monitoring system includes the following steps:

[0016] Step S1: Obtain the names of surveillance cameras involved in the absence detection model scenarios in the absence monitoring system;

[0017] Step S2: Obtain monitoring data from the target surveillance camera, obtain a list of information from several target cameras, and receive images through storage.

[0018] Step S3: Process image data using AI:

[0019] AI services include algorithm analysis services, algorithm analysis subroutines, and algorithm analysis threads;

[0020] Step S4: Call the image analysis function:

[0021] The image analysis function calls the image detection function to perform image analysis based on the input algorithm name and image, returns the analysis results, initializes the algorithm name, reads the image according to the input image path, passes the parameters to step S5 of the image detection function, and returns the detection results.

[0022] Step S5: Call the image detection function:

[0023] If the model can run, proceed to step S6 of the detection function. The image detection function receives a dictionary cmdJson containing the algorithm name and the image, and a confidence threshold conf_thres as input parameters. It calls the corresponding detection model to perform image detection and finally returns the detection result.

[0024] Step S6: Use the detection function to make inferences, return the detection results, and store the image of the person leaving their post to the specified path;

[0025] Step S61: Preprocessing of monitoring images;

[0026] Step S62: Load the model to perform inference on the image;

[0027] Step S63: Eliminate redundancy and adjust the detection results, return the detection results, and store the off-duty image to the specified path.

[0028] Preferably, step S2 specifically includes the following steps:

[0029] Step S21: Obtain the CarmeraID of the surveillance camera from the interface according to the surveillance camera name;

[0030] Step S22: Initialize the log module and output logs;

[0031] Step S23: Load the target camera information list;

[0032] Step S24: Start a thread based on the camera information, retrieve images from the video stream, and save them to the image saving location.

[0033] Preferably, step S3 specifically includes the following steps:

[0034] Step S31: Enable AI service:

[0035] If it starts normally, it will run normally; if there is an error in the configuration file, it will output that the configuration file is incorrect.

[0036] Step S32: Start up the various services:

[0037] Get a list of all cameras and a list of all algorithms. If an algorithm exists in the list of invalid algorithms, the program will report an error and exit.

[0038] Step S33: Start the algorithm analysis service:

[0039] Load the algorithm module, start the algorithm analysis subroutine for all algorithms, allocate the number of algorithm analysis scheduling threads, and each thread enters step S35;

[0040] Step S34: Start the algorithm analysis subroutine:

[0041] If the motion model initialization function is successfully executed, the output will be "Successfully loaded person detection model" and a success message will be returned. If the model initialization function is not successfully executed, the output will be "Failed to load person detection model. Please check if the weight path matches and if the called library is installed" and a failure message will be returned.

[0042] Step S35: The algorithm analysis thread retrieves the image from the queue, performs analysis, and returns the result to the queue.

[0043] Preferably, step S35 specifically includes the following steps:

[0044] Step S351: Obtain the image to be analyzed, initialize the image name, algorithm name, and process name, call the image analysis function in step S4, pass in the parameters, and then determine whether the recognition object has been found.

[0045] Step S352: If no object is found, output "Analysis complete, XX algorithm did not find the object";

[0046] Step S353: If the object to be identified is found, add the identification result to the queue, output "Found XX algorithm identification result", and add the identification result to the identification result list;

[0047] Step S354: If the above process throws an exception, output "Error in algorithm analysis thread concurrency".

[0048] Preferably, step S5 specifically includes the following steps:

[0049] S51. In the image detection function, the main task is to load the corresponding detection model and pass the input image and confidence threshold to the model for detection.

[0050] Step S52: If the detection model is not loaded, call the image detection function to load the model.

[0051] Preferably, step S6 specifically includes the following steps:

[0052] The detection function is entered to run the detection program. It receives an image, confidence threshold conf_thres, IoU threshold iou_thres, and whether to save the output image save_img as input. It uses the loaded detection model to perform inference and finally returns the detection result.

[0053] Preferably, step S6 specifically includes the following steps:

[0054] First, record the start time of inference, and use the letterbox function to fill and scale the input image img0;

[0055] Secondly, a detection model is used for detection, the detection results are converted into a more readable format, and only the detection results for the "person" category are retained;

[0056] Then, the detection results are returned, and the time when the inference ends is recorded. In the detection function, the main task is to convert the input image into a format acceptable to the model and use the intelligent detection model for detection and inference. After returning the detection results, the code also performs coordinate transformation and category filtering on the detection results.

[0057] Preferably, step S63 specifically includes the following steps:

[0058] If there is no label "person", res will be empty, the console will print "No one is in the monitoring room", and the image of the person being absent from their post will be saved to the specified path.

[0059] If there is a label "person", res is not empty, the console prints "Someone is in the monitoring room", and no photos of people leaving their posts are stored;

[0060] If an exception is thrown, the console will print "Failed to open video" and the image of the person leaving their post will not be stored.

[0061] Preferably, step S6 further includes:

[0062] The detection function adopts the intelligent identification model for personnel leaving their posts. By introducing BOT into the YOLOv5 architecture, the improved YOLOv5 backbone network is used to extract features from the target image, resulting in feature maps of different scales.

[0063] Multiple feature maps are fused, and the fused feature map is input into the YOLOV5 detection head for processing to obtain the off-duty recognition results at different image scales.

[0064] Compared with the prior art, the beneficial effects of the present invention are:

[0065] The detection function of this invention adopts an innovative intelligent identification model for absenteeism. By introducing a BOT into the YOLOv5 architecture, the neural network is enhanced in feature extraction capabilities and can focus more on key features of the target area, thereby improving the model's performance. At the same time, data augmentation strategies such as HSV enhancement and MOSAIC enhancement are used to train the model to reduce the similarity between data and enhance the diversity of data. The model is experimentally verified using real-world enterprise production data. The results show that this intelligent identification model for absenteeism has good performance in terms of recognition accuracy, specificity, and F1 score. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the structure of the present invention;

[0067] Figure 2 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0068] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0069] like Figure 1 , Figure 2 As shown, an absence monitoring system includes:

[0070] The target surveillance camera is used to acquire image data collected by the target surveillance camera in the area of ​​the work post. The target surveillance camera is one or more of several target surveillance cameras involved in the absence detection model scenario in the risk monitoring and early warning system.

[0071] The storage device is used to store image data of the work area collected by the target surveillance camera during a certain period of time;

[0072] The camera interface of the target surveillance camera is used to provide a data transmission interface;

[0073] The processor is used to analyze images or pictures captured by the target surveillance camera through AI services and output image detection results;

[0074] The algorithm loading module is used to start the algorithm analysis subroutine for the algorithms running in the off-duty monitoring system;

[0075] The communication module is used to remotely transmit image data captured by the target surveillance camera;

[0076] The target surveillance camera is connected to the cloud platform. The target surveillance camera belongs to the enterprise. Data transmission is achieved between the target surveillance camera and the cloud platform. The information corresponding to the target surveillance camera and the enterprise is entered into the cloud platform. The cloud platform summarizes the data of several enterprises and target surveillance cameras. The subsystem is the off-duty monitoring system of this application belonging to the enterprise, and the main system is the cloud platform for remote summary monitoring.

[0077] A method for identifying absenteeism in a monitoring system includes the following steps:

[0078] Step S1: Obtain the names of surveillance cameras involved in the absence detection model scenarios in the absence monitoring system;

[0079] The name includes, but is not limited to, the names of various enterprise surveillance cameras that include the following keywords (such as monitoring room, duty room, central control room, etc.) (e.g., the main entrance of the oil depot monitoring room of a certain city's petrochemical company, the monitoring room on the 1st floor of the power workshop of a certain city's pharmaceutical chemical company, the duty room of a certain city's pharmaceutical chemical company, etc.).

[0080] The acquired data is stored in text format. The surveillance camera name information is stored in the format "Company Name\nMonitor Name", for example:

[0081] XX City XX Petrochemical Co., Ltd.

[0082] Oil depot monitoring room main entrance

[0083] A certain technology company in a certain city

[0084] Monitoring Room

[0085] A certain oil depot of a certain company's branch

[0086] [Company Name] Branch Company [Name] Oil Depot [Name] Fire Brigade Monitoring Room

[0087] [Company Name] [Company Name] [Reserve Warehouse Name]

[0088] Monitoring room of a certain reserve warehouse.

[0089] Step S2: Obtain monitoring data from the target surveillance camera, obtain a list of information from several target cameras, and receive images through storage.

[0090] The acquired data is stored in text format. The CarmeraID information of the surveillance camera is stored in the format "Organizational Structure\nMonitor Name\nCarmeraID", for example:

[0091] [City Name] / BJ34 / [City Name] Petroleum Sales Co., Ltd.

[0092] Central control room 11340080021314000412

[0094] [City Name] / BJ34 / [City Name] Aquatic Products Market Co., Ltd.

[0095] Fire control room 11340080021314003817

[0097] [City Name] / BJ34 / [City Name] Aquatic Products Market Co., Ltd.

[0098] Central control room

[0099] 11340080021314000453.

[0100] Step S21: Obtain the CarmeraID of the surveillance camera from the interface according to the surveillance camera name;

[0101] Step S22: Initialize the log module and output logs;

[0102] Step S23: Load the target camera information list;

[0103] Step S24: Start a thread based on the camera information, retrieve the image from the video stream and save it to the image save location img_path.

[0104] Step S3: Process image data using AI: The AI ​​service includes algorithm analysis service, algorithm analysis subroutine, and algorithm analysis thread.

[0105] Step S31: Enable AI service:

[0106] If it starts normally, it will run normally; if there is an error in the configuration file, it will output that the configuration file is incorrect.

[0107] Step S32: Start up the various services:

[0108] Get a list of all cameras and a list of all algorithms. If an algorithm exists in the list of invalid algorithms, the program will report an error and exit.

[0109] Step S33: Start the algorithm analysis service start_alg_analyse():

[0110] Load the algorithm module, start the algorithm analysis subroutine for all algorithms, allocate the number of algorithm analysis scheduling threads, and each thread enters step S35;

[0111] Step S34: Start the algorithm analysis subroutine start_algorithm():

[0112] If the motion model initialization function is successfully executed, the output will be "Successfully loaded person detection model" and a success message will be returned. If the model initialization function is not successfully executed, the output will be "Failed to load person detection model. Please check if the weight path matches and if the called library is installed" and a failure message will be returned.

[0113] Step S35: The algorithm analysis thread retrieves an image from the queue (acquires the thread lock, retrieves an image from the list of images to be analyzed, and releases the thread lock), calls the analysis, and returns the result to the queue.

[0114] Step S351: Obtain the image to be analyzed, initialize the image name, algorithm name, and process name, call the image analysis function in step S4, pass in the parameters, and then determine whether the recognition object has been found.

[0115] Step S352: If no object is found, output "Analysis complete, XX algorithm did not find the object";

[0116] Step S353: If the object to be identified is found, add the identification result to the queue, output "Found identification result of XX algorithm", and add the identification result to the identification result list analyze_result_list;

[0117] Step S354: If the above process throws an exception, output "Error in algorithm analysis thread concurrency".

[0118] Step S4: Call the image analysis function image_analyse():

[0119] The image analysis function `image_analyse()` calls the image detection function to perform image analysis based on the passed-in algorithm name and image, returns the analysis results, initializes the algorithm name, reads the image according to the passed-in image path, passes the parameters to step S5 of the image detection function, and returns the detection results.

[0120] Step S5: Call the image detection function modelTool.detect_image():

[0121] If the model can run, proceed to step S6 of the detection function. The image detection function receives a dictionary cmdJson containing the algorithm name and the image, and a confidence threshold conf_thres as input parameters. It calls the corresponding detection model to perform image detection and finally returns the detection result.

[0122] S51. In the image detection function, the main task is to load the corresponding detection model and pass the input image and confidence threshold to the model for detection.

[0123] Step S52: If the detection model is not loaded, call the image detection function self.load_model to load the model.

[0124] Step S6: Use the detection function to make inferences, return the detection results, and store the image of the person leaving their post to the specified path;

[0125] The detection function is entered to run the detection program. It receives an image, confidence threshold conf_thres, IoU threshold iou_thres, and whether to save the output image save_img as input. It uses the loaded detection model to perform inference and finally returns the detection result.

[0126] The detection function uses an intelligent identification model for personnel leaving their posts. By introducing a BOT into the YOLOv5 architecture, the improved YOLOv5 backbone network is used to extract features from the target image, resulting in feature maps of different scales.

[0127] Multiple feature maps are fused, and the fused feature map is input into the YOLOV5 detection head for processing to obtain the off-duty recognition results at different image scales.

[0128] Step S61: Preprocessing of monitoring images;

[0129] Step S62: Load the model to perform inference on the image;

[0130] Step S63: Eliminate redundancy and adjust the detection results, return the detection results, and store the off-duty image to the specified path.

[0131] First, record the start time of inference, and use the letterbox function to fill and scale the input image img0;

[0132] Secondly, a detection model is used for detection, the detection results are converted into a more readable format, and only the detection results for the "person" category are retained;

[0133] Then, the detection results are returned, and the time when the inference ends is recorded. In the detection function, the main task is to convert the input image into a format acceptable to the model and use the intelligent detection model for detection and inference. After returning the detection results, the code also performs coordinate transformation and category filtering on the detection results.

[0134] Test results:

[0135] If there is no label "person", res will be empty, the console will print "No one is in the monitoring room", and the image of the person being absent from their post will be saved to the specified path.

[0136] If there is a label "person", res is not empty, the console prints "Someone is in the monitoring room", and no photos of people leaving their posts are stored;

[0137] If an exception is thrown, the console will print "Failed to open video" and the image of the person leaving their post will not be stored.

[0138] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for identifying absenteeism in a monitoring system, characterized in that, include: The target surveillance camera is used to acquire image data collected by the target surveillance camera in the area of ​​the work post. The target surveillance camera is one or more of several target surveillance cameras involved in the absence detection model scenario in the risk monitoring and early warning system. The storage device is used to store image data of the work area collected by the target surveillance camera during a certain period of time; The camera interface of the target surveillance camera is used to provide a data transmission interface; The processor is used to analyze images or pictures captured by the target surveillance camera through AI services and output image detection results; The algorithm loading module is used to start the algorithm analysis subroutine for the algorithms running in the off-duty monitoring system; The communication module is used to remotely transmit image data captured by the target surveillance camera; It also includes the following steps: Step S1: Obtain the names of the surveillance cameras involved in the absence detection model scenarios in the absence monitoring system; Step S2: Obtain monitoring data from the target surveillance camera, obtain a list of information from several target cameras, and receive images through storage. Step S3: Process image data using AI: AI services include algorithm analysis services, algorithm analysis subroutines, and algorithm analysis threads; Step S4: Call the image analysis function: The image analysis function calls the image detection function to perform image analysis based on the input algorithm name and image, returns the analysis results, initializes the algorithm name, reads the image according to the input image path, passes the parameters to step S5 of the image detection function, and returns the detection results. Step S5: Call the image detection function: If the model can run, proceed to step S6 of the detection function. The image detection function receives a dictionary cmdJson containing the algorithm name and the image, and a confidence threshold conf_thres as input parameters. It calls the corresponding detection model to perform image detection and finally returns the detection result. Step S6: Use the detection function to make inferences, return the detection results, and store the image of the person leaving their post to the specified path; Step S61: Preprocessing of monitoring images; Step S62: Load the model to perform inference on the image; Step S63: Eliminate redundancy and adjust the detection results, return the detection results, and store the off-duty image to the specified path.

2. The identification method of the absenteeism monitoring system according to claim 1, characterized in that, Step S2 specifically includes the following steps: Step S21: Obtain the CarmeraID of the surveillance camera from the interface according to the surveillance camera name; Step S22: Initialize the log module and output logs; Step S23: Load the target camera information list; Step S24: Start a thread based on the camera information, retrieve images from the video stream, and save them to the image saving location.

3. The identification method of the absenteeism monitoring system according to claim 1, characterized in that, Step S3 specifically includes the following steps: Step S31: Enable AI service: If it starts normally, it will run normally; if there is an error in the configuration file, it will output that the configuration file is incorrect. Step S32: Start up the various services: Get a list of all cameras and a list of all algorithms. If an algorithm exists in the list of invalid algorithms, the program will report an error and exit. Step S33: Start the algorithm analysis service: Load the algorithm module, start the algorithm analysis subroutine for all algorithms, allocate the number of algorithm analysis scheduling threads, and each thread enters step S35; Step S34: Start the algorithm analysis subroutine: If the motion model initialization function is successfully executed, the output will be "Successfully loaded person detection model" and a success message will be returned. If the model initialization function is not successfully executed, the output will be "Failed to load person detection model. Please check if the weight path matches and if the called library is installed" and a failure message will be returned. Step S35: The algorithm analysis thread retrieves the image from the queue, performs analysis, and returns the result to the queue.

4. The identification method of the absenteeism monitoring system according to claim 3, characterized in that, Step S35 specifically includes the following steps: Step S351: Obtain the image to be analyzed, initialize the image name, algorithm name, and process name, call the image analysis function in step S4, pass in the parameters, and then determine whether the recognition object has been found. Step S352: If no object is found, output "Analysis complete, XX algorithm did not find the object"; Step S353: If the object to be identified is found, add the identification result to the queue, output "Found the identification result of XX algorithm", and add the identification result to the identification result list; Step S354: If the above process throws an exception, output "There is an error in the algorithm analysis of thread concurrency".

5. The method for identifying absenteeism in a monitoring system according to claim 1, characterized in that, Step S5 specifically includes the following steps: S51. In the image detection function, the main task is to load the corresponding detection model and pass the input image and confidence threshold to the model for detection. Step S52: If the detection model is not loaded, call the image detection function to load the model.

6. The identification method of an absenteeism monitoring system according to claim 1, characterized in that, Step S6 specifically includes the following steps: The detection function runs the detection program and receives an image, a confidence threshold (conf_thres), an IoU threshold (iou_thres), and a parameter indicating whether to save the output image (save_img) as input. It then uses the loaded detection model to perform inference and finally returns the detection result.

7. The identification method of an absenteeism monitoring system according to claim 1, characterized in that, Step S6 specifically includes the following steps: First, record the start time of inference, and use the letterbox function to fill and scale the input image img0; Secondly, a detection model is used for detection, the detection results are converted into a more readable format, and only the detection results for the "person" category are retained; Then, the detection results are returned, and the time of inference end is recorded. In the detection function, the main task is to convert the input image into a format acceptable to the model, and to use the intelligent detection model for detection and inference. After returning the detection results, the coordinates of the detection results are transformed and the categories are filtered.

8. The identification method of an absenteeism monitoring system according to claim 1, characterized in that, Step S63 specifically includes the following steps: If there is no label "person", res will be empty, the console will print "No one is in the monitoring room", and the image of the person being absent from their post will be saved to the specified path. If the label is "person", res is not empty, the console prints "Someone is in the monitoring room", and no photos of people being absent from their posts are stored. If an exception is thrown, the console will print "Failed to open video" and the image of the person leaving their post will not be stored.

9. The identification method of an absenteeism monitoring system according to claim 1, characterized in that, Step S6 further includes: The detection function adopts the intelligent identification model for personnel leaving their posts. By introducing BOT into the YOLOv5 architecture, the improved YOLOv5 backbone network is used to extract features from the target image, resulting in feature maps of different scales. Multiple feature maps are fused, and the fused feature map is input into the YOLOV5 detection head for processing to obtain the off-duty recognition results at different image scales.

Images