System and method for monitoring departure of personnel in AI visual area based on real-time video analysis

Abstract

The invention discloses an AI visual area personnel departure monitoring system and method based on real-time video analysis, and the method comprises the steps: receiving a factory starting signal through a Raspberry Pi, transmitting a starting operation signal to a departure detection model, and triggering the model to start real-time video data analysis. The system is integrated with a high-performance monitoring camera to capture real-time video streams in a factory environment. And through a YOLO algorithm and a human body posture estimation algorithm, performing intelligent analysis on personnel positions, actions and behavior modes in the video data so as to judge whether an off-post behavior exists or not. And when personnel leaving or other abnormal behaviors are detected, the system generates and uploads alarm information, including abnormal types, time, positions and risk assessment, and meanwhile, the system automatically pushes the early warning information to related personnel. The Raspberry Pi is also responsible for analyzing signals from the server. And if the early warning information is received, the Raspberry Pi controls the audible and visual alarm to give an alarm. And if no alarm information exists, the Raspberry Pi continuously monitors the working state of the factory.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence, and in particular to an AI visual area personnel off-duty monitoring system and method based on real-time video analysis Background Art

[0002] In industrial environments, regional personnel off-duty detection has become a vital need, which is directly related to industrial safety, production efficiency and personnel management. With the complexity of industrial production processes and the improvement of automation, traditional off-duty detection methods have gradually shown a series of limitations and shortcomings. These include challenges such as environmental conditions, lack of accuracy and poor real-time performance. Therefore, in order to improve factory automation, personnel safety monitoring and production efficiency, an efficient, accurate and real-time regional personnel off-duty detection model is designed.

[0003] The regional personnel absence detection model uses advanced computer vision technology and deep learning algorithms to process video data from surveillance cameras in real time. Through automated data analysis and processing, personnel absence is quickly detected, reducing manual intervention and processing time. Combined with high-speed data transmission technology and real-time video analysis capabilities, the system can instantly capture and analyze the dynamic status of personnel. This real-time performance can not only quickly generate alarms, but also provide timely data support to help managers respond quickly. This is crucial to avoiding industrial accidents, improving production efficiency, and ensuring personnel safety. With the advancement of industrial automation and the continuous improvement of personnel safety management, the demand for high-precision, real-time, and reliable regional personnel absence detection models has become increasingly prominent in the engineering field. Summary of the invention

[0004] In view of this, the present invention provides a leave detection system based on video analysis, which monitors the factory start signal through the Raspberry Pi. When the start signal is received, the real-time video data obtained by the monitoring camera is intelligently analyzed to detect whether the staff is off duty. When it is found that a person is off duty, an alarm message is automatically sent to the constructed smart inspection service platform, and at the same time, it is pushed to relevant personnel, including abnormal type, time, location and other information. By analyzing the monitoring data over a long period of time, the robustness and reliability of event detection and identification are improved. Using video data, an automated event alarm response is realized through a linkage algorithm. And it also supports alarm processing records and statistics, which is convenient for subsequent analysis and management. At the same time, when the early warning information in the server is received, the Raspberry Pi will control the sound and light alarm to make an on-site sound and light alarm.

[0005] In a first aspect, the present invention provides an AI visual area personnel leaving their post monitoring system based on real-time video analysis, comprising:

[0006] The information collection and start signal processing module includes a data collection module and a start signal processing module. The data collection module is used to collect data set information of the factory off-duty detection area. The data collection module transmits the collected data to the server and performs relevant processing on the data set later. The start signal processing module is a Raspberry Pi that receives the factory start signal, passes the signal to the off-duty detection model, and triggers the model to start real-time video data analysis.

[0007] The real-time video data analysis module includes a high-definition camera and a stable local area network. The camera is installed in the working area as a factory security device to capture the real-time video stream in the factory environment. Through the YOLO model and the human posture estimation model, the position, movement and behavior pattern of the personnel in the video data are intelligently analyzed to determine whether there is off-duty behavior.

[0008] The abnormality detection and alarm module, the startup signal processing module transmits a signal to start the off-duty detection model. When it detects that a person is off-duty or other abnormal behavior occurs, the system generates and uploads alarm information to the constructed smart inspection service platform, including the abnormality type, time, location and risk assessment. At the same time, the system automatically pushes the warning information to relevant personnel.

[0009] The linkage response and management module combines linkage algorithms to realize automatic alarm response and preset measures, improving on-site management efficiency. The Raspberry Pi analyzes the signal from the server, controls the sound and light alarm to send on-site alarms, and continuously monitors the working status of the factory.

[0010] As a further improvement of the above technical solution, the data acquisition module and the start signal processing module are used to collect regional data sets and obtain start signal information, including:

[0011] Connect the USB camera module to the camera port of the Raspberry Pi and make sure the connection is secure. Execute commands in the terminal to start the camera module to collect data sets. Use labellmg to annotate the data.

[0012] The 220V operation signal indicator AC power controls the path of the AC relay, so that the high and low level jumps output from the Raspberry Pi pin of the path can be detected by the Raspberry Pi itself. The Raspberry Pi monitors the state changes of the relay, identifies the start signal, and transmits this signal to the server. After receiving the start signal, the server sends a reply signal back to the Raspberry Pi, and the Raspberry Pi starts to monitor the operation status after receiving the reply signal. In this way, the Raspberry Pi can accurately detect and respond to the start signal in the factory environment according to the state changes of the AC relay, and then perform real-time status monitoring.

[0013] As a further improvement of the above technical solution, the interquartile range is used to eliminate outliers in the monitoring data. The preset monitoring data is x 1 ,x 2 ...x n , the expression for calculating its arithmetic mean is Calculate the quartiles, sort the data, and calculate the 25th percentile Q1 and the 75th percentile Q3. Calculate the interquartile range IQR = Q3-Q1. In eliminating abnormal data, set the threshold to Q1-1.5IQR and Q3+1.5IQR. If the data point exceeds this range, it is considered abnormal data and eliminated. Repeat the above calculation steps for the remaining data until all abnormal data are eliminated.

[0014] As a further improvement of the above technical solution, the real-time video data analysis part includes a YOLOv5 model and a human posture estimation model. YOLOv5 is based on CSPDarknet53 as the backbone network, and uses technologies such as Adaptive Anchor Boxes and PANet (Path Aggregation Network) to improve detection accuracy and efficiency. The human posture estimation model is based on HRNet as the backbone network, and combines information at different resolutions through multi-scale feature fusion to enhance the detection ability of key points of the human body. First, the bottleneck layer of the high-resolution network is redesigned using dynamic split convolution and dynamic kernel aggregation operations to avoid the computational cost caused by excessive use of large convolution kernels while enhancing the network's ability to extract useful features; then, a basic module of partial convolution and normalized attention mechanism is proposed to reduce computational redundancy and memory access while retaining channel and spatial information to enhance cross-dimensional interaction; finally, the network output feature fusion method is redesigned based on multi-resolution features and deconvolution to improve the network's heat map regression prediction accuracy.

[0015] As a further improvement of the above technical solution, the anomaly detection and alarm module also includes a video surveillance playback function, an early warning push function, an information query function and a system management function. The video surveillance playback function is used to start the function to complete event storage when the model recognizes an abnormality, which is convenient for later query. The early warning push function is used to identify the abnormality and push it to relevant personnel as information after completing the storage. The information query function is used to query the time and location of historical abnormal events. The system setting function is used to set the relevant parameters of the security subsystem.

[0016] As a further improvement of the above technical solution, when the model judges that the behavior is abnormal, the server will upload the alarm information to the smart inspection platform, and at the same time require the model to perform another identification. If abnormal behavior still exists this time, and the comprehensive judgment shows that an early warning is required, the system will automatically push the early warning information to relevant personnel.

[0017] As a further improvement of the above technical solution, during the monitoring period, the Raspberry Pi receives the alarm information from the server, and controls the relay to control the sound and light alarm. If a stop signal is detected, a stop message is sent to the server, and the monitoring ends after receiving the reply. Continue to wait for the start of the next operation.

[0018] In a second aspect, the present invention provides an AI visual area personnel off-duty monitoring method based on real-time video analysis, comprising the following steps:

[0019] Acquire monitoring data of the analysis area and preprocess the data set. The Raspberry Pi monitors factory signals in real time. When it receives a start signal, it passes it to the off-duty detection model, triggering the model to start real-time video data analysis.

[0020] Intelligent analysis of real-time video data obtained by surveillance cameras can be used to detect whether staff members are away from their posts. The model uses the YOLOv5 algorithm and the human posture estimation algorithm to intelligently analyze the position, movement, and behavior patterns of personnel in the video data to determine whether there is any off-duty behavior.

[0021] When an abnormality is detected, the system generates and uploads alarm information to the regional personnel off-duty monitoring system, and automatically sends alarm information to relevant personnel to implement linkage early warning.

[0022] The Raspberry Pi analyzes the signals from the server and controls the sound and light alarms to sound on-site alarms, while continuously monitoring the factory's working status.

[0023] The present invention provides an AI visual area personnel off-duty monitoring system and method based on real-time video analysis. The factory start signal is received by Raspberry Pi, and the signal is transmitted to the off-duty detection model, triggering the model to start real-time video data analysis. The system integrates high-performance surveillance cameras to capture real-time video streams in the factory environment. Through the YOLO algorithm and the human posture estimation algorithm, the personnel position, action and behavior pattern in the video data are intelligently analyzed to determine whether there is off-duty behavior. When personnel off-duty or other abnormal behaviors are detected, the system generates and uploads alarm information, including abnormal type, time, location and risk assessment, and the system automatically pushes the warning information to relevant personnel. Raspberry Pi is also responsible for analyzing signals from the server. If the warning information is detected, the Raspberry Pi will control the sound and light alarm to alarm. If there is no alarm information, the Raspberry Pi will continue to monitor the working status of the factory. The safety management level of the factory is improved, and the work process and emergency response capabilities are also optimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A structural block diagram of the AI ​​visual area personnel leaving their posts monitoring system based on real-time video analysis provided by the present invention;

[0026] Figure 2 A flowchart of the AI ​​visual area personnel off-duty monitoring method based on real-time video analysis provided by the present invention. DETAILED DESCRIPTION

[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0028] See also Figure 1 The present invention provides an AI visual area personnel leaving their post monitoring system based on real-time video analysis, comprising:

[0029] The data acquisition and signal acquisition module includes a data acquisition module and a signal acquisition module. The data acquisition module is used to collect data set information of the factory off-duty detection area. The data acquisition module transmits the collected data to the server and performs relevant processing on the data set later. The start signal processing module is a Raspberry Pi that receives the factory start signal, passes the signal to the off-duty detection model, and triggers the model to start real-time video data analysis.

[0030] A real-time video data analysis module is provided, including a high-definition camera and a stable local area network. The camera is installed in the working area as a factory security device to capture real-time video streams in the factory environment. Through the YOLO model and the human posture estimation model, the position, movement and behavior pattern of personnel in the video data are intelligently analyzed to determine whether there is off-duty behavior.

[0031] The abnormality detection and alarm module, the startup signal processing module transmits a signal to start the off-duty detection model. When it detects that a person is off-duty or other abnormal behavior occurs, the system generates and uploads alarm information to the constructed smart inspection service platform, including the abnormality type, time, location and risk assessment. At the same time, the system automatically pushes the warning information to relevant personnel.

[0032] The linkage response and management module combines linkage algorithms to realize automatic alarm response and preset measures, improving on-site management efficiency. The Raspberry Pi analyzes the signal from the server, controls the sound and light alarm to send on-site alarms, and continuously monitors the working status of the factory.

[0033] In this embodiment, the data acquisition and signal acquisition module includes a data acquisition unit and a signal acquisition unit. The signal acquisition unit is connected to the real-time data analysis module. After the signal unit obtains the start signal, the model is started to analyze the real-time monitoring video data. The real-time data analysis module uses the YOLOv5 model and the human posture estimation model. YOLOv5 is based on CSPDarknet53 as the backbone network, and uses technologies such as Adaptive Anchor Boxes and PANet (Path Aggregation Network) to improve detection accuracy and efficiency. The human posture estimation model is based on HRNet as the backbone network. It combines information at different resolutions through multi-scale feature fusion to enhance the detection ability of key points of the human body. First, the bottleneck layer of the high-resolution network is redesigned using dynamic split convolution and dynamic kernel aggregation operations to avoid the computational cost caused by excessive use of large convolution kernels while enhancing the network's ability to extract useful features; then, a basic module of partial convolution and normalized attention mechanism is proposed to reduce computational redundancy and memory access while retaining channel and spatial information to enhance cross-dimensional interaction; finally, the network output feature fusion method is redesigned based on multi-resolution features and deconvolution to improve the network's heat map regression prediction accuracy. When there is an abnormal situation, the system generates and uploads alarm information, including the abnormal type, time, location and risk assessment, and the system automatically pushes the warning information to relevant personnel. The abnormal detection and alarm module also includes video surveillance playback function, warning push function, information query function and system management function. The video surveillance playback function is used to start the function when the model recognizes an abnormality to complete event storage, which is convenient for later query. The warning push function is used to identify the abnormality and complete the storage and push it to relevant personnel as information. The information query function is used to query the time and location of historical abnormal events. The system setting function is used to set relevant parameters of the security subsystem.

[0034] It should be noted that when the model determines that the behavior is abnormal, the server will upload the alarm information to the smart inspection platform, and at the same time, the model will be required to perform another identification. If there is still abnormal behavior this time, the comprehensive judgment requires an early warning, and the system will automatically push the early warning information to relevant personnel. During the monitoring period, the Raspberry Pi receives the alarm information from the server, and controls the relay to control the sound and light alarm. If a stop signal is detected, a stop message is sent to the server, and the monitoring ends after receiving a reply. Continue to wait for the start of the next operation.

[0035] It should be understood that after identifying an abnormality, sending a request signal and asking the model to identify again can reduce false alarms. If an alarm is still required after two judgments, an early warning will be issued and pushed to relevant personnel. Reduce manual intervention and processing time. Combined with high-speed data transmission technology and real-time video analysis capabilities, ensure that the system can instantly capture and analyze the dynamic status of personnel. Avoid industrial accidents, improve production efficiency, and ensure personnel safety.

[0036] Optionally, the data collection and signal acquisition module is used to collect regional data sets and acquire signals, including:

[0037] The line interpolation algorithm is used to complete the missing data, piecewise linear interpolation, and each segment of the linear polynomial is interpolated between the data points.

[0038] In this embodiment, the interquartile range is used to eliminate outlier data in the monitoring data. The preset monitoring data is x 1 ,x 2 ...x n , the expression for calculating its arithmetic mean is Calculate the quartiles, sort the data, and calculate the 25th percentile Q1 and the 75th percentile Q3. Calculate the interquartile range IQR = Q3-Q1. In eliminating abnormal data, set the threshold to Q1-1.5IQR and Q3+1.5IQR. If the data point exceeds this range, it is considered abnormal data and eliminated. Repeat the above calculation steps for the remaining data until all abnormal data are eliminated.

[0039] See also Figure 2 , an AI visual area personnel leaving post monitoring method based on real-time video analysis, comprising the following steps:

[0040] S1: Obtain monitoring data of the analysis area and preprocess the data set. The Raspberry Pi monitors factory signals in real time. When it receives a start signal, it passes it to the off-duty detection model, triggering the model to start real-time video data analysis.

[0041] S2: Intelligently analyze the real-time video data obtained by the surveillance camera to detect whether the staff is away from their post. The model uses the yolov5 algorithm and the human posture estimation algorithm to intelligently analyze the position, movement and behavior pattern of the personnel in the video data to determine whether there is any off-duty behavior.

[0042] S3: When an abnormality is detected, the system generates and uploads alarm information to the regional personnel off-duty monitoring system. It also automatically sends alarm information to relevant personnel to implement linkage early warning.

[0043] S4: The Raspberry Pi analyzes the signal from the server and controls the sound and light alarm to sound an on-site alarm, while continuously monitoring the factory's working status.

[0044] In this embodiment, advanced computer vision technology and deep learning algorithms are used to process video data from surveillance cameras in real time. Through automated data analysis and processing, the absence of personnel can be quickly detected, reducing manual intervention and processing time. Combining high-speed data transmission technology and real-time video analysis capabilities ensures that the system can instantly capture and analyze the dynamic status of personnel. This real-time nature can not only quickly generate alarms, but also provide timely data support to help managers respond quickly. It greatly avoids industrial accidents, improves production efficiency, and ensures personnel safety.

[0045] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0046] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0047] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. An AI visual area personnel off-duty monitoring system and method based on real-time video analysis, characterized in that: It includes a data acquisition module, a start signal processing module, a real-time video data analysis module, an anomaly detection and alarm module, and a linkage response and management module; the data acquisition module is used to collect data set information of the factory off-duty detection area, transmit the collected data to the server, and perform relevant processing on the data set later; The startup signal processing module is a Raspberry Pi that receives the factory startup signal, transmits the signal to the off-duty detection model, and triggers the model to start real-time video data analysis; A real-time video data analysis module, including a high-definition camera and a stable local area network. The camera is installed in the work area as a factory security device to capture real-time video streams in the factory environment. Through the YOLO model and human posture estimation model, the position, movement and behavior pattern of personnel in the video data are intelligently analyzed to determine whether there is off-duty behavior; Abnormal detection and alarm module, the start signal processing module transmits a signal to start the off-duty detection model. When it detects that a person is off-duty or other abnormal behavior, the system generates and uploads alarm information, including the abnormal type, time, location and risk assessment. At the same time, the system automatically pushes the warning information to relevant personnel; The linkage response and management module combines the linkage algorithm in the system. The Raspberry Pi analyzes the signals from the server and controls the sound and light alarm to sound on-site alarms, while continuously monitoring the factory's working status.

2. The AI ​​visual area personnel leaving the post monitoring system based on real-time video analysis according to claim 1 is characterized in that: The data acquisition module is used to collect data set information of the factory off-duty detection area, and adopts a line interpolation algorithm to fill in the missing data, piecewise linear interpolation, and each linear polynomial is interpolated between data points.

3. The AI ​​visual area personnel leaving the post monitoring system based on real-time video analysis according to claim 1 is characterized in that: The interquartile range is used to eliminate outliers in the monitoring data. The preset monitoring data is x1, x2...x n , the expression for calculating its arithmetic mean is Calculate the quartiles, sort the data, calculate the 25th percentile Q1 and the 75th percentile Q3; calculate the interquartile range IQR = Q3-Q1; in eliminating abnormal data, set the thresholds to Q1-1.5IQR and Q3+1.5IQR. If the data point exceeds this range, it is considered as abnormal data and eliminated; repeat the above calculation steps for the remaining data until all abnormal data are eliminated. The data acquisition module transmits the collected data to the server, and the data set is processed later; The startup signal processing module is a Raspberry Pi that receives the factory startup signal, transmits the signal to the off-duty detection model, and triggers the model to start real-time video data analysis.

4. The AI ​​visual area personnel leaving the post monitoring system based on real-time video analysis according to claim 1 is characterized in that: The video analysis module uses the YOLOv5 model and the human posture estimation model. YOLOv5 is based on CSPDarknet53 as the backbone network, and uses technologies such as Adaptive Anchor Boxes and PANet (Path Aggregation Network) to improve detection accuracy and efficiency. The human posture estimation model is based on HRNet as the backbone network, and combines information at different resolutions through multi-scale feature fusion to enhance the detection capability of key points of the human body. First, the bottleneck layer of the high-resolution network is redesigned using dynamic split convolution and dynamic kernel aggregation operations to avoid the computational cost caused by excessive use of large convolution kernels while enhancing the network's ability to extract useful features; then, a basic module of partial convolution and normalized attention mechanism is proposed to reduce computational redundancy and memory access while retaining channel and spatial information to enhance cross-dimensional interaction; finally, the network output feature fusion method is redesigned based on multi-resolution features and deconvolution to improve the network's heat map regression prediction accuracy.

5. The AI ​​visual area personnel leaving the post monitoring system based on real-time video analysis according to claim 1 is characterized in that: Also includes: The abnormal detection and alarm module also includes a video surveillance playback function, an early warning push function, an information query function and a system management function. The video surveillance playback function is used to start the function when the model recognizes an abnormality to complete event storage, which is convenient for later query. The early warning push function is used to identify the abnormality and complete the storage and then push it to relevant personnel as information. The information query function is used to query the time and location of historical abnormal events, and the system setting function is used to set the relevant parameters of the security subsystem.

6. The AI ​​visual area personnel leaving the post monitoring system based on real-time video analysis according to claim 1 is characterized in that: When the model determines that the behavior is abnormal, the server will upload the alarm information to the smart inspection platform and require the model to perform another identification. If abnormal behavior still exists this time, and the comprehensive judgment shows that a warning is required, the system will automatically push the warning information to relevant personnel.

7. The AI ​​visual area personnel leaving the post monitoring system based on real-time video analysis according to claim 1 is characterized in that: During the monitoring period, the Raspberry Pi receives the alarm information from the server, controls the relay, and thus controls the sound and light alarm. If a stop signal is detected, it sends a stop message to the server. After receiving the reply, it ends the monitoring and continues to wait for the start of the next operation.

8. The AI ​​visual area personnel leaving the post monitoring system based on real-time video analysis according to claim 1 is characterized in that: After identifying an abnormality, a request signal is sent to ask the model to identify it again, which can reduce false alarms. If an alarm is still needed after two judgments, an early warning will be issued and pushed to relevant personnel to reduce manual intervention and processing time. Combined with high-speed data transmission technology and real-time video analysis capabilities, it ensures that the system can instantly capture and analyze the dynamic status of personnel, avoid industrial accidents, improve production efficiency, and ensure personnel safety.

9. An AI visual area personnel leaving post monitoring method based on real-time video analysis according to the AI ​​visual area personnel leaving post monitoring system based on real-time video analysis according to any one of claims 1 to 8, characterized in that: The following steps are involved: Acquire and analyze regional monitoring data, and pre-process and fuse the environmental monitoring data to generate a data set; The Raspberry Pi monitors factory signals in real time. When it receives a start signal, it passes it to the off-duty detection model, triggering the model to start real-time video data analysis. When an abnormality is detected, the system generates and uploads alarm information to the regional personnel off-duty monitoring system, and automatically sends alarm information to relevant personnel to implement linkage early warning. The Raspberry Pi analyzes the signal from the server, controls the sound and light alarm to sound an on-site alarm, and continuously monitors the factory's working status.

Images