AI-based video analytics-based method and system for monitoring safe production behaviors

Abstract

This application provides a method and system for monitoring safe production behavior based on AI video analysis. The method includes: acquiring real-time video data streams from the production area; inputting each frame of video image from the real-time video data stream into a target detection model for target detection, obtaining the personnel target output by the target detection model and the target position coordinates of the personnel target in each frame of video image; cropping a local image region of the personnel target from each frame of video image based on the target position coordinates, performing feature recognition based on the local image region to obtain personnel features; comparing the personnel features with the personnel's standardized behavior features to obtain the personnel's behavior state; performing trajectory association based on the personnel behavior states corresponding to multiple consecutive frames of video images to obtain the personnel's behavior trajectory; and monitoring safe production behavior based on the personnel's behavior state and personnel behavior trajectory to generate abnormal behavior warning signals. This application improves the real-time performance and accuracy of safety monitoring in production scenarios.

Description

Technical Field

[0001] This application relates to the field of behavior monitoring technology, and in particular to a method and system for monitoring safe production behavior based on AI video analysis. Background Technology

[0002] In industrial production, construction, and mining, workplace safety is a core element in ensuring the safety of personnel and preventing property damage. With the expansion of production scale and the increase in technological complexity, the number of safety risks on production sites is growing. Unsafe acts by operators (such as failing to wear protective equipment as required, operating equipment improperly, and entering hazardous areas) have become one of the main causes of safety accidents.

[0003] To promptly detect and correct such unsafe behaviors and reduce the probability of accidents, various safety production behavior monitoring methods have gradually developed within the industry. Existing safety production behavior monitoring methods largely rely on manual inspections or a combination of traditional video surveillance and manual review. Manual inspections suffer from limited coverage, insufficient inspection frequency, and susceptibility to human fatigue or subjective judgment. While traditional video surveillance can achieve 24 / 7 coverage, it requires monitoring personnel to view video footage in real time to identify unsafe behaviors, necessitating significant manpower and exhibiting drawbacks such as recognition delays and high missed detection rates. With the increasing number of surveillance cameras and the expansion of monitored areas, the efficiency of manually processing massive amounts of video data is extremely low, making it difficult to meet the real-time and accuracy requirements of safety monitoring in modern production scenarios. Summary of the Invention

[0004] This application provides a method and system for monitoring safe production behavior based on AI video analysis, in order to improve the real-time performance and accuracy of safety monitoring in production scenarios.

[0005] Firstly, this application provides a method for monitoring safe production behavior based on AI video analysis, including:

[0006] Based on high-definition network cameras deployed in various monitoring areas of the production area, real-time video data streams of the production area are collected; the real-time video data streams include images of personnel activities within the production area;

[0007] Each frame of video image in the real-time video data stream is input into a pre-trained target detection model for target detection, and the person target output by the target detection model and the target position coordinates of the person target in each frame of video image are obtained.

[0008] Based on the target location coordinates, a local image region of the person target is cropped from each frame of video image, and feature recognition is performed based on the local image region to obtain the person's features;

[0009] The personnel characteristics are compared with the preset personnel behavior characteristics to obtain the personnel behavior state of each video image. The trajectory is then associated with the personnel behavior states corresponding to multiple consecutive video images to obtain the personnel behavior trajectory of the target in the production area.

[0010] Based on the personnel's behavior status and behavior trajectory, the safety production behavior of the personnel target in the production area is monitored, and abnormal behavior warning signals are generated.

[0011] The target detection model is trained based on sample training images, their corresponding target labels, and the position coordinates of the target labels in the sample training images.

[0012] Secondly, this application also provides a safety production behavior monitoring system based on AI video analysis, applied to the safety production behavior monitoring method based on AI video analysis as described in any of the first aspects, wherein the safety production behavior monitoring system based on AI video analysis includes:

[0013] The video data acquisition module is used to acquire real-time video data streams of the production area based on high-definition network cameras deployed in various monitoring areas of the production area; the real-time video data streams include images of personnel activities within the production area;

[0014] The target detection module is used to input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the personnel target output by the target detection model and the target position coordinates of the personnel target in each frame of video image;

[0015] The feature recognition module is used to crop out a local image region of the person target in each frame of video image based on the target location coordinates, and to perform feature recognition based on the local image region to obtain the person features;

[0016] The behavior analysis module is used to compare the personnel characteristics with preset personnel norm behavior characteristics to obtain the personnel behavior state of each frame of video image, and to perform trajectory association based on the personnel behavior states corresponding to multiple consecutive video images to obtain the personnel behavior trajectory of the personnel target in the production area.

[0017] The behavior monitoring module is used to monitor the personnel's behavior in the production area based on the personnel's behavior status and behavior trajectory, and generate abnormal behavior early warning signals.

[0018] This application also provides an electronic device, including: a memory for storing computer software programs; and a processor for reading and executing the computer software programs, thereby realizing the AI ​​video analysis-based safety production behavior monitoring method described above.

[0019] This application also provides a non-transitory computer-readable storage medium storing a computer software program, which, when executed by a processor, implements the AI ​​video analysis-based safety production behavior monitoring method described above.

[0020] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the AI ​​video analysis-based safety production behavior monitoring method described above.

[0021] The safety production behavior monitoring method and system based on AI video analysis provided in this application can accurately identify personnel targets in each frame of video images and their target position coordinates in each frame through a target detection model, avoiding the subjective bias of manual identification. Furthermore, accurate personnel features are extracted from each frame of video images by cropping and feature extraction based on the target position coordinates. Based on personnel feature comparison, automated judgment of personnel and equipment status is realized, replacing the inefficient mode of manual observation. Furthermore, trajectory tracking is performed by associating continuous multi-frame behavior trajectories, which makes up for the instantaneous misjudgment problem that may exist in single-frame analysis and improves the accuracy of behavior judgment. Furthermore, early warning of safety production behavior monitoring is carried out based on personnel behavior status and personnel behavior trajectory, ensuring the real-time nature of monitoring. Therefore, manual intervention is greatly reduced and the real-time nature and accuracy of safety production behavior monitoring are improved. Attached Figure Description

[0022] Figure 1 This is a flowchart of the safety production behavior monitoring method based on AI video analysis provided in this application;

[0023] Figure 2 This is a structural diagram of the safety production behavior monitoring system based on AI video analysis provided in this application;

[0024] Figure 3 A schematic diagram illustrating an embodiment of the electronic device provided in this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0027] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessarily obscuring the description of this application. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0028] See Figure 1 , Figure 1 This is a flowchart of the safety production behavior monitoring method based on AI video analysis provided in this application. In this embodiment, the executing entity of the safety production behavior monitoring method based on AI video analysis is the behavior monitoring system. Therefore, the safety production behavior monitoring method based on AI video analysis includes:

[0029] Step 10: Based on the high-definition network cameras deployed in various monitoring areas of the production area, collect real-time video data streams of the production area. The real-time video data streams contain images of personnel activities within the production area.

[0030] Optionally, the high-definition network cameras in this embodiment need to be placed in key locations according to the layout characteristics of the production area, such as both sides of the production line, the entrance to the raw material storage area, the vicinity of equipment operating stations, and the boundaries of hazardous areas, to ensure comprehensive coverage of all personnel activity areas within the production area and eliminate blind spots. Therefore, the behavior monitoring system continuously captures scenes within the monitoring range using high-definition network cameras deployed in various monitoring areas of the production area, converting light signals into electrical signals, and then using encoding technology to convert the electrical signals into digital video stream formats, such as common formats like H.264 and H.265. Subsequently, the digital video stream is transmitted in real-time to the behavior monitoring system via network transmission protocols such as RTSP (Real-Time Streaming Protocol) and HTTP, completing the acquisition of real-time video data streams.

[0031] In one embodiment, to monitor the safe production behavior of personnel in an automotive parts manufacturing workshop, high-definition network cameras with a resolution of 1920×1080 are deployed in key monitoring areas such as the stamping machine operating area, the perimeter of welding stations, corners of material transport channels, and the workshop entrance. The camera frame rate is set to 25 frames per second. When workshop workers are loading and unloading parts in the stamping machine operating area, the high-definition network cameras record the workers' operations in real time. After converting the light signals into electrical signals, the video data is encoded into a digital video stream using the H.265 encoding format. The real-time video data stream is continuously transmitted to the behavior monitoring system through the workshop's local area network, completing the collection of real-time video data streams of personnel activities in the stamping machine operating area.

[0032] Step 20: Input each frame of video image in the real-time video data stream into the pre-trained target detection model for target detection, and obtain the personnel target output by the target detection model and the target position coordinates of the personnel target in each frame of video image.

[0033] Furthermore, the behavior monitoring system parses the real-time video data stream frame by frame to obtain independent video images for each frame. Then, each video image is preprocessed according to preset image preprocessing requirements, such as size adjustment (adjusting to the size required by the target detection model) and image normalization (normalizing pixel values ​​to a specific range, such as 0-1). After preprocessing, each video image is input into a pre-trained target detection model. In this embodiment, the target detection model is trained using deep learning algorithms (such as YOLO, Faster R-CNN, etc.) based on a large number of sample training images and their corresponding target labels (labeled as "person") and the position coordinates of the target labels in the sample training images (usually represented by the coordinates of the top left corner (x1, y1) and the bottom right corner (x2, y2) of a rectangle). The target detection model extracts and analyzes features from each input video image, identifies the personnel targets in the image, calculates the target position coordinates of each personnel target in that video image frame, and finally outputs the personnel targets and their corresponding target position coordinates.

[0034] Continuing with the above embodiment, the behavior monitoring system analyzes the real-time video data stream of the stamping machine operation area frame by frame, obtaining video images with a resolution of 1920×1080 per frame. Each frame is adjusted to the input size of 640×640 required by the target detection model, and the pixel values ​​are normalized from 0-255 to 0-1. The preprocessed images are input into a target detection model trained using the YOLOv5 algorithm. This model was trained using 50,000 sample training images containing workshop personnel activities. Each person in the sample training images is labeled "person" and has corresponding location coordinates. When a frame containing a worker operating in front of the stamping machine is input into the model, the model successfully identifies the worker as a target by extracting and analyzing image features, and calculates the worker's location coordinates as the upper left corner (320, 200) and lower right corner (480, 500). The target detection model outputs the target person and their corresponding location coordinates.

[0035] Step 30: Based on the target location coordinates, crop out the local image region of the person target in each frame of video image, and perform feature recognition based on the local image region to obtain the person features.

[0036] Furthermore, the behavior monitoring system performs image cropping in the corresponding video frame based on the target's position coordinates within each frame. Specifically, using a rectangle defined by the target's position coordinates as the boundary, the image region within this rectangle is cropped from the entire video frame to obtain a local image region of the target person. The cropped local image regions may have inconsistent sizes, therefore, they need to be resized to meet the input size requirements of the feature recognition model. Then, the resized local image region is input into the feature recognition model (such as a feature extraction model based on a convolutional neural network). The feature recognition model performs deep feature extraction on the local image region, extracting information that characterizes the person, such as clothing color, body outline, features of protective equipment worn (such as safety helmets and gloves), and limb movements and postures, thus obtaining the person's features.

[0037] Continuing with the above embodiment, the behavior monitoring system, based on the worker's location coordinates (320, 200) - (480, 500), crops a local image region within a rectangle in the corresponding video image. This region is the local image containing the worker. The cropped local image is adjusted to a size of 224×224 and input into a feature recognition model built on ResNet50. The feature recognition model processes the local image and extracts personnel features such as the worker wearing blue overalls, a yellow safety helmet, a black protective glove on their right hand, and being in a bent-over posture, thus completing the personnel feature recognition.

[0038] Step 40: Based on the personnel characteristics, compare them with the preset personnel normative behavior characteristics to obtain the personnel behavior status of each frame of video image, and perform trajectory association based on the personnel behavior status corresponding to multiple consecutive frames of video images to obtain the personnel behavior trajectory of the personnel target in the production area.

[0039] Furthermore, the behavior monitoring system compares personnel characteristics with preset personnel behavioral norms. These preset norms include standardized clothing requirements (e.g., wearing designated color work clothes), mandatory protective equipment (e.g., safety helmets, gloves, safety glasses), standardized operational actions, and boundary features of prohibited areas. Through comparison, the system determines whether personnel in each video frame are not wearing protective equipment, are violating regulations (e.g., not operating equipment according to procedures), or have entered prohibited areas, thus obtaining the personnel behavior status corresponding to each video frame, as detailed in steps 401 to 404.

[0040] Furthermore, the behavior monitoring system tracks the position coordinates of the same person in multiple consecutive video frames. Combining the person's behavior status in each frame, the system uses a trajectory association algorithm (such as a trajectory prediction and association algorithm based on Kalman filtering) to connect the continuous person's position and behavior status, forming the person's behavior trajectory in the production area, as described in steps 405 to 408. The person's behavior trajectory can clearly reflect the person's movement path in the production area and their behavior at different locations, which is used to determine whether the person has a continuous unsafe behavior or abnormal movement path.

[0041] Step 50: Based on personnel behavior status and personnel behavior trajectory, monitor the personnel target's safe production behavior in the production area and generate abnormal behavior early warning signals.

[0042] Furthermore, the behavior monitoring system comprehensively analyzes personnel behavior status and behavior trajectory. When abnormal behaviors such as not wearing protective equipment, violating regulations, or entering prohibited areas are detected in personnel behavior status, or when personnel behavior trajectory shows continuous unsafe behaviors (such as staying in dangerous areas for a long time with abnormal behavior status) or abnormal movement paths (such as frequently entering and leaving prohibited areas or moving routes that deviate from normal working areas), the behavior monitoring system determines that there is a safety production risk. At this time, it automatically generates an abnormal behavior warning signal according to the preset warning rules, as shown in steps 501 to 504.

[0043] Optionally, the abnormal behavior warning signal in this embodiment includes key information such as the time of occurrence of the abnormal behavior, its specific location (determined based on the camera deployment location and personnel location coordinates), the type of abnormal behavior (e.g., not wearing a safety helmet, violation of regulations, entering a prohibited area, etc.), and corresponding personnel information (e.g., personnel identifiers associated through feature recognition). The generated warning signal is output through the system's warning module in various ways (e.g., displaying warning information on the monitoring center screen, issuing audible and visual alarms, sending warning SMS messages or APP push notifications to relevant management personnel's mobile phones, etc.) to remind relevant personnel to handle abnormal situations in a timely manner.

[0044] This application embodiment can accurately identify personnel targets and their location coordinates in each frame of video images through a target detection model, avoiding the subjective bias of manual identification. Furthermore, it extracts accurate personnel features by cropping and feature extraction from each frame of video images using the target location coordinates. Based on personnel feature comparison, it achieves automated judgment of personnel and equipment status, replacing the inefficient mode of manual observation. Furthermore, it performs trajectory tracking by associating continuous multi-frame behavioral trajectories, compensating for potential momentary misjudgments in single-frame analysis and improving the accuracy of behavioral judgment. Finally, it provides early warnings for safety production behavior monitoring based on personnel behavior status and trajectory, ensuring real-time monitoring and thus improving the real-time performance and accuracy of safety production behavior monitoring.

[0045] In one embodiment, steps 401 to 404 include:

[0046] Step 401: Compare each type of feature in the personnel characteristics with the corresponding type of feature in the personnel normative behavior characteristics one by one to obtain the comparison results of the same type of feature.

[0047] Optionally, the behavior monitoring system categorizes personnel characteristics according to feature types. Feature types include protective equipment wearing characteristics (such as safety helmets, protective gloves, and safety glasses), operational action characteristics (such as equipment operation posture and tool usage), and area location characteristics (such as whether the current area is a permitted area). Simultaneously, preset personnel behavioral norms are also categorized and stored according to the same feature types. For each feature type, the behavior monitoring system compares the specific features of that type in the personnel characteristics with the corresponding specific features of the same type in the personnel behavioral norms.

[0048] Optionally, in the comparison process of this application embodiment, the matching degree between the actual feature and the standard feature is calculated by a feature matching algorithm (such as a matching algorithm based on feature vector distance). When the matching degree reaches or exceeds a preset threshold, the comparison result of this type of feature is determined to be "compliant with the standard"; when the matching degree is lower than the preset threshold, the comparison result of this type of feature is determined to be "non-compliant with the standard".

[0049] In one embodiment, in the operating area of ​​a stamping machine, the behavior monitoring system extracts the personnel characteristic type and specific characteristics of a worker as follows: protective equipment characteristics (safety helmet: none; protective gloves: yes, black), operation action characteristics (hand position: not inserted into the machine tool; operation rhythm: in accordance with the procedure), and area location characteristics (location: safe operating zone). Among the preset standardized personnel behavior characteristics for this area, the same type of characteristics are: protective equipment characteristics (safety helmet: yellow safety helmet must be worn; protective gloves: black protective gloves must be worn), operation action characteristics (hand position: not allowed to be inserted into the machine tool; operation rhythm: operated according to the prescribed procedure), and area location characteristics (location: only within the safe operating zone). The system compares each type of characteristic one by one: the comparison result for the "safety helmet" characteristic in the protective equipment characteristics is "non-compliant," while the comparison result for the "protective gloves" characteristic is "compliant"; the comparison result for both sub-characteristics in the operation action characteristics is "compliant"; the comparison result for the area location characteristic is "compliant," thus obtaining the comparison results for each type of characteristic.

[0050] Step 402: Based on the comparison results of similar features, identify the deviation feature items to obtain the deviation feature set.

[0051] Furthermore, the behavior monitoring system sorts out the comparison results of similar features and filters out the features that are "non-compliant with the standard". These "non-compliant" features are the deviation features. All deviation features are summarized and integrated to obtain the deviation feature set. The deviation feature set clearly lists the specific features of the personnel in the current frame video image that are inconsistent with the standard behavior features.

[0052] Continuing with the above embodiment, based on the comparison results of step 401, the behavior monitoring system filters out the feature item "Safety Helmet: None" from the protective equipment features as the only feature item with a comparison result of "non-compliant". This feature item is summarized and integrated to form a deviation feature set, i.e., deviation feature set = {protective equipment features - missing safety helmet}. If, in another worker's comparison result, both the protective equipment feature "protective gloves: None" and the operation action feature "hand position: inside the machine tool" are "non-compliant", then the deviation feature set = {protective equipment features - missing protective gloves, operation action feature - hand inside the machine tool}.

[0053] Step 403: Based on the number of times any two deviation features appear simultaneously in the deviation feature set and the total number of times each of the two deviation features appears, determine the correlation degree between any two deviation features, and construct a deviation correlation degree matrix based on the correlation degree between any two deviation features in the deviation feature set.

[0054] Furthermore, the behavior monitoring system counts the number of times any two deviation features appear simultaneously in the set of statistical deviation features. Here, i and j represent two different bias characteristics. Simultaneously, the total number of occurrences of bias characteristic i in historical data or within the current analysis period is counted. And the total number of times the deviation feature j appears in historical data or the current analysis period. Furthermore, the behavior monitoring system determines the correlation between any two deviation features based on the number of times they occur simultaneously and the total number of times each deviation feature occurs individually. The specific formula is as follows:

[0055] .

[0056] Furthermore, the behavior monitoring system arranges the correlation degree of any two deviation features in the deviation feature set in the form of a matrix, where the rows and columns are all deviation features in the deviation feature set, and the elements in the matrix are the corresponding correlation degree values, thus constructing a deviation correlation degree matrix.

[0057] Step 404: Based on the deviation correlation matrix, identify the behavior state to obtain the personnel behavior state.

[0058] Furthermore, the behavior monitoring system identifies the behavior status based on the deviation correlation matrix to obtain the personnel behavior status, as detailed in steps 4041 to 4044.

[0059] This application embodiment can accurately identify deviation features by comparing personnel characteristics with standard features, and mine the inherent correlation between deviation features by constructing a deviation correlation matrix, ultimately achieving accurate identification of personnel behavior status. It can not only identify single abnormal behaviors such as not wearing protective equipment or violating regulations, but also discover composite abnormal behaviors with strong correlation through correlation analysis, effectively improving the accuracy of personnel behavior monitoring and risk identification capabilities in production areas.

[0060] In one embodiment, steps 4041 to 4044 include:

[0061] Step 4041: Based on the deviation correlation matrix, the deviation features with a correlation degree greater than or equal to the preset correlation threshold are sequentially combined according to the order of occurrence and logical relationship of the deviation features to construct a deviation behavior sequence.

[0062] Optionally, the behavior monitoring system extracts deviation feature pairs from the constructed deviation correlation matrix whose correlation degree is greater than or equal to a preset correlation threshold. Then, combining the actual temporal order of each deviation feature in the real-time video data stream (determined by frame number or timestamp) and the causal relationship between deviation features in the production operation logic (e.g., the logical order of entering a prohibited area followed by a violation), these deviation features are arranged and combined in an orderly manner. During the combination process, it is ensured that the sequence order of the deviation features conforms to the actual occurrence process and operational logic, ultimately constructing a deviation behavior sequence with temporal order and logical relationships.

[0063] Continuing with the above embodiment, in the stamping machine operation area, the preset association threshold is 0.5. The deviation association degree matrix shows that the association degree between deviation feature A (missing safety helmet) and deviation feature B (hand reaching into the machine tool) is 0.7 (>0.5), and the association degree between deviation feature B and deviation feature C (failure to follow emergency stop procedures) is 0.6 (>0.5). Through video frame timestamp analysis, deviation feature A first appears in frame 10, deviation feature B appears in frame 20, and deviation feature C appears in frame 30. Logically, there is an operational association of "not wearing a safety helmet → hand illegally reaching in → failure to follow emergency stop procedures". The behavior monitoring system combines these three deviation features according to their time sequence and logical relationship to construct a deviation behavior sequence of [A (frame 10) → B (frame 20) → C (frame 30)].

[0064] Step 4042: Based on the deviation behavior sequence, match it with the preset typical deviation behavior state pattern library to determine the matching similarity between the deviation behavior sequence and each typical deviation behavior state pattern.

[0065] Furthermore, the behavior monitoring system pre-establishes a typical deviation behavior state pattern library. This library contains various abnormal behavior patterns commonly found in the production area, each pattern consisting of a specific sequence of deviation behaviors, such as "not wearing a safety helmet → entering a prohibited area" and "operating equipment improperly → failing to stop the machine." The system compares the deviation behavior sequence constructed in step 4041 with each typical deviation behavior state pattern in the pattern library, calculating the matching similarity between the two using a sequence matching algorithm. The formula for calculating the matching similarity is: S = M / L. Where M represents the number of deviation features and feature pairs with the same order as the deviation behavior sequence and typical patterns, and L represents the total number of deviation features in the typical pattern. The value of S ranges from 0 to 1, with a value closer to 1 indicating a higher degree of matching.

[0066] Continuing with the above embodiment, the typical deviation behavior state pattern library contains pattern P1: [Helmet missing → Hand inserted into machine tool → Failure to follow emergency stop procedure], which contains 3 deviation features. The deviation behavior sequence constructed in step 4041 is [A (Helmet missing) → B (Hand inserted into machine tool) → C (Failure to follow emergency stop procedure)]. The behavior monitoring system compares this sequence with pattern P1 and finds that the deviation features and order of the two are completely consistent, i.e., M=3, L=3. The matching similarity is calculated according to the formula S=3 / 3=1. If the sequence is [A→B], when compared with pattern P1, M=2, L=3, then the matching similarity S=2 / 3=0.67.

[0067] Step 4043: Based on the matching similarity between the deviation behavior sequence and each typical deviation behavior state pattern, determine the behavior state pattern type corresponding to the deviation behavior sequence.

[0068] Furthermore, the behavior monitoring system sorts the similarity scores between the deviation behavior sequence calculated in step 4042 and each typical deviation behavior state pattern, and selects the typical pattern with the highest similarity. When the highest similarity score is greater than or equal to a preset pattern matching threshold (e.g., 0.8), it is determined that the behavior state pattern type corresponding to the deviation behavior sequence is consistent with the type of the typical pattern; if the highest similarity score is lower than the pattern matching threshold, it is determined to be an "abnormal behavior pattern with an unclear type." This process clarifies whether the deviation behavior sequence belongs to a known typical pattern type or an unknown pattern type.

[0069] In one embodiment, the preset pattern matching threshold is 0.8. In step 4042, the similarity between the deviation behavior sequence and pattern P1 (high-risk machinery operation violation pattern) is 1 (>0.8), while the similarity with other patterns in the pattern library (such as P2: [not wearing protective gloves → improper handling of materials]) is all below 0.5. The behavior monitoring system determines that the behavior state pattern type corresponding to the deviation behavior sequence is the "high-risk machinery operation violation pattern" to which pattern P1 belongs. If the highest similarity between a sequence and all typical patterns is 0.7 (<0.8), it is determined to be an "abnormal behavior pattern of undefined type".

[0070] Step 4044: Based on the behavioral state pattern type, occurrence frequency and impact degree of the deviation behavior sequence, identify the behavioral state to obtain the personnel behavioral state.

[0071] Furthermore, the behavior monitoring system combines walking as a state pattern type, counts the frequency of occurrence of this pattern type per unit time (e.g., number of occurrences per minute), and assesses the impact of the pattern by referring to the preset behavior impact level standard (classified according to the accident risk, equipment damage, production interruption impact, etc. that the behavior may cause, such as minor, moderate, and severe).

[0072] Furthermore, the behavior monitoring system comprehensively considers the nature of the mode type, the frequency of occurrence (higher frequency means higher risk), and the degree of impact (higher level means higher risk). According to the preset behavior status judgment rules, it classifies the personnel behavior status into specific types such as "normal", "minor abnormality", "moderate abnormality", and "serious abnormality", and finally obtains the personnel behavior status corresponding to each frame of video image.

[0073] Continuing with the above embodiment, step 4043 determines the behavioral status pattern type as "High-risk machinery operation violation mode" (mode P1). Statistics show that this mode occurs 3 times within 5 minutes (high frequency). According to the impact severity standard, this mode may lead to a mechanical injury accident, with an impact severity level of "serious". According to the judgment rule: High-risk mode + high frequency occurrence + serious impact → "Severe abnormality". Therefore, the behavior monitoring system identifies the person's behavioral status as "Severe abnormality (high-risk machinery operation violation)". If a certain pattern type is "Minor lack of protection mode", occurring once within 1 hour, with an impact severity of "minor", then the behavioral status is "Minor abnormality (lack of protective equipment)".

[0074] This application's embodiments realize a complete process from deviation feature correlation analysis to accurate identification of behavioral states. It can not only capture the temporal logical relationship of deviation behavior through sequence combination, but also realize the typological identification of abnormal behavior with the help of typical pattern library. At the same time, it combines the frequency of occurrence and the degree of impact to classify risks, and finally outputs refined personnel behavioral states, which effectively improves the logic and accuracy of behavioral state identification. It can accurately locate high-risk behavioral patterns, provide more targeted decision-making basis for safety management and control in production areas, and enhance the ability to predict and deal with safety production risks.

[0075] In one embodiment, steps 405 to 408 include:

[0076] Step 405: Based on the consistency and spatiotemporal correlation of the human behavior state in two adjacent video frames, determine the set of inter-frame correlation between the human behavior state in multiple consecutive video frames.

[0077] Optionally, the behavior monitoring system extracts the personnel behavior status between every two adjacent frames of continuous multi-frame video images. First, it determines the consistency of the behavior status between two adjacent frames. If the behavior status mode types of the two frames are the same (e.g., both are "high-risk machinery operation violation mode"), the consistency parameter is set to 1; if the types are different but logically related (e.g., a change from "minor lack of protection" to "high-risk operation violation"), the consistency parameter is set to 0.5; if the types are completely unrelated (e.g., a sudden change from "normal" to "entering a prohibited area"), the consistency parameter is set to 0. Simultaneously, the spatiotemporal correlation between the two frames is calculated. The spatiotemporal correlation is calculated based on the distance between the position coordinates of the personnel target in two adjacent frames, using the following formula: .in, Let be the Euclidean distance between the position coordinates of the same person target in frame i and frame j. This represents the maximum possible distance within the monitoring range of the production area. Inter-frame correlation R ij The calculation formula is:

[0078] .in, For consistency parameters, The weighting coefficient (with a value of 0.5) is calculated sequentially for all adjacent frames to form a set of inter-frame correlation coefficients.

[0079] In one embodiment, in the stamping machine operating area, the behavioral state of the same worker in five consecutive video frames is as follows: Frame 1 (minor lack of protection), Frame 2 (minor lack of protection), Frame 3 (high-risk machine operation violation), Frame 4 (high-risk machine operation violation), and Frame 5 (high-risk machine operation violation). Consistency parameters for adjacent frames: =1, =0.5, =1, =1. The distances between adjacent frames are respectively =0.5 meters, =1 meter =0.3 meters, =0.2 meters, =5 meters. Spatiotemporal correlation: =1-0.5 / 5=0.9, =1 - 1 / 5 = 0.8 =0.94, =0.96. Inter-frame correlation: =0.5×1+0.5×0.9=0.95, =0.5×0.5+0.5×0.8=0.65, =0.97, =0.98, forming the inter-frame correlation set {0.95, 0.65, 0.97, 0.98}.

[0080] Step 406: Determine the continuity of the behavior state of multiple consecutive frames based on the inter-frame correlation set and the preset correlation threshold to obtain a set of continuous behavior state segments.

[0081] Furthermore, a preset correlation threshold is set, and the behavior monitoring system performs element-by-element judgment on the inter-frame correlation set. When the correlation between a certain frame is greater than or equal to the preset threshold, it is determined that the behavior states of the corresponding two adjacent frames are continuous, and the latter frame is merged into the current continuous segment;

[0082] When the inter-frame correlation is less than a preset threshold, continuity is determined to be interrupted, the current continuous segment terminates, and a new continuous segment count begins from the next frame. Therefore, the behavior monitoring system traverses the entire set of inter-frame correlations, dividing the continuously correlated frame sequence into multiple continuous behavior state segments. Each segment contains a continuous range of frame numbers and a corresponding behavior state sequence, ultimately forming a set of continuous behavior state segments.

[0083] In one embodiment, the preset correlation threshold is 0.6, and the inter-frame correlation set is {0.95, 0.65, 0.97, 0.98}. Element-by-element judgment: =0.95>0.6, Frame 2 is merged into the segment containing Frame 1; =0.65>0.6, frame 3 is merged into this segment; =0.97>0.6, frame 4 is merged; =0.98>0.6, frame 5 is incorporated. Therefore, a continuous behavioral state segment is formed: segment 1 contains frames 1-5, and the behavioral state sequence is [minor protection deficiency → minor protection deficiency → high-risk machinery operation violation → high-risk machinery operation violation → high-risk machinery operation violation]. If the correlation between frames is 0.5 (<0.6), the preceding frame forms a segment, and the subsequent frame starts a new segment, eventually resulting in a set of multiple segments.

[0084] Step 407: Determine the spatial location of the segment based on the image coordinate boundaries of the frames contained in each continuous behavior state segment in the set of continuous behavior state segments.

[0085] Furthermore, for each segment in the set of continuous behavioral state segments, the behavior monitoring system extracts the location coordinates of the person target in the video images of all frames contained in that segment.

[0086] Furthermore, for each segment, the behavior monitoring system calculates the boundary values ​​for all location coordinates: the minimum value of the x-coordinate. Maximum value of x-coordinate Minimum value of the ordinate Maximum value of the ordinate The rectangular region defined by these four boundary values ​​is taken as the spatial location range of the segment, that is, the spatial location of the segment is represented by the coordinates of the upper left corner of the rectangular region (…). , ) and the coordinates of the lower right corner ( , (), used to characterize the range of activity space of a person during the continuous segment.

[0087] Continuing with the above embodiment, the continuous behavior state segment 1 includes frames 1-5, and the worker's position coordinates in each frame are: frame 1 (10, 20), frame 2 (12, 22), frame 3 (15, 25), frame 4 (14, 24), and frame 5 (13, 23). The behavior monitoring system calculates the boundary values: =10, =15, =20, =25. Therefore, the spatial location of this segment is a rectangular area with the upper left corner (10, 20) and the lower right corner (15, 25). This area corresponds to the core working range of the stamping machine tool's operating area, indicating that the worker is always active in the operating area during this segment.

[0088] Step 408: Based on the spatial location and behavioral characteristics of each continuous behavioral state segment, trajectory association is performed to obtain the personnel behavior trajectory. The segment behavioral characteristics include the duration of the segment, the proportion of each behavioral state pattern type, and the frequency of behavioral state changes within the segment.

[0089] Furthermore, the behavior monitoring system acquires the segment behavior characteristics of each continuous behavior state segment, wherein the segment behavior characteristics include the segment duration, the proportion of each behavior state pattern type, and the frequency of behavior state changes within the segment. Further, the behavior monitoring system performs trajectory correlation based on the segment spatial location and segment behavior characteristics of each continuous behavior state segment to obtain the personnel behavior trajectory, as detailed in steps 4081 to 4084.

[0090] This application embodiment captures behavioral continuity by calculating inter-frame correlation, clarifies behavioral stages by segmenting segments, and achieves segment correlation by combining spatial location and behavioral characteristics. The resulting trajectory not only includes the movement path of personnel in the production area, but also integrates key information such as behavioral state patterns, duration, and risk percentage at different stages. This solves the limitations of single-frame behavioral analysis and can comprehensively reflect the spatiotemporal evolution and risk trends of personnel behavior. It provides a basis for judging whether personnel have continuous unsafe behaviors or abnormal movement paths, and improves the completeness and depth of personnel behavior monitoring in the production area.

[0091] In one embodiment, steps 4081 to 4084 include:

[0092] Step 4081: Determine the spatial connection degree between any two consecutive behavioral state segments based on the segment spatial position of each consecutive behavioral state segment, and determine the behavioral feature similarity between any two consecutive behavioral state segments based on the segment behavioral features of each consecutive behavioral state segment.

[0093] Optionally, for any two segments in a set of continuous behavioral state segments, the behavior monitoring system first calculates the spatial coherence. The spatial connectivity is calculated based on the overlapping and adjacent areas of the two segments, using the following formula: ,in, Let be the area of ​​the overlapping region between fragment a and fragment b in terms of spatial location. This represents the area of ​​adjacent regions whose edges meet but do not overlap. and Let F represent the total area of ​​the spatial locations of fragments a and b, respectively. Simultaneously, behavioral feature similarity is calculated based on the behavioral characteristics of the fragments. Three indicators are selected: the matching degree of fragment duration, the consistency of the trend of high-risk pattern proportion, and the similarity of behavioral state change frequency. The mean of each indicator after normalization is taken as the behavioral feature similarity F. ab The value ranges from 0 to 1.

[0094] In one embodiment, the production workshop contains segment a and segment b. Segment a has a spatial location between (10, 20) and (15, 25) and an area of... =25 square meters; the spatial location of segment b is (14, 24) - (18, 28), and its area is... =24 square meters. The overlapping area of ​​the two regions. =6 square meters, area of ​​adjacent area =3 square meters, spatial connectivity =(6+3) / (25+24)≈0.18. Regarding behavioral characteristics, segment a lasts 10 seconds, has a high-risk percentage of 60%, and a change frequency of 0.2 times / second; segment b lasts 8 seconds, has a high-risk percentage of 100%, and a change frequency of 0 times / second. The duration matching degree is 0.8, the percentage trend consistency is 1, the frequency similarity is 0.5, and the behavioral characteristic similarity... = (0.8 + 1 + 0.5) / 3 = 0.77.

[0095] Step 4082: Determine the trajectory correlation between any two consecutive behavioral state segments based on the spatial location connectivity and behavioral feature similarity between any two consecutive behavioral state segments.

[0096] Furthermore, the behavior monitoring system calculates the trajectory correlation degree between any two consecutive behavioral state segments based on spatial location connectivity and behavioral feature similarity. The calculation formula is:

[0097] .in, This is the spatial location weighting coefficient (value 0.4). For spatial connectivity, This represents the similarity of behavioral characteristics.

[0098] Therefore, the correlation between spatial location and behavioral characteristics can be integrated into a single trajectory correlation value, which ranges from 0 to 1. The higher the value, the stronger the correlation between the two segments.

[0099] Continuing with the above embodiments, for fragment a and fragment b, it is known that... =0.18, =0.77, =0.4. The trajectory correlation degree is calculated according to the formula. =0.4×0.18+(1-0.4)×0.77=0.072+0.462=0.534. If the spatial location connectivity between another set of segments c and d is 0.3 and the behavioral feature similarity is 0.9, then the trajectory correlation degree is... =0.4×0.3+0.6×0.9=0.12+0.54=0.66, indicating that the two segments are more closely related.

[0100] Step 4083: Construct a segment correlation matrix based on the trajectory correlation degree between any two consecutive behavioral state segments.

[0101] Furthermore, the behavior monitoring system constructs a segment correlation matrix by arranging the trajectory correlation degrees between any two consecutive behavior state segments in matrix form. The rows and columns of the matrix are the numbers of the consecutive behavior state segments, and the elements in the matrix... This represents the trajectory correlation between the i-th segment and the j-th segment. The correlation value is 1 for the same segment itself; and 0 for combinations of segments that do not have temporal continuity. This matrix clearly shows the degree of correlation between all segments.

[0102] Step 4084: Based on the fragment correlation matrix, connect the continuous behavioral state fragments with a trajectory correlation degree greater than or equal to the preset correlation threshold in chronological order to obtain the personnel behavior trajectory.

[0103] Furthermore, the behavior monitoring system analyzes the constructed segment correlation matrix based on a preset trajectory correlation threshold, selecting segment pairs with a trajectory correlation degree greater than or equal to the preset threshold. These segments are then connected sequentially according to their chronological order (determined by the range of frame numbers contained within each segment). During the connection process, it is ensured that the end time of the preceding segment is earlier than the start time of the following segment, forming a continuous time series. By sequentially connecting all segments that meet the correlation conditions, a complete human behavior trajectory is ultimately formed, which includes a timeline, a sequence of spatial location changes, and the corresponding evolution of behavioral characteristics.

[0104] Continuing with the above embodiments, the preset trajectory association threshold is 0.5, and the segment association degree matrix is... =0.534>0.5, =0.65>0.5, =0.2 < 0.5. The time sequence of each segment is segment 1 (0-10 seconds) → segment 2 (10-18 seconds) → segment 3 (18-25 seconds). The behavior monitoring system connects segment 1, segment 2, and segment 3 in chronological order to form the personnel behavior trajectory: 0-10 seconds in the (10, 20)-(15, 25) area, with a high risk ratio of 60%; 10-18 seconds in the (14, 24)-(18, 28) area, with a high risk ratio of 100%; 18-25 seconds in the (17, 26)-(20, 30) area, with a high risk ratio of 100%, fully reflecting the worker's movement path and risk change trend.

[0105] This application's embodiments calculate the trajectory correlation between segments by considering both spatial location connectivity and behavioral feature similarity. By using matrix analysis to clarify the segment correlation, the trajectory formed by connecting segments in chronological order fully presents the spatiotemporal activity patterns and behavioral feature evolution of personnel in the production area. It can accurately capture the continuity of personnel behavior and the risk accumulation trend, providing trajectory evidence for identifying continuous unsafe behaviors and abnormal movement paths, and improving the accuracy and effectiveness of personnel safety monitoring in the production area.

[0106] In one embodiment, steps 501 to 504 include:

[0107] Step 501: Construct a behavior state feature matrix based on the personnel behavior state, and calculate the behavior state risk value of the personnel target based on the behavior state feature matrix.

[0108] Optionally, the behavior monitoring system determines a behavior status indicator system, including key indicators such as not wearing protective equipment, violation of regulations, and entering prohibited areas. Each indicator corresponds to a different risk level value (e.g., 0 for normal, 1 for minor abnormality, 2 for moderate abnormality, and 3 for severe abnormality).

[0109] Furthermore, the behavior monitoring system constructs a behavior state feature matrix based on the personnel behavior state of each video image frame. The number of rows in the matrix equals the total number of video images within the analysis period, the number of columns equals the number of behavior state indicators, and the matrix element values ​​are the risk level values ​​of the corresponding behavior state indicator in the corresponding frame. Optionally, the calculation formula for the behavior state risk value in this embodiment is as follows:

[0110] .in, This represents the number of rows (frames) in the matrix. This represents the number of columns in the matrix (number of indicators). Let be the value of the j-th index in the i-th frame. The higher the value, the higher the risk of the behavioral state.

[0111] Step 502: Determine the degree of deviation between the movement distance of the personnel target within a unit time and the normal movement distance threshold within the production area, and the deviation angle between the trajectory path and the preset safety path, based on the personnel behavior trajectory.

[0112] Furthermore, the behavior monitoring system extracts the actual distance a person moves within a unit of time (e.g., 1 minute) based on their behavioral trajectory. Normal movement distance threshold within the production area. Based on the pre-set operating procedures for different work areas (e.g., a threshold of 5 meters per minute for the stamping zone), the formula for calculating the deviation degree E is: E = ( - ) / .

[0113] Meanwhile, the behavior monitoring system extracts the actual direction vector of the trajectory path based on the person's behavior trajectory and compares it with the standard direction vector of the preset safe path, and calculates the deviation angle using the vector angle formula. The formula is:

[0114] .in, This is the actual trajectory direction vector. The safe path direction vector, The value ranges from 0° to 180°.

[0115] Continuing with the above experiment, in the stamping machine operating area, set the unit time to 1 minute and the normal travel distance threshold. =5 meters / minute. The actual distance a worker moves in 1 minute. =8 meters, deviation E=|8-5| / 5=0.6. Actual trajectory direction vector =(3,4), Preset safe path direction vector =(5,0), vector dot product =15, =5, =5, then =15 / (5×5)=0.6, deviation angle ≈53.13°.

[0116] Step 503: Determine the initial trajectory anomaly degree based on the degree of deviation and the deviation angle.

[0117] Furthermore, the behavior monitoring system normalizes the deviation angle, i.e., deviation angle / 180, and calculates the initial trajectory anomaly by weighting the normalized deviation angle and the deviation angle.

[0118] Step 504: Monitor safe production behavior based on behavioral state risk value and initial trajectory anomaly degree, and generate abnormal behavior early warning signal.

[0119] Furthermore, the behavior monitoring system monitors safe production behavior based on the behavior status risk value and the initial trajectory anomaly degree, and generates abnormal behavior warning signals, as specifically in steps 5041 to 5044.

[0120] This application embodiment combines the dual dimensions of behavioral state risk value and trajectory anomaly degree, which not only quantitatively assesses the compliance risk of personnel behavior, but also analyzes the degree of anomaly in movement trajectory. Through a dual threshold judgment mechanism, it effectively distinguishes between severe anomalies and general anomalies, so that the generated warning signal contains rich risk parameters and location information, which can accurately locate high-risk behaviors, improve the efficiency of discovering safety hazards in production areas and the accuracy of warnings, and effectively reduce the probability of safety production accidents.

[0121] In one embodiment, steps 5041 to 5044 include:

[0122] Step 5041: Correct the initial trajectory anomaly based on the behavioral state risk value to obtain the corrected trajectory anomaly, and then perform a weighted fusion based on the behavioral state risk value and the corrected trajectory anomaly to obtain the comprehensive risk index of the personnel target.

[0123] Optionally, the behavior monitoring system corrects the initial trajectory anomaly level based on the behavior state risk value. The correction formula is: = *(1+R). Wherein, To correct the trajectory anomaly degree, The initial trajectory anomaly degree, This represents the risk value for the behavioral state.

[0124] Furthermore, the behavior monitoring system uses a weighted fusion formula to calculate the comprehensive risk index of personnel objectives:

[0125] .in, This is the weighting coefficient (with a value of 0.5). This is a comprehensive risk index, with a value ranging from 0 to 2. The higher the value, the higher the comprehensive risk.

[0126] In one embodiment, the behavioral state risk value R = 0.87, and the initial trajectory anomaly degree... =0.4475. The corrected trajectory anomaly degree is calculated using the correction formula. =0.837. The comprehensive risk index is then calculated using a weighted fusion formula. =0.5*0.87+(1-0.5)*0.837=0.435+0.4185=0.8535.

[0127] Step 5042: Calculate the rate of change of the comprehensive risk index over three time intervals based on the comprehensive risk index at different time points, and determine the cumulative value of the risk trend.

[0128] Furthermore, the behavior monitoring system selects three consecutive time intervals (e.g., each interval is 1 minute) and records the comprehensive risk index corresponding to each time point. The comprehensive risk index of the first interval is... The comprehensive risk index of the second interval The comprehensive risk index of the third interval Calculate the rate of change of the comprehensive risk index within each time interval: =( - ) / , =( - ) / .

[0129] The formula for calculating the cumulative risk trend value Q is: Q = + A positive and larger cumulative risk trend value (Q) indicates an upward trend in risk and a higher degree of accumulation.

[0130] In one embodiment, the combined risk index for the three time intervals are respectively =0.6、 =0.75、 =0.8535. Calculate the rate of change. =(0.75-0.6) / 0.6=0.25, =(0.8535-0.75) / 0.75=0.138. Based on the formula, the cumulative risk trend value Q=0.25+0.138=0.388.

[0131] Step 5043: If the comprehensive risk index is greater than or equal to the preset risk threshold, or the cumulative risk trend value is greater than or equal to the preset trend threshold, then a first-level abnormal behavior warning signal is triggered.

[0132] Furthermore, preset risk thresholds and trend threshold The behavior monitoring system judges the comprehensive risk index I and the cumulative risk trend value Q: if I , or Q If the condition is met, the triggering conditions for a Level 1 abnormal behavior warning signal are met. The behavior monitoring system immediately generates a Level 1 abnormal behavior warning signal, which includes information such as the comprehensive risk index, cumulative risk trend value, anomaly type, and location of occurrence.

[0133] In one embodiment, a preset risk threshold is used. =0.7, trend threshold =0.3, I=0.8535>0.7, satisfying I> The condition is: Q = 0.388 > 0.3, which also satisfies Q > 0.3. The conditions are met. At this time, the behavior monitoring system triggers a Level 1 abnormal behavior warning signal. The signal content is: "August 10, 2025, 14:33, stamping machine tool operation area (camera CY-001), Level 1 abnormal behavior warning: comprehensive risk index 0.85, risk trend cumulative value 0.39".

[0134] Step 5044: If the comprehensive risk index is greater than or equal to the preset risk threshold and the cumulative risk trend value is greater than or equal to the preset trend threshold, then a level 2 abnormal behavior warning signal is triggered.

[0135] Furthermore, based on its judgment, the behavior monitoring system conducts further checks: if the comprehensive risk index I... And the cumulative value of risk trend Q If the conditions for triggering a Level 2 abnormal behavior warning signal are met, then the Level 2 warning signal is also met. Compared to Level 1 warnings, Level 2 warnings indicate a higher level of risk and a continuously rising trend. Therefore, the behavior monitoring system generates Level 2 abnormal behavior warning signals. In addition to the information from Level 1 warnings, the signal content also emphasizes the urgency and severity of the risk, and the warning methods are more intense (such as high-frequency audible and visual alarms, and notifications to multiple levels of management personnel).

[0136] Continuing with the above embodiment, and combining the previous calculation results, I=0.8535>0.7 and Q=0.388>0.3, simultaneously satisfying both threshold conditions. The behavior monitoring system triggers a Level 2 abnormal behavior warning signal, the content of which is: "August 10, 2025, 14:33, stamping machine operation area (camera CY-001), Level 2 abnormal behavior warning: comprehensive risk index 0.85, risk trend cumulative value 0.39, high risk and continuously rising, please take emergency measures!" This signal triggers a high-frequency audible and visual alarm in the monitoring center and simultaneously sends warning information to the workshop safety officer, production supervisor, and head of the safety management department.

[0137] It should be noted that when the comprehensive risk index I... And the cumulative value of risk trend Q This indicates that there is no risk, and no abnormal behavior warning signal will be generated at this time.

[0138] This application's embodiments obtain a comprehensive risk index by correcting trajectory anomalies and fusing behavioral state risk values. This index, combined with the cumulative risk trend value, comprehensively assesses the risk situation and triggers Level 1 and Level 2 warning signals based on different conditions. The dual-condition judgment of Level 2 warnings accurately identifies high-risk and continuously deteriorating situations, while Level 1 warnings cover scenarios where a single risk exceeds the standard. This makes the warnings more targeted and hierarchical, improving the accuracy and timeliness of safety warnings. It helps managers quickly distinguish risk levels and take corresponding measures, minimizing the possibility of safety accidents in production areas.

[0139] Furthermore, the safety production behavior monitoring system based on AI video analysis provided in this application will be described below. The safety production behavior monitoring system based on AI video analysis described below can be referred to in correspondence with the safety production behavior monitoring method based on AI video analysis described above.

[0140] Optional, refer to Figure 2 , Figure 2 This is a structural diagram of the AI-based video analytics-based safety production behavior monitoring system provided in this application. The AI-based video analytics-based safety production behavior monitoring system includes:

[0141] The video data acquisition module 210 is used to acquire real-time video data streams of the production area based on high-definition network cameras deployed in various monitoring areas of the production area; the real-time video data streams include images of personnel activities within the production area.

[0142] The target detection module 220 is used to input each frame of video image in the real-time video data stream into the pre-trained target detection model for target detection, and obtain the personnel target output by the target detection model and the target position coordinates of the personnel target in each frame of video image;

[0143] The feature recognition module 230 is used to crop out a local image region of the person target in each frame of video image based on the target position coordinates, and to perform feature recognition based on the local image region to obtain the person features;

[0144] The behavior analysis module 240 is used to compare personnel characteristics with preset personnel norm behavior characteristics to obtain the personnel behavior status of each frame of video image, and to perform trajectory association based on the personnel behavior status corresponding to multiple consecutive frames of video images to obtain the personnel behavior trajectory of the personnel target in the production area.

[0145] The behavior monitoring module 250 is used to monitor the safe production behavior of personnel in the production area based on their behavior status and behavior trajectory, and generate early warning signals for abnormal behavior.

[0146] This application embodiment can accurately identify personnel targets and their location coordinates in each frame of video images through a target detection model, avoiding the subjective bias of manual identification. Furthermore, it extracts accurate personnel features by cropping and feature extraction from each frame of video images using the target location coordinates. Based on personnel feature comparison, it achieves automated judgment of personnel and equipment status, replacing the inefficient mode of manual observation. Furthermore, it performs trajectory tracking by associating continuous multi-frame behavioral trajectories, compensating for potential momentary misjudgments in single-frame analysis and improving the accuracy of behavioral judgment. Finally, it provides early warnings for safety production behavior monitoring based on personnel behavior status and trajectory, ensuring real-time monitoring and thus improving the real-time performance and accuracy of safety production behavior monitoring.

[0147] Please see Figure 3 , Figure 3 A schematic diagram illustrating an embodiment of the electronic device provided in this application. For example... Figure 3 As shown, this application provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it performs the following steps:

[0148] Based on high-definition network cameras deployed in various monitoring areas of the production area, real-time video data streams of the production area are collected; the real-time video data streams include images of personnel activities within the production area;

[0149] Each frame of video image in the real-time video data stream is input into a pre-trained target detection model for target detection, and the output of the target detection model is the person target and the target position coordinates of the person target in each frame of video image.

[0150] Based on the target location coordinates, a local image region of the person target is cropped from each frame of video image, and feature recognition is performed based on the local image region to obtain the person's features;

[0151] By comparing personnel characteristics with preset personnel behavioral characteristics, the personnel behavior status of each video image is obtained. Based on the personnel behavior status corresponding to multiple consecutive video images, the trajectory is correlated to obtain the personnel behavior trajectory of the target in the production area.

[0152] Based on personnel behavior status and behavior trajectory, the system monitors personnel behavior in the production area to ensure safe production and generates early warning signals for abnormal behavior.

[0153] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0154] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0155] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0159] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for monitoring safe production behavior based on AI video analysis, characterized in that, include: Based on high-definition network cameras deployed in various monitoring areas of the production area, real-time video data streams of the production area are collected. The real-time video data stream includes images of personnel activity within the production area; Each frame of video image in the real-time video data stream is input into a pre-trained target detection model for target detection, and the person target output by the target detection model and the target position coordinates of the person target in each frame of video image are obtained. Based on the target location coordinates, a local image region of the person target is cropped from each frame of video image, and feature recognition is performed based on the local image region to obtain the person's features; The personnel characteristics are compared with the preset personnel behavior characteristics to obtain the personnel behavior state of each video image. The trajectory is then associated with the personnel behavior states corresponding to multiple consecutive video images to obtain the personnel behavior trajectory of the target in the production area. Based on the personnel's behavior status and behavior trajectory, the safety production behavior of the personnel target in the production area is monitored, and abnormal behavior warning signals are generated. The target detection model is trained based on sample training images, their corresponding target labels, and the position coordinates of the target labels in the sample training images. The step of comparing the personnel characteristics with preset personnel behavioral norms to obtain the personnel behavior state of each video frame includes: Based on the comparison of each type of feature in the personnel characteristics with the corresponding type of feature in the personnel normative behavior characteristics, the comparison results of the same type of feature are obtained; Based on the comparison results of the same type of features, deviation feature items are identified to obtain a set of deviation features; Based on the number of times any two deviation features appear simultaneously in the deviation feature set and the total number of times each of the two deviation features appears, the correlation degree between any two deviation features is determined, and a deviation correlation degree matrix is ​​constructed based on the correlation degree between any two deviation features in the deviation feature set. The behavior state of the person is obtained by identifying the behavior state based on the deviation correlation matrix.

2. The method for monitoring safe production behavior based on AI video analysis according to claim 1, characterized in that, The method of monitoring the safety production behavior of the personnel target in the production area based on the personnel's behavior status and behavior trajectory, and generating abnormal behavior early warning signals, includes: A behavior state feature matrix is ​​constructed based on the personnel's behavior state, and the behavior state risk value of the personnel target is calculated based on the behavior state feature matrix; the rows of the behavior state feature matrix represent each frame of video image, the tables represent behavior state indicators, and the matrix element values ​​represent the behavior state values ​​corresponding to the personnel's behavior state. Based on the personnel behavior trajectory, determine the degree of deviation between the personnel target's movement distance per unit time and the normal movement distance threshold within the production area, as well as the deviation angle between the trajectory path and the preset safety path. The initial trajectory anomaly degree is determined based on the degree of deviation and the angle of deviation; Based on the behavioral state risk value and the initial trajectory anomaly degree, safety production behavior is monitored, and the abnormal behavior early warning signal is generated.

3. The method for monitoring safe production behavior based on AI video analysis according to claim 2, characterized in that, The process of monitoring safe production behavior based on the behavioral state risk value and the initial trajectory anomaly degree, and generating the abnormal behavior early warning signal, includes: The initial trajectory anomaly is corrected based on the behavioral state risk value to obtain the corrected trajectory anomaly. The behavioral state risk value and the corrected trajectory anomaly are then weighted and fused to obtain the comprehensive risk index of the personnel target. Calculate the rate of change of the comprehensive risk index over three time intervals based on the comprehensive risk index at different time points, and determine the cumulative value of risk trend. If the comprehensive risk index is greater than or equal to the preset risk threshold, or if the cumulative risk trend value is greater than or equal to the preset trend threshold, a first-level abnormal behavior warning signal is triggered. If the comprehensive risk index is greater than or equal to the preset risk threshold, and the cumulative risk trend value is greater than or equal to the preset trend threshold, then a level-two abnormal behavior warning signal is triggered.

4. The method for monitoring safe production behavior based on AI video analysis according to claim 1, characterized in that, The process of associating the trajectory of the personnel target in the production area based on the personnel behavior states corresponding to consecutive multi-frame video images to obtain the personnel behavior trajectory includes: Based on the consistency and spatiotemporal correlation of the behavior state of people in two adjacent video frames, the set of inter-frame correlation degrees among the behavior states of people in multiple consecutive video frames is determined. Based on the inter-frame correlation set and the preset correlation threshold, the continuity of the behavior state of multiple consecutive frames is determined to obtain a set of continuous behavior state segments. Based on the image coordinate boundaries of the frames contained in each continuous behavior state segment in the set of continuous behavior state segments, the spatial location of the segment is determined; Trajectory association is performed based on the spatial location and behavioral characteristics of each continuous behavioral state segment to obtain the personnel behavior trajectory; the segment behavioral characteristics include the duration of the segment, the proportion of each behavioral state pattern type, and the frequency of behavioral state changes within the segment.

5. The method for monitoring safe production behavior based on AI video analysis according to claim 4, characterized in that, The process of associating the spatial location and behavioral features of each continuous behavioral state segment to obtain the personnel behavior trajectory includes: The spatial connection degree between any two consecutive behavioral state segments is determined based on the segment spatial location of each consecutive behavioral state segment, and the behavioral feature similarity between any two consecutive behavioral state segments is determined based on the segment behavioral features of each consecutive behavioral state segment. Based on the spatial location connectivity and behavioral feature similarity between any two consecutive behavioral state segments, the trajectory correlation between any two consecutive behavioral state segments is determined; Construct a segment correlation matrix based on the trajectory correlation degree between any two consecutive behavioral state segments; Based on the segment correlation matrix, continuous behavioral state segments with a trajectory correlation degree greater than or equal to a preset correlation threshold are connected in chronological order to obtain the personnel behavior trajectory.

6. The method for monitoring safe production behavior based on AI video analysis according to claim 1, characterized in that, The step of identifying the behavior state based on the deviation correlation matrix to obtain the personnel behavior state includes: Based on the deviation correlation matrix, deviation features with a correlation degree greater than or equal to a preset correlation threshold are sequentially combined according to the order of occurrence and logical relationship of the deviation features to construct a deviation behavior sequence. The similarity between the deviation behavior sequence and each typical deviation behavior state pattern is determined by matching the deviation behavior sequence with a preset typical deviation behavior state pattern. Based on the matching similarity between the deviation behavior sequence and each typical deviation behavior state pattern, the behavior state pattern type corresponding to the deviation behavior sequence is determined. Based on the behavioral state pattern type, frequency of occurrence, and degree of impact of the deviation behavior sequence, the behavioral state is identified to obtain the personnel's behavioral state.

7. A safety production behavior monitoring system based on AI video analysis, characterized in that, The safety production behavior monitoring system based on AI video analysis, as described in any one of claims 1 to 6, comprises: The video data acquisition module is used to acquire real-time video data streams of the production area based on high-definition network cameras deployed in various monitoring areas of the production area; the real-time video data streams include images of personnel activities within the production area; The target detection module is used to input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the personnel target output by the target detection model and the target position coordinates of the personnel target in each frame of video image; The feature recognition module is used to crop out a local image region of the person target in each frame of video image based on the target location coordinates, and to perform feature recognition based on the local image region to obtain the person features; The behavior analysis module is used to compare the personnel characteristics with preset personnel norm behavior characteristics to obtain the personnel behavior state of each frame of video image, and to perform trajectory association based on the personnel behavior states corresponding to multiple consecutive video images to obtain the personnel behavior trajectory of the personnel target in the production area. The behavior monitoring module is used to monitor the personnel target's safe production behavior in the production area based on the personnel's behavior status and behavior trajectory, and generate abnormal behavior early warning signals. The step of comparing the personnel characteristics with preset personnel behavioral norms to obtain the personnel behavior state of each video frame includes: Based on the comparison of each type of feature in the personnel characteristics with the corresponding type of feature in the personnel normative behavior characteristics, the comparison results of the same type of feature are obtained; Based on the comparison results of the same type of features, deviation feature items are identified to obtain a set of deviation features; Based on the number of times any two deviation features appear simultaneously in the deviation feature set and the total number of times each of the two deviation features appears, the correlation degree between any two deviation features is determined, and a deviation correlation degree matrix is ​​constructed based on the correlation degree between any two deviation features in the deviation feature set. The behavior state of the person is obtained by identifying the behavior state based on the deviation correlation matrix.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the safety production behavior monitoring method based on AI video analysis as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium, wherein a computer software program is stored therein, characterized in that, When the processor executes the program, it implements the safety production behavior monitoring method based on AI video analysis as described in any one of claims 1 to 6.

Images