Safety production behavior monitoring method and system based on AI video analysis

Abstract

The invention provides a safety production behavior monitoring method and system based on AI video analysis. The method comprises the steps of collecting a real-time video data stream of a production area; inputting each frame of video image in the real-time video data stream into a target detection model for target detection to obtain a personnel target output by the target detection model and a target position coordinate of the personnel target in each frame of video image; cutting out a local image area of the personnel target in each frame of video image based on the target position coordinate, and performing feature recognition based on the local image area to obtain personnel features; performing comparison on the basis of the personnel characteristics and the personnel standard behavior characteristics to obtain personnel behavior states, and performing track association on the basis of the personnel behavior states corresponding to the continuous multi-frame video images to obtain personnel behavior tracks; and performing safety production behavior monitoring based on the personnel behavior state and the personnel behavior track, and generating an abnormal behavior early warning signal. According to the method and the device, the real-time performance and the accuracy of safety monitoring in a production scene are improved.

Description

Technical Field

[0001] The present application relates to the field of behavior monitoring technology, and in particular to a method and system for monitoring safe production behavior based on AI video analysis. Background Art

[0002] In industrial production, construction, mining, and other fields, safe production is a core component of ensuring human safety and preventing property damage. With the expansion of production scale and the increase in process complexity, safety risks on production sites are increasing. Unsafe operator behavior (such as failure to wear protective gear as required, improper equipment operation, and entering hazardous areas) has become a major cause of safety accidents.

[0003] To promptly detect and correct such unsafe behaviors and reduce the probability of accidents, the industry has gradually developed a variety of safety production behavior monitoring methods. Existing safety production behavior monitoring methods mostly rely on manual inspections or traditional video surveillance combined with manual review. Manual inspections have problems such as limited coverage, insufficient inspection frequency, and susceptibility to human fatigue or subjective judgment. Although traditional video surveillance can achieve all-weather coverage, it relies on monitoring personnel to watch video footage in real time to identify unsafe behaviors. This not only requires a large amount of manpower, but also has the defects of recognition delays and high missed detection rates. With the increase in the number of surveillance cameras and the expansion of monitoring areas, the efficiency of manually processing massive amounts of video data is extremely low, making it difficult to meet the real-time and accuracy requirements of safety monitoring in modern production scenarios. Summary of the Invention

[0004] This application provides a method and system for monitoring production safety behaviors based on AI video analysis, which is used to improve the real-time and accuracy of safety monitoring in production scenarios.

[0005] In a first aspect, the present application provides a method for monitoring safe production behavior based on AI video analysis, comprising: Based on high-definition network cameras deployed in each monitoring area of ​​the production area, real-time video data streams of the production area are collected; the real-time video data streams include images of personnel activities in the production area; Input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the human target output by the target detection model and the target position coordinates of the human target in each frame of video image; Based on the target position coordinates, a local image region of the person target is cropped out in each frame of the video image, and feature recognition is performed based on the local image region to obtain person features; Based on the comparison of the personnel characteristics with the preset personnel standard behavior characteristics, the personnel behavior status of each frame of the video image is obtained, and the trajectory association is performed based on the personnel behavior status corresponding to the continuous multiple frames of video images to obtain the personnel behavior trajectory of the personnel target in the production area; Based on the personnel behavior status and the personnel behavior trajectory, the personnel target is monitored for safe production behavior in the production area, and an abnormal behavior warning signal is generated; The target detection model is trained based on sample training images and their corresponding target labels as well as position coordinate labels of the target labels in the sample training images.

[0006] In a second aspect, the present application further provides a safety production behavior monitoring system based on AI video analysis, which is applied to the safety production behavior monitoring method based on AI video analysis as described in any one of the first aspects, and the safety production behavior monitoring system based on AI video analysis includes: A video data acquisition module is configured to acquire real-time video data streams of the production area based on high-definition network cameras deployed in each monitoring area of ​​the production area; the real-time video data streams include images of personnel activities in the production area; A target detection module is used to input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the human target output by the target detection model and the target position coordinates of the human target in each frame of video image; A feature recognition module is used to crop a local image area of ​​a person target in each frame of video image based on the target position coordinates, and perform feature recognition based on the local image area to obtain person features; A behavior analysis module is used to compare the personnel characteristics with preset personnel standard behavior characteristics to obtain the personnel behavior status of each frame of video image, and to associate the trajectory of the personnel behavior status corresponding to multiple consecutive frames of video images to obtain the personnel behavior trajectory of the personnel target in the production area; A behavior monitoring module is used to monitor the safety production behavior of the personnel target in the production area based on the personnel behavior status and the personnel behavior trajectory, and generate an abnormal behavior early warning signal.

[0007] The present application also provides an electronic device, including: a memory for storing computer software programs; a processor for reading and executing the computer software programs, thereby implementing the above-mentioned method for monitoring safe production behavior based on AI video analysis.

[0008] The present application also provides a non-transitory computer-readable storage medium, in which a computer software program is stored. When the computer software program is executed by a processor, the method for monitoring safe production behavior based on AI video analysis as described above is implemented.

[0009] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the above-mentioned method for monitoring safe production behavior based on AI video analysis.

[0010] The embodiments of the present application provide a safe production behavior monitoring method and system based on AI video analysis, which can accurately identify the human target in each frame of video image and the target position coordinates of the human target in each frame of video image through the target detection model, thereby avoiding the subjective bias of manual identification, and further cropping and feature extraction of accurate human features in each frame of video image through the target position coordinates, and realizing automatic judgment of the status of personnel and equipment based on personnel feature comparison, replacing the inefficient mode of manual observation, and further tracking the trajectory through the association of continuous multi-frame behavior trajectories, thereby compensating for the instantaneous misjudgment problem that may exist in single-frame analysis, improving the accuracy of behavior judgment, and further performing early warning of safe production behavior monitoring based on the personnel behavior status and personnel behavior trajectory, thereby ensuring the real-time nature of monitoring, thereby greatly reducing human intervention and improving the real-time nature and accuracy of safe production behavior monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flowchart of the safety production behavior monitoring method based on AI video analysis provided by this application; Figure 2 This is a structural diagram of the safety production behavior monitoring system based on AI video analysis provided by this application; Figure 3 Schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0013] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "plurality" means two or more, unless otherwise specifically specified.

[0014] In the description of this application, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0015] See Figure 1 , Figure 1 : is a flowchart of the safety production behavior monitoring method based on AI video analysis provided by this application. In the embodiment of this application, the execution subject of the safety production behavior monitoring method based on AI video analysis is a behavior monitoring system. Therefore, the safety production behavior monitoring method based on AI video analysis includes: Step 10: Using high-definition network cameras deployed in each monitoring area of ​​the production area, real-time video data streams of the production area are collected. The real-time video data streams include images of people's activities in the production area.

[0016] Optionally, the high-definition network cameras in the embodiments of the present application need to be set up in key locations according to the layout characteristics of the production area, such as on both sides of the production line, the entrance to the raw material storage area, around the equipment operating station, the boundary of the dangerous area, etc., to ensure that all personnel activity areas in the production area can be fully covered and there are no monitoring blind spots. Therefore, the behavior monitoring system uses high-definition network cameras deployed in each monitoring area of ​​the production area to continuously shoot the scenes within the monitoring range, convert the optical signal into an electrical signal, and then convert the electrical signal into a digital video stream format through encoding technology, such as H.264, H.265 and other common formats. Subsequently, the digital video stream is transmitted to the behavior monitoring system in real time through network transmission protocols such as RTSP (Real-time Streaming Protocol), HTTP, etc., to complete the collection of real-time video data streams.

[0017] In one embodiment, to monitor the safety production behavior of personnel in an automotive parts production workshop, high-definition network cameras with a resolution of 1920×1080 and a frame rate of 25 frames per second were deployed in key monitoring areas within the workshop, such as the stamping machine operating area, the vicinity of welding stations, the corners of material transportation channels, and the workshop entrance. When workshop workers load and unload parts in the stamping machine operating area, the high-definition network cameras capture the workers' operations in real time. After converting the optical signal into an electrical signal, the video data is encoded into a digital video stream using the H.265 encoding format. The real-time video data stream is continuously transmitted to the behavior monitoring system via the workshop's local area network, completing the collection of real-time video data streams of personnel activities in the stamping machine operating area.

[0018] In step 20, each frame of video image in the real-time video data stream is input into a pre-trained target detection model for target detection, and the target detection model outputs the human target and the target position coordinates of the human target in each frame of video image.

[0019] Furthermore, the behavior monitoring system parses the real-time video data stream frame by frame, obtaining an independent video image for each frame. Each video frame is then preprocessed according to preset image preprocessing requirements, such as resizing (adjusting to the size required for the target detection model input) and image normalization (normalizing pixel values ​​to a specific range, such as 0-1). After preprocessing, each video frame is input into a pre-trained target detection model. The target detection model in the present embodiment is trained using a deep learning algorithm (such as the YOLO algorithm or Faster R-CNN algorithm) based on a large number of sample training images, their corresponding target labels (labeled "person"), and the position coordinate labels of the target labels in the sample training images (typically represented by the upper left corner coordinates (x1, y1) and lower right corner coordinates (x2, y2) of the rectangular box). The target detection model extracts and analyzes features from each input video frame, identifies the person targets in the image, and calculates the target position coordinates of each person target in the video frame, ultimately outputting the person targets and their corresponding target position coordinates.

[0020] Continuing with the above example, the behavior monitoring system analyzes the real-time video data stream of the stamping machine operating area frame by frame, generating video images with a resolution of 1920×1080 per frame. Each frame is resized to the 640×640 input size required by the target detection model, and the pixel values ​​are normalized from 0-255 to 0-1. The preprocessed images are then fed into a target detection model trained with the YOLOv5 algorithm. This model was trained using 50,000 sample training images of workshop personnel activities. Each person in the sample training images is labeled with a "person" label and corresponding location coordinates. When a video frame containing a worker operating a stamping machine is fed into the model, the model extracts and analyzes image features to successfully identify the worker as a target. The model then calculates the worker's position in the image as (320, 200) in the upper left corner and (480, 500) in the lower right corner. The target detection model then outputs the target and its corresponding location coordinates.

[0021] Step 30 : cropping a local image region of the person target in each frame of the video image based on the target position coordinates, and performing feature recognition based on the local image region to obtain person features.

[0022] Furthermore, the behavior monitoring system performs an image cropping operation in the video image of the corresponding frame based on the target position coordinates of the target person in each frame of the video image. Specifically, the image area within the rectangular frame determined by the target position coordinates is cropped from the entire frame of the video image to obtain a local image area of ​​the target person. The cropped local image area may have inconsistent sizes, so it is necessary to unify the size and adjust it to the input size required by the feature recognition model. After that, the unified local image area is input into the feature recognition model (such as a feature extraction model based on a convolutional neural network). The feature recognition model performs deep feature extraction on the local image area to extract information that can represent the characteristics of the person, such as the color of the person's clothing, body shape, characteristics of the protective equipment worn (such as a helmet, protective gloves, etc.), body movements and postures, etc., to obtain the person's characteristics.

[0023] Continuing with the above example, the behavior monitoring system crops the local image region within the rectangular frame corresponding to the worker's position coordinates (320, 200)-(480, 500) from the video image. This region represents the local image containing the worker. The cropped local image is resized to 224×224 and fed into a feature recognition model built on ResNet50. The feature recognition model processes this local image and extracts personal features, such as the worker's blue overalls, yellow hard hat, black protective gloves on his right hand, and a bent-over posture, completing the identification of the worker's features.

[0024] Step 40, based on the comparison of personnel characteristics with preset personnel standard behavior characteristics, the personnel behavior status of each frame of video image is obtained, and the trajectory is associated based on the personnel behavior status corresponding to multiple consecutive frames of video images to obtain the personnel behavior trajectory of the personnel target in the production area.

[0025] Furthermore, the behavior monitoring system compares the personnel characteristics with pre-set standardized personnel behavior characteristics. These include standardized clothing requirements (such as wearing work clothes of a specified color), required protective equipment characteristics (such as hard hats, protective gloves, and safety glasses), standardized operating motion characteristics, and the boundaries of prohibited areas. Through this comparison, the system determines whether the personnel in each frame of video image are not wearing protective equipment, operating in violation of regulations (such as operating equipment not according to prescribed procedures), or entering prohibited areas. The corresponding personnel behavior status for each frame of video image is then determined, as shown in steps 401 to 404.

[0026] Furthermore, the behavior monitoring system tracks the position coordinates of the same person target in multiple consecutive frames of video images, combines the person behavior status corresponding to each frame, and connects the continuous person positions and behavior status through a trajectory association algorithm (such as a trajectory prediction and association algorithm based on Kalman filtering) to form a person behavior trajectory of the person target in the production area, such as the process from step 405 to step 408. The person behavior trajectory can clearly reflect the movement path of the person in the production area and the behavior at different positions, and is used to determine whether the person has continuous unsafe behavior or abnormal movement path.

[0027] Step 50: Monitor the safety production behavior of personnel targets in the production area based on the personnel behavior status and personnel behavior trajectory, and generate abnormal behavior warning signals.

[0028] Furthermore, the behavior monitoring system comprehensively analyzes personnel behavior status and trajectory. If abnormal behavior is detected, such as not wearing protective gear, operating in violation of regulations, or entering prohibited areas, or if the personnel's trajectory indicates persistent unsafe behavior (such as prolonged stays in hazardous areas with abnormal behavior) or abnormal movement paths (such as frequent entry and exit of prohibited areas or movement routes that deviate from normal working areas), the behavior monitoring system determines that a production safety risk exists. At this point, it automatically generates an abnormal behavior warning signal according to pre-set warning rules, as detailed in steps 501 through 504.

[0029] Optionally, the abnormal behavior warning signal in the embodiments of this application includes key information such as the time the abnormal behavior occurred, the specific location (determined by the camera deployment position and the personnel's location coordinates), the type of abnormal behavior (such as not wearing a helmet, illegal operation, entering a prohibited area, etc.), and the corresponding personnel information (such as the personnel identification associated through feature recognition). The generated warning signal is output by the system's warning module in various ways (such as displaying the warning information on the monitoring center screen, issuing an audible and visual alarm, sending a warning text message or app push notification to the relevant management personnel's mobile phones, etc.), reminding relevant personnel to promptly address the abnormal situation.

[0030] The embodiment of the present application can accurately identify the human target in each frame of video image and the target position coordinates of the human target in each frame of video image through the target detection model, thereby avoiding the subjective bias of manual identification, and further cropping and feature extraction of accurate human features in each frame of video image through the target position coordinates, and realizes automatic judgment of the status of personnel and equipment based on personnel feature comparison, replacing the inefficient mode of manual observation, and further tracking the trajectory through the association of continuous multi-frame behavior trajectories, making up for the instantaneous misjudgment problem that may exist in single-frame analysis, improving the accuracy of behavior judgment, and further performing early warning of safe production behavior monitoring based on personnel behavior status and personnel behavior trajectory, ensuring the real-time nature of monitoring, thereby improving the real-time nature and accuracy of safe production behavior monitoring.

[0031] In one embodiment, the process from step 401 to step 404 includes: Step 401 : Compare each type of feature in the personnel feature with the corresponding same type feature in the personnel standard behavior feature one by one to obtain a same type feature comparison result.

[0032] Optionally, the behavior monitoring system categorizes personnel characteristics by characteristic type. These types include protective equipment wearing characteristics (such as helmets, protective gloves, and protective glasses), operating action characteristics (such as equipment operating posture and tool usage), and regional location characteristics (such as whether the current area is a permitted area). Preset standardized personnel behavior characteristics are also categorized and stored according to the same characteristic type. For each characteristic type, the behavior monitoring system compares the specific characteristics of that type in the personnel characteristics with the corresponding specific characteristics of the same type in the standardized personnel behavior characteristics.

[0033] Optionally, during the comparison process in the embodiment of the present application, the degree of matching between the actual features and the standard features is calculated through a feature matching algorithm (such as a matching algorithm based on feature vector distance). When the matching degree reaches or exceeds a preset threshold, the result of the feature comparison of this type is judged to be "compliant with the specification"; when the matching degree is lower than the preset threshold, the result of the feature comparison of this type is judged to be "not compliant with the specification".

[0034] In one embodiment, in the stamping machine operating area, the behavior monitoring system extracts the following types of personnel characteristics and specific features of a worker: protective equipment characteristics (safety helmet: no helmet; protective gloves: yes, black), operating action characteristics (hand position: not reaching into the machine; operating rhythm: in accordance with the specified procedure), and regional location characteristics (area: safe operating zone). Within the preset standardized personnel behavior characteristics for this area, similar characteristics include: protective equipment characteristics (safety helmet: must wear a yellow helmet; protective gloves: must wear black protective gloves), operating action characteristics (hand position: prohibited from reaching into the machine; operating rhythm: operate according to the specified procedure), and regional location characteristics (area: safe operating zone only). The system compares these characteristics one by one: the comparison result for the "safety helmet" feature in the protective equipment characteristics is "not in compliance with the specification," and the comparison result for the "protective gloves" feature is "in compliance with the specification." The comparison results for both sub-features in the operating action characteristics are "in compliance with the specification," and the comparison result for the regional location characteristics is "in compliance with the specification," thus obtaining the comparison results for each similar feature.

[0035] Step 402 : Identify deviation feature items based on the result of feature comparison of the same type to obtain a deviation feature set.

[0036] Furthermore, the behavior monitoring system sorts out the results of feature comparisons of the same type, and filters out feature items whose comparison results are "not in compliance with the standards". These "not in compliance with the standards" feature items are deviation feature items. All deviation feature items are summarized and integrated to obtain a deviation feature set. The deviation feature set clearly lists the specific feature content of the person in the current frame video image that does not conform to the standard behavior characteristics.

[0037] Continuing with the above example, based on the comparison results from step 401, the behavior monitoring system identifies the only feature item with a "non-compliant" comparison result: "Safety helmet: absent" in the protective gear feature. This feature item is aggregated and integrated to form a deviation feature set, i.e., Deviation Feature Set = {Protective Gear Feature - Safety Helmet Missing}. If, in the comparison results for another worker, both the Protective Gear Feature "Protective Gloves: absent" and the Operation Action Feature "Hand Position: Reaching into the Machine Tool" are "non-compliant," then the Deviation Feature Set = {Protective Gear Feature - Protective Gloves Missing, Operation Action Feature - Hand Reaching into the Machine Tool}.

[0038] Step 403 : Determine the correlation between any two deviation features based on the number of times any two deviation features in the deviation feature set appear simultaneously and the total number of times any two deviation features appear individually, and construct a deviation correlation matrix based on the correlation between any two deviation features in the deviation feature set.

[0039] Furthermore, the behavior monitoring system counts the number of times any two deviation features in the deviation feature set appear simultaneously. , where i and j represent two different deviation features. At the same time, the total number of times the deviation feature i appears in the historical data or the current analysis period is counted. , and the total number of times the deviation feature j appears in the historical data or the current analysis period Furthermore, the behavior monitoring system determines the correlation between any two deviation features based on the number of times any two deviation features appear at the same time and the total number of times any two deviation features appear separately. , the specific formula is: .

[0040] Furthermore, the behavior monitoring system arranges the correlation between any two deviation features in the deviation feature set in the form of a matrix, where the rows and columns are the deviation features in the deviation feature set, and the elements in the matrix are the corresponding correlation values, thereby constructing a deviation correlation matrix.

[0041] Step 404 : Perform behavioral status recognition based on the deviation correlation matrix to obtain the personnel behavioral status.

[0042] Furthermore, the behavior monitoring system identifies the behavior status according to the deviation correlation matrix to obtain the behavior status of the person, as specifically described in the process from step 4041 to step 4044 .

[0043] The embodiment of the present application can accurately identify deviation feature items by comparing personnel characteristics with standard characteristics, and explore the intrinsic correlation between deviation features by constructing a deviation correlation matrix, and finally achieve accurate identification of personnel behavior status. It can not only identify single abnormal behaviors such as not wearing protective equipment and illegal operations separately, but also discover complex abnormal behaviors with strong correlation through correlation analysis, effectively improving the accuracy of personnel behavior monitoring and risk identification capabilities in production areas.

[0044] In one embodiment, the process from step 4041 to step 4044 includes: Step 4041 : Based on the deviation correlation matrix, deviation features with correlation greater than or equal to a preset correlation threshold are sequentially combined according to the order and logical relationship of occurrence of the deviation features to construct a deviation behavior sequence.

[0045] Optionally, the behavior monitoring system extracts deviation feature pairs from the constructed deviation correlation matrix whose correlation is greater than or equal to a preset correlation threshold. Subsequently, these deviation features are arranged and combined in an orderly manner, combining the actual chronological order of occurrence of each deviation feature in the real-time video data stream (determined by frame number or timestamp) and the causal relationship between deviation features in the production operation logic (such as the logical sequence of entering a prohibited area followed by an illegal operation). During the combination process, the sequence of deviation features is ensured to conform to the actual occurrence process and operation logic, ultimately constructing a deviation behavior sequence with a chronological order and logical relationship.

[0046] Continuing with the above example, in the stamping machine operating area, the preset correlation threshold is 0.5. The deviation correlation matrix shows that the correlation between deviation feature A (missing helmet) and deviation feature B (hand inserted into the machine tool) is 0.7 (>0.5), and the correlation between deviation feature B and deviation feature C (failure to follow emergency stop procedures) is 0.6 (>0.5). Analysis of video frame timestamps shows that deviation feature A appears as early as frame 10, deviation feature B appears at frame 20, and deviation feature C appears at frame 30. Logically, the operational association "not wearing a helmet → illegally inserting hands → failure to follow emergency stop procedures" exists. The behavior monitoring system combines these three deviation features according to their chronological order and logical relationship, constructing a deviation behavior sequence: [A (frame 10) → B (frame 20) → C (frame 30)].

[0047] Step 4042 : Based on matching the deviation behavior sequence with a preset typical deviation behavior state pattern library, determine the matching similarity between the deviation behavior sequence and each typical deviation behavior state pattern.

[0048] Furthermore, the behavior monitoring system pre-establishes a typical deviation behavior state pattern library, which contains various abnormal behavior patterns commonly seen in production areas. Each pattern is composed of a specific deviation behavior sequence, such as typical sequence patterns such as "not wearing a safety helmet → entering a prohibited area" and "illegal operation of equipment → not shutting down for processing". The system compares the deviation behavior sequence constructed in step 4041 with each typical deviation behavior state pattern in the pattern library, and calculates the matching similarity between the two through a sequence matching algorithm. The formula for calculating matching similarity is: S=M / L. Among them, M represents the number of deviation features and feature logarithms of the same order as those in the deviation behavior sequence and the typical pattern, and L represents the total number of deviation features in the typical pattern. The value range of S is 0 to 1, and the closer the value is to 1, the higher the degree of matching.

[0049] Continuing with the above example, the typical deviation behavior state pattern library contains pattern P1: [Missing helmet → Hand inserted into machine tool → Failure to follow emergency stop procedures], which contains three deviation features. The deviation behavior sequence constructed in step 4041 is [A (Missing helmet) → B (Hand inserted into machine tool) → C (Failure to follow emergency stop procedures)]. The behavior monitoring system compares this sequence with pattern P1 and finds that the deviation features and sequence of the two are exactly the same, namely, M=3 and L=3. The matching similarity is calculated according to the formula S=3 / 3=1. If the sequence is [A→B], when compared with pattern P1, M=2 and L=3, the matching similarity S=2 / 3=0.67.

[0050] Step 4043: Determine the behavior state pattern type corresponding to the deviant behavior sequence based on the matching similarity between the deviant behavior sequence and each typical deviant behavior state pattern.

[0051] Furthermore, the behavior monitoring system ranks the similarities between the deviant behavior sequence calculated in step 4042 and each typical deviant behavior state pattern, selecting the typical pattern with the highest similarity. When the highest similarity is greater than or equal to a preset pattern matching threshold (e.g., 0.8), the behavior state pattern type corresponding to the deviant behavior sequence is determined to be consistent with the type of the typical pattern. If the highest similarity is less than the pattern matching threshold, the behavior state pattern is determined to be an "abnormal behavior pattern of unknown type." This process clarifies whether the deviant behavior sequence belongs to a known typical pattern type or an unknown pattern type.

[0052] In one embodiment, the preset pattern matching threshold is 0.8. The similarity between the deviant behavior sequence in step 4042 and pattern P1 (high-risk machinery operation violation pattern) is 1 (>0.8), while the similarity with other patterns in the pattern library (e.g., P2: [Not wearing protective gloves → Illegal material handling]) is less than 0.5. The behavior monitoring system determines that the behavioral state pattern type corresponding to this deviant behavior sequence is the "high-risk machinery operation violation pattern" to which pattern P1 belongs. If the highest similarity between a sequence and all typical patterns is 0.7 (<0.8), it is determined to be an "abnormal behavior pattern of undefined type."

[0053] Step 4044 , based on the behavioral state pattern type, occurrence frequency, and impact degree of the deviant behavior sequence, behavioral state identification is performed to obtain the personnel behavioral state.

[0054] Furthermore, the behavior monitoring system combines the walking behavior status pattern type, counts the frequency of occurrence of the pattern type in unit time (such as the number of occurrences per minute), and refers to the preset behavior impact level standard (based on the accident risk, equipment damage degree, production interruption impact, etc. that the behavior may cause, such as minor, moderate, and severe) to evaluate the impact of the pattern.

[0055] Furthermore, the behavior monitoring system integrates the nature of the pattern type, the frequency of occurrence (the higher the frequency, the higher the risk) and the degree of impact (the higher the level, the higher the risk), and divides the personnel behavior status into specific types such as "normal", "slightly abnormal", "moderately abnormal", and "severely abnormal" according to the preset behavior status judgment rules, and finally obtains the personnel behavior status corresponding to each frame of video image.

[0056] Continuing with the above embodiment, step 4043 determines that the behavior status pattern type is "high-risk mechanical operation violation pattern" (pattern P1). Statistics show that the frequency of occurrence of this pattern is 3 times within 5 minutes (high frequency occurrence). According to the impact level standard, this pattern may cause mechanical injury accidents, and the impact level is "serious". According to the judgment rule: high-risk pattern + high frequency occurrence + serious impact → "serious abnormality". Therefore, the behavior monitoring system identifies the behavior status of the person as "serious abnormality (high-risk mechanical operation violation)". If a certain pattern type is "minor protection loss pattern", it occurs once within 1 hour, and the impact level is "minor", then the behavior status is "minor abnormality (protective equipment missing)".

[0057] The embodiment of the present application realizes a complete process from deviation feature correlation analysis to accurate identification of behavioral status. It can not only capture the temporal logical relationship of deviation behavior through sequence combination, but also realize typological identification of abnormal behavior with the help of typical pattern library, and at the same time perform risk grading based on the frequency of occurrence and degree of impact, and finally output refined personnel behavior status, effectively improving the logic and accuracy of behavior status identification, and can accurately locate high-risk behavior patterns, providing more targeted decision-making basis for safety management and control of production areas, and enhancing the ability to predict and deal with production safety risks.

[0058] In one embodiment, the process from step 405 to step 408 includes: Step 405 : Based on the consistency and spatiotemporal correlation of the behavior states of the personnel in two adjacent frames of video images, a set of inter-frame correlations between the behavior states of the personnel in a plurality of consecutive frames of video images is determined.

[0059] Optionally, the behavior monitoring system extracts the behavior status of personnel in each two adjacent frames for continuous multi-frame video images. First, the consistency of the behavior status of the two adjacent frames is determined. If the behavior status pattern types of the two frames are the same (such as both are "high-risk mechanical operation violation patterns"), the consistency parameter value is 1; if the types are different but there is a logical association (such as from "minor protection loss" to "high-risk operation violation"), the consistency parameter value is 0.5; if the types are completely unrelated (such as suddenly changing from "normal" to "entering the prohibited area"), the consistency parameter value is 0. At the same time, the spatiotemporal correlation of the two frames is calculated. ,The spatiotemporal correlation is calculated based on the position coordinate distance of the personnel target in two adjacent frames, and the formula is: .in, is the Euclidean distance between the position coordinates of the same person target in the i-th frame and the j-th frame, The maximum possible distance within the monitoring range of the production area. ij The calculation formula is: .in, is the consistency parameter, is the weight coefficient (value is 0.5), which is calculated for all adjacent frames in sequence to form a set of inter-frame correlation degrees.

[0060] In one embodiment, in the stamping machine operation area, the behavior status of the same worker in 5 consecutive video frames is as follows: frame 1 (minor protection loss), frame 2 (minor protection loss), frame 3 (high-risk mechanical operation violation), frame 4 (high-risk mechanical operation violation), frame 5 (high-risk mechanical operation violation). Consistency parameters of adjacent frames: =1, =0.5, =1, =1. The distances between adjacent frames are = 0.5 m, =1 meter, = 0.3 m, = 0.2 m, =5 meters. Space-time correlation: =1-0.5 / 5=0.9, =1-1 / 5=0.8, =0.94, =0.96. Inter-frame correlation: =0.5×1+0.5×0.9=0.95, =0.5×0.5+0.5×0.8=0.65, =0.97, =0.98, forming the inter-frame correlation degree set {0.95, 0.65, 0.97, 0.98}.

[0061] Step 406 : determining the continuity of the behavior states of the personnel in the consecutive multiple frames based on the inter-frame correlation degree set and a preset correlation degree threshold, and obtaining a set of continuous behavior state segments.

[0062] Furthermore, a correlation threshold is preset, and the behavior monitoring system performs element-by-element judgment on the inter-frame correlation set. When the correlation between a certain frame is greater than or equal to the preset threshold, the behavior states of the corresponding two adjacent frames are determined to be continuous, and the latter frame is merged into the current continuous segment; When the inter-frame correlation falls below a preset threshold, continuity is determined to be interrupted, the current continuous segment terminates, and a new continuous segment count begins from the next frame. Therefore, the behavior monitoring system traverses the entire inter-frame correlation set and divides the continuously correlated frame sequence into multiple continuous behavior state segments. Each segment contains a continuous frame number range and a corresponding behavior state sequence, ultimately forming a continuous behavior state segment set.

[0063] In one embodiment, the preset correlation threshold is 0.6, and the inter-frame correlation set is {0.95, 0.65, 0.97, 0.98}. Element-by-element judgment: =0.95>0.6, frame 2 is merged into the segment where frame 1 is located; =0.65>0.6, frame 3 is merged into the segment; =0.97>0.6, frame 4 incorporated; =0.98>0.6, frame 5 is merged. This forms a continuous behavioral state segment: Segment 1 contains frames 1-5, and the behavioral state sequence is [minor protection failure → minor protection failure → high-risk mechanical operation violation → high-risk mechanical operation violation → high-risk mechanical operation violation]. If the correlation between frames is 0.5 (<0.6), the preceding frame forms a segment, and the subsequent frame starts a new segment, ultimately resulting in a collection of multiple segments.

[0064] Step 407 : Determine the spatial position of each segment based on the image coordinate boundary of the frame included in each continuous behavior state segment in the continuous behavior state segment set.

[0065] Furthermore, the behavior monitoring system extracts the position coordinates of the human target in the video images of all frames contained in each segment in the set of continuous behavior state segments.

[0066] Furthermore, for each segment, the behavior monitoring system calculates the boundary values ​​of all position coordinates: the minimum value of the horizontal coordinate , maximum value of the horizontal axis , minimum value of vertical axis , maximum value of vertical axis The rectangular area determined by these four boundary values ​​is used as the spatial position range of the fragment, that is, the spatial position of the fragment is represented by the coordinates of the upper left corner of the rectangular area ( , ) and the lower right corner coordinates ( , ), which is used to characterize the spatial range of a person's activities during this continuous segment.

[0067] Continuing with the above example, the continuous behavior state segment 1 includes frames 1-5, and the worker's position coordinates in each frame are: frame 1 (10, 20), frame 2 (12, 22), frame 3 (15, 25), frame 4 (14, 24), and frame 5 (13, 23). The behavior monitoring system calculates the boundary value: =10, =15, =20, = 25. Therefore, the spatial position of this segment is the rectangular area with the upper left corner (10, 20) and the lower right corner (15, 25). This area corresponds to the core working range of the stamping machine operation area, indicating that the worker is always active in the operation area during this segment.

[0068] Step 408: Correlate the trajectory of the person's behavior based on the spatial position and behavior characteristics of each continuous behavior state segment. The segment behavior characteristics include the duration of the segment, the proportion of each behavior state pattern type, and the frequency of behavior state changes within the segment.

[0069] Furthermore, the behavior monitoring system obtains segment behavior features for each continuous behavior state segment. These segment behavior features include the segment duration, the proportion of each behavior state pattern type, and the frequency of behavior state changes within the segment. Furthermore, the behavior monitoring system associates the spatial position and segment behavior features of each continuous behavior state segment to obtain the individual's behavior trajectory, as specifically described in steps 4081 to 4084.

[0070] The embodiment of the present application captures the continuity of behavior through inter-frame correlation calculation, clarifies the behavior stages with the help of segment division, and realizes segment association by combining spatial position and behavior characteristics. The final trajectory not only includes the movement path of personnel in the production area, but also integrates key information such as behavior state patterns, duration, risk ratio, etc. in different stages. It solves the limitations of single-frame behavior analysis, can comprehensively reflect the spatiotemporal evolution laws and risk trends of personnel behavior, and provides a basis for judging whether personnel have continuous unsafe behaviors or abnormal movement paths, thereby improving the integrity and depth of personnel behavior monitoring in production areas.

[0071] In one embodiment, the process from step 4081 to step 4084 includes: Step 4081: determine the spatial position connection between any two continuous behavior state segments based on the segment spatial position of each continuous behavior state segment, and determine the behavior feature similarity between any two continuous behavior state segments based on the segment behavior feature of each continuous behavior state segment.

[0072] Optionally, the behavior monitoring system first calculates the spatial position connectivity of any two segments in the set of continuous behavior state segments. , where the spatial position connectivity is calculated based on the overlapping area and adjacent areas of the two fragments’ spatial positions, and the formula is: ,in, is the overlapping area of ​​the spatial positions of fragment a and fragment b, is the area of ​​adjacent regions where the edges of the two regions touch but do not overlap, and are the total spatial areas of segments a and b, respectively. At the same time, the behavioral feature similarity is calculated based on the behavioral features of the segments. Three indicators are selected: the matching degree of the duration of the segments, the consistency of the trend of the change of the proportion of high-risk patterns, and the proximity of the frequency of behavioral state changes. After normalization, the average of each indicator is taken as the behavioral feature similarity F. ab , with a value range of 0 to 1.

[0073] In one embodiment, there are fragments a and b in the production workshop. The spatial position of fragment a is (10, 20)-(15, 25), and the area is = 25 square meters; the spatial position of fragment b is (14, 24)-(18, 28), and the area =24 square meters. The overlapping area of ​​the two areas =6 square meters, adjacent area =3 square meters, spatial connection =(6+3) / (25+24)≈0.18. In terms of behavioral characteristics, segment a lasts 10 seconds, high risk accounts for 60%, and changes at a frequency of 0.2 times / second; segment b lasts 8 seconds, high risk accounts for 100%, and changes at a frequency of 0 times / second. Duration matching is 0.8, proportion trend consistency is 1, frequency proximity is 0.5, and behavioral characteristics similarity is 0. =(0.8+1+0.5) / 3=0.77.

[0074] Step 4082: Determine the trajectory correlation between any two consecutive behavior state segments based on the spatial position connectivity and behavior feature similarity between any two consecutive behavior state segments.

[0075] Furthermore, the behavior monitoring system calculates the trajectory correlation between any two consecutive behavior state segments based on the spatial position connection and behavior feature similarity. The calculation formula is: .in, is the spatial position weight coefficient (value is 0.4), is the spatial position cohesion, is the behavioral feature similarity.

[0076] Therefore, the degree of correlation between spatial position and behavioral features can be integrated into a single trajectory correlation value, which ranges from 0 to 1. The higher the value, the stronger the correlation between the two segments.

[0077] Continuing with the above example, for fragment a and fragment b, it is known that =0.18, =0.77, =0.4. Calculate the trajectory correlation according to the formula =0.4×0.18+(1-0.4)×0.77=0.072+0.462=0.534. If the spatial position connection degree of another group of segments c and d is 0.3 and the behavioral feature similarity is 0.9, then the trajectory correlation degree =0.4×0.3+0.6×0.9=0.12+0.54=0.66, indicating that the two fragments are more closely related.

[0078] Step 4083: construct a segment correlation matrix based on the trajectory correlation between any two consecutive behavior state segments.

[0079] Furthermore, the behavior monitoring system arranges the trajectory correlation between any two consecutive behavior state segments in a matrix form to construct a segment correlation matrix. The rows and columns of the matrix are the numbers of the consecutive behavior state segments, and the elements in the matrix are represents the degree of correlation between the trajectories of the i-th segment and the j-th segment. The correlation between the same segment itself is 1; for a combination of segments without temporal continuity, the correlation is 0. This matrix clearly shows the degree of correlation between all segments.

[0080] Step 4084 : Based on the segment correlation matrix, the continuous behavior state segments with trajectory correlation greater than or equal to a preset correlation threshold are connected in chronological order to obtain the personnel behavior trajectory.

[0081] Furthermore, the behavior monitoring system analyzes the constructed segment correlation matrix using a preset trajectory correlation threshold. It then selects segment pairs within the matrix whose trajectory correlation is greater than or equal to the preset threshold. These segments are then linked sequentially according to their chronological order (determined by the frame number range within each segment). During the linking process, the end time of the previous segment is ensured to be before the start time of the next segment, forming a continuous time series. By sequentially linking all segments that meet the association criteria, a complete individual behavior trajectory is ultimately formed, encompassing the timeline, the sequence of spatial position changes, and the corresponding evolution of behavioral characteristics.

[0082] Continuing with the above example, the preset trajectory correlation threshold is 0.5, and the segment correlation matrix =0.534>0.5, =0.65>0.5, =0.2<0.5. The time sequence of each segment is segment 1 (0-10 seconds) → segment 2 (10-18 seconds) → segment 3 (18-25 seconds). The behavior monitoring system connects segments 1, 2, and 3 in chronological order to form a person's behavior trajectory: 0-10 seconds, activities in the (10, 20)-(15, 25) area, with a high risk ratio of 60%; 10-18 seconds, activities in the (14, 24)-(18, 28) area, with a high risk ratio of 100%; 18-25 seconds, activities in the (17, 26)-(20, 30) area, with a high risk ratio of 100%, fully reflecting the worker's movement path and risk change trend.

[0083] The embodiment of the present application calculates the trajectory correlation between fragments by dual considerations of spatial position connectivity and behavioral feature similarity, clarifies the fragment correlation relationship with the help of matrix analysis, and finally forms trajectories connected in chronological order to fully present the spatiotemporal activity patterns and behavioral feature evolution of personnel in the production area. It can accurately capture the continuity of personnel behavior and the risk accumulation trend, provide a trajectory basis for identifying persistent unsafe behaviors and abnormal movement paths, and improve the accuracy and effectiveness of personnel safety monitoring in production areas.

[0084] In one embodiment, the process from step 501 to step 504 includes: Step 501: construct a behavior state characteristic matrix based on the behavior state of the personnel, and calculate the behavior state risk value of the personnel target based on the behavior state characteristic matrix.

[0085] Optionally, the behavior monitoring system determines a behavior status indicator system, including key indicators such as failure to wear protective equipment, illegal operation, and entering prohibited areas. Each indicator corresponds to a different risk level value (such as 0 for normal, 1 for slight abnormality, 2 for moderate abnormality, and 3 for severe abnormality).

[0086] Furthermore, the behavior monitoring system constructs a behavior state feature matrix based on the behavior state of the person in each frame of video image. The number of rows of the matrix is ​​equal to the total number of video image frames in the analysis period, the number of columns is equal to the number of behavior state indicators, and the matrix element value is the risk level value of the behavior state indicator in the corresponding frame. Optionally, the calculation formula of the behavior state risk value in the embodiment of the present application is: .in, is the number of matrix rows (number of frames), is the number of matrix columns (number of indices), is the value of the jth indicator in the i-th frame. A larger value indicates a higher risk behavior state.

[0087] Step 502 : Determine the degree of deviation between the moving distance of the personnel target per unit time and the normal moving distance threshold in the production area, and the deviation angle between the trajectory path and the preset safety path based on the personnel behavior trajectory.

[0088] Furthermore, the behavior monitoring system extracts the actual moving distance of the target person in a unit time (such as 1 minute) based on the person's behavior trajectory. Normal movement distance threshold within the production area According to the preset operating specifications of different working areas (such as the threshold value of the stamping area is 5 meters / minute). The calculation formula of the deviation degree E is: E=( - ) / .

[0089] At the same time, the behavior monitoring system extracts the actual direction vector of the trajectory path and the standard direction vector of the preset safe path based on the personnel behavior trajectory, and calculates the deviation angle through the vector angle formula , the formula is: .in, is the actual trajectory direction vector, is the safe path direction vector, The value range is 0° to 180°.

[0090] Continuing the above test, in the stamping machine operation area, the unit time is set to 1 minute, and the normal moving distance threshold is =5 m / min. The actual distance a worker moves in 1 minute =8 meters, the deviation degree E=|8-5| / 5=0.6. Actual trajectory direction vector =(3, 4), preset safe path direction vector =(5,0), vector dot product =15, =5, =5, then =15 / (5×5)=0.6, deviation angle ≈53.13°.

[0091] Step 503: Determine the initial trajectory abnormality based on the deviation degree and the deviation angle.

[0092] Furthermore, the behavior monitoring system normalizes the deviation angle, i.e., deviation angle / 180, and performs weighted calculation on the normalized deviation angle and the deviation angle to obtain the initial trajectory abnormality.

[0093] Step 504 : Monitor production safety behaviors based on the behavior state risk value and the initial trajectory abnormality, and generate abnormal behavior warning signals.

[0094] Furthermore, the behavior monitoring system monitors production safety behavior based on the behavior state risk value and the initial trajectory abnormality, and generates an abnormal behavior warning signal, as specifically shown in the process from step 5041 to step 5044.

[0095] The embodiment of the present application combines the dual dimensions of behavioral state risk value and trajectory abnormality, which not only quantitatively evaluates the compliance risk of personnel behavior, but also analyzes the degree of abnormality of the movement trajectory. Through the dual-threshold judgment mechanism, it effectively distinguishes between serious abnormalities and general abnormalities, so that the generated early warning signal contains rich risk parameters and location information, can accurately locate high-risk behaviors, improve the efficiency of discovering safety hazards in production areas and the accuracy of early warnings, and effectively reduce the probability of production safety accidents.

[0096] In one embodiment, the process from step 5041 to step 5044 includes: Step 5041: The initial trajectory abnormality is corrected based on the behavior state risk value to obtain a corrected trajectory abnormality, and a weighted fusion is performed based on the behavior state risk value and the corrected trajectory abnormality to obtain a comprehensive risk index of the personnel target.

[0097] Optionally, the behavior monitoring system corrects the initial trajectory abnormality based on the behavior state risk value. The correction formula is: = *(1+R). Where, is the trajectory abnormality after correction, is the initial trajectory abnormality, is the behavior state risk value.

[0098] Furthermore, the behavior monitoring system uses a weighted fusion formula to calculate the comprehensive risk index of the personnel target: .in, is the weight coefficient (value is 0.5), It is a comprehensive risk index, ranging from 0 to 2. A larger value indicates a higher comprehensive risk.

[0099] In one embodiment, the behavior state risk value R=0.87, the initial trajectory abnormality =0.4475. Calculate the corrected trajectory abnormality according to the correction formula =0.837. Then calculate the comprehensive risk index through the weighted fusion formula =0.5*0.87+(1-0.5)*0.837=0.435+0.4185=0.8535.

[0100] Step 5042: Calculate the rate of change of the comprehensive risk index within three time intervals based on the comprehensive risk index at different time points to determine the cumulative value of the risk trend.

[0101] Furthermore, the behavior monitoring system selects three consecutive time intervals (e.g., each interval is 1 minute) and records the comprehensive risk index corresponding to each time point. The comprehensive risk index of the first interval is , the comprehensive risk index of the second interval and the comprehensive risk index of the third interval , calculate the rate of change of the comprehensive risk index in each time interval: =( - ) / , =( - ) / .

[0102] The calculation formula of the risk trend cumulative value Q is: Q= + The larger the risk trend cumulative value Q is, the more positive it is, indicating that the risk is on an upward trend and the higher the degree of accumulation.

[0103] In one embodiment, the comprehensive risk indices for the three time intervals are =0.6, =0.75, =0.8535. Calculate the rate of change =(0.75-0.6) / 0.6=0.25, =(0.8535-0.75) / 0.75=0.138. Calculate the cumulative risk trend value using the formula Q=0.25+0.138=0.388.

[0104] Step 5043: If the comprehensive risk index is greater than or equal to the preset risk threshold, or the cumulative value of the risk trend is greater than or equal to the preset trend threshold, a first-level abnormal behavior warning signal is triggered.

[0105] Furthermore, a preset risk threshold and trend thresholds , the behavior monitoring system judges the comprehensive risk index I and the risk trend cumulative value Q: If I , or Q , then the triggering conditions for the first-level abnormal behavior warning signal are met. The behavior monitoring system immediately generates a first-level abnormal behavior warning signal, which includes information such as the comprehensive risk index, the cumulative value of the risk trend, the type of abnormality, and the location of the occurrence.

[0106] In one embodiment, a risk threshold is preset. =0.7, trend threshold =0.3, I=0.8535>0.7, satisfying I> The condition; Q = 0.388 > 0.3, also satisfies Q > At this point, the behavior monitoring system triggers a Level 1 abnormal behavior warning signal, which reads: "August 10, 2025, 14:33, stamping machine operation area (camera CY-001), Level 1 abnormal behavior warning: comprehensive risk index 0.85, cumulative risk trend value 0.39."

[0107] Step 5044: If the comprehensive risk index is greater than or equal to the preset risk threshold, and the cumulative value of the risk trend is greater than or equal to the preset trend threshold, a secondary abnormal behavior warning signal is triggered.

[0108] Furthermore, the behavior monitoring system further checks based on the judgment: If the comprehensive risk index I , and the cumulative risk trend value Q , then the triggering conditions for a Level 2 abnormal behavior warning signal are also met. Compared to Level 1 warnings, Level 2 warnings indicate a higher level of risk and a continuously increasing trend. Therefore, the behavior monitoring system generates a Level 2 abnormal behavior warning signal. In addition to the Level 1 warning information, the signal emphasizes the urgency and severity of the risk and uses a more intensive warning method (such as high-frequency sound and light alarms and notifications to multiple levels of management personnel).

[0109] Continuing with the above example, combined with the previous calculation results, I = 0.8535 > 0.7 and Q = 0.388 > 0.3, satisfying both threshold conditions. The behavior monitoring system triggers a Level 2 abnormal behavior warning signal with the following content: "August 10, 2025, 2:33 PM, Stamping Machine Operation Area (Camera CY-001), Level 2 abnormal behavior warning: Comprehensive risk index 0.85, cumulative risk trend value 0.39, high risk and continuing to rise, please take urgent action!" This signal triggers a high-frequency audible and visual alarm at the monitoring center, and simultaneously sends a warning message to the workshop safety officer, production supervisor, and the head of the safety management department.

[0110] It should be noted that when the comprehensive risk index I , and the cumulative risk trend value Q , indicating that there is no risk and no abnormal behavior warning signal is generated.

[0111] The embodiment of the present application corrects the trajectory anomaly and integrates the behavior state risk value to obtain a comprehensive risk index, combines the risk trend cumulative value to comprehensively assess the risk situation, and triggers the first and second level warning signals according to different conditions. The dual condition judgment of the second level warning can accurately identify high-risk and continuously deteriorating situations, while the first level warning covers scenarios where a single risk exceeds the standard, making the warning more targeted and hierarchical, improving the accuracy and timeliness of safety warnings, helping managers quickly distinguish risk levels and take corresponding disposal measures, and minimizing the possibility of safety accidents in production areas.

[0112] Furthermore, the safety production behavior monitoring system based on AI video analysis provided in this application is described below. The safety production behavior monitoring system based on AI video analysis described below and the safety production behavior monitoring method based on AI video analysis described above can be referenced to each other.

[0113] Optional, see Figure 2 , Figure 2 This is a structural diagram of the safety production behavior monitoring system based on AI video analysis provided by this application. The safety production behavior monitoring system based on AI video analysis includes: The video data acquisition module 210 is used to collect real-time video data streams of the production area based on high-definition network cameras deployed in various monitoring areas of the production area; the real-time video data streams include images of personnel activities in the production area; The target detection module 220 is used to input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the human target output by the target detection model and the target position coordinates of the human target in each frame of video image; A feature recognition module 230 is configured to crop a local image region of a person target in each frame of video image based on the target position coordinates, and perform feature recognition based on the local image region to obtain person features; Behavior analysis module 240 is used to compare personnel characteristics with preset personnel standard behavior characteristics to obtain the personnel behavior status of each frame of video image, and to associate the trajectory of the personnel behavior status corresponding to multiple consecutive frames of video images to obtain the personnel behavior trajectory of the target in the production area; The behavior monitoring module 250 is used to monitor the safety production behavior of personnel targets in the production area based on the personnel behavior status and personnel behavior trajectory, and generate abnormal behavior warning signals.

[0114] The embodiment of the present application can accurately identify the human target in each frame of video image and the target position coordinates of the human target in each frame of video image through the target detection model, thereby avoiding the subjective bias of manual identification, and further cropping and feature extraction of accurate human features in each frame of video image through the target position coordinates, and realizes automatic judgment of the status of personnel and equipment based on personnel feature comparison, replacing the inefficient mode of manual observation, and further tracking the trajectory through the association of continuous multi-frame behavior trajectories, making up for the instantaneous misjudgment problem that may exist in single-frame analysis, improving the accuracy of behavior judgment, and further performing early warning of safe production behavior monitoring based on personnel behavior status and personnel behavior trajectory, ensuring the real-time nature of monitoring, thereby improving the real-time nature and accuracy of safe production behavior monitoring.

[0115] See also Figure 3 , Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present application. Figure 3 As shown, an embodiment of the present application provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented: Based on high-definition network cameras deployed in various monitoring areas of the production area, real-time video data streams of the production area are collected; the real-time video data streams contain images of personnel activities in the production area; Input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the human target output by the target detection model and the target position coordinates of the human target in each frame of video image; Based on the target position coordinates, a local image area of ​​the person target is cropped in each frame of the video image, and feature recognition is performed based on the local image area to obtain the person features; By comparing personnel characteristics with preset personnel standard behavior characteristics, the personnel behavior status of each frame of video image is obtained, and the trajectory of the personnel behavior status corresponding to multiple consecutive frames of video images is associated to obtain the personnel behavior trajectory of the target in the production area; Based on the personnel behavior status and personnel behavior trajectory, the safety production behavior of personnel targets in the production area is monitored, and abnormal behavior warning signals are generated.

[0116] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0119] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that is implemented in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0121] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0122] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if such changes and modifications of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such changes and modifications.

Claims

1. A safety production behavior monitoring method based on AI video analysis, characterized in that: include: Based on high-definition network cameras deployed in various monitoring areas of the production area, real-time video data streams of the production area are collected; The real-time video data stream includes images of personnel activities in the production area; Input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the human target output by the target detection model and the target position coordinates of the human target in each frame of video image; Based on the target position coordinates, a local image region of the person target is cropped out in each frame of the video image, and feature recognition is performed based on the local image region to obtain person features; Based on the comparison of the personnel characteristics with the preset personnel standard behavior characteristics, the personnel behavior status of each frame of the video image is obtained, and the trajectory association is performed based on the personnel behavior status corresponding to the continuous multiple frames of video images to obtain the personnel behavior trajectory of the personnel target in the production area; Based on the personnel behavior status and the personnel behavior trajectory, the personnel target is monitored for safe production behavior in the production area, and an abnormal behavior warning signal is generated; The target detection model is trained based on sample training images and their corresponding target labels as well as position coordinate labels of the target labels in the sample training images.

2. The method for monitoring safe production behavior based on AI video analysis according to claim 1, characterized in that: The step of monitoring the safety production behavior of the personnel target in the production area based on the personnel behavior state and the personnel behavior trajectory and generating an abnormal behavior warning signal includes: A behavior state characteristic matrix is ​​constructed based on the behavior state of the personnel, and a behavior state risk value of the personnel target is calculated based on the behavior state characteristic matrix; the rows of the behavior state characteristic matrix represent each frame of video image, the columns represent behavior state indicators, and the matrix element values ​​represent behavior state values ​​corresponding to the behavior state of the personnel; Determine, based on the personnel behavior trajectory, the degree of deviation between the movement distance of the personnel target per unit time and the normal movement distance threshold within the production area, as well as the deviation angle between the trajectory path and the preset safety path; determining an initial trajectory abnormality based on the deviation degree and the deviation angle; Safety production behavior monitoring is performed based on the behavior state risk value and the initial trajectory abnormality, and the abnormal behavior early warning signal is generated.

3. The method for monitoring safe production behavior based on AI video analysis according to claim 2 is characterized in that: The step of monitoring the production safety behavior based on the behavior state risk value and the initial trajectory abnormality to generate the abnormal behavior warning signal includes: Correcting the initial trajectory abnormality based on the behavior state risk value to obtain a corrected trajectory abnormality, and performing weighted fusion based on the behavior state risk value and the corrected trajectory abnormality to obtain a comprehensive risk index for the personnel target; Based on the comprehensive risk index at different time points, the rate of change of the comprehensive risk index within three time intervals is calculated to determine the cumulative value of the risk trend; If the comprehensive risk index is greater than or equal to the preset risk threshold, or the cumulative value of the risk trend is greater than or equal to the preset trend threshold, a first-level abnormal behavior warning signal is triggered; If the comprehensive risk index is greater than or equal to the preset risk threshold, and the risk trend cumulative value is greater than or equal to the preset trend threshold, a secondary abnormal behavior warning signal is triggered.

4. The method for monitoring safe production behavior based on AI video analysis according to claim 1, characterized in that: The process of performing trajectory association based on the behavior states of personnel corresponding to the video images of the continuous multiple frames to obtain the behavior trajectory of the personnel target in the production area includes: Based on the consistency and spatiotemporal correlation of the behavior states of the personnel in two adjacent frames of video images, a set of inter-frame correlations between the behavior states of the personnel in multiple consecutive frames of video images is determined; Determining the continuity of the behavior states of the personnel in multiple consecutive frames based on the inter-frame correlation degree set and a preset correlation degree threshold, and obtaining a set of continuous behavior state segments; determining a segment spatial position based on an image coordinate boundary of a frame included in each continuous behavior state segment in the set of continuous behavior state segments; The person's behavior trajectory is obtained by performing trajectory association based on the spatial position and behavior characteristics of each continuous behavior state segment; the behavior characteristics of the segment include the duration of the segment, the proportion of each behavior state pattern type, and the frequency of behavior state changes within the segment.

5. The method for monitoring safe production behavior based on AI video analysis according to claim 4 is characterized in that: The step of performing trajectory association based on the segment spatial position and segment behavior features of each continuous behavior state segment to obtain the personnel behavior trajectory includes: Determining the spatial position connection between any two continuous behavior state segments based on the segment spatial position of each continuous behavior state segment, and determining the behavioral feature similarity between any two continuous behavior state segments based on the segment behavioral feature of each continuous behavior state segment; Determine the trajectory correlation between any two consecutive behavior state segments based on the spatial position connection and behavior feature similarity between any two consecutive behavior state segments; Based on the trajectory correlation between any two consecutive behavior state segments, a segment correlation matrix is ​​constructed; Based on the segment correlation matrix, continuous behavior state segments with trajectory correlation greater than or equal to a preset correlation threshold are connected in chronological order to obtain the personnel behavior trajectory.

6. The method for monitoring production safety behavior based on AI video analysis according to any one of claims 1 to 5, characterized in that: The comparison between the personnel characteristics and the preset personnel standard behavior characteristics is performed to obtain the personnel behavior status of each frame of the video image, including: Comparing each type of feature in the personnel characteristics with the corresponding same type features in the personnel standard behavior characteristics one by one, obtaining a same type feature comparison result; Identify deviation feature items based on the same type feature comparison results to obtain a deviation feature set; Determining the correlation between any two deviation features based on the number of times any two deviation features in the deviation feature set appear simultaneously and the total number of times any two deviation features appear individually, and constructing a deviation correlation matrix based on the correlation between any two deviation features in the deviation feature set; Behavior status is identified based on the deviation correlation matrix to obtain the personnel behavior status.

7. The method for monitoring safe production behavior based on AI video analysis according to claim 6, characterized in that: The performing of behavioral state identification based on the deviation correlation matrix to obtain the personnel behavioral state includes: Based on the deviation correlation matrix, the deviation features with a correlation greater than or equal to a preset correlation threshold are sequentially combined according to the order and logical relationship of occurrence of the deviation features to construct a deviation behavior sequence; Based on matching the deviant behavior sequence with a preset typical deviant behavior state pattern library, determining a matching similarity between the deviant behavior sequence and each typical deviant behavior state pattern; Determining the behavior state pattern type corresponding to the deviant behavior sequence based on the matching similarity between the deviant behavior sequence and each typical deviant behavior state pattern; Behavior state identification is performed based on the behavior state pattern type, occurrence frequency and impact degree of the deviant behavior sequence to obtain the personnel behavior state.

8. A safety production behavior monitoring system based on AI video analysis, characterized in that: The method for monitoring safe production behavior based on AI video analysis according to any one of claims 1 to 7 is applied, wherein the system for monitoring safe production behavior based on AI video analysis comprises: A video data acquisition module is configured to acquire real-time video data streams of the production area based on high-definition network cameras deployed in each monitoring area of ​​the production area; the real-time video data streams include images of personnel activities in the production area; A target detection module is used to input each frame of video image in the real-time video data stream into a pre-trained target detection model for target detection, and obtain the human target output by the target detection model and the target position coordinates of the human target in each frame of video image; A feature recognition module is used to crop a local image area of ​​a person target in each frame of video image based on the target position coordinates, and perform feature recognition based on the local image area to obtain person features; A behavior analysis module is used to compare the personnel characteristics with preset personnel standard behavior characteristics to obtain the personnel behavior status of each frame of video image, and to associate the trajectory of the personnel behavior status corresponding to multiple consecutive frames of video images to obtain the personnel behavior trajectory of the personnel target in the production area; A behavior monitoring module is used to monitor the safety production behavior of the personnel target in the production area based on the personnel behavior status and the personnel behavior trajectory, and generate an abnormal behavior early warning signal.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the safe production behavior monitoring method based on AI video analysis as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing a computer software program, wherein: When the processor executes the program, it implements the safe production behavior monitoring method based on AI video analysis as described in any one of claims 1 to 7.

Images