A posture recognition management system and method for factory workers

Abstract

The present invention discloses a posture recognition management system and method for factory operators, and relates to the technical field of factory operation safety management. The system comprises a video image acquisition module, a target detection module, a behavior recognition module, a violation detection module, an alarm module, a recording module and a QT5 integration module which are operated in sequence. The application scenario of the whole management system is to identify and manage operators and automation equipment in the factory. The technical key points are as follows: in view of the problem that the behavior recognition in the factory scenario in the prior art is prone to false alarms, the present invention proposes a system for detecting violation behaviors of human-object interaction in the factory. The system integrates target detection and behavior recognition algorithms, can monitor the interaction between factory operators and equipment in real time, accurately identify possible violation behaviors, and send out alarm signals in time to improve the safety management level of the factory.

Description

Technical Field

[0001] The present invention relates to the technical field of factory operation safety management, and in particular to a posture recognition management system and method for factory operators. Background Art

[0002] With the development of industry and intelligent manufacturing, the degree of factory automation and intelligence has been continuously improved, but the corresponding safety management issues of factory operators have also become more prominent; traditional safety management methods mainly rely on manual monitoring and passive protection measures, such as wearing protective equipment and setting safety signs. Although these methods have reduced the incidence of accidents to a certain extent, there are still many shortcomings. Especially in the complex and changeable factory environment, manual monitoring is easily affected by subjective judgment and fatigue, resulting in unsatisfactory monitoring results;

[0003] With the rapid development of computer vision technology and artificial intelligence, gesture recognition and behavior analysis technologies based on video surveillance are gradually being applied to industrial safety management. These technologies can monitor and analyze the real-time behavior of factory operators, thereby timely detecting abnormal situations and issuing early warnings.

[0004] However, the current posture recognition system still faces certain challenges in terms of accuracy and real-time performance. In particular, when dealing with complex scenarios and multiple behavior patterns, traditional algorithms often find it difficult to meet high-precision recognition requirements. In addition, with the widespread use of automated equipment within factories, such as AGVs (automated guided vehicles), collaborative operations between workers and equipment have become another key point in safety management. How to effectively identify the interaction between factory personnel and equipment and respond promptly when danger may occur has become an important issue in smart factory safety management. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a posture recognition management system and method for factory workers. The system has more intelligent and real-time posture recognition and behavior analysis functions, can accurately monitor the behavior of workers in a complex factory environment, identify potential safety hazards, and effectively coordinate with the factory's automation equipment, thereby improving the overall safety management level and solving the problems raised in the background technology.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A posture recognition management system for factory operators, the system comprising:

[0010] The video image acquisition module selects the target factory, deploys cameras in key areas, and acquires video data;

[0011] The target detection module pre-processes the video data of each key area based on the YOLOv8 algorithm, identifies the relevant information of the operators and the trajectory information of the automation equipment in the target factory, and outputs the relevant information and trajectory information;

[0012] The behavior recognition module uses the slowFast algorithm to synchronously process the video data of each key area, analyze the actions and postures of the operators, and trigger the key point sub-module built into the behavior recognition module when the operators interact with the automation equipment, identify the interactive behaviors of the operators, and output the interactive behaviors;

[0013] The illegal behavior detection module receives the output results of the target detection module and the behavior recognition module, determines whether the operator has performed illegal operations, and extracts the type of illegal operations when it is determined that there are illegal operations; when the automated equipment is in a moving state, the risk prediction mechanism is triggered synchronously, and when the prediction result meets the standard, a prompt instruction is issued;

[0014] The alarm module, when it is determined that the operator has violated the regulations, will issue a warning signal of the corresponding level according to the type of illegal operation; under the condition of receiving a prompt instruction, it will issue a prompt signal.

[0015] Furthermore, the process of preprocessing the video data is as follows:

[0016] Perform processing on images or video frames in video data, including denoising and contrast enhancement.

[0017] Furthermore, the relevant information of the operator includes: identity information and appearance status;

[0018] The identity information includes the name, position and coordinates of each operator;

[0019] Appearance status includes wearing of factory uniforms, safety helmets and safety belts;

[0020] The automated equipment includes an AGV vehicle, and the trajectory information of the automated equipment includes a motion trajectory, a speed, and a position.

[0021] Furthermore, the slowFast algorithm is used to build a model to analyze the movements and postures of the operators. The model is trained to identify several different interactive behaviors. After the video data is input, the model obtains the predicted probability distribution of the category to which the operator's behavior belongs, and calculates a vector containing multiple probability values. Each probability value corresponds to an interactive behavior category. By comparing several probability values, the corresponding interactive behavior category with the largest probability value is selected and output as the classification result.

[0022] Furthermore, in the key point sub-module, combined with the key point capture technology, the number of key points of the operator's body is 15, and the positions of the 15 key points are: head, neck, two shoulders, two elbows, two wrists, hips, two waists, two knees and two ankles; the interactive behavior represents: the type of key point when the key point of the corresponding operator enters the selection area of ​​the calibration operation interface.

[0023] Furthermore, the process of determining whether the operator has violated the regulations is as follows:

[0024] Extract the appearance status of the operator's relevant information. If any one or more of the uniforms, helmets, and safety belts are not worn, it is determined that the corresponding operator has violated the regulations and marked the wearing abnormality.

[0025] Build a rule engine, retrieve the specifications of the corresponding category of interactive behaviors, and analyze the interactive behaviors;

[0026] When the equipment is being adjusted, if the two wrist key points do not overlap with the boxed area of ​​the calibration operation interface, the corresponding operator is judged to have violated the regulations and the calibration operation is abnormal; if any key point overlaps with the dangerous area calibrated on the automated equipment, the corresponding operator is judged to have violated the regulations and the calibration contact is abnormal.

[0027] Furthermore, the process of triggering the risk prediction mechanism is as follows:

[0028] Taking the automated equipment in a moving state as the center, obtain the coordinates of i operators that coincide with the trajectory of the automated equipment. Starting from the operator closest to the center, calculate the time required for the automated equipment to reach the nearest operator based on the trajectory information of the automated equipment. Use the probe on the automated equipment to detect whether the facial image of the operator closest to the center can be collected.

[0029] If yes, mark it as 1;

[0030] If not, mark it as 0;

[0031] This process is repeated until the i-th operator coincides with the motion trajectory of the automated equipment;

[0032] The required duration and mark value are used as parameters to construct a risk assessment model, generate risk assessment values ​​corresponding to each operator who overlaps with the motion trajectory of the automated equipment, and extract the maximum value as the prediction result.

[0033] Furthermore, when constructing the risk assessment model, the formula is as follows:

[0034] ;

[0035] Define variables:

[0036] : The time required for the automated equipment to reach the i-th operator;

[0037] : Whether the probe can detect the face of the i-th operator, 1 means it is detected, 0 means it is not detected;

[0038] N: The total number of operators who overlap with the motion trajectory of the automated equipment;

[0039] R: risk assessment value;

[0040] τ: preset time coefficient, the value range is [2, 4];

[0041] θ: preset risk threshold;

[0042] w1, w2: weight coefficients, both in the range of [0, 1];

[0043] When R<θ, it means that the prediction result meets the standard.

[0044] Furthermore, the process of issuing a warning signal of a corresponding level according to the type of illegal operation is as follows:

[0045] When the illegal operation type is: abnormal wearing, a No. 1 warning signal is issued;

[0046] When the illegal operation type is: abnormal operation, a second warning signal is issued;

[0047] When the illegal operation type is: abnormal contact, a third warning signal is issued;

[0048] Warning signals can be expressed in the form of sound and light;

[0049] The prompt signal is expressed in the form of: using a player installed on the corresponding automation equipment to play the prompt message "Please pay attention to dodge".

[0050] A posture recognition management method for factory workers includes the following steps:

[0051] S1. Select the target factory, deploy cameras in key areas, and obtain video data;

[0052] S2. Based on the YOLOv8 algorithm, the video data of each key area is pre-processed to identify the relevant information of the operators and the trajectory information of the automation equipment in the target factory, and the relevant information and trajectory information are output;

[0053] S3, using the slowFast algorithm to synchronously process the video data of each key area, analyze the actions and postures of the operators, trigger and identify the interactive behaviors of the operators under the condition that the operators interact with the automation equipment, and output the interactive behaviors;

[0054] S4, receiving the output results of S2 and S3, judging whether the operator has any illegal operation, and extracting the type of illegal operation when it is judged that there is any illegal operation; when the automated equipment is in a moving state, the risk prediction mechanism is synchronously triggered, and when the prediction result meets the standard, a prompt instruction is issued;

[0055] S5. When it is determined that the operator has violated the regulations, a warning signal of a corresponding level is issued according to the type of illegal operation; a prompt signal is issued under the condition that a prompt instruction is received.

[0056] (III) Beneficial effects

[0057] The present invention provides a posture recognition management system and method for factory workers, which has the following beneficial effects:

[0058] 1) This solution provides a system for detecting violations of factory human-object interaction. The system integrates target detection and behavior recognition algorithms, can monitor the interaction between factory operators and equipment in real time, accurately identify possible violations, and issue alarm signals in a timely manner to improve the factory's safety management level;

[0059] 2) This solution can accurately identify the appearance, interactive behavior and contact of operators with automated equipment, so as to accurately determine whether there are any illegal operations. The realization of this function relies on advanced target detection, behavior recognition and key point capture technology to ensure the accuracy and timeliness of monitoring;

[0060] 3) This solution introduces a risk prediction mechanism to predict and evaluate the risks that may arise during the movement of automated equipment. By calculating the relative position and required time between the automated equipment and the operator, combined with face detection technology, the system can generate a risk assessment value and issue a prompt instruction when the prediction result meets the standard. This mechanism helps to detect potential risks in advance and provide strong protection for the safety of operators;

[0061] In addition, the system also has a complete alarm function, which can issue early warning signals of different levels according to the type of illegal operation, as well as prompt signals for the risk of movement of automated equipment; these signals are presented in the form of sound and light, which can not only attract the attention of on-site operators, but also accurately convey the type and risk level of illegal operations, so that operators can respond quickly; in summary, this technical solution integrates a variety of advanced technologies to achieve comprehensive monitoring and intelligent management of factory operators and automated equipment behaviors; the system not only improves the accuracy and timeliness of monitoring, but also enhances the safety and reliability of the working environment, providing strong support for the factory's intelligent transformation and safe production. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a real scene demonstration diagram of capturing key points of the present invention;

[0063] Figure 2 This is a diagram showing the key point capture of the present invention;

[0064] Figure 3 It is a schematic diagram of the system operation flow of the present invention;

[0065] Figure 4 It is a schematic diagram of the overall steps of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] Example 1

[0068] See also Figures 1 to 3 , this embodiment provides a posture recognition management system for factory workers, the system includes a video image acquisition module, a target detection module, a behavior recognition module, a violation detection module and an alarm module that run in sequence; the application scenario of the entire management system is to identify and manage workers and automation equipment in the factory; in view of the problem that the existing technology is prone to false alarms in behavior recognition in factory scenarios, the present invention proposes a system for detecting violations of factory human-object interaction, the system integrates target detection and behavior recognition algorithms, can monitor the interaction between factory workers and equipment in real time, accurately identify possible violations, and issue alarm signals in time to improve the safety management level of the factory;

[0069] It should be noted that:

[0070] Figure 3 The video and image acquisition module in is the video image acquisition module, and the posture recognition and target recognition are performed by the behavior recognition module and the target detection module respectively;

[0071] Video image acquisition module:

[0072] Select the target factory, deploy cameras in the set key areas, and obtain video data of the corresponding key areas;

[0073] The images captured by the camera completely cover the corresponding key areas, and the video data obtained by the camera is real-time; if conditions permit, the camera should have sufficient resolution and frame rate to ensure that the detailed movements of factory workers and automation equipment can be clearly captured; at the same time, the camera should also have night vision, waterproof, dustproof and other features to adapt to various factory environments;

[0074] Technical implementation:

[0075] Camera selection: Choose cameras with high definition, high frame rate, wide dynamic range and other characteristics; Deployment location: Deploy cameras in key areas such as production lines, warehouses, and equipment operation areas according to factory layout and monitoring requirements; Data transmission: The camera transmits video data via wired or wireless methods, and the specific method is selected according to actual needs;

[0076] Object detection module:

[0077] Based on the YOLOv8 algorithm, the video data of each key area is pre-processed to identify the relevant information of the operators and the trajectory information of the automation equipment in the target factory, and the relevant information and trajectory information are output;

[0078] The process of preprocessing video data is as follows:

[0079] Processing images or video frames in the video data, including denoising and contrast enhancement, to improve the accuracy of subsequent key point detection;

[0080] The relevant information of the operator includes: identity information and appearance status;

[0081] Identity information includes the name, position and coordinates of each operator, which can be obtained through face recognition technology; appearance status includes the wearing of factory uniforms, safety helmets and safety belts; automation equipment includes AGV vehicles, and the trajectory information of automation equipment includes movement trajectory, speed and position;

[0082] It should be noted that the module for identifying factory workers and automated equipment (such as AGV vehicles) in surveillance videos based on the YOLOv8 algorithm is an intelligent module that integrates advanced computer vision and deep learning technologies;

[0083] YOLOv8 is the latest version in the YOLO (You Only Look Once) series. The algorithm is known for its fast and accurate object detection capabilities, especially for real-time video stream processing and object recognition in complex scenes. YOLOv8 introduces many innovations and improvements based on previous versions, including a new backbone network, anchor-free detection head, and a new loss function, which further improves performance and flexibility.

[0084] The core features of the YOLOv8 algorithm are as follows:

[0085] New backbone network: YOLOv8 adopts a new backbone network architecture, which is expected to improve feature extraction and processing capabilities, thereby contributing to more accurate target detection and segmentation; Anchor-Free detection head: The introduction of the Anchor-Free detection head no longer relies on anchor boxes, providing greater flexibility and better adapting to various target shapes and sizes; New loss function: YOLOv8 adopts a brand-new loss function, which is expected to further improve the convergence speed and performance of the model; Multi-platform support: YOLOv8 has the ability to support multiple platforms and can run on different hardware platforms, including CPU and GPU; Scalability: YOLOv8's open source library is not limited to the YOLO series of models, but also has excellent scalability, supporting other types of models and various tasks, such as classification, segmentation, pose estimation, etc.

[0086] The application of YOLOv8 algorithm in factory monitoring video is as follows:

[0087] Personnel identity detection: Through the YOLOv8 algorithm, the system can accurately detect and identify the identities of factory workers in surveillance videos. This is usually achieved by training models to identify the facial features of specific people or the identification badges they wear. Factory uniform wearing detection: The system can detect the factory uniform wearing of workers, including whether they are wearing them and whether they are wearing helmets correctly. This helps ensure that workers comply with safety regulations during the production process and reduce the risk of accidents. Appearance status detection: The YOLOv8 algorithm can also be used to detect the appearance status of workers, such as whether they are wearing glasses, masks, etc. This helps ensure that workers have appropriate protective measures in specific environments. Equipment motion trajectory detection: In addition to detecting personnel, the system can also accurately detect the motion trajectory of automated equipment, such as the driving path, position and speed of AGV vehicles. This helps ensure that the equipment runs according to the predetermined trajectory and speed during the production process, improving production efficiency.

[0088] Technical implementation:

[0089] Algorithm training: Use video data from inside the target factory to train the YOLOv8 algorithm to improve recognition accuracy; Real-time detection: Input real-time video data into the trained YOLOv8 model for target detection and recognition; Information extraction: Extract relevant information of the operator and trajectory information of the automation equipment from the recognition results and pass them to subsequent modules.

[0090] Behavior recognition module:

[0091] The slowFast algorithm is used to synchronously process the video data of each key area, analyze the actions and postures of the operators, and trigger the key point submodule built into the behavior recognition module when the operators interact with the automation equipment, identify the interactive behaviors of the operators, and output the interactive behaviors;

[0092] The slowFast algorithm is a deep learning model framework for video behavior recognition. It uses a dual-path network structure. The slow path is responsible for capturing spatial information in the video (such as shape, environment, etc.), while the fast path is responsible for capturing dynamic information in the video (such as fast movement, small changes, etc.). This structure enables the slowFast algorithm to capture both fine dynamic and key spatial information in the video, improving the recognition ability and efficiency of complex actions.

[0093] The principle of slowFast algorithm to analyze the movements and postures of workers:

[0094] Video frame sampling and processing: The slowFast algorithm first samples the video data and selects key frames and non-key frames according to the preset time interval; key frames (slow frames) are deeply processed through the slow path to extract advanced spatial features; non-key frames (fast frames) are quickly processed through the fast path to capture dynamic changes in the video;

[0095] Feature extraction and fusion: In the slow path, the algorithm uses a deeper network structure to extract features from key frames, and these features mainly reflect the spatial information in the video; in the fast path, the algorithm uses a shallower network structure to extract features from non-key frames, and these features mainly reflect the dynamic information in the video; at certain stages of the network, the features of the fast path and the slow path will be fused to enhance the model's discrimination ability; this fusion strategy helps the algorithm to more accurately understand complex actions and postures in the video;

[0096] Behavior recognition and output: After feature extraction and fusion, the slowFast algorithm will input the extracted feature information into the fully connected layer for further processing; the fully connected layer will input the calculated feature vector into the sigmoid regression layer for regression calculation to obtain the classification result; the classification result mentioned in the SlowFast algorithm refers to the model's predicted probability distribution of the category to which the operator's behavior belongs in the input video frame; the model is trained to identify a variety of different interactive behaviors, so the classification result will be a vector containing multiple probability values, each value corresponding to an interactive behavior category; by comparing these probability values, the interactive behavior category corresponding to the maximum probability is selected and output as the classification result;

[0097] Under the condition that the operator interacts with the automated equipment, the interaction behavior category has been determined before, which facilitates the determination of the specification to which the corresponding interaction behavior category belongs. For example, when the equipment is adjusted, both wrist key points coincide with the framed area of ​​the calibration operation interface, which belongs to the corresponding specification;

[0098] In the key point submodule, the key point capture technology is combined, referring to Figure 1 to Figure 2 As shown in the figure, there are 15 key points on the operator's body, which constitute a detailed description of the body's posture and motion trajectory. The 15 key points are located at the head, neck, shoulders (one on each side), elbows (one on each side), wrists (one on each side), hips, waist (one on each side), knees (one on each side), and ankles (one on each side). In addition, more detailed key points such as finger joints may also be included depending on the specific application scenario.

[0099] The interactive behavior indicates the type of key point when the key point of the corresponding operator enters the selection area of ​​the calibration operation interface.

[0100] Through the above technical solutions, the system deploys high-definition, high-frame-rate cameras and integrates the YOLOv8 algorithm for target detection, achieving real-time and accurate identification of operators and automated equipment (such as AGV vehicles) in key areas of the factory. This not only improves monitoring efficiency, but also ensures the comprehensiveness and accuracy of monitoring data, providing a solid foundation for subsequent behavior analysis and violation detection;

[0101] Secondly, the system uses the slowFast algorithm to conduct in-depth analysis of the operator's movements and postures. Combined with key point capture technology, it can accurately identify the operator's behavior patterns during the interaction with automated equipment. This capability is of great significance for timely detection and correction of illegal operations and prevention of potential safety risks.

[0102] In addition, the system has strong scalability and flexibility, and can be customized according to different factory environments and monitoring requirements; for example, the deployment location and number of cameras can be adjusted according to actual needs, and algorithm parameters can be optimized to improve recognition accuracy;

[0103] In summary, this technical solution realizes comprehensive, real-time monitoring and intelligent management of factory operators and automation equipment by integrating advanced computer vision and deep learning technologies. This not only improves the production efficiency and safety of the factory, but also provides strong support for the intelligent transformation and upgrading of the factory. At the same time, the scalability and flexibility of the system also lay a solid foundation for its wide application in different industries and scenarios.

[0104] Violation detection module:

[0105] Receive the output results of the target detection module and the behavior recognition module, determine whether the operator has violated the regulations, and extract the type of violation if it is determined that there is a violation; when the automated equipment is in a moving state, the risk prediction mechanism is triggered synchronously, and when the prediction result meets the standard, a prompt instruction is issued;

[0106] The process of determining whether an operator has violated regulations is as follows:

[0107] Extract the appearance status of the relevant information of the operator. If any one or more of the uniforms, helmets and safety belts are not worn (wearing uniforms, helmets and safety belts is the norm), it is determined that the corresponding operator has violated the rules and marked the wearing abnormality. The wearing abnormality is the type of illegal operation.

[0108] Build a rule engine, retrieve the specifications of the corresponding category of interactive behaviors, and analyze the interactive behaviors;

[0109] When the equipment is being adjusted, if the two wrist key points do not overlap with the framed area of ​​the calibration operation interface (the overlap of the two wrist key points with the framed area of ​​the calibration operation interface is standard), the corresponding operator is judged to have violated the rules and the calibration operation is abnormal, which is the type of illegal operation; if any key point overlaps with the dangerous area calibrated on the automation equipment (the non-overlap of any key point with the dangerous area calibrated on the automation equipment is standard), the corresponding operator is judged to have violated the rules and the calibration contact is abnormal;

[0110] It should be noted that there is usually at most one automated device in a key area, namely an AGV vehicle;

[0111] The process of triggering the risk prediction mechanism is as follows:

[0112] Taking the automated equipment in a moving state as the center, obtain the coordinates of i operators that coincide with the motion trajectory of the automated equipment. Starting from the operator closest to the center, calculate the time required for the automated equipment to reach the nearest operator based on the trajectory information of the automated equipment. Use the probe on the automated equipment to detect whether the facial image of the operator closest to the center can be collected. If so, mark it as 1, if not, mark it as 0. Repeat this process until the i-th operator coincides with the motion trajectory of the automated equipment.

[0113] The required duration and the mark value are used as parameters to construct a risk assessment model, generate risk assessment values ​​corresponding to each operator who overlaps with the motion trajectory of the automated equipment, and extract the maximum value as the prediction result;

[0114] Among them, the following are the steps to calculate the required time:

[0115] S1.1. Determine the relative position of the automation equipment and the operator:

[0116] Get the current position (assumed to be (x0, y0)) and speed (assumed to be v, in meters per second, and a constant speed) of the automation equipment; get the coordinates of the operator closest to the center of the automation equipment (assumed to be (x1, y1)).

[0117] S1.2. Calculate distance:

[0118] Use the Euclidean distance formula to calculate the straight-line distance d between the automated equipment and the operator:

[0119] ;

[0120] S1.3. Calculation time required:

[0121] Assume that the automated equipment moves in a straight line toward the operator (in actual situations, the actual motion trajectory of the automated equipment needs to be considered, but here, to simplify the calculation, we assume linear motion); based on the relationship between speed, distance, and time, calculate the time t required for the automated equipment to reach the operator: t=d / v;

[0122] Here, the unit of t is seconds.

[0123] It should be noted that in actual applications, the movement of automated equipment may not be a simple linear motion, but may follow a preset trajectory or path; therefore, when calculating the time, it is necessary to consider the time required for the automated equipment to reach the operator along its actual motion trajectory. This usually involves geometric calculations and path planning algorithms, so I will not go into details here.

[0124] The same principle is used to calculate the time required for other operators;

[0125] When building a risk assessment model, the formula used is as follows:

[0126] ;

[0127] Define variables:

[0128] : The time required for the automated equipment to reach the i-th operator;

[0129] : Whether the probe can detect the face of the i-th operator, 1 means it is detected, 0 means it is not detected;

[0130] N: The total number of operators who overlap with the motion trajectory of the automated equipment;

[0131] R: risk assessment value;

[0132] τ: preset time coefficient, with a value range of [2, 4] (for example, if the automated equipment can reach the operator in a very short time, the risk is higher, and the time coefficient is set based on the reaction time provided to the operator);

[0133] θ: The preset risk threshold, which is set according to the actual situation and serves as the critical value for determining whether the prediction result meets the standard;

[0134] w1 and w2 are weight coefficients, which are used to adjust the impact of duration and face detection on risk assessment, and their value range is [0, 1]. The weight coefficient is determined by the coefficient of variation method, which is a method of weighting each indicator according to the degree of variation between the current value and the target value of each evaluation indicator. If the numerical difference of an indicator is large, it can clearly distinguish the evaluated objects, indicating that the indicator has rich information for distinguishing, and thus the indicator should be given a larger weight. On the contrary, if the numerical difference of each evaluated object on a certain indicator is small, then the ability of this indicator to distinguish the evaluated objects is weak, and thus the indicator should be given a smaller weight. This method directly uses the information contained in each indicator to obtain the weight of the indicator through calculation, so it is objective.

[0135] The "if" after the formula means under the corresponding conditions;

[0136] Logic description:

[0137] / τ represents the time risk component (if is very small, the time risk component is large); represents the risk component of face detection (if , then the face detection risk component is 1); take the maximum value of all risk components as the final risk assessment value, indicating that we only care about the most dangerous situation;

[0138] Time risk: If the automation equipment can reach a certain operator very quickly, the time risk component is large, which reflects the high risk in an emergency situation. Face detection risk: If the automation equipment cannot detect the face of a certain operator, the face detection risk component is 1, which reflects that the operator is facing away from the automation equipment, which increases the risk. If the face is detected, it means that the automation equipment is within the operator's line of sight, which is convenient for timely avoidance.

[0139] When the prediction result reaches the target, a prompt instruction is issued;

[0140] The above prediction result meets the standard, indicating that: when R<θ, a prompt instruction is issued.

[0141] Alarm module:

[0142] When it is determined that the operator has violated the rules, a warning signal of the corresponding level will be issued according to the type of illegal operation; when it is determined that the operator has not violated the rules, no response action will be taken; under the condition of receiving the prompt instruction, a prompt signal will be issued;

[0143] The process of issuing warning signals of corresponding levels according to the types of illegal operations is as follows:

[0144] When the illegal operation type is: abnormal wearing, a No. 1 warning signal is issued;

[0145] When the illegal operation type is: abnormal operation, a second warning signal is issued;

[0146] When the illegal operation type is: abnormal contact, a third warning signal is issued;

[0147] The warning signal reminds the workers on site through sound, light and other forms. Different warning signals have different manifestations to distinguish them, so that the operators can respond quickly and accurately know the type of illegal operation.

[0148] The prompt signal is expressed in the form of: the player installed on the corresponding automation equipment plays the prompt message "Please pay attention to dodge" three times;

[0149] In addition, it should be noted that for the specific area selected in the frame in the image, an alarm will be triggered when an unauthorized person enters. The alarm form is different from the early warning signal and prompt signal to make a distinction.

[0150] Through the above technical solutions, this system effectively solves the problems of real-time monitoring and early warning of operators' illegal behaviors in the factory working environment, as well as the prediction and prompting of potential risks in the process of moving automated equipment;

[0151] First, the system can accurately identify the operator's appearance, interactive behavior, and contact with automated equipment, so as to accurately determine whether there is any illegal operation. The realization of this function relies on advanced target detection, behavior recognition and key point capture technology to ensure the accuracy and timeliness of monitoring.

[0152] Secondly, the system introduces a risk prediction mechanism to predict and evaluate the risks that may arise during the movement of automated equipment. By calculating the relative position and required time between the automated equipment and the operator, combined with face detection technology, the system can generate a risk assessment value and issue a prompt instruction when the prediction result meets the standard. This mechanism helps to detect potential risks in advance and provide strong protection for the safety of operators.

[0153] In addition, the system also has a complete alarm function, which can issue different levels of early warning signals according to the type of illegal operation, as well as prompt signals for the risk of automated equipment movement. These signals are presented in the form of sound, light, etc., which can not only attract the attention of on-site operators, but also accurately convey the type and risk level of illegal operation, so that operators can respond quickly;

[0154] In summary, this technical solution realizes comprehensive monitoring and intelligent management of the behavior of factory operators and automated equipment by integrating multiple advanced technologies; the system not only improves the accuracy and timeliness of monitoring, but also enhances the safety and reliability of the working environment, providing strong support for the factory's intelligent transformation and safe production.

[0155] The system also includes a recording module and a QT5 integration module;

[0156] Logging module:

[0157] All data generated during the violation detection process, including but not limited to video data, detection results and alarm records (corresponding to the issuance records of warning signals), will be stored in the recording module. This data can be used for subsequent safety analysis to help factory managers understand and improve safety measures;

[0158] QT5 integrated modules:

[0159] The QT5 integrated module integrates all functional modules (i.e., video image acquisition module, target detection module, behavior recognition module, violation detection module, alarm module, and recording module) into a multifunctional system, and provides operation and management functions through the user interface; managers can monitor the status of operators and automation equipment in real time, receive alarm information (including early warning signals and prompt signals), and adjust system settings to adapt to different factory environments. This is the existing setting adjustment action, which will not be elaborated here.

[0160] Example 2

[0161] See also Figure 4 Based on Example 1, this embodiment further provides a posture recognition management method for factory workers, including the following specific steps:

[0162] S1. Select the target factory, deploy cameras in key areas, and obtain video data;

[0163] S2. Based on the YOLOv8 algorithm, the video data of each key area is pre-processed to identify the relevant information of the operators and the trajectory information of the automation equipment in the target factory, and the relevant information and trajectory information are output;

[0164] S3, using the slowFast algorithm to synchronously process the video data of each key area, analyze the actions and postures of the operators, trigger and identify the interactive behaviors of the operators under the condition that the operators interact with the automation equipment, and output the interactive behaviors;

[0165] S4, receiving the output results of S2 and S3, judging whether the operator has any illegal operation, and extracting the type of illegal operation when it is judged that there is any illegal operation; when the automated equipment is in a moving state, the risk prediction mechanism is synchronously triggered, and when the prediction result meets the standard, a prompt instruction is issued;

[0166] S5. When it is determined that the operator has violated the regulations, a warning signal of a corresponding level is issued according to the type of illegal operation; a prompt signal is issued under the condition that a prompt instruction is received;

[0167] S6. All data generated during the violation detection process, including video data, detection results, and alarm records, are stored for subsequent security analysis;

[0168] S7: Integrate the processes of S1 to S6 into the same system and provide operation and management functions through the user interface, monitor the status of operators and automation equipment in real time, receive alarm information, and adjust system settings to adapt to different factory environments.

[0169] In the application, the several formulas involved are all calculated by taking their numerical values ​​after removing the dimensions, and the formulas are established by collecting a large amount of data and performing software simulation to obtain a formula for the most recent actual situation. Some coefficients or weights in the formulas are set by technical personnel in this field according to actual conditions, so they will not be elaborated here.

[0170] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. A person of ordinary skill in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A posture recognition management system for factory workers, characterized in that: The system includes: The video image acquisition module selects the target factory, deploys cameras in key areas, and acquires video data; The target detection module pre-processes the video data of each key area based on the YOLOv8 algorithm, identifies the relevant information of the operators and the trajectory information of the automation equipment in the target factory, and outputs the relevant information and trajectory information; The relevant information of the operator includes: identity information and appearance status; The identity information includes the name, position and coordinates of each operator; Appearance status includes wearing of factory uniforms, safety helmets and safety belts; The automation equipment includes an AGV vehicle, and the trajectory information of the automation equipment includes motion trajectory, speed and position; The behavior recognition module uses the slowFast algorithm to synchronously process the video data of each key area, analyze the actions and postures of the operators, and trigger the key point sub-module built into the behavior recognition module when the operators interact with the automation equipment, identify the interactive behaviors of the operators, and output the interactive behaviors; In the key point submodule, combined with the key point capture technology, the number of key points of the operator's body is 15, and the positions of the 15 key points are: head, neck, two shoulders, two elbows, two wrists, hips, two waists, two knees and two ankles; interactive behavior represents: the type of key point when the key point of the corresponding operator enters the frame selection area of ​​the calibration operation interface; The illegal behavior detection module receives the output results of the target detection module and the behavior recognition module, determines whether the operator has performed illegal operations, and extracts the type of illegal operations when it is determined that there are illegal operations; when the automated equipment is in a moving state, the risk prediction mechanism is triggered synchronously, and when the prediction result meets the standard, a prompt instruction is issued; The process of determining whether an operator has violated regulations is as follows: Extract the appearance status of the operator's relevant information. If any one or more of the uniforms, helmets, and safety belts are not worn, it is determined that the corresponding operator has violated the regulations and marked the wearing abnormality. Build a rule engine, retrieve the specifications of the corresponding category of interactive behaviors, and analyze the interactive behaviors; When the equipment is being adjusted, if the two wrist key points do not overlap with the framed area of ​​the calibration operation interface, the corresponding operator is judged to have violated the rules and the calibration operation is abnormal; if any key point overlaps with the dangerous area calibrated on the automation equipment, the corresponding operator is judged to have violated the rules and the calibration contact is abnormal; The process of triggering the risk prediction mechanism is as follows: Taking the automated equipment in a moving state as the center, obtain the coordinates of i operators that coincide with the trajectory of the automated equipment. Starting from the operator closest to the center, calculate the time required for the automated equipment to reach the nearest operator based on the trajectory information of the automated equipment. Use the probe on the automated equipment to detect whether the facial image of the operator closest to the center can be collected. If yes, mark it as 1; If not, mark it as 0; This process is repeated until the i-th operator coincides with the motion trajectory of the automated equipment; The required duration and the mark value are used as parameters to construct a risk assessment model, generate risk assessment values ​​corresponding to each operator who overlaps with the motion trajectory of the automated equipment, and extract the maximum value as the prediction result; When building a risk assessment model, the formula used is as follows: ； Define variables: : The time required for the automated equipment to reach the i-th operator; : Whether the probe can detect the face of the i-th operator, 1 means it is detected, 0 means it is not detected; N: The total number of operators who overlap with the motion trajectory of the automated equipment; R: risk assessment value; : The preset time coefficient, the value range is [2, 4]; : Preset risk threshold; w1, w2: weight coefficients, both in the range of [0, 1]; When R<θ, it means the prediction result meets the standard; The alarm module, when it is determined that the operator has violated the regulations, will issue a warning signal of the corresponding level according to the type of illegal operation; under the condition of receiving a prompt instruction, it will issue a prompt signal.

2. A posture recognition management system for factory workers according to claim 1, characterized in that: The process of preprocessing video data is as follows: Perform processing on images or video frames in video data, including denoising and contrast enhancement.

3. The posture recognition management system for factory workers according to claim 1, characterized in that: The slowFast algorithm is used to build a model to analyze the movements and postures of operators. The model is trained to identify several different interactive behaviors. After inputting video data, the model obtains the predicted probability distribution of the category to which the operator's behavior belongs, and calculates a vector containing multiple probability values. Each probability value corresponds to an interactive behavior category. By comparing several probability values, the corresponding interactive behavior category with the largest probability value is selected and output as the classification result.

4. The posture recognition management system for factory workers according to claim 1, characterized in that: The process of issuing warning signals of corresponding levels according to the types of illegal operations is as follows: When the illegal operation type is: abnormal wearing, a No. 1 warning signal is issued; When the illegal operation type is: abnormal operation, a second warning signal is issued; When the illegal operation type is: abnormal contact, a third warning signal is issued; Warning signals can be expressed in the form of sound and light; The prompt signal is expressed in the form of: using a player installed on the corresponding automation equipment to play the prompt message "Please pay attention to dodge".

5. A posture recognition management method for factory workers, using any system described in claim 1 to claim 4, characterized in that: The steps include: S1. Select the target factory, deploy cameras in key areas, and obtain video data; S2. Based on the YOLOv8 algorithm, the video data of each key area is pre-processed to identify the relevant information of the operators and the trajectory information of the automation equipment in the target factory, and the relevant information and trajectory information are output; The relevant information of the operator includes: identity information and appearance status; The identity information includes the name, position and coordinates of each operator; Appearance status includes wearing of factory uniforms, safety helmets and safety belts; The automation equipment includes an AGV vehicle, and the trajectory information of the automation equipment includes motion trajectory, speed and position; S3, using the slowFast algorithm to synchronously process the video data of each key area, analyze the actions and postures of the operators, trigger and identify the interactive behaviors of the operators under the condition that the operators interact with the automation equipment, and output the interactive behaviors; Among them, combined with the key point capture technology, the number of key points of the operator's body is 15, and the positions of the 15 key points are: head, neck, two shoulders, two elbows, two wrists, hips, two waists, two knees and two ankles; interactive behavior represents: the type of key point when the key point of the corresponding operator enters the frame selection area of ​​the calibration operation interface; S4, receiving the output results of S2 and S3, judging whether the operator has any illegal operation, and extracting the type of illegal operation when it is judged that there is any illegal operation; when the automated equipment is in a moving state, the risk prediction mechanism is synchronously triggered, and when the prediction result meets the standard, a prompt instruction is issued; The process of determining whether an operator has violated regulations is as follows: Extract the appearance status of the operator's relevant information. If any one or more of the uniforms, helmets, and safety belts are not worn, it is determined that the corresponding operator has violated the regulations and marked the wearing abnormality. Build a rule engine, retrieve the specifications of the corresponding category of interactive behaviors, and analyze the interactive behaviors; When the equipment is being adjusted, if the two wrist key points do not overlap with the framed area of ​​the calibration operation interface, the corresponding operator is judged to have violated the rules and the calibration operation is abnormal; if any key point overlaps with the dangerous area calibrated on the automation equipment, the corresponding operator is judged to have violated the rules and the calibration contact is abnormal; The process of triggering the risk prediction mechanism is as follows: Taking the automated equipment in a moving state as the center, obtain the coordinates of i operators that coincide with the trajectory of the automated equipment. Starting from the operator closest to the center, calculate the time required for the automated equipment to reach the nearest operator based on the trajectory information of the automated equipment. Use the probe on the automated equipment to detect whether the facial image of the operator closest to the center can be collected. If yes, mark it as 1; If not, mark it as 0; This process is repeated until the i-th operator coincides with the motion trajectory of the automated equipment; The required duration and the mark value are used as parameters to construct a risk assessment model, generate risk assessment values ​​corresponding to each operator who overlaps with the motion trajectory of the automated equipment, and extract the maximum value as the prediction result; When building a risk assessment model, the formula used is as follows: ； Define variables: : The time required for the automated equipment to reach the i-th operator; : Whether the probe can detect the face of the i-th operator, 1 means it is detected, 0 means it is not detected; N: The total number of operators who overlap with the motion trajectory of the automated equipment; R: risk assessment value; : The preset time coefficient, the value range is [2, 4]; : Preset risk threshold; w1, w2: weight coefficients, both in the range of [0, 1]; When R<θ, it means the prediction result meets the standard; S5. When it is determined that the operator has violated the regulations, a warning signal of a corresponding level is issued according to the type of illegal operation; a prompt signal is issued under the condition that a prompt instruction is received.

Images