Method and system for monitoring work behavior in smart factory based on human pose estimation

Abstract

The application discloses a smart factory operation behavior monitoring method and system based on human body pose estimation, relates to the technical field of computer vision, and comprises the following steps: acquiring factory site multi-dimensional perception data, wherein the factory site multi-dimensional perception data comprises visual perception data, personnel physiological state and motion data, and personnel position data; inputting the visual perception data into a pre-established lightweight AI model to output an unsafe behavior type; performing multi-modal fusion correlation analysis on the unsafe behavior type, the personnel physiological state and motion data, and the personnel position data; verifying whether the unsafe behavior is established; if not, there is no unsafe behavior; otherwise, outputting a real-time early warning instruction; and controlling an execution device based on the real-time early warning instruction to perform early warning monitoring, so that accurate identification and timely early warning of unsafe behavior are realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method and system for monitoring intelligent factory operations based on human pose estimation. Background Technology

[0002] Currently, safety monitoring technologies for smart factory operations can be mainly divided into three categories: computer vision-based behavior recognition, sensor-based behavior monitoring, and location-based behavior analysis. Computer vision-based behavior recognition technology primarily uses cameras to collect video data and analyzes personnel behavior through deep learning algorithms. Sensor-based behavior monitoring technology collects physiological and motion data of personnel through wearable devices. Location-based behavior analysis systems use technologies such as UWB, RFID, or Bluetooth beacons to accurately locate personnel and assets.

[0003] Existing technologies have certain shortcomings. Existing behavior recognition algorithms lack adaptability and generalization ability in complex environments. In actual industrial environments, factors such as changes in lighting, occlusion, and cluttered backgrounds lead to a low accuracy rate in the final recognition and monitoring. Summary of the Invention

[0004] To address the shortcomings mentioned in the background section, the present invention aims to provide a method and system for monitoring intelligent factory work behavior based on human pose estimation.

[0005] Firstly, the objective of this invention can be achieved through the following technical solution: a method for monitoring intelligent factory work behavior based on human pose estimation, the method comprising the following steps: Acquire multi-dimensional sensing data from the factory site, including visual sensing data, personnel physiological state and motion data, and personnel location data; Visual perception data is input into a pre-built lightweight AI model, and the output is the type of unsafe behavior. The unsafe behavior type is fused and correlated with personnel physiological state, motion data, and personnel location data through multimodal analysis to verify whether the unsafe behavior is valid. If it is not valid, there is no unsafe behavior; otherwise, a real-time warning command is output, and the execution equipment is controlled to perform warning monitoring based on the real-time warning command.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: an expression for the pre-established lightweight AI model, as follows: The lightweight AI model adopts a Transformer-based encoder and decoder architecture, as follows: Where Z is the attention-weighted output feature, Q is the query matrix, K is the key matrix, V is the value matrix, and d k Let be the dimension of the key vector, and Softmax() be the attention weight distribution function; The lightweight AI model extracts features from the input visual perception data through a multi-layer, multi-head attention mechanism and outputs the unsafe behavior type. Its classification decision expression is as follows: In the formula, For unsafe behavior types, FFN() is a feedforward neural network, and argmax() is the maximum output value.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the lightweight AI model is constructed and improved by a Transformer-based sequence analysis algorithm, which accurately identifies the stopping position, eliminates the phase difference between pedestrians and vehicles, and quantifies the ambiguity in violation decision-making through an attention mechanism to achieve automated violation detection.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the unsafe behavior types include not wearing a safety helmet, entering a dangerous area, personnel falling, and abnormal operating posture.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the process of performing multimodal fusion correlation analysis of unsafe behavior types with personnel physiological state and motion data and personnel location data, including: If the unsafe behavior type is that a person is not wearing a safety helmet, the person's identity is confirmed based on the person's location data, physiological status and movement data, and it is determined whether the person has permission to enter the corresponding area. If the person has permission to enter the corresponding area and the person's physiological status and movement data are normal, then the person has permission to enter the corresponding area; otherwise, the person does not have permission to enter the corresponding area.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the multi-dimensional perception data and unsafe behavior types at the factory site are aggregated and processed through a cloud platform to obtain training sample data; and a lightweight AI model is trained and optimized based on the training sample data to obtain a trained lightweight AI model.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the process of acquiring the multi-dimensional sensing data at the factory site, as follows: The visual perception equipment uses high-definition network cameras with motion capture and facial recognition functions, which are deployed at workshop entrances and exits, green channels and various work areas to collect visual perception data. Sensor devices collect physiological and motion data of people by deploying wearable devices such as smart bracelets and smart safety helmets; Positioning devices use UWB, RFID, or Bluetooth beacon technologies to accurately locate people and mobile assets, and are used to collect personnel location data.

[0012] Secondly, in order to achieve the above objectives, this invention discloses an intelligent factory operation behavior monitoring system based on human pose estimation, comprising: The data acquisition module is used to acquire multi-dimensional sensing data of the factory site, including visual sensing data, personnel physiological state and motion data, and personnel location data. The data processing module is used to input visual perception data into a pre-built lightweight AI model and output the unsafe behavior type. The monitoring output module is used to perform multimodal fusion correlation analysis on unsafe behavior types with personnel physiological state, motion data, and personnel location data to verify whether an unsafe behavior is established. If it is not established, there is no unsafe behavior; otherwise, a real-time warning command is output, and the execution equipment is controlled to perform warning monitoring based on the real-time warning command.

[0013] The beneficial effects of this invention are: This invention achieves accurate identification and timely warning of unsafe behaviors through multi-source information collection at the terminal perception layer, real-time analysis and response at the edge processing layer, and global optimization at the cloud platform layer. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the overall architecture of the present invention; Figure 3 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Example 1: like Figure 1 As shown, a smart factory operation behavior monitoring method based on human pose estimation includes the following steps: S101: Acquire multi-dimensional sensing data of the factory site, including visual sensing data, personnel physiological state and motion data, and personnel location data; Specifically, the process of multi-dimensional sensing data in the factory includes the following steps: ① Visual perception equipment: High-definition network cameras with motion capture and facial recognition capabilities are deployed at workshop entrances and exits, green channels, and various work areas. In addition, thermal imaging cameras and depth cameras are also equipped for special environments to identify personnel positions and analyze operational postures under low-light conditions.

[0017] ② Sensor devices: These include wearable devices such as smart bracelets and smart helmets, used to monitor people's physiological state (e.g., heart rate, body temperature) and motion data (e.g., falls, violent shaking). Simultaneously, various IoT sensors are deployed in the devices and environment to collect device status parameters (e.g., rotation speed, temperature) and environmental parameters (e.g., concentration of harmful gases).

[0018] ③ Positioning equipment: Based on UWB, RFID or Bluetooth beacon technology, it can accurately locate personnel. By deploying positioning base stations in key areas of the factory, it can realize real-time tracking of personnel location and electronic fence function.

[0019] The terminal perception layer provides a comprehensive and multi-dimensional data foundation for behavior analysis through multi-source information perception, solving the problems of misjudgment and missed judgment that may be caused by a single data source.

[0020] S102: Input visual perception data into a pre-built lightweight AI model and output the unsafe behavior type; The expression for the pre-built lightweight AI model is as follows: The lightweight AI model adopts a Transformer-based encoder and decoder architecture, as follows: Where Z is the attention-weighted output feature, Q is the query matrix, K is the key matrix, V is the value matrix, and d k Let be the dimension of the key vector, and Softmax() be the attention weight distribution function; The lightweight AI model extracts features from the input visual perception data through a multi-layer, multi-head attention mechanism and outputs the unsafe behavior type. Its classification decision expression is as follows: In the formula, For unsafe behavior types, FFN() is a feedforward neural network, and argmax() is the maximum output value.

[0021] The lightweight AI model is built and improved through a Transformer-based sequence analysis algorithm. It accurately identifies the stopping position, eliminates the phase difference between pedestrians and vehicles, and quantifies the ambiguity in violation decision-making through an attention mechanism, thereby achieving automated violation detection.

[0022] Specifically, quantifying the ambiguity in violation decisions through the attention mechanism involves extracting the attention weight matrix of the Transformer and calculating its information entropy. When occlusion in the factory environment leads to the loss of key features, the model's attention distribution tends to be more dispersed and uniform, and the information entropy increases accordingly, thereby quantifying the degree of ambiguity at the visual feature level. Specifically, the process of building a lightweight AI model is as follows: A Transformer-based encoder-decoder architecture is constructed as the base network. To eliminate the motion phase difference between pedestrians and vehicles in complex factory scenarios, spatiotemporal kinematic position encoding is introduced at the encoder input. This maps the skeletal keypoint sequence in the visual perception data to the vehicle motion trajectory sequence to a unified temporal feature space, and aligns the temporal phase of human-vehicle interaction through an attention mechanism. To address the computing power limitations of edge devices in factories, a lightweight improvement and optimization of the standard Transformer's multi-head attention mechanism is made. A linear attention mechanism is adopted to replace the standard quadratic complexity attention calculation. Combined with a channel pruning strategy, redundant attention heads that contribute less than a preset threshold to the classification of unsafe behaviors are removed, resulting in a lightweight AI model. The lightweight AI model is trained by constructing a multi-task joint loss function, which includes cross-entropy classification loss for unsafe behavior types, fuzziness penalty loss based on uncertainty estimation branch calculation, and temporal phase alignment constraint loss. The fuzziness penalty loss based on uncertainty estimation branch calculation is used to constrain the model's overconfident prediction in fuzzy scenarios; the temporal phase alignment constraint loss is used to penalize phase misalignment in human-vehicle interaction sequences. Specifically, unsafe behaviors include not wearing a safety helmet, entering a dangerous area, falling, and abnormal operating posture.

[0023] S103: Perform multimodal fusion correlation analysis on unsafe behavior types with personnel physiological state, motion data, and personnel location data to verify whether unsafe behavior is established. If it is not established, there is no unsafe behavior; otherwise, output a real-time warning command and control the execution equipment to perform warning monitoring based on the real-time warning command.

[0024] Specifically, if the unsafe behavior type is that a person is not wearing a safety helmet, the person's identity is confirmed based on the person's location data, physiological status and movement data, and it is determined whether the person has permission to enter the corresponding area. If the person has permission to enter the corresponding area and the person's physiological status and movement data are normal, then the person has permission to enter the corresponding area; otherwise, the person does not have permission to enter the corresponding area.

[0025] When the unsafe behavior type is trespassing into a dangerous area: By mapping the coordinates of the person's image extracted from the visual perception data to the physical coordinate system, the real-time physical coordinates fed back by the positioning device are obtained simultaneously; if both coordinates fall within the preset danger zone electronic fence, then spatial verification is passed.

[0026] Based on the confirmed personnel identity query security management system, if the personnel do not have permission to enter the dangerous area and there is no accompanying authorized device status, it is determined that the entry into the dangerous area has been established, and an early warning signal is output; if the personnel have permission (e.g., they are performing authorized maintenance tasks), or the location data shows that their actual physical coordinates are outside the fence (e.g., due to visual misjudgment caused by obstruction), it is determined that the entry into the dangerous area has not been established.

[0027] When the unsafe behavior type is a person falling: Spatiotemporal synchronization feature verification: Extract the vertical height drop feature of key points of human skeleton in visual perception data, and synchronously retrieve motion data collected by the inertial measurement unit (IMU) built into the wearable device; if the error between the timestamp of the visual height drop and the timestamp of the triaxial acceleration change detected by the IMU is within a preset threshold, then the spatiotemporal synchronization verification is passed. In this embodiment, the preset threshold is set to ±0.2 seconds.

[0028] If the visual characteristics, acceleration abrupt change, and physiological abnormality are logically consistent, then a fall is determined and the highest level of emergency medical warning is immediately triggered. If a fall is visually determined but there is no acceleration abrupt change in IMU, for example, if the person is only squatting down to work or tying shoelaces, then the determination is not valid.

[0029] When the unsafe behavior type is abnormal operating posture: Job status association verification: Based on personnel location data, confirm that the personnel are currently at the operating station of a specific device, and simultaneously obtain the IoT operating status of the device; Fatigue and Load Fusion Decision Making: Extracting postural features from visual perception data indicating prolonged bending, twisting, or other non-standard work procedures, and simultaneously retrieving physiological state data collected by wearable devices, such as fatigue level assessed based on heart rate variability (HRV) or muscle load feedback from electromyography (EMG) sensors; if the visual posture is abnormal and physiological data indicates that fatigue or muscle load exceeds the safety threshold, and the device is in operation, then the abnormal operating posture is determined, and a forced rest or intervention warning is output; if the visual assessment is abnormal but physiological data shows that the person is energetic, or the device is in a shutdown maintenance state, then the determination is not valid.

[0030] The process of performing multimodal fusion and correlation analysis of unsafe behavior types with personnel physiological state, motion data, and personnel location data includes: Specifically, it includes: Step 1: Spatiotemporal matching of vision and localization The target personnel image coordinates extracted from the visual perception data are mapped to the factory's unified physical coordinate system; the positioning device IDs located within the preset error range of the coordinates are retrieved simultaneously to establish a preliminary binding between the visual target and the physical device, with the preset error range set at ±0.5 meters.

[0031] Step 2: Logical self-consistency verification of physiological and motor characteristics Retrieve the real-time physiological state and motion data associated with the aforementioned bound positioning device ID, and verify whether the data is physically consistent with the current person's motion state captured by vision. The current person's motion state includes being still, walking, or violently shaking.

[0032] Step 3: Unique Identity Verification and Authorization Verification If the spatiotemporal coordinates match and the physiological or motion characteristics are logically consistent, the personal information registered in the system with the location device ID is confirmed as the unique identity of the current target person. Then, based on this unique identity, the security management system is queried to verify whether the person has the right to enter the current physical area, thereby completing the final verification of the multimodal fusion correlation analysis.

[0033] In this embodiment, the physiological state and motion data of the person are used as specific pose data.

[0034] The process of verifying whether an unsafe act has occurred includes: Step 1: Visual initial screening trigger The lightweight AI model, based on visual perception data, outputs suspected unsafe behavior types and confidence levels as initial events to be verified.

[0035] Step 2: Identity and Location Verification Based on personnel location data, visual targets are spatiotemporally matched with physical positioning devices to confirm the unique identity of the personnel and verify whether the personnel have the work permissions for the current area.

[0036] Step 3: State Logic Verification The physiological state and motion data of the person are retrieved simultaneously to verify whether the multidimensional data is consistent with the action characteristics recognized by vision in terms of timestamps and physical logic. For example, in this embodiment, when the visual judgment of a fall occurs, the acceleration data should show a sudden peak.

[0037] Step 4: Integration Decision Making Determination not valid: If the personnel have legal working authority in the area, or if there is a logical conflict between physiological / motor data and visual characteristics, such as abnormal visual judgment but normal physiological data, then the unsafe behavior is determined not to be valid and no warning is issued.

[0038] Valid judgment: If personnel are unauthorized to enter, or if multi-dimensional data are highly self-consistent in time, space, and logic, jointly confirming the violation, then the unsafe behavior is deemed valid, and a real-time warning signal is output.

[0039] Multi-dimensional perception data and unsafe behavior types at the factory site are aggregated and processed through a cloud platform to obtain training sample data. The lightweight AI model is then trained and optimized based on the training sample data to obtain the trained lightweight AI model.

[0040] The training process for a lightweight AI model is as follows: Multi-dimensional perception data from the factory site is acquired, and multimodal fusion correlation analysis is used to cross-validate the pure visual recognition results. The multi-dimensional perception data from the factory site is divided into a training set and a validation set. The high-confidence visual data that has passed validation is used as standard samples; at the same time, low-confidence, highly ambiguous, or validation conflict visual data uploaded from the edge side is aggregated through the cloud platform to construct a difficult sample set. The specific training process, including the design of hyperparameters and training epochs, is as follows: To ensure the inference efficiency of the lightweight AI model at the edge and its global convergence performance on the cloud platform, differentiated training hyperparameters and optimization strategies are configured, specifically including: (1) Optimizer and learning rate scheduling: The AdamW optimizer is used to update the model parameters to effectively address the sparse gradient problem in the Transformer architecture and prevent overfitting. In this embodiment, an adaptive moment estimation optimizer with weight decay is used. The learning rate scheduling adopts a cosine annealing strategy with preheating, and the initial base learning rate is set to 1×10. -4 Up to 5×10 -4 Between these stages, the preheating rounds are set to 5 to 10, and the weight decay coefficient is set to 1×10.-4 ; (2) Batch Size Configuration: The training batch size is configured differently to accommodate the difference in computing power between the edge processing layer and the cloud platform layer. When the edge side uses local data for initial training or fine-tuning, the batch size is set to 16 or 32 to adapt to the memory limitations of edge devices; when the cloud platform layer uses globally aggregated data for retraining, the batch size is set to 64 or 128 to utilize the large computing power of the cloud to accelerate the stability of global gradient descent. (3) Stage setting of training epochs: The initial full training epochs of the lightweight AI model are set to 100 to 200 epochs to ensure that the model fully learns the basic spatiotemporal features; when the model is deployed to the edge, the cloud platform layer performs incremental retraining based on periodically aggregated difficult samples, and the single training epochs are set to 10 to 50 epochs to achieve rapid iteration of the model and avoid catastrophic forgetting, and finally output the trained lightweight AI model.

[0041] Specifically, the cloud platform layer is responsible for centralized data management, in-depth analysis, and model optimization. The main functions of this layer include: Centralized Management and Visualization: Provides a system management backend and a large visual dashboard to display the plant's overall safety status, alarm statistics, risk heat maps, etc.

[0042] Model Training and Optimization: The cloud platform aggregates challenging samples from various edge processing layers, leveraging the powerful computing capabilities of the cloud to retrain and optimize the security behavior model, generating an optimized security behavior model, which is then distributed to the edge processing layer through a closed-loop update mechanism. This mechanism enables the system to continuously learn from real-world applications, constantly improving its recognition accuracy and adaptability to new scenarios.

[0043] Big Data Analytics: Stores all historical alarm and operational data, performs macro-level statistical analysis, and uncovers the root causes of accidents and safety patterns. By analyzing the spatiotemporal distribution characteristics of unsafe behaviors, it identifies high-risk periods and areas, providing data support for safety management decisions.

[0044] The cloud platform layer compensates for the limitations of local optimization in the edge processing layer through global optimization and macro analysis, enabling the system to continuously evolve itself.

[0045] The overall structure of this application, as follows: Figure 2 As shown, through distributed computing and multimodal data fusion, real-time and accurate identification and early warning of unsafe behaviors are achieved. The system architecture consists of four parts: terminal perception layer, edge processing layer, cloud platform layer, and application layer.

[0046] Example 2: To achieve the above objective, such as Figure 3As shown, based on Embodiment 1, this invention discloses an intelligent factory operation behavior monitoring system based on human pose estimation, comprising: Data acquisition module 11 is used to acquire multi-dimensional sensing data of the factory site, including visual sensing data, personnel physiological state and motion data, and personnel location data. The data processing module 12 is used to input visual perception data into a pre-built lightweight AI model and output the unsafe behavior type. The monitoring output module 13 is used to perform multimodal fusion correlation analysis on unsafe behavior types, personnel physiological state and motion data, and personnel location data to verify whether unsafe behavior is established. If it is not established, there is no unsafe behavior; otherwise, a real-time warning command is output, and the execution equipment is controlled to perform warning monitoring based on the real-time warning command.

[0047] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.

[0048] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0049] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0050] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.

Claims

1. A method for monitoring intelligent factory work behavior based on human pose estimation, characterized in that, The method includes the following steps: Acquire multi-dimensional sensing data from the factory site, including visual sensing data, personnel physiological state and motion data, and personnel location data; Visual perception data is input into a pre-built lightweight AI model, and the output is the type of unsafe behavior. The unsafe behavior type is fused and correlated with personnel physiological state, motion data, and personnel location data through multimodal analysis to verify whether the unsafe behavior is valid. If it is not valid, there is no unsafe behavior; otherwise, a real-time warning command is output, and the execution equipment is controlled to perform warning monitoring based on the real-time warning command.

2. The intelligent factory operation behavior monitoring method based on human pose estimation according to claim 1, characterized in that, The expression for the pre-built lightweight AI model is as follows: The lightweight AI model adopts a Transformer-based encoder and decoder architecture, as follows: Where Z is the attention-weighted output feature, Q is the query matrix, K is the key matrix, V is the value matrix, and d k Let be the dimension of the key vector, and Softmax() be the attention weight distribution function; The lightweight AI model extracts features from the input visual perception data through a multi-layer, multi-head attention mechanism and outputs the unsafe behavior type. Its classification decision expression is as follows: In the formula, For unsafe behavior types, FFN() is a feedforward neural network, and argmax() is the maximum output value.

3. The intelligent factory operation behavior monitoring method based on human pose estimation according to claim 2, characterized in that, The lightweight AI model is constructed and improved using a Transformer-based sequence analysis algorithm. It achieves automated violation detection by accurately identifying stopping positions and quantifying the ambiguity in violation decision-making through an attention mechanism.

4. The intelligent factory operation behavior monitoring method based on human pose estimation according to claim 1, characterized in that, The unsafe behaviors include not wearing a safety helmet, entering a dangerous area, falling, and abnormal operating posture.

5. The intelligent factory operation behavior monitoring method based on human pose estimation according to claim 1, characterized in that, The process of performing multimodal fusion correlation analysis between unsafe behavior types and personnel physiological state, motion data, and personnel location data includes: If the unsafe behavior type is that a person is not wearing a safety helmet, the person's identity is confirmed based on the person's location data, physiological status and movement data, and it is determined whether the person has permission to enter the corresponding area. If the person has permission to enter the corresponding area and the person's physiological status and movement data are normal, then the person has permission to enter the corresponding area; otherwise, the person does not have permission to enter the corresponding area.

6. The intelligent factory operation behavior monitoring method based on human pose estimation according to claim 1, characterized in that, The multi-dimensional perception data and unsafe behavior types at the factory site are aggregated and processed through a cloud platform to obtain training sample data. The lightweight AI model is then trained and optimized based on the training sample data to obtain the trained lightweight AI model.

7. The intelligent factory operation behavior monitoring method based on human pose estimation according to claim 1, characterized in that, The process of collecting multi-dimensional sensing data at the factory site is as follows: The visual perception equipment uses high-definition network cameras with motion capture and facial recognition functions, which are deployed at workshop entrances and exits, green channels and various work areas to collect visual perception data. Sensor devices collect physiological and motion data of people by deploying wearable devices such as smart bracelets and smart safety helmets; Positioning devices use UWB, RFID, or Bluetooth beacon technologies to accurately locate people and mobile assets, and are used to collect personnel location data.

8. A smart factory operation behavior monitoring system based on human pose estimation, employing the smart factory operation behavior monitoring method based on human pose estimation as described in any one of claims 1 to 7, characterized in that, include: The data acquisition module is used to acquire multi-dimensional sensing data of the factory site, including visual sensing data, personnel physiological state and motion data, and personnel location data. The data processing module is used to input visual perception data into a pre-built lightweight AI model and output the unsafe behavior type. The monitoring output module is used to perform multimodal fusion correlation analysis on unsafe behavior types with personnel physiological state, motion data, and personnel location data to verify whether an unsafe behavior is established. If it is not established, there is no unsafe behavior; otherwise, a real-time warning command is output, and the execution equipment is controlled to perform warning monitoring based on the real-time warning command.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, it employs the intelligent factory operation behavior monitoring method based on human pose estimation as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the intelligent factory operation behavior monitoring method based on human pose estimation as described in any one of claims 1 to 7.

Images