Personal protection equipment compliance detection method

Abstract

The invention discloses a personal protection equipment compliance detection method, and relates to the technical field of image data processing. Comprising the steps of constructing a two-dimensional human body sensing network, a three-dimensional human body shape reconstruction network and a layered protection equipment judgment network; obtaining video data, performing feature extraction on the video data, and generating accurate coordinates of human skeleton key points and body joints; based on the human skeleton key points, a working state result is generated through a three-dimensional human body shape reconstruction network; based on the human skeleton key points and the video data, generating protective equipment wearing features through a layered protective equipment judgment network, judging whether a working state result of the target person meets a preset result, if so, continuing to judge whether the protective equipment wearing features meet a preset condition, and if the protective equipment wearing features meet the preset condition, continuing to judge whether the protective equipment wearing features meet the preset condition. And outputting wearing compliance. The invention aims to judge whether an experimenter regularly wears personal protection equipment or not by analyzing video stream data in combination with a modern image recognition technology and a video monitoring means.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a compliance detection method for personal protective equipment. Background Art

[0002] With the continuous development of science and technology and the increasing complexity of modern laboratory environments, laboratory safety issues have gradually attracted widespread attention. Personal protective equipment (PPE) is an important measure to protect the lives of laboratory personnel and prevent accidents. The compliance of its use directly affects the safety of the laboratory and the accuracy of experimental results.

[0003] However, in modern laboratories, monitoring the compliance of PPE use faces many technical challenges. Traditional manual inspection and manual recording are not only inefficient, but also easily affected by subjective factors, making it difficult to ensure real-time and accuracy. Therefore, how to improve the level of laboratory safety management through intelligent means and ensure the correct use of PPE has become an urgent problem to be solved in the current laboratory information management system (LIMS).

[0004] In order to solve the above technical problems, some intelligent monitoring technologies currently attempt to determine the wearing status of PPE by installing sensors in the laboratory. These sensors usually include hardware connected to protective equipment, such as position sensors, temperature sensors, pressure sensors, etc., which are designed to monitor the wearer's status in real time and ensure that the PPE equipment is worn correctly. However, in the actual laboratory environment, due to the frequent activities of personnel and the complex laboratory scenes, it is difficult to accurately determine whether the experimenters are wearing all protective equipment in a standardized manner by relying on a single detection method of sensors. For example, when experimenters wear multiple protective equipment (such as protective glasses, gloves, masks, protective clothing, etc.), sensors often cannot fully and effectively monitor the wearing status of all equipment, and are prone to misjudgment or missed judgments. At the same time, the installation and maintenance of these sensor devices requires a lot of capital investment, which increases the construction and operation costs of the system.

[0005] Therefore, how to combine modern image recognition technology and video surveillance methods to determine whether the experimenters are wearing personal protective equipment properly by analyzing video stream data has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The main purpose of the present invention is to provide a personal protective equipment compliance detection method, which aims to combine modern image recognition technology and video monitoring means to determine whether the experimenter is wearing personal protective equipment in a standardized manner by analyzing video stream data.

[0007] In order to achieve the above object, the present invention proposes a personal protective equipment compliance detection method, comprising the following steps: Construct a 2D human body perception network, a 3D human body shape reconstruction network, and a layered protective equipment judgment network; Acquire video data, extract features from the video data through a two-dimensional human body perception network, and generate precise coordinates of key points of human skeleton and body joints; Based on the human skeleton key points, a working status result of the target person is generated through a three-dimensional human shape reconstruction network; Based on the human skeleton key points and video data, a layered protective equipment judgment network is used to generate protective equipment wearing features to determine whether the target person's working status results meet the preset results. If so, continue to determine whether the protective equipment wearing features meet the preset conditions. If the protective equipment wearing features meet the preset conditions, output personal protective equipment wearing compliance.

[0008] In one embodiment of the present application, the video data is subjected to feature extraction through a two-dimensional human body perception network to generate accurate coordinates of human skeleton key points and body joints, further comprising the following steps: Split the video data into continuous video frames, perform human body detection on each video frame, and segment the region of interest corresponding to the human target; Perform feature extraction in each region of interest to obtain global posture information and local posture information; Extracting skeleton key points according to the global posture information and the local posture information; The skeleton key points are subjected to feature fusion through a multi-layer perceptron to generate precise coordinates of body joints.

[0009] In one embodiment of the present application, based on the human skeleton key points, a three-dimensional human body shape reconstruction network is used to generate a working status result of a target person, and the following steps are also included: Projecting the skeleton key points into three-dimensional space to form a skeleton representation; Based on the skeleton representation, generating body shape parameters and joint angle parameters of the target person through the SMPL model, and generating an initial mesh based on the body shape parameters and joint angle parameters of the target person; Optimizing the initial mesh to generate a detailed human body mesh; The working status results of the target person are generated based on the detailed human body mesh, global posture information, and the corresponding video frames.

[0010] In one embodiment of the present application, optimizing the initial human body network includes the following steps: Selecting a local mesh from the initial mesh, and aligning the geometric consistency and surface smoothness of the local mesh vertices with the initial image perception of the corresponding video frame; The visibility of each joint of the character's body in the initial mesh is judged. When the joint is invisible, the position information of the corresponding invisible joint is derived according to the prior knowledge encoded in the SMPL model and the contextual information of the visible joints to generate a detailed human body mesh.

[0011] In one embodiment of the present application, based on the human skeleton key points and video data, a layered protective equipment judgment network is used to generate protective equipment wearing features, including the following steps: Determine the key parts of the human body through the key points of the skeleton; Generate an overall mask of the target person according to the key parts through the SAM algorithm, so as to achieve segmentation of the target person and the background; Obtaining a local mask according to the precise coordinates of the body joints and the global mask; Generate protective equipment wearing features based on the local mask and the corresponding video frame.

[0012] In an embodiment of the present application, the local mask includes at least one of a face mask, a hand mask, a torso mask, and a foot mask.

[0013] In one embodiment of the present application, the two-dimensional human body perception network is constructed based on the Alpha gesture architecture.

[0014] In one embodiment of the present application, the three-dimensional human body shape reconstruction network is constructed based on the Hybrik framework.

[0015] In one embodiment of the present application, after constructing a two-dimensional human body perception network, a three-dimensional human body shape reconstruction network, and a layered protective equipment judgment network, the following is further included: Obtain training videos, preprocess the training data, and generate enhanced data; Input the enhanced data into the 2D human perception network and 3D human shape reconstruction network, and set the training hyperparameters; Through the two-dimensional human perception network, three-dimensional human shape reconstruction network, and layered protective equipment judgment network, the working status of the target person and the wearing status of the protective equipment are obtained, the working status of the target person and the wearing status of the protective equipment are input into the loss function, and the two-dimensional human perception network and three-dimensional human shape reconstruction network are trained through back propagation.

[0016] In one embodiment of the present application, the preprocessing includes: at least one of perturbation, flipping, and stretching of the training video.

[0017] By adopting the above technical solution, by constructing a two-dimensional human perception network, a three-dimensional human shape reconstruction network and a layered protective equipment judgment network, it is possible to accurately extract the precise coordinates of the key points of the human skeleton and the body joints from the video data, and generate the working status results of the target person through the three-dimensional human shape reconstruction network. The layered protective equipment judgment network further combines the working status and protective equipment wearing characteristics to accurately judge whether the wearing of protective equipment meets the preset conditions and output the compliance results. This method can realize efficient and accurate compliance detection of personal protective equipment in complex scenarios, greatly improving the detection efficiency and reliability of the results. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention is described in detail below with reference to specific embodiments and accompanying drawings, wherein: Figure 1 It is a schematic structural diagram of the first embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the following specific embodiments are only used to explain the present invention and do not constitute a limitation to the present invention.

[0020] like Figure 1 As shown, in order to achieve the above purpose, the present invention proposes a personal protective equipment compliance detection method, comprising the following steps: Construct a 2D human body perception network, a 3D human body shape reconstruction network, and a layered protective equipment judgment network; Acquire video data, extract features from the video data through a two-dimensional human body perception network, and generate precise coordinates of key points of human skeleton and body joints; Based on the human skeleton key points, a working status result of the target person is generated through a three-dimensional human shape reconstruction network; Based on the human skeleton key points and video data, a layered protective equipment judgment network is used to generate protective equipment wearing features to determine whether the target person's working status results meet the preset results. If so, continue to determine whether the protective equipment wearing features meet the preset conditions. If the protective equipment wearing features meet the preset conditions, output personal protective equipment wearing compliance.

[0021] Specifically, a two-dimensional human perception network, a three-dimensional human shape reconstruction network, and a layered protective equipment judgment network are constructed. The two-dimensional human perception network is modified and extended based on the Alpha posture architecture, and can generate accurate coordinates of human skeleton key points and body joints. The three-dimensional human shape reconstruction network is extended based on the Hybrik (inverse dynamics algorithm) framework to generate a three-dimensional mesh of the human body and the working status results of the target person. The layered protective equipment judgment network is constructed through a multi-layer perceptron algorithm, which can generate wearing features for protective equipment in different parts, and realize the working status of the target person and the compliance judgment of the wearing of protective equipment.

[0022] The video data is obtained, split into frames and then input into a two-dimensional human perception network. The two-dimensional human perception network generates the precise coordinates of the human skeleton key points and body joints from each frame by extracting the multi-scale spatial features of the video frames.

[0023] Based on the human skeleton key point data generated above, the 3D human body shape reconstruction network is input. The 3D human body shape reconstruction network first maps the 2D skeleton key points to the standard 3D space to generate the 3D human skeleton structure. Then, combined with the human body mesh reconstruction algorithm, a 3D mesh representation of the human body is generated to extract the 3D working state features. By fusing the human body mesh information with the global posture features of the video frame, the working state results of the target person are finally generated.

[0024] Based on the above human skeleton key points and video data, they are input into the layered protective equipment judgment network. The layered protective equipment judgment network generates protective equipment wearing features based on the protective equipment related features extracted from the video frame. Specifically, the layered protective equipment judgment network combines the wearing area of ​​the protective equipment (such as face, hands, torso, etc.) and the target working status result to judge whether the working status result of the target person meets the preset result. If the working status result meets the preset result, it is further judged whether the protective equipment wearing of the target person meets the preset conditions based on the protective equipment wearing characteristics. If the conditions are met, the result of personal protective equipment wearing compliance is output.

[0025] By adopting the above technical solution, by constructing a two-dimensional human perception network, a three-dimensional human shape reconstruction network and a layered protective equipment judgment network, it is possible to accurately extract the precise coordinates of the key points of the human skeleton and the body joints from the video data, and generate the working status results of the target person through the three-dimensional human shape reconstruction network. The layered protective equipment judgment network further combines the working status and protective equipment wearing characteristics to accurately judge whether the wearing of protective equipment meets the preset conditions and output the compliance results. This method can realize efficient and accurate compliance detection of personal protective equipment in complex scenarios, greatly improving the detection efficiency and reliability of the results.

[0026] In one embodiment of the present application, the video data is subjected to feature extraction through a two-dimensional human body perception network to generate accurate coordinates of human skeleton key points and body joints, further comprising the following steps: Split the video data into continuous video frames, perform human body detection on each video frame, and segment the region of interest corresponding to the human target; Perform feature extraction in each region of interest to obtain global posture information and local posture information; Extracting skeleton key points according to the global posture information and the local posture information; The skeleton key points are subjected to feature fusion through a multi-layer perceptron to generate precise coordinates of body joints.

[0027] Specifically, the acquired video data is decomposed into a plurality of continuous video frames according to the frame rate, and each frame of data represents an independent time slice for subsequent frame-by-frame processing.

[0028] Human body detection is performed on each video frame, and the target detection algorithm is used to locate the human body area contained in the frame. The region of interest corresponding to the human body target is segmented according to the detection results to eliminate background interference and focus on the human body target.

[0029] Multi-scale feature extraction is performed in each region of interest, and the global posture information and local posture information are extracted using a two-dimensional human perception network. The global posture information captures the overall posture characteristics of the human body through large-scale features, and the local posture information captures the fine features of key parts of the human body through small-scale features, such as the detailed posture of the head, hands, and torso.

[0030] Based on global and local posture information, a two-dimensional human perception network is used to generate human skeleton key points. The extraction of skeleton key points is based on the prior knowledge of human joint structure, and the multi-stage regression method is used to gradually optimize the prediction results of key points to ensure the high accuracy and stability of skeleton key points.

[0031] The extracted skeleton key points are input into the multi-layer perceptron to fuse the features of the skeleton key points and combine the spatial correlation and contextual information between the key points. The multi-layer perceptron further optimizes and improves the data representation of the skeleton key points through hierarchical feature learning, and generates body joint data with precise coordinates.

[0032] By adopting the above technical solution, the video data is split into continuous video frames and human body detection and region of interest segmentation are performed to ensure the focus of feature extraction and minimize background interference. Global and local posture information is extracted in the region of interest, and the skeleton key points are combined with a multi-layer perceptron to perform feature fusion, achieving high-precision extraction of human skeleton key points and generation of precise coordinates of body joints. This method has high robustness and can adapt to background changes and occlusion in complex scenes, effectively improving the efficiency and accuracy of extracting human posture information.

[0033] In one embodiment of the present application, based on the human skeleton key points, a three-dimensional human body shape reconstruction network is used to generate a working status result of a target person, and the following steps are also included: Projecting the skeleton key points into three-dimensional space to form a skeleton representation; Based on the skeleton representation, generating body shape parameters and joint angle parameters of the target person through the SMPL model, and generating an initial mesh based on the body shape parameters and joint angle parameters of the target person; Optimizing the initial mesh to generate a detailed human body mesh; The working status results of the target person are generated based on the detailed human body mesh, global posture information, and the corresponding video frames.

[0034] Specifically, the skeleton key point data extracted by the 2D human perception network is input into the 3D human shape reconstruction network, and the 2D skeleton key points are projected into the standard 3D space through the 3D space mapping algorithm to generate a 3D skeleton representation. The 3D skeleton representation accurately describes the joint positions of the target person and their geometric relationships in 3D space.

[0035] Based on the 3D skeleton representation, the SMPL (Skinned Multi-Person Linear) model is combined to calculate the target person's body shape parameters and joint angle parameters. The SMPL model uses the prior knowledge of human anatomy and parametric representation to model the target person's body characteristics (such as height, weight, etc.), generate complete body shape parameters, and generate joint angle parameters based on the skeletal joint positions. The initial 3D mesh of the target person is generated based on the above parameters.

[0036] The initial mesh is adjusted using a geometric consistency optimization algorithm, including optimizing the geometric details of local areas to ensure the smoothness of the mesh surface and the perceptual consistency with the original video data. At the same time, for the occluded or partially visible joint areas in the video frame, the prior knowledge of the SMPL model and the contextual information of the visible joints are used to infer the reasonable positions of the invisible joints, thereby generating a complete and detailed human body mesh.

[0037] The optimized detailed human body mesh is combined with the global posture information, and the posture and motion characteristics of the target person are analyzed through the 3D human body shape reconstruction network. Combined with the corresponding video frame data, the 3D features of the target person are extracted and input into the multi-layer perceptron to generate the working status results of the target person through classification or regression models. For example, it can identify whether the target person is in a working state such as standing, walking, bending, carrying, etc., and generate accurate working state classification results or motion parameters.

[0038] By adopting the above technical solution, the 3D human body shape of the target person can be accurately restored by projecting the key points of the skeleton into the 3D space and combining the SMPL model to generate body shape parameters and joint angle parameters. By optimizing the initial mesh, the generated detailed human body mesh is ensured to have high precision and surface smoothness to adapt to occlusion and lighting changes in complex scenes. The action characteristics of the target person are further analyzed by combining the 3D mesh, global posture information and video frame data to generate accurate working status results.

[0039] In one embodiment of the present application, optimizing the initial human body network includes the following steps: Selecting a local mesh from the initial mesh, and aligning the geometric consistency and surface smoothness of the local mesh vertices with the initial image perception of the corresponding video frame; The visibility of each joint of the character's body in the initial mesh is judged. When the joint is invisible, the position information of the corresponding invisible joint is derived according to the prior knowledge encoded in the SMPL model and the contextual information of the visible joints to generate a detailed human body mesh.

[0040] Specifically, from the initial mesh generated by the 3D human body shape reconstruction network, specific local meshes are selected for optimization. Local meshes are joints (such as shoulders, knees, elbows, etc.) or areas with large posture changes in the human body mesh. For these local meshes, the vertex positions are adjusted through the geometric consistency optimization algorithm to ensure that the geometric structure of these local meshes is consistent with the target shape of the human body in the video frame. The optimization process also includes the adjustment of surface smoothness to ensure that the surface of the mesh presents a natural and smooth effect and avoids obvious geometric defects.

[0041] During the local mesh optimization process, the optimized local mesh vertices are aligned with the initial image perception data corresponding to the video frame. The image perception information of the human target area in the video frame is combined with the texture features and color information of the human target, and the image features extracted by the deep learning model (such as convolutional neural network) are used to further refine the vertex position of the mesh to make it more consistent with the shape of the person in the video frame. This process not only ensures geometric consistency, but also improves the adaptability of the mesh to changes in the posture of the person in the video frame.

[0042] The visibility of each joint position in the initial mesh is judged. The image features and the positions of the skeleton key points in the video frame are analyzed to determine whether each joint is visible. For invisible joints, their positions are deduced using the prior knowledge based on the SMPL model and the contextual information of the visible joints.

[0043] The process is as follows: first, the reasonable position of the joints is inferred based on the human skeleton structure and the relative position relationship of the joints encoded in the SMPL model; then, the reasonable position of the invisible joints is derived by combining the spatial position information and movement pattern of the visible joints, and the corresponding joint coordinates in the grid are updated. Through this derivation mechanism, the occluded areas in the image can be effectively filled to generate a more complete and realistic 3D grid representation.

[0044] During the optimization process, a detailed 3D human body mesh is finally generated through local adjustments to the initial mesh and derivation of joint visibility. This mesh not only has precise geometric form, but also accurately reflects the motion posture and detailed features of the target person. The generated detailed mesh is consistent in both space and time, and can be used continuously in multiple frames of video data, ensuring the smoothness and stability of the mesh over time.

[0045] The above technical solution ensures geometric consistency and surface smoothness by optimizing local mesh vertices in the initial human body mesh, and further improves the accuracy and naturalness of the mesh by aligning with the image perception in the video frame. By judging the visibility of joints and inferring the positions of invisible joints, the occlusion problem is solved and the integrity of the mesh is ensured.

[0046] In one embodiment of the present application, based on the human skeleton key points and video data, a layered protective equipment judgment network is used to generate protective equipment wearing features, including the following steps: Determine the key parts of the human body through the key points of the skeleton; Generate an overall mask of the target person according to the key parts through the SAM algorithm, so as to achieve segmentation of the target person and the background; Obtaining a local mask according to the precise coordinates of the body joints and the global mask; Generate protective equipment wearing features based on the local mask and the corresponding video frame.

[0047] Specifically, using the skeleton key point data extracted in the previous steps, first determine the key parts of the human body, including the head, torso, arms, legs and other important parts. Through the spatial relationship of the skeleton key points, the relative position and geometric shape of each key part are accurately located to provide an accurate reference for subsequent mask generation. For example, the head can be determined by the skeleton key points of the top of the head and neck, and the hands can be determined by the shoulder and wrist key points.

[0048] Based on the key parts determined above, the SAM (Segment Anything Model) algorithm is used to generate the overall mask of the target person. The SAM algorithm constructs the accurate outline of the human target by extracting the contextual information of the skeleton key points and the target parts. This mask not only effectively distinguishes the area of ​​the human body and the background, but also can cope with the target detection task under complex backgrounds. Through the generation of the overall mask, the segmentation of the target person and the background is successfully achieved, providing a clean image area for the subsequent local mask generation and protective equipment judgment.

[0049] Based on the precise coordinates of the body joints and the generated overall mask, the local mask of the human body is further extracted. The local mask is extracted for specific parts of the human body (such as the head, hands, torso, etc.) to ensure that background interference is minimized when analyzing the wearing characteristics of protective equipment in these areas. For example, for the local mask of the hand, the mask area of ​​the hand can be extracted through the skeletal key point information of the wrist and elbow, and the area can be matched with the actual image in the video frame to detect the wearing of protective equipment such as gloves and wristbands.

[0050] The extracted local mask is combined with the corresponding video frame data to generate protective equipment wearing features. This process extracts fine features related to protective equipment through convolutional neural networks (CNN) or other deep learning methods. For different protective equipment (such as masks, gloves, goggles, etc.), the network will analyze its coverage, shape, material and other features in the local mask area to determine whether the protective equipment is worn correctly. For example, the mask detection branch will evaluate whether the facial area is covered and determine the completeness of the coverage; the glove detection branch will check whether the hand area is completely covered by the gloves and whether the wearing meets the safety standards. The protective equipment wearing features finally generated are used to determine whether the target person meets the preset protective equipment wearing standards.

[0051] The above technical solution is adopted to accurately locate the key points of the skeleton and generate an overall mask using the SAM algorithm to ensure the accurate segmentation of the human body and the background and reduce background interference. The key parts of the human body are extracted through local masks, and the protective equipment wearing features are generated by combining the video frame data, realizing the accurate assessment of the wearing status of the protective equipment.

[0052] In an embodiment of the present application, the local mask includes at least one of a face mask, a hand mask, a torso mask, and a foot mask.

[0053] Specifically, in the aforementioned steps, the overall mask generated by the skeleton key points and the SAM algorithm can be further subdivided into multiple local masks, including masks of multiple local areas such as face mask, hand mask, torso mask, and foot mask.

[0054] The mask of the facial area is determined by the key points of the facial bones (such as eyes, nose, mouth and other key positions), which is particularly suitable for the wearing detection of protective equipment such as masks and respirators.

[0055] The mask area of ​​the hand is determined by the key bone points of the hand (such as the wrist, elbow, etc.), which is suitable for judging whether the gloves are worn.

[0056] The mask of the torso area is extracted through the skeleton key points related to the torso (such as shoulders, chest, abdomen, etc.), which is suitable for the detection of protective clothing, chest protectors and other equipment.

[0057] The mask of the foot area is extracted through the key bone points of the foot (such as ankles, knees, etc.), which is suitable for the detection of footwear or ankle protection equipment.

[0058] By adopting the above technical solution, the wearing condition of protective equipment can be evaluated more accurately by subdividing local masks such as face, hands, torso, and feet.

[0059] In one embodiment of the present application, the two-dimensional human body perception network is constructed based on the Alpha gesture architecture.

[0060] Specifically, AlphaPose is an efficient and accurate multi-person pose estimation algorithm, which is widely used in human key point detection tasks. The two-dimensional human perception network improves the detection ability of human skeleton key points by modifying and extending the AlphaPose architecture. AlphaPose extracts features from images through a deep convolutional neural network (CNN) and predicts the coordinates of each human body part through key point regression.

[0061] The above technical solution can efficiently and accurately extract the coordinates of the human skeleton key points and key parts in the video frame through the two-dimensional human perception network built on the Alpha Pose architecture. By modifying and expanding the Alpha Pose architecture, the performance of the network in complex scenes is optimized, and high accuracy can be maintained in the case of dense crowds or large changes in posture.

[0062] In one embodiment of the present application, the three-dimensional human body shape reconstruction network is constructed based on the Hybrik framework.

[0063] Specifically, the Hybrik framework is a deep learning architecture for 3D human body reconstruction that can accurately reconstruct the 3D shape of the human body by combining image data and posture information. The framework integrates the processing capabilities of multimodal information, including 2D image information and 3D skeleton data, making it highly efficient in human body shape reconstruction.

[0064] The Hybrik framework uses a multi-stage neural network design, including a deep convolutional neural network (CNN) to extract features from input video frames and map these features into 3D space. The framework establishes a mapping relationship between image features and 3D human body parameters, enabling the network to generate accurate 3D human body meshes based on 2D human skeleton key point data.

[0065] First, based on the skeleton key point data generated by the 2D human perception network, these key points are projected into 3D space through the Hybrik framework. The projected skeleton representation serves as the basis for 3D shape reconstruction. This projection process not only calculates the skeleton structure based on the joint positions, but also combines 3D prior models (such as the SMPL model) to generate a more accurate initial shape.

[0066] Next, based on the 3D skeleton representation, the framework estimates the body shape parameters and joint angle parameters of the target person through a regression network. These parameters mainly include the body shape information, the angles of each joint, and the relative positions of different body parts. By regressing these parameters, the framework generates an initial 3D mesh model of the target person.

[0067] The generated initial mesh then enters the optimization phase. Through local optimization algorithms, the mesh is adjusted for geometric consistency and surface smoothness, making it more consistent with the actual image perception in the input video frame. At this stage, the framework also adjusts the temporal consistency of the mesh based on the dynamic changes of skeletal key points and the continuity of human posture.

[0068] By introducing recurrent neural networks (RNN) or long short-term memory networks (LSTM), the Hybrik framework can effectively model the temporal dependencies between video frames. When processing consecutive video frames, the network can maintain the consistency of the 3D mesh in the temporal dimension, avoiding mesh deformation or unnatural reconstruction results due to posture changes. During the optimization process, the Hybrik framework also uses refinement algorithms to further adjust the details of the 3D mesh to ensure the accuracy and realism of the mesh. This process will involve fine-tuning of joints and key parts to accurately reflect changes in human posture in dynamic videos.

[0069] After completing the reconstruction of the 3D mesh, the 3D human shape reconstruction network can generate the target person's working state results based on the detailed 3D mesh, global posture information and input data in the video frame. These working state results can determine whether the target person is in a specific working state (such as standing, walking, sitting, etc.) by analyzing the 3D mesh.

[0070] Using the above technical solution, the 3D human shape reconstruction network based on the Hybrik framework can efficiently map the skeletal key point data generated by the 2D human perception network into 3D space and accurately reconstruct the 3D shape of the target person. By combining multimodal features, the Hybrik framework can not only generate high-precision 3D meshes, but also ensure the consistency of the meshes in time.

[0071] In one embodiment of the present application, after constructing a two-dimensional human body perception network, a three-dimensional human body shape reconstruction network, and a layered protective equipment judgment network, the following is further included: Obtain training videos, preprocess the training data, and generate enhanced data; Input the enhanced data into the 2D human perception network and 3D human shape reconstruction network, and set the training hyperparameters; Through the two-dimensional human perception network, three-dimensional human shape reconstruction network, and layered protective equipment judgment network, the working status of the target person and the wearing status of the protective equipment are obtained, the working status of the target person and the wearing status of the protective equipment are input into the loss function, and the two-dimensional human perception network and three-dimensional human shape reconstruction network are trained through back propagation.

[0072] Specifically, first, collect video data containing the target person's activities. The video data can come from different scenes and different working environments, including but not limited to indoors, outdoors, workplaces, construction sites, etc. These video data should cover the diverse performances of the target person in different postures and different working states.

[0073] For the original training videos obtained, necessary preprocessing operations are performed, including video frame extraction, image size adjustment, color standardization, background removal, etc. Through these preprocessing operations, it is ensured that the video data can achieve high quality when entering the network model, and noise and redundant information are prevented from affecting the training effect of the model.

[0074] In order to enhance the robustness of the model, especially when dealing with large posture changes and complex environments, a variety of data enhancement methods are used to expand the original training data. Common data enhancement techniques include: Image perturbation randomly changes the brightness, contrast, color and other parameters of the video frame to simulate scenes under different lighting conditions.

[0075] Image flipping and rotation: flipping or rotating the input image horizontally or vertically to simulate different perspective changes.

[0076] Scale stretching and cropping, randomly crop the image and adjust the scale to simulate target people of different sizes and postures.

[0077] Noise addition, adding Gaussian noise to the image, etc., increases the model's tolerance to noise.

[0078] Through these enhancement operations, the generated enhanced data can effectively increase the diversity of training samples during the training process, so that the trained model has stronger generalization ability and can adapt to various postures and environmental changes.

[0079] The generated enhanced data is input into the constructed 2D human perception network and 3D human shape reconstruction network for network training. The enhanced data will provide diverse input information, enabling the network to learn multi-level features such as human key points, bone positions, and body postures.

[0080] During the training process, set appropriate hyperparameters, including learning rate, batch size, optimization algorithm (such as Adam, SGD, etc.), training rounds, loss function, etc.

[0081] Through the two-dimensional human perception network and the three-dimensional human shape reconstruction network, combined with the skeleton key points, body joint information and three-dimensional mesh representation extracted from the video frame, the working status results of the target person are generated. Through the layered protective equipment judgment network, combined with the posture information of the target person and the characteristics of the protective equipment, the protective equipment wearing status is generated. The network can identify whether the target person is wearing compliant protective equipment.

[0082] During the training process, the working status and protective equipment wearing status output by the network will be compared with the preset standard labels to calculate the loss function. The loss function consists of multiple parts, including classification loss, regression loss, smoothing loss, and cosine similarity loss. Through the back propagation algorithm, the weights of the two-dimensional human perception network, the three-dimensional human shape reconstruction network, and the layered protective equipment judgment network are updated according to the gradient calculated by the loss function. Through multiple rounds of iterative training, the network gradually adjusts parameters to improve the accuracy of working status recognition and protective equipment wearing detection.

[0083] The above technical solution can greatly improve the diversity of training data through effective data preprocessing and data enhancement technology, so that the model can still maintain high accuracy when dealing with different working environments and complex postures. Through the joint training of the two-dimensional human perception network and the three-dimensional human shape reconstruction network, the working status of the target person can be accurately identified, and combined with the layered protective equipment judgment network, the wearing of protective equipment can be effectively evaluated.

[0084] In one embodiment of the present application, the preprocessing includes: at least one of perturbation, flipping, and stretching of the training video.

[0085] Specifically, the perturbation is to randomly perturb parameters such as brightness, contrast, and saturation of each frame image in the training video to simulate different lighting conditions.

[0086] Flipping is to flip the training video horizontally or vertically. By flipping the image horizontally, you can simulate observing the target person in reverse.

[0087] Stretching refers to stretching each frame of the training video, that is, changing the proportion of the target person by compressing or expanding the horizontal or vertical size of the image.

[0088] By using the above technical solution, the diversity of training data can be greatly increased and the generalization ability of the model can be improved by performing data enhancement processing such as perturbation, flipping, and stretching on the training videos. The enhanced data can cover a variety of situations such as different lighting conditions, shooting angles, and scale changes, improving the model's ability to cope with complex scenes in actual environments.

[0089] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for detecting compliance of personal protective equipment, characterized in that: The following steps are involved: Construct a 2D human body perception network, a 3D human body shape reconstruction network, and a layered protective equipment judgment network; Acquire video data, extract features from the video data through a two-dimensional human body perception network, and generate precise coordinates of key points of human skeleton and body joints; Based on the human skeleton key points, a working status result of the target person is generated through a three-dimensional human body shape reconstruction network; Based on the human skeleton key points and video data, a layered protective equipment judgment network is used to generate protective equipment wearing features to determine whether the target person's working status results meet the preset results. If so, continue to determine whether the protective equipment wearing features meet the preset conditions. If the protective equipment wearing features meet the preset conditions, output personal protective equipment wearing compliance.

2. The personal protective equipment compliance detection method according to claim 1, characterized in that: The video data is subjected to feature extraction through a two-dimensional human body perception network to generate accurate coordinates of human skeleton key points and body joints, and the following steps are also included: Split the video data into continuous video frames, perform human body detection on each video frame, and segment the region of interest corresponding to the human target; Perform feature extraction in each region of interest to obtain global posture information and local posture information; Extracting skeleton key points according to the global posture information and the local posture information; The skeleton key points are subjected to feature fusion through a multi-layer perceptron to generate precise coordinates of body joints.

3. The method for detecting compliance of personal protective equipment according to claim 2, characterized in that: Based on the human skeleton key points, a working state result of the target person is generated through a three-dimensional human body shape reconstruction network, and the following steps are also included: Projecting the skeleton key points into three-dimensional space to form a skeleton representation; Based on the skeleton representation, generating body shape parameters and joint angle parameters of the target person through the SMPL model, and generating an initial mesh based on the body shape parameters and joint angle parameters of the target person; Optimizing the initial mesh to generate a detailed human body mesh; The working status results of the target person are generated based on the detailed human body mesh, global posture information, and the corresponding video frames.

4. The method for detecting compliance of personal protective equipment according to claim 3, characterized in that: Optimizing the human body initial network comprises the following steps: Selecting a local mesh from the initial mesh, and aligning the geometric consistency and surface smoothness of the local mesh vertices with the initial image perception of the corresponding video frame; The visibility of each joint of the character's body in the initial mesh is judged. When the joint is invisible, the position information of the corresponding invisible joint is derived according to the prior knowledge encoded in the SMPL model and the contextual information of the visible joints to generate a detailed human body mesh.

5. The method for detecting compliance of personal protective equipment according to claim 2, characterized in that: Based on the human skeleton key points and video data, a protective equipment wearing feature is generated through a layered protective equipment judgment network, including the following steps: Determine the key parts of the human body through the key points of the skeleton; Generate an overall mask of the target person according to the key parts through the SAM algorithm, so as to achieve segmentation of the target person and the background; Obtaining a local mask according to the precise coordinates of the body joints and the global mask; Generate protective equipment wearing features based on the local mask and the corresponding video frame.

6. The method for detecting compliance of personal protective equipment according to claim 5, characterized in that: The local mask includes at least one of a face mask, a hand mask, a torso mask, and a foot mask.

7. The method for detecting compliance of personal protective equipment according to claim 1, characterized in that: The two-dimensional human body perception network is constructed based on the Alpha gesture architecture.

8. The method for detecting compliance of personal protective equipment according to claim 1, characterized in that: The three-dimensional human body shape reconstruction network is constructed based on the Hybrik framework.

9. The method for detecting compliance of personal protective equipment according to claim 1, characterized in that: After building a 2D human perception network, a 3D human shape reconstruction network, and a layered protective equipment judgment network, it also includes: Obtain training videos, preprocess the training data, and generate enhanced data; Input the enhanced data into the 2D human perception network and 3D human shape reconstruction network, and set the training hyperparameters; Through the two-dimensional human perception network, three-dimensional human shape reconstruction network, and layered protective equipment judgment network, the working status of the target person and the wearing status of the protective equipment are obtained, the working status of the target person and the wearing status of the protective equipment are input into the loss function, and the two-dimensional human perception network and three-dimensional human shape reconstruction network are trained through back propagation.

10. The method for detecting compliance of personal protective equipment according to claim 9, characterized in that: The preprocessing includes: at least one of perturbation, flipping, and stretching of the training video.

Images