Factory personnel dangerous behavior identification method and device, computer equipment and storage medium

Abstract

The application provides a factory personnel dangerous behavior recognition method and device, computer equipment and storage medium, belongs to the field of industrial safety monitoring, specifically includes obtaining factory real scene video data, obtaining training sample data set; the skeleton key points of dynamic behavior in the training sample data set are captured, and the enhanced multiple scale backbone features are output; the enhanced multiple scale backbone features are filtered by the equipment background, and the skeleton point features corresponding to the skeleton key points are extracted; the small size skeleton point of the distant personnel is extracted to obtain the distant skeleton point feature; the skeleton point feature, the distant skeleton point feature and the dangerous behavior label are learned and trained to obtain a dangerous behavior model; the factory real scene video data to be detected is input into the trained dangerous behavior model, and an analysis result is output. Through the processing scheme of the application, the accurate identification of various dangerous behaviors in the dynamic operation scene of the factory is effectively ensured.

Description

Technical Field

[0001] This application relates to the field of industrial safety monitoring, and in particular to a method, apparatus, computer equipment, and storage medium for identifying hazardous behaviors of factory personnel. Background Technology

[0002] Industrial sites such as manufacturing plants, chemical industrial parks, and warehousing and logistics centers present complex environments, and various equipment and operations carry inherent risks. In high-risk work environments (such as chemical, metallurgical, and heavy machinery operating areas), a person falling (e.g., fainting, injury, poisoning) can rapidly endanger their life. Running, grappling, and other dangerous behaviors in high-risk environments not only easily lead to fatal injuries but also greatly increase the risk of slipping or even being caught in equipment. However, factory areas are vast, with numerous shifts and dispersed personnel, making it difficult to cover every corner with manual patrols. Therefore, real-time monitoring of various dangerous behaviors such as falling, tailgating, running, grappling, and unnatural climbing is necessary for timely warnings. Summary of the Invention

[0003] Therefore, in order to overcome the shortcomings of the prior art, the present invention provides a method, device, computer equipment and storage medium for identifying dangerous behaviors of factory personnel that can solve the problem of insufficient multi-behavior classification ability and achieve accurate identification of multiple types of dangerous behaviors at the same time.

[0004] To achieve the above objectives, this invention provides a method for identifying hazardous behaviors of factory personnel based on an improved YOLO 10 network, comprising: acquiring real-world factory video data as training samples, and acquiring hazardous behavior labels corresponding to dynamic behaviors in the real-world factory video data, as well as multiple skeletal key points corresponding to personnel in the dynamic behaviors, to obtain a training sample dataset; capturing the skeletal key points of the dynamic behaviors in the training sample dataset using the C2f-DCNV3 module of the YOLO 10 network backbone, and outputting enhanced multi-scale backbone features; using an adaptive sparse self-attention module to filter the enhanced multi-scale backbone features against the equipment background, and applying YOLO... The neck feature layer of the YOLO10 network extracts skeletal point features corresponding to key skeletal points from the filtered skeletal features at multiple scales. The neck feature layer of the YOLO10 network is then fused with a small target augmentation pyramid to extract features from small skeletal points of distant individuals, resulting in distant skeletal point features. A task-aligned dynamic detection head is used to train the skeletal point features, the distant skeletal point features, and the hazardous behavior labels to obtain a hazardous behavior model. The real-world video data of the factory to be detected is input into the trained hazardous behavior model, and the analysis results are output.

[0005] In one embodiment, obtaining the factory scene video data as training samples includes: correcting image distortion in the factory scene video data using OpenCV; focusing on the area of ​​human activity and performing dynamic ROI cropping on the images in the corrected factory scene video data; enhancing the texture difference between the cropped image area and other areas to obtain the processed training sample image.

[0006] In one embodiment, the step of capturing the skeletal keypoints of the dynamic behavior in the training sample dataset through the C2f-DCNV3 module of the YOLO 10 network backbone and outputting enhanced multi-scale skeletal features includes: performing convolutional learning on the training sample maps of the training sample dataset through the DCNV3 module to capture the skeletal keypoints of the dynamic behavior and obtain deformable convolutional output; and fusing the deformable convolutional output with the feature residual through a BN layer and a SiLU activation function to output enhanced multi-scale skeletal features, which include S3 features focusing on detailed skeletal points, S4 features focusing on trunk bones, and S5 features focusing on global behavior.

[0007] In one embodiment, the step of using an adaptive sparse self-attention module to filter the enhanced multi-scale backbone features against the device background, and using the neck feature layer of the YOLO 10 network to extract the skeletal point features corresponding to the skeletal keypoints, includes: performing layer normalization and linear projection on the S5 features output by the backbone network to generate a projection matrix; one branch uses a sparse function to filter and remove background feature pairs with low matching scores in the background of the projection matrix to strengthen the spatial correlation between skeletal point features; another branch uses a dense function to retain the key skeletal point features of the projection matrix to avoid information loss due to excessive sparsity of the sparse function; learning normalized weights, and extracting the multi-scale backbone features after processing by the two branches to output the skeletal point features corresponding to the skeletal keypoints.

[0008] In one embodiment, the sparse function is either the ReLU function or the Sigmoid function; the dense function is either the Softmax function or the Tanh function.

[0009] In one embodiment, the method of using the neck feature layer of the YOLO 10 network to fuse small target augmentation pyramids to extract features of small-sized skeletal points of distant persons to obtain distant skeletal point features includes: performing spatial-to-depth transformation on the S2 features output by the backbone network to preserve the details of small-sized skeletal points of distant persons; fusing the transformed S2 and S3 features across scales and outputting multi-scale fused features: F3 features focusing on detailed skeletal points, F4 features focusing on torso bones, and F5 features focusing on global behavior; and extracting key region features from the fused features of the F3, F4, and F5 features to extract distant skeletal point features.

[0010] In one embodiment, the step of using a task-aligned dynamic detection head to learn and train the skeletal point features, the distant skeletal point features, and the dangerous behavior label to obtain a dangerous behavior model includes: using the task-aligned dynamic detection head to associate the skeletal point features, the distant skeletal point features, and the dangerous behavior label respectively through shared convolution and dynamic scaling layers; outputting the behavior probability distributions corresponding to each of the skeletal point features and the distant skeletal point features with the dangerous behavior label through a classification branch; training a coordinate correction function corresponding to the coordinates of the skeletal point features through a regression branch to correct the coordinates of each skeletal point in the factory real-scene video data to be detected; and obtaining the dangerous behavior model corresponding to the dangerous behavior label based on the coordinate correction function and the behavior probability distribution.

[0011] A device for identifying hazardous behaviors of factory personnel, the device comprising: a sample acquisition module, used to acquire real-world video data of the factory as training samples, and to acquire hazardous behavior labels corresponding to dynamic behaviors in the real-world video data and multiple skeletal key points corresponding to personnel in the dynamic behaviors, to obtain a training sample dataset; a backbone feature enhancement module, used to capture the skeletal key points of the dynamic behaviors in the training sample dataset through the C2f-DCNV3 module of the YOLO 10 network backbone, and output enhanced multi-scale backbone features; a skeletal point feature extraction module, used to filter the enhanced multi-scale backbone features against the device background using an adaptive sparse self-attention module, and to extract the filtered multi-scale backbone features from the skeletal key points using the neck feature layer of the YOLO 10 network; and a distant skeletal point feature extraction module, used to extract the skeletal point features corresponding to the skeletal key points using YOLO 10 network. In the network, the neck feature layer is fused with a small target enhancement pyramid to extract features from small-sized skeletal points of distant personnel, thus obtaining distant skeletal point features; the learning and training module is used to learn and train the skeletal point features, the distant skeletal point features, and the dangerous behavior labels using a task-aligned dynamic detection head to obtain a dangerous behavior model; the analysis module is used to input the factory real-scene video data to be detected into the trained dangerous behavior model and output the analysis results.

[0012] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described method.

[0013] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0014] Compared with existing technologies, the advantages of this invention are as follows: Based on an improved YOLO-trained dangerous behavior model, it enhances dynamic features through the C2f-DCNV3 module, suppresses background interference through the adaptive sparse self-attention module, improves small target detection through the small target enhancement pyramid module, and strengthens behavior-skeleton association through the task-aligned dynamic detection head module. This multi-faceted approach achieves a comprehensive improvement in detection accuracy, anti-interference capability, real-time performance, and scene adaptability, effectively ensuring accurate identification of various dangerous behaviors such as falling, tailgating, running, wrestling, and abnormal climbing in scenarios such as dynamic factory operations, long-distance monitoring, and dense equipment interference. Taking the model trained on five dangerous behaviors—falling, tailgating, running, wrestling, and abnormal climbing—as an example, the overall mAP@0.5 accuracy for these five types of dangerous behaviors can reach 96.2%, with a false negative rate of <8% and a false positive rate of <6%.

[0015] Furthermore, in response to interference from factors such as dense factory equipment, glare, and fluctuating lighting, the adaptive sparse filtering and dynamic adjustment mechanism of the attention weight in the adaptive sparse self-attention module significantly enhance the anti-interference capability of this hazardous behavior model, reducing the misjudgment rate of skeletal points in densely equipped areas from 15% to 6%.

[0016] Moreover, the number of model parameters has been reduced from 32M to 26.2M (a reduction of 18%), and the computational cost has been reduced from 80 GFLOPs to 65 GFLOPs (a reduction of 18.7%). The deployment cost is low, and it balances real-time performance with industrial applicability, making it easier to promote on a large scale in various factories. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the method for identifying hazardous behaviors of factory personnel in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the method for identifying hazardous behaviors of factory personnel in an embodiment of the present invention; Figure 3 This is a structural block diagram of the factory personnel hazardous behavior identification device in an embodiment of the present invention; Figure 4 This is an internal structural diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0019] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0020] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] It should be noted that the following description covers various aspects of embodiments within the scope of protection of this invention. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0022] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0023] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the aspects described can be practiced without these specific details.

[0024] Existing methods for detecting abnormal behavior in factories track personnel moving within a specific area using bounding boxes. However, these methods can only detect single abnormal behaviors in specific scenarios. When multiple dynamic behaviors exist, their ability to distinguish and detect these behaviors is limited. They cannot distinguish between them, cannot guarantee accurate identification, and cannot guarantee continuous detection results.

[0025] Furthermore, existing factory abnormal behavior detection methods have limited applicability to specific scenarios. For example, models trained for specific chemical plant scenarios have limited adaptability to other industrial scenarios and cannot accurately distinguish between various behaviors. Moreover, the detection accuracy of skeleton points is greatly affected by the environment. For example, changes in equipment background, lighting, and personnel occlusion can easily lead to a high false alarm rate.

[0026] In view of this, the present invention proposes a method for identifying dangerous behaviors of factory personnel: based on a dangerous behavior model trained with improved YOLO, the method enhances dynamic features through the C2f-DCNV3 module, suppresses background interference through the adaptive sparse self-attention module, improves the detection of small targets through the small target enhancement pyramid module, and strengthens the behavior-skeleton association through the task-aligned dynamic detection head module. This combination from multiple perspectives achieves a comprehensive improvement in detection accuracy, anti-interference ability, real-time performance, and scene adaptability, effectively ensuring the accurate identification of various dangerous behaviors such as falling, tailgating, running, fighting, and abnormal climbing in scenarios such as dynamic factory operations, long-distance monitoring, and dense equipment interference.

[0027] The method for identifying hazardous behaviors of factory personnel can be applied to servers or terminals. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, and portable smart devices. Servers can be implemented using independent servers or server clusters composed of multiple servers.

[0028] like Figure 1 As shown, this application provides a method for identifying hazardous behaviors of factory personnel based on an improved YOLO 10 network. Taking the application of this method to a server as an example, the method includes the following steps: Step 101: Obtain the factory real-scene video data as training samples, and obtain the dangerous behavior labels corresponding to the dynamic behavior in the factory real-scene video data, as well as the multiple skeletal key points corresponding to the personnel in the dynamic behavior, to obtain the training sample dataset.

[0029] The server acquires real-world factory video data as training samples. The server can divide the factory video data into multiple consecutive sub-video segments according to a pre-set detection interval. Each sub-video segment may or may not contain some overlapping frames with adjacent sub-videos at the beginning or end.

[0030] The server can obtain dangerous behavior labels corresponding to dynamic behaviors in the factory's real-world video data, as well as multiple skeletal key points corresponding to the personnel in the dynamic behaviors, to obtain a training sample dataset. The server can extract sub-videos corresponding to the dynamic behaviors from the factory's real-world video data and associate the sub-videos with the dangerous behavior labels. Based on the obtained multiple skeletal key points corresponding to the personnel in the dynamic behaviors, the server can annotate the skeletal key points of the personnel in each page frame of the sub-video and construct a training sample dataset. Skeletal key points can describe human skeletal information. In one embodiment, to ensure that each person in each page frame is accurately identified, each person corresponds to 17 skeletal key points, including 5 points for the head and face (nose, left eye, right eye, left ear, right ear) for head posture and facial orientation judgment, 6 points for the upper limbs (left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist) for arm movements and gesture interaction analysis, and 6 points for the lower limbs (left hip, right hip, left knee, right knee, left ankle, right ankle) for gait analysis and leg movement recognition.

[0031] Hazardous behaviors can be any abnormal behavior in a factory that is prone to causing various accidents. The number of abnormal behaviors can be adjusted according to the actual situation. Hazardous behavior labels are labels corresponding to abnormal behaviors. Specific names can be dynamic behaviors such as "falling down," "tailing," "running," "grappling," "abnormal climbing," or other specific dynamic behaviors.

[0032] Step 102: Capture the skeletal key points of dynamic behavior in the training sample dataset through the C2f-DCNV3 module of the YOLO 10 network backbone, and output the enhanced multi-scale backbone features.

[0033] The server captures skeletal keypoints of dynamic behavior in the training sample dataset using the C2f-DCNV3 module of the YOLO 10 network backbone, and outputs enhanced skeletal features at multiple scales. For example... Figure 2 As shown, the server replaces the original backbone network's C2f module with a C2f-DCNV3 module, which enhances the ability to capture skeletal features of dynamic behavior through deformable convolution (DCNV3). Figure 2 In Chinese, the full name of a Conv convolutional layer is ConvBNSiLU convolutional layer, which is usually composed of three modules: a convolutional network, batch normalization, and an activation function.

[0034] Batch normalization helps convolutional networks learn and converge better. It normalizes each mini-batch of data, making the input distribution of each layer in the convolutional network more stable. Batch normalization reduces the problems of vanishing and exploding gradients, making the network easier to train. Furthermore, batch normalization increases the network's robustness, making it more stable to changes in the input data.

[0035] Activation functions perform non-linear transformations on input data, thereby enhancing the expressive power of convolutional networks. Commonly used activation functions in convolutional neural networks include ReLU, LeakyReLU, Sigmoid, and Tanh. These activation functions can all introduce non-linear transformations by setting negative values ​​to 0 or by scaling, and they have different properties and advantages and disadvantages.

[0036] The server performs convolutional sampling on page frames representing dynamic behaviors in the training dataset using first and second Conv convolutional layers. Then, it performs deep feature learning on the skeletal features of the dynamic behaviors using the C2f-DCNV3 module (a first in this application). Next, it downsamples the skeletal features learned from the deep features of the C2f-DCNV3 module using Conv convolutional layers, and performs even deeper feature learning using the C2f-DCNV3 module (outputting S3 features, 1 / 8 resolution, focusing on detailed skeletal points). It then downsamples the skeletal features learned from the deep features of the C2f-DCNV3 module using Conv convolutional layers, and performs deep semantic feature learning using the C2f-DCNV3 module (outputting S4 features, 1 / 16 resolution, focusing on torso bones). Finally, it downsamples the skeletal features learned from the deep features of the C2f-DCNV3 module using Conv convolutional layers, and performs the deepest semantic feature learning using the C2f-DCNV3 module. Finally, it implements spatial pyramid pooling using the SPPF module (outputting S5 features, 1 / 32 resolution, focusing on global behaviors).

[0037] Step 103: The device background is filtered by the enhanced multi-scale backbone features using an adaptive sparse self-attention module, and the neck feature layer of the YOLO 10 network is used to extract the bone point features corresponding to the bone key points.

[0038] The server employs an adaptive sparse self-attention module to filter device backgrounds from enhanced multi-scale backbone features, and uses the neck feature layer of the YOLO 10 network to extract skeletal point features corresponding to skeletal keypoints from the filtered multi-scale backbone features. The server replaces the original AIFI module of YOLOv10 with an adaptive sparse self-attention module, using interactive attention features based on the adaptive sparse self-attention module to focus on the human skeletal region and filter out device background interference.

[0039] Taking the adaptive sparse self-attention module (ASSA module) as an example, the server uses the ASSA module, Concat module, Conv convolutional layer, Upsample module, and C2f module to closely cooperate through a bidirectional feature pyramid architecture to form a dual-path fusion mechanism that is both bottom-up and top-down.

[0040] In the bottom-up path, the Upsample module performs nearest-neighbor smoothing sampling on the S5 features; then the Concat module fuses the S4 features with the features processed by the Upsample module; then the C2f module performs feature learning, and the Upsample module performs nearest-neighbor smoothing sampling on the features learned by the C2f module; the Concat module fuses the S3 features with the features processed by the Upsample module, and then the C2f module performs feature learning to output the fused F3 features for the task-aligned dynamic detection head.

[0041] In the top-down path, the Conv convolutional layer performs convolutional sampling, and then the Concat module fuses the features processed by the Upsample module with the features sampled by the Conv convolutional layer; the Conv convolutional layer resamples the features fused by the Concat module, and the ASSA module learns and outputs the fused F4 features for the task-aligned dynamic detection head; the Conv convolutional layer resamples the features processed by the ASSA module, and the Concat module fuses the S5 features, and the ASSA module learns and outputs the fused F5 features for the task-aligned dynamic detection head.

[0042] Step 104: Use the neck feature layer in the YOLO 10 network to fuse the small target augmentation pyramid to extract features of small-sized skeletal points of distant people and obtain distant skeletal point features.

[0043] The server employs the neck feature layer of the YOLO 10 network, fusing the Small Object Enhanced Pyramid (SOEP) to extract features from small skeletal points of distant individuals. The server further enhances the Neck feature layer by using the SOEP module, strengthening feature fusion for distant individuals (small skeletal points). The SOEP module is a feature enhancement module specifically designed for small object detection, improving detection performance through strategies such as high-resolution preservation, cross-scale feature fusion, and detail enhancement.

[0044] The SOEP module obtains input feature data from the backbone network module and outputs the fused features to the detection head module.

[0045] Step 105: The task-aligned dynamic detection head is used to learn and train the skeletal point features, distant skeletal point features and dangerous behavior labels to obtain a dangerous behavior model.

[0046] The server uses a Task Aligned Dynamic Detection Head (TADDH) to learn and train a dangerous behavior model by combining skeletal point features, distant skeletal point features, and dangerous behavior labels. The server replaces the original YOLOv10 detection head with the Task Aligned Dynamic Detection Head, associating skeletal point features with dangerous behavior categories.

[0047] The server analyzes multiple fusion features (F3, F4, and F5 features), extracts the skeletal point features of each person in each page frame image, and learns and trains the skeletal points and dynamic behaviors of each person in each page frame image based on the dangerous behavior labels to obtain a dangerous behavior model.

[0048] Step 106: Input the real-time video data of the factory to be detected into the trained hazardous behavior model and output the analysis results.

[0049] The server inputs the real-time video data of the factory to be detected into the trained hazardous behavior model and outputs the analysis results.

[0050] When the analysis results contain dangerous behavior tags, the server outputs the analysis results and stores them along with the corresponding sub-videos for that time period. The server can also generate warning messages based on these results and send them to the monitoring terminals held by regulatory personnel for timely alerts. When the analysis results do not contain dangerous behavior tags, the server can generate safety information based on these results and send all analysis results for a given period to the monitoring terminals held by regulatory personnel at preset time points.

[0051] The above method, based on an improved YOLO-trained dangerous behavior model, enhances dynamic features through the C2f-DCNV3 module, suppresses background interference through the adaptive sparse self-attention module, improves small target detection through the small target enhancement pyramid module, and strengthens behavior-skeleton association through the task-aligned dynamic detection head module. It achieves a comprehensive improvement in detection accuracy, anti-interference ability, real-time performance, and scene adaptability from multiple perspectives, effectively ensuring the accurate identification of various dangerous behaviors such as falling, tailgating, running, fighting, and abnormal climbing in scenarios such as dynamic factory operations, long-distance monitoring, and dense equipment interference.

[0052] Furthermore, in response to interference from factors such as dense factory equipment, glare, and fluctuating lighting, the adaptive sparse filtering and dynamic adjustment mechanism of the attention weight in the adaptive sparse self-attention module significantly enhance the anti-interference capability of this hazardous behavior model, reducing the misjudgment rate of skeletal points in densely equipped areas from 15% to 6%.

[0053] Moreover, the number of model parameters has been reduced from 32M to 26.2M (a reduction of 18%), and the computational cost has been reduced from 80 GFLOPs to 65 GFLOPs (a reduction of 18.7%). The deployment cost is low, and it balances real-time performance with industrial applicability, making it easier to promote on a large scale in various factories.

[0054] In one embodiment, acquiring factory real-scene video data as training samples includes the following steps: Step 1: Correct image distortion in the factory live video data using OpenCV.

[0055] Image distortion includes radial distortion and tangential distortion. Radial distortion is caused by the shape of the lens, resulting in curved lines (barrel or pincushion distortion). Tangential distortion is caused by the lens not being parallel to the image plane.

[0056] The server can directly obtain the camera's intrinsic parameter matrix K and image distortion coefficients D (D includes radial distortion coefficients k1, k2, k3, and tangential distortion coefficients p1, p2) or determine the camera's intrinsic parameter matrix K and image distortion coefficients D through camera calibration (camera calibration is a technique that calculates and determines the parameters of the camera's imaging geometric model to establish the correspondence between three-dimensional space and two-dimensional image coordinates). This allows the server to determine the conversion relationship between physical dimensions and pixels. Then, the server uses OpenCV to correct image distortion in the factory scene video data to obtain the physical dimension data. OpenCV can use mathematical models... Describe the radial distortion of the image coordinates along the x and y axes respectively; use mathematical models. These describe the tangential distortion along the x and y axes of the image, respectively, with "distorted" indicating distortion.

[0057] Step two: Focus on the area of ​​human activity and perform dynamic ROI cropping on the images in the corrected factory live video data.

[0058] The server focuses on areas of human activity, dynamically cropping Regions of Interest (ROIs) from sub-videos within the corrected factory scene video data. The ROIs adjust in real-time based on changes in human position. The server can utilize the YOLO 10 network to detect human figures in images, extracting their bounding boxes as ROIs. This allows processing only sub-regions containing people, focusing on areas of human activity, excluding static backgrounds like equipment, and reducing unnecessary computation. Specifically, the server uses YOLO 10 to detect human figures in page frames, obtaining the detection box with the highest confidence; it then crops the ROI using the box's coordinates (x, y, w, h); and sends the cropped ROI to subsequent processing. If multiple people appear in a page frame, the server can process multiple ROIs in parallel.

[0059] Step 3: Enhance the texture difference between the image regions of the cropped factory scene video data and other regions to obtain the processed training sample image.

[0060] The server enhances the texture difference between cropped factory video data and other regions, obtaining processed training sample images. The server receives the original factory video data, automatically identifies the Regions of Interest (ROIs) in each frame, and enhances the texture difference between the ROIs and the background using algorithms to generate high-quality training sample images. The server can execute differential enhancement algorithms to process the factory video data. Enhancement processing of ROI areas: Apply unsharpened mask (USM) to enhance edge sharpness; improve local contrast (CLAHE algorithm, limiting contrast adaptive histogram equalization); moderately increase high-frequency details to enhance surface texture. Background area weakening process: Apply a light Gaussian blur to reduce texture complexity; reduce saturation and detail sharpness to make it smoother; Optimize the boundary blending of the ROI and background areas: Apply gradient blending at the boundary between the ROI and the background (5-10 pixels outside the bounding box); use Laplacian pyramid blending technology to eliminate hard edge artifacts.

[0061] In one embodiment, the server can also calculate the Texture Difference Index (TDI) between the enhanced ROI and the background to automatically evaluate the enhancement effect. When the TDI does not reach a preset threshold, the enhancement parameters are adjusted and the process is repeated.

[0062] The above method preprocesses the factory live video data, which not only improves the image quality of the data but also reduces background interference, ensuring that the ROI area where the personnel are located is accurately identified, thereby improving the ability to distinguish and analyze the factory live video data.

[0063] In one embodiment, the C2f-DCNV3 module of the YOLO 10 network backbone captures the skeletal keypoints of dynamic behavior in the training sample dataset, and outputs enhanced skeletal features at multiple scales, including: The DCNV3 module performs convolutional learning on the training sample images of the training sample dataset and captures the skeletal key points of dynamic behavior to obtain deformable convolutional output. By using a BN layer and the SiLU activation function, the deformable convolution output is fused with the feature residuals to output enhanced backbone features at multiple scales. These backbone features include S3 features focusing on detailed skeletal points, S4 features focusing on trunk bones, and S5 features focusing on global behavior.

[0064] The server performs convolutional learning on the training sample images of the training sample dataset. The training sample images can be used... (H is height, W is width, C is number of channels, and time is the time of the page frame). The dynamic sampling offset is learned through DCNV3 convolution, and the formula is: .

[0065] Where K is the number of sampling points, which is a constant, for example, K=12; w i For learnable weights, p i This is the original sampling location. This is the deformable offset (the change in the position of bone points caused by the dynamic movement of the learner).

[0066] The server fuses the deformable convolutional output with the feature residuals through a Batch Normalization (BN) layer and the SiLU activation function, outputting enhanced backbone features at multiple scales. The fusion formula is as follows: , , where X res X1 is the final output feature of the module, X is the feature tensor from the upper layer, θ is a learnable parameter, and + is a residual connection. The server outputs enhanced multi-scale backbone features through the computation of the C2f-DCNV3 module and Conv convolutional layers. The multi-scale backbone features include S3 features (1 / 8 resolution) focusing on detailed bone points, S4 features (1 / 16 resolution) focusing on trunk bones, and S5 features (1 / 32 resolution) focusing on global behavior.

[0067] The above method enhances the ability to capture skeletal features of dynamic behavior through deformable convolution (DCNV3), strengthens the representation of the relative position, angle and distance of joints, not only clarifies the spatial relationship of each person, but also enhances the continuity of skeletal point displacement in adjacent frames, reduces the interference of different body types and clothing on skeletal point recognition, and focuses on the essential features of dynamic behavior.

[0068] In one embodiment, an adaptive sparse self-attention module is used to filter the enhanced multi-scale skeleton features for device background, and the neck feature layer of the YOLO 10 network is used to extract the filtered multi-scale skeleton features to obtain the skeletal point features corresponding to the skeletal keypoints, including the following steps: The S5 features output by the backbone network are subjected to layer normalization and linear projection to generate a projection matrix; One branch filters and removes low-matching background feature pairs in the background of the projection matrix using a sparse function to enhance the spatial correlation between skeletal point features; Another branch preserves the key skeletal features of the projection matrix through a dense function to avoid information loss caused by excessive sparsity of the sparse function. By learning normalized weights, the two branches are processed and the backbone features at multiple scales are extracted and output to obtain the skeletal point features corresponding to the skeletal key points.

[0069] The server performs layer normalization and linear projection on the S5 features output by the backbone to generate a projection matrix, which contains query (Q), key (K), and value (V) matrices.

[0070]

[0071] Where, θ Q θ K θ V These are learnable parameters; ; N is the number of feature points; D is the feature dimension.

[0072] The server performs a two-branch attention computation. One branch uses a sparse function to filter out low-matching background feature pairs in the projection matrix, thereby strengthening the spatial correlation between skeletal point features. The other branch uses a dense function to preserve key skeletal point features in the projection matrix, avoiding information loss due to excessive sparsity of the sparse function. The sparse function can be either ReLU or Sigmoid, and the dense function can be either Softmax or Tanh.

[0073] Taking the sparse function, specifically the sigmoid function, as an example, the sparse branch is used to remove the background. The sigmoid activation filters out background feature pairs with low matching scores. The formula is: , where Δ is the learnable relative positional deviation, which strengthens the spatial relationship between skeletal points.

[0074] Taking the Softmax function as an example, dense functions preserve key skeletal point features and avoid information loss due to excessive sparsity. The formula is as follows: .

[0075] The server learns normalized weights α and β (α+β=1) and outputs the attention-enhanced feature AttnASSA, adapting to different background complexities. The formula is as follows: .

[0076] The above method achieves Pareto optimality through a dual-branch approach of "hard screening + soft retention," which improves the convergence speed of the training process, significantly optimizes the gradient signal-to-noise ratio, decouples the feature space to avoid foreground-background entanglement, and ultimately achieves a simultaneous leap in detection accuracy and recall in complex scenarios such as dense crowds and fast movement.

[0077] In one embodiment, the neck feature layer of the YOLO 10 network is used to fuse a small target augmentation pyramid to extract features of small-sized skeletal points of a distant person, including the following steps: The S2 features output by the backbone network are spatially to depth-oriented to preserve small-scale skeletal details of distant people. The transformed S2 and S3 features are fused across scales to output multi-scale fused features: F3 features focusing on detailed skeleton points, F4 features focusing on trunk skeletons, and F5 features focusing on global behavior. The key region features are extracted from the fused features of F3, F4 and F5 features, and the distant skeletal point features are extracted.

[0078] The server performs a spatial-to-depth transformation on the S2 features output from the backbone network to preserve the details of small skeletal points of distant figures. The server receives the S2 features (1 / 4 resolution, this application is the first to fuse these features with other YOLO 10 features) output from the second layer of the backbone network and performs SPDConv (spatial-to-depth transformation) to preserve the details of small skeletal points. SPDConv is an optimization technique for convolutional neural networks (CNNs). It decomposes standard convolution operations, breaking down spatial convolutions into multiple convolutions with smaller kernels, thereby reducing computational complexity and the number of parameters. This decomposition method not only preserves the spatial receptive field of the original convolutions but also enhances the model's ability to extract features from small targets. The server reorganizes the spatial dimension of the S2 feature map into the channel dimension, compressing the spatial information into channels through non-convolutional pixel unshuffle, and outputs the transformed feature S. 2' Specifically, the following formula can be used: SPDConv reduces downsampling loss and enhances the feature representation of small skeletal points through dimensional rearrangement. After space-depth transformation, skeletal point details are transferred from the easily lost spatial dimension to the depth channel, effectively preserving multiple times the spatial resolution and improving the integrity of small target features.

[0079] The server performs cross-scale fusion of the transformed S2 and S3 features, outputting multi-scale fused features: F3 features focusing on detailed skeleton points, F4 features focusing on trunk skeletons, and F5 features focusing on global behavior. The server first performs channel alignment: using a 1×1 convolution to... 2' The dimensions are compressed from 1024 to 512, matched to the number of S3 channels, and then cross-scale splicing is performed: concatenate along the channel dimensions of the Concat module to output 1024-dimensional mixed features; in C2f deepening processing, the fused features are extracted through 3 Bottleneck units to enhance cross-layer information interaction, and three-level feature outputs are provided: F3 features, S...2' The upsampling branch focuses on detailed skeletal points (fingers, ankles); the F4 feature, the main branch of fused features, focuses on trunk bones (shoulders, hips, knees); and the F5 feature, the S4 / S5 downsampling branch, focuses on global behavioral patterns.

[0080] The server extracts key region features from the fused features of F3, F4, and F5, extracting distant skeletal point features. The server inputs the three-level fused features F3 (256×80×80), F4 (512×40×40), and F5 (1024×10×10), employing a key region extractor architecture: Phase A: Multi-scale attention gating The F3 branch employs Detail Enhancement Attention (DEA) to enhance the edge response of skeletal points through Laplacian edge detection. The F4 branch uses coordinate attention (CA) to embed horizontal / vertical position codes for precise location of key points on the torso. The F5 branch employs global self-attention (AIFI) to capture long-distance dependencies between limbs.

[0081] Phase B: Distant Area Focusing Mechanism "Probability map of the presence of people at a distance" based on F5 global feature prediction; Use Sigmoid activation to generate a spatial weight mask (0-1 binarization). Upsample the mask to F3 / F4 resolution for use as gated key areas.

[0082] Stage C: Feature Refinement Output The F3 feature is multiplied by the mask, and only the detailed skeletal features of the distant personnel area are preserved; The F4 feature is multiplied by the mask, and only the torso skeleton features of people in the distance are preserved; By using C2f cross-layer fusion, a purified feature map of distant skeletal points is output.

[0083] Output results, which include distant skeletal features, trunk skeletal features, and global behavioral features.

[0084] The distant skeletal features include a fine-scale heatmap of 17 keypoints (80×80 resolution). The torso skeletal features include a medium-resolution heatmap of 5 torso keypoints (40×40). The global behavioral features include a semantic vector of behavioral category probabilities (10×10 resolution).

[0085] The server will S 2' The fusion with S3 via the CSP-OmniKernel module (including global, local, and large receptive field branches) is as follows: .

[0086] Where Cat represents channel splicing, and θ Omni The learnable parameters for the OmniKernel branch are used to finally output multi-scale fusion features: F3, F4, and F5 features (corresponding to 1 / 8, 1 / 16, and 1 / 32 resolutions).

[0087] The above method preserves minute skeletal details through space-depth transformation, which significantly improves the detection rate of distant people and significantly improves the accuracy of key details. Moreover, cross-scale fusion combines high-resolution details with deep semantic information, avoiding the dilution of small target features by deep layers.

[0088] In one embodiment, a task-aligned dynamic detection head is used to learn and train skeletal point features, distant skeletal point features, and dangerous behavior labels to obtain a dangerous behavior model, including: A task-aligned dynamic detection head is adopted to associate skeletal point features, distant skeletal point features, and dangerous behavior labels through shared convolution and dynamic scaling layers. The classification branch outputs the probability distribution of behavior corresponding to dangerous behavior labels for each skeleton point feature and the distant skeleton point feature; The coordinate correction function corresponding to the feature coordinates of the skeleton points is trained by the regression branch in order to correct the coordinates of each skeleton point in the factory scene video data to be detected. Based on the coordinate correction function and the behavior probability distribution, a dangerous behavior model corresponding to the dangerous behavior label is obtained.

[0089] Based on the YOLO network architecture, the server constructs a Target-Aware Dual Detection Head (TADDH module) to extract multi-scale features through shared convolutional layers. This head includes a dynamic scaling layer that adaptively adjusts the feature scale. Skeleton point features (near-field worker keypoints) and distant-field worker keypoints are input into the shared convolutional network, and the dynamic scaling layer normalizes the scale of features from workers at different distances. The server performs RoI Align on the fused F3, F4, and F5 features to extract the skeletal point features for each worker. (17 key points, each corresponding to a D-dimensional feature): .

[0090] Here, bbox is the bounding box of the person target, which is generated by clustering skeletal points.

[0091] The server associates the processed features with dangerous behavior labels (such as "falling down," "tailgating," "running," "grappling," "abnormal climbing," etc.) to establish a preliminary feature-label correspondence. The server learns the association features between skeletal points and behaviors through shared convolutional and dynamic scaling layers. , where γ t This is a dynamic scaling factor (adaptively adjusted according to the behavior category, such as increasing the weight of the trunk skeleton in "fighting" behavior).

[0092] The server outputs the probability distributions of each skeleton feature and distant skeleton features, respectively, along with the corresponding dangerous behavior labels, through a classification branch. The classification branch employs a decoupled head design to classify and predict the skeleton features. The output layer can use the Softmax activation function. , , where P cls It is the classification probability, Δ kp These are the coordinates of the skeletal points, F int It is a high-dimensional feature tensor after feature fusion, where θ is a learnable parameter, generating the probability distribution of each skeletal point feature and the corresponding dangerous behavior label. For example, for the label "falling down", the output probability of the skeletal point feature of the nearby worker's head belonging to this category is 0.92, and the corresponding feature probability of the worker at a distance is 0.85, forming a complete probability distribution matrix.

[0093] The server trains the coordinate correction function through a regression branch. This branch takes the feature coordinates of skeletal points (such as the coordinates of the neck keypoints (x, y)) as input, uses the CIoU loss function for supervised training, and constructs the coordinate correction function Δ(x, y) = f(original coordinates, feature scale, distance parameter), where f is a lightweight fully connected network. During training, labeled skeletal points from real-world factory video data are used as ground truth to correct coordinate deviations caused by perspective distortion and occlusion. For example, for a worker 15 meters away from the camera, their original detection coordinates (320, 240) are adjusted to (318, 238) after the correction function, improving long-distance detection accuracy.

[0094] The server integrates coordinate correction functions and behavior probability distributions to construct the final hazardous behavior model. The output of the hazardous behavior model includes: ① the precise skeleton point positions after coordinate correction; ② the probability distribution of each behavior category; and ③ the confidence score (≥0.8 indicates a valid warning). For input factory scene video frames, the model first extracts YOLO backbone network features, processes them through a dynamic detection head, and outputs the corrected skeleton point coordinates and their corresponding hazardous behavior category probabilities. This ultimately forms detection results that correspond one-to-one with hazardous behavior labels, enabling real-time identification of hazardous behaviors of workers at multiple distances and in various postures within a factory environment.

[0095] The above method achieves collaborative optimization of classification and localization tasks through a task-aligned dynamic detection head, and effectively solves the problem of decreased accuracy in long-distance target detection in factory scenarios by utilizing a coordinate correction function, thereby improving the accuracy and robustness of dangerous behavior detection.

[0096] When the working steps of this application embodiment are executed according to "data preprocessing → feature extraction → attention interaction → multi-scale fusion → behavior classification → early warning output", the precise improvements of "C2f-DCNV3 module to enhance dynamic features, ASSA module to suppress background interference, SOEP module to improve small target detection, and TADDH module to enhance behavior-skeleton association" have the following effects: 1. Significantly Improved Detection Accuracy: The C2f-DCNV3 module enhances dynamic skeleton capture rate, the ASSA module focuses on effective features, the SOEP module strengthens the representation of long-distance skeleton points, and the TADDH module optimizes multi-behavior differentiation. The overall mAP@0.5 for five types of dangerous behaviors reaches 96.2% (mAP@0.5, or mean Average Precision at IoU threshold 0.5, refers to the average precision calculated when the Intersection over Union (IoU) threshold is set to 0.5. IoU measures the overlap between the predicted bounding box and the ground truth bounding box, calculated as: IoU = Intersection area of ​​predicted and ground truth boxes / Union area of ​​predicted and ground truth boxes. When IoU ≥ 0.5, the prediction result is considered "correct". AP value is obtained by plotting the Precision-Recall curve and calculating the area under the curve (AUC). The average AP value for all categories is the mAP value. mAP reflects the model's average performance across all categories), with a false negative rate of <8% and a false positive rate of <6%.

[0097] 2. Significantly enhanced anti-interference capability: In response to interference such as dense factory equipment, reflection, and light fluctuations, the ASSA module's adaptive sparse filtering and dynamic attention weight adjustment mechanism reduces the misjudgment rate of skeletal points in densely equipped areas from 15% to 6%.

[0098] 3. Balancing real-time performance with industrial applicability: The number of model parameters has been reduced from 32M to 26.2M (a reduction of 18%), and the computational cost has been reduced from 80GFLOPs to 65GFLOPs (a reduction of 18.7%). The deployment cost is low and it is easy to promote on a large scale in various factories.

[0099] In one embodiment, such as Figure 3As shown, a device for identifying hazardous behaviors of factory personnel is provided. The device includes a sample acquisition module 301, a backbone feature enhancement module 302, a skeletal point feature extraction module 303, a distant skeletal point feature extraction module 304, a learning and training module 305, and an analysis module 306.

[0100] The sample acquisition module 301 is used to acquire factory real-scene video data as training samples, and to acquire dangerous behavior labels corresponding to dynamic behaviors in the factory real-scene video data, as well as multiple skeletal key points corresponding to personnel in the dynamic behaviors, to obtain a training sample dataset.

[0101] The backbone feature enhancement module 302 is used to capture the skeletal key points of dynamic behavior in the training sample dataset through the C2f-DCNV3 module of the YOLO 10 network backbone, and output the enhanced backbone features at multiple scales.

[0102] The skeletal point feature extraction module 303 is used to filter the enhanced multi-scale skeletal features using an adaptive sparse self-attention module, and to extract the skeletal point features corresponding to the skeletal key points using the neck feature layer of the YOLO 10 network.

[0103] The distant skeletal point feature extraction module 304 is used to extract the features of small-sized skeletal points of distant people by fusing the neck feature layer of the YOLO 10 network with the small target augmentation pyramid.

[0104] The learning and training module 305 is used to learn and train the skeletal point features, distant skeletal point features and dangerous behavior labels using a task-aligned dynamic detection head to obtain a dangerous behavior model.

[0105] Analysis module 306 is used to input the real-time video data of the factory to be detected into the trained hazardous behavior model and output the analysis results.

[0106] In one embodiment, the sample acquisition module includes the following units: The distortion correction unit is used to correct image distortion in factory scene video data using OpenCV.

[0107] The ROI cropping unit is used to focus on the area of ​​human activity and perform dynamic ROI cropping on images in the corrected factory live video data.

[0108] The enhancement unit is used to enhance the texture difference between the image regions of the cropped factory scene video data and other regions, resulting in a processed training sample image.

[0109] In one embodiment, the backbone feature enhancement module includes: The convolutional output unit is used to perform convolutional learning on the training sample maps of the training sample dataset through the DCNV3 module, capture the skeletal key points of dynamic behavior, and obtain deformable convolutional output.

[0110] The backbone feature output unit is used to fuse deformable convolution output with feature residuals through BN layer and SiLU activation function to output enhanced backbone features at multiple scales. These backbone features include S3 features focusing on detailed skeletal points, S4 features focusing on trunk bones, and S5 features focusing on global behavior.

[0111] In one embodiment, the skeletal point feature extraction module includes: The projection unit is used to perform layer normalization and linear projection on the S5 features output by the backbone network to generate a projection matrix.

[0112] The sparse unit is used in one branch to filter and remove low-matching background feature pairs in the background of the projection matrix through a sparse function, so as to strengthen the spatial correlation between skeletal point features.

[0113] Dense units are used in another branch to preserve key skeletal features of the projection matrix through a dense function, in order to avoid information loss due to excessive sparsity of the sparse function.

[0114] The branch merging unit is used to learn normalized weights and extracts the skeleton point features corresponding to the skeleton key points after processing the two branches at multiple scales.

[0115] In one embodiment, the distant skeletal point feature extraction module includes: The depth transformation unit is used to perform spatial-to-depth transformation on the S2 features output by the backbone network to preserve small-scale skeletal details of distant people.

[0116] The cross-scale fusion unit is used to fuse the transformed S2 and S3 features across scales and output multi-scale fused features: F3 features focusing on detail skeleton points, F4 features focusing on trunk skeletons, and F5 features focusing on global behavior.

[0117] The fusion feature extraction unit is used to extract key region features from the fusion features of F3, F4 and F5 features, and to extract distant skeletal point features.

[0118] In one embodiment, the learning and training module includes: The association unit is used to associate skeletal point features, distant skeletal point features, and dangerous behavior labels separately by using a task-aligned dynamic detection head through shared convolution and dynamic scaling layers.

[0119] The probability distribution unit is used to output the probability distribution of each skeleton point feature and the distant skeleton point feature with the dangerous behavior label through the classification branch.

[0120] The coordinate correction unit is used to train a coordinate correction function corresponding to the feature coordinates of the skeleton points through a regression branch, so as to correct the coordinates of each skeleton point in the factory scene video data to be detected.

[0121] The model training unit is used to obtain a dangerous behavior model corresponding to the dangerous behavior label based on the coordinate correction function and the behavior probability distribution.

[0122] Specific limitations regarding the factory personnel hazardous behavior identification device can be found in the limitations of the factory personnel hazardous behavior identification method described above, and will not be repeated here. Each module in the aforementioned factory personnel hazardous behavior identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0123] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data such as real-time factory video data. The network interface communicates with external terminals via a network. When executed by the processor, the computer program implements a method for identifying hazardous behaviors of factory personnel.

[0124] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring factory real-scene video data as training samples, and acquiring dangerous behavior labels corresponding to dynamic behaviors in the factory real-scene video data and multiple skeletal keypoints corresponding to personnel in the dynamic behaviors, to obtain a training sample dataset; capturing the skeletal keypoints of dynamic behaviors in the training sample dataset using the C2f-DCNV3 module of the YOLO 10 network backbone, and outputting enhanced multi-scale backbone features; using an adaptive sparse self-attention module to filter the enhanced multi-scale backbone features for device background, and using the neck feature layer of the YOLO 10 network to extract the filtered multi-scale backbone features to obtain skeletal point features corresponding to the skeletal keypoints; and using YOLO... In the 10 network, the neck feature layer is fused with a small target augmentation pyramid to extract features from small-sized skeletal points of distant personnel. The skeletal point features are obtained by extracting features from distant skeletal points. A task-aligned dynamic detection head is used to learn and train the skeletal point features, distant skeletal point features and dangerous behavior labels to obtain a dangerous behavior model. The real-world video data of the factory to be detected is input into the trained dangerous behavior model and the analysis results are output.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: acquiring factory real-scene video data as training samples, and acquiring dangerous behavior labels corresponding to dynamic behaviors in the factory real-scene video data and multiple skeletal keypoints corresponding to personnel in the dynamic behaviors, to obtain a training sample dataset; capturing the skeletal keypoints of dynamic behaviors in the training sample dataset through the C2f-DCNV3 module of the YOLO 10 network backbone, and outputting enhanced multi-scale backbone features; using an adaptive sparse self-attention module to filter the enhanced multi-scale backbone features for device background, and using the neck feature layer of the YOLO 10 network to extract the filtered multi-scale backbone features to obtain skeletal point features corresponding to the skeletal keypoints; using the neck feature layer of the YOLO 10 network to fuse small target augmentation pyramids to extract features of small-sized skeletal points of distant personnel to obtain distant skeletal point features; using a task-aligned dynamic detection head to learn and train the skeletal point features, distant skeletal point features, and dangerous behavior labels to obtain a dangerous behavior model; inputting the factory real-scene video data to be detected into the trained dangerous behavior model and outputting the analysis results.

[0127] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for identifying hazardous behaviors of factory personnel based on an improved YOLO 10 network, characterized in that, include: Obtain real-world video data of the factory as training samples, and obtain dangerous behavior labels corresponding to dynamic behaviors in the real-world video data of the factory, as well as multiple skeletal key points corresponding to personnel in the dynamic behaviors, to obtain the training sample dataset. The C2f-DCNV3 module of the YOLO 10 backbone network is used to capture the skeletal key points of the dynamic behavior in the training sample dataset and output the enhanced multi-scale backbone features. An adaptive sparse self-attention module is used to filter the enhanced multi-scale backbone features for device background, and the neck feature layer of the YOLO 10 network is used to extract the filtered multi-scale backbone features to obtain the bone point features corresponding to the bone key points. The neck feature layer in the YOLO 10 network is used to fuse a small target augmentation pyramid to extract features of small-sized skeletal points of distant people, thus obtaining distant skeletal point features; A task-aligned dynamic detection head is used to learn and train the skeletal point features, the distant skeletal point features, and the dangerous behavior labels to obtain a dangerous behavior model. The real-world video data of the factory to be detected is input into the trained hazardous behavior model, and the analysis results are output.

2. The method for identifying hazardous behaviors of factory personnel according to claim 1, characterized in that, The acquisition of factory real-scene video data as training samples includes: Image distortion in the factory real-scene video data was corrected using OpenCV; Focusing on areas of human activity, dynamic ROI cropping is performed on images in the corrected factory real-scene video data; Enhance the texture difference between the image regions of the cropped factory scene video data and other regions to obtain the processed training sample image.

3. The method for identifying hazardous behaviors of factory personnel according to claim 1, characterized in that, The C2f-DCNV3 module of the YOLO 10 network backbone captures the skeletal keypoints of the dynamic behavior in the training sample dataset, and outputs enhanced skeletal features at multiple scales, including: The DCNV3 module performs convolutional learning on the training sample images of the training sample dataset and captures the skeletal key points of dynamic behavior to obtain deformable convolutional output. By using a BN layer and the SiLU activation function, the deformable convolution output is fused with the feature residuals to output enhanced backbone features at multiple scales. These backbone features include S3 features focusing on detailed skeletal points, S4 features focusing on trunk bones, and S5 features focusing on global behavior.

4. The method for identifying hazardous behaviors of factory personnel according to claim 1, characterized in that, The process employs an adaptive sparse self-attention module to filter the enhanced multi-scale skeleton features from the device background, and uses the neck feature layer of the YOLO 10 network to extract the skeleton point features corresponding to the skeletal keypoints, including: The S5 features output by the backbone network are subjected to layer normalization and linear projection to generate a projection matrix; One branch filters and removes low-matching background feature pairs in the background of the projection matrix using a sparse function to enhance the spatial correlation between skeletal point features; Another branch preserves the key skeletal point features of the projection matrix through a dense function to avoid information loss due to excessive sparsity of the sparse function. By learning normalized weights, the two branches are processed and the backbone features at multiple scales are extracted and output to obtain the skeletal point features corresponding to the skeletal key points.

5. The method for identifying hazardous behaviors of factory personnel according to claim 4, characterized in that, The sparse function is either the ReLU function or the Sigmoid function; The dense function can be either the Softmax function or the Tanh function.

6. The method for identifying hazardous behaviors of factory personnel according to claim 1, characterized in that, The method employs the neck feature layer of the YOLO 10 network to fuse small target augmentation pyramids and extract features from small-sized skeletal points of distant individuals, including: The S2 features output by the backbone network are spatially to depth-oriented to preserve small-scale skeletal details of distant people. The transformed S2 and S3 features are fused across scales to output multi-scale fused features: F3 features focusing on detailed skeleton points, F4 features focusing on trunk skeletons, and F5 features focusing on global behavior. Key region features are extracted from the fused features of F3, F4, and F5, and distant skeletal point features are extracted.

7. The method for identifying hazardous behaviors of factory personnel according to claim 1, characterized in that, The step of using a task-aligned dynamic detection head to learn and train the skeletal point features, the distant skeletal point features, and the dangerous behavior labels to obtain a dangerous behavior model includes: A task-aligned dynamic detection head is used to associate the skeletal point features, the distant skeletal point features, and the dangerous behavior labels through shared convolution and dynamic scaling layers. The classification branch outputs the probability distribution of behavior corresponding to each of the skeletal point features and the distant skeletal point features, respectively, and the dangerous behavior label. The coordinate correction function corresponding to the feature coordinates of the skeleton points is trained by the regression branch in order to correct the coordinates of each skeleton point in the factory scene video data to be detected. Based on the coordinate correction function and the behavior probability distribution, a dangerous behavior model corresponding to the dangerous behavior label is obtained.

8. A device for identifying hazardous behaviors of factory personnel, characterized in that, The device includes: The sample acquisition module is used to acquire factory real-scene video data as training samples, and to acquire dangerous behavior labels corresponding to dynamic behaviors in the factory real-scene video data, as well as multiple skeletal key points corresponding to personnel in the dynamic behaviors, to obtain the training sample dataset. The backbone feature enhancement module is used to capture the skeletal key points of the dynamic behavior in the training sample dataset through the C2f-DCNV3 module of the YOLO 10 network backbone, and output the enhanced backbone features at multiple scales. The skeletal point feature extraction module is used to filter the enhanced multi-scale skeletal features using an adaptive sparse self-attention module, and to extract the skeletal point features corresponding to the skeletal key points using the neck feature layer of the YOLO 10 network. The distant skeletal point feature extraction module is used to extract the features of small-sized skeletal points of distant people by fusing the neck feature layer of the YOLO 10 network with the small target augmentation pyramid. The learning and training module is used to learn and train the skeletal point features, the distant skeletal point features and the dangerous behavior labels using a task-aligned dynamic detection head to obtain a dangerous behavior model. The analysis module is used to input the real-time video data of the factory to be detected into the trained hazardous behavior model and output the analysis results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images