Mine personnel behavior recognition method and system based on YOLOv5

Abstract

The application relates to the technical field of image processing, and discloses a mine personnel behavior recognition method and system based on YOLOv5. The method comprises the following steps: acquiring a video frame sequence of a mine site, extracting an environmental interference vector from the video frame sequence, performing optimization processing on the video frame sequence based on the environmental interference vector, and obtaining a clear frame sequence; extracting human body key point coordinates and an attitude vector from the clear frame sequence, analyzing time sequence changes of the key point coordinates, and obtaining preliminary action features; calculating a similarity score of the preliminary action features and a preset standard template, refining the preliminary action features based on the similarity score, and obtaining a refined action component vector; matching the refined action component vector from a preset action library, obtaining an action recognition result, and generating a corresponding action response output based on the action recognition result. The application improves the accuracy of personnel action recognition in a complex mine environment.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for recognizing the behavior of miners based on YOLOv5. Background Technology

[0002] In the complex environment of mining sites, motion recognition technology faces a core technical challenge: how to accurately and robustly recognize human movements under dynamic environmental factors such as uneven lighting and dust interference to support safety monitoring and production efficiency optimization. This problem stems from the unique business scenarios in mining sites, where video frame sequences often suffer from image quality degradation due to drastic changes in lighting and rapid dust diffusion, making motion feature extraction difficult and reducing motion recognition accuracy, thus affecting the timeliness of safety warnings and the reliability of equipment control commands. Specifically, uneven lighting causes fluctuations in the brightness and contrast of video frames, making key point detection susceptible to interference and making it difficult to stably extract time-series features of human movements; dust diffusion further blurs the image, increasing the difficulty of noise reduction and leading to inaccurate refinement of motion component vectors. In addition, the dynamic changes in environmental interference conflict with the real-time requirements of motion recognition: rapidly changing environments require complex preprocessing algorithms to eliminate interference, but real-time requirements limit the computational complexity of the algorithms; at the same time, motion recognition needs to perform high-precision matching with a preset motion library, but the similarity calculation between environmental interference vectors and standard templates is easily interfered with and difficult to adaptively adjust. These seemingly minor issues collectively constitute the core challenge of robustness and accuracy in action recognition under complex environments. Especially in mining sites, misjudgments or omissions in action recognition can directly lead to safety accidents or a decline in production efficiency, highlighting the uniqueness and importance of this technical problem. Summary of the Invention

[0003] To address the aforementioned technical issues, this application provides a method and system for identifying personnel behavior in mines based on YOLOv5, which aims to improve the accuracy of personnel action recognition in complex mining environments.

[0004] Firstly, this application provides a method for identifying the behavior of miners based on YOLOv5, the method comprising:

[0005] Step S1: Obtain a video frame sequence of the mine site, extract an environmental interference vector from the video frame sequence, and optimize the video frame sequence based on the environmental interference vector to obtain a clear frame sequence;

[0006] Step S2: Extract the coordinates of human key points and the pose vector from the clear frame sequence, analyze the time series changes of the key point coordinates, and obtain preliminary motion features;

[0007] Step S3: Calculate the similarity score between the preliminary action features and the preset standard template, and refine the preliminary action features based on the similarity score to obtain a refined action component vector;

[0008] Step S4: Match the refined action component vector from the preset action library to obtain the action recognition result, and generate the corresponding action response output based on the action recognition result.

[0009] In conjunction with the first aspect, in a first implementation of the first aspect of this application, optimizing the video frame sequence based on the environmental interference vector includes:

[0010] Analyze the uniform illumination distribution and dust particle concentration of each frame in the video frame sequence; calculate the dynamic range of illumination based on the uniform illumination distribution, and calculate the dust diffusion rate based on the dust particle concentration; fuse the dynamic range of illumination and the dust diffusion rate to generate the environmental interference vector;

[0011] The video frame sequence is processed using a convolutional neural network, and the illumination color temperature deviation is adjusted according to the illumination dynamic range in the environmental interference vector.

[0012] Based on the dust diffusion rate in the environmental interference vector, the contrast of the video image is adjusted to obtain a clear frame sequence.

[0013] In conjunction with the first aspect, in the second implementation of the first aspect of this application, adjusting the contrast of the video image based on the dust diffusion rate in the environmental interference vector includes:

[0014] By applying an optical flow field estimation algorithm to multiple consecutive video images, the displacement vector of pixels in the dust region is obtained;

[0015] The direction and velocity distribution of dust movement are calculated based on the statistical results of the displacement vectors.

[0016] The contrast enhancement coefficient is adaptively set according to the dust density gradient, and the local area is subjected to regional histogram equalization to enhance the overall picture and highlight the edge features of dust particles.

[0017] In conjunction with the first aspect, in the third implementation of the first aspect of this application, the time series changes of the key point coordinates are analyzed to obtain preliminary action features, including:

[0018] A target detection model based on YOLOv5 is established to detect and locate human body regions in the clear frame sequence and extract target bounding boxes.

[0019] Input the target bounding box into the pose estimation network model to obtain the coordinates of human key points and the pose vector.

[0020] The continuous time series of the key point coordinates is input into the recurrent neural network model to analyze the changes in the displacement and velocity of the key points and output the dynamic characteristics of the action evolution over time.

[0021] Obtain the real-time acquisition frequency, and multiply the dynamic feature by the acquisition frequency to obtain a weighted dynamic value;

[0022] If the weighted dynamic value is greater than a preset threshold, the action corresponding to the dynamic feature is marked as a significant action, and preliminary action features are generated.

[0023] In conjunction with the first aspect, in the fourth implementation of the first aspect of this application, calculating the similarity score between the preliminary action features and the preset standard template includes:

[0024] A standardized set of posture vectors is extracted from the mine motion recognition database as a preset standard template, and the initial similarity between the preliminary motion features and the preset standard template is calculated.

[0025] If the initial similarity is lower than a preset threshold, the weight coefficient of each action component in the preliminary action feature is adjusted according to the uniform distribution of illumination and the concentration of dust particles in the environmental interference vector.

[0026] The similarity score is generated adaptively based on the updated weight coefficients.

[0027] In conjunction with the first aspect, in the fifth implementation of the first aspect of this application, the preliminary action features are refined based on the similarity score, including:

[0028] Determine whether the similarity score is greater than the target threshold. If so, determine that the preliminary action feature matches the preset template.

[0029] Otherwise, the preliminary action features are decomposed into multiple action components, each corresponding to a time segment in the action sequence;

[0030] Extract key regions from the decomposed motion components. The key regions are the parts that can reflect differences in gestures or subtle changes in posture.

[0031] Weights are assigned to the key regions using an attention mechanism to generate an attention-weighted vector;

[0032] The target threshold is adaptively adjusted based on the environmental interference vector;

[0033] Based on the similarity score, a dynamic gain coefficient is calculated, and the key features in the attention weighting vector are corrected and amplified to obtain a refined motion component vector, which is a local-level feature of velocity, angle, and trajectory change.

[0034] In conjunction with the first aspect, in the sixth implementation of the first aspect of this application, calculating the dynamic gain coefficient based on the similarity score includes:

[0035] The similarity score is input into a combined model of a linear mapping function and an exponential smoothing function to obtain a gain factor. If the similarity score is less than a first threshold, the gain factor is automatically increased; if the similarity score is greater than the first threshold, the gain factor is decreased.

[0036] In conjunction with the first aspect, in the seventh implementation of the first aspect of this application, matching the refined action component vector from a preset action library to obtain the action recognition result includes:

[0037] A preset action library is constructed based on historical mine video data, and the preset action library includes hand waving signals and crouching inspection action patterns.

[0038] The refined action component vector is matched with the preset action library, and the action recognition result is generated by combining the update iteration cycle and weight normalization processing.

[0039] In conjunction with the first aspect, in the eighth implementation of the first aspect of this application, generating a corresponding action response output based on the action recognition result includes:

[0040] Based on the action recognition results, generate safety warning signals or production efficiency optimization instructions;

[0041] If the action recognition result indicates an abnormal action, extract the multi-dimensional vector from the refined action component vector of the abnormal action, assign dimension coefficients to the multi-dimensional vector, and perform weighted fusion on the multi-dimensional vector based on the dimension coefficients to obtain a fused vector;

[0042] The stability of the dimensional coefficients is verified by calculating the variance of each dimensional coefficient. If the variance is lower than a preset threshold, the dimensional coefficient is determined to be stable. Otherwise, the dimensional coefficients are iteratively adjusted until they are stable, and an instruction to trigger device operation adjustment is generated.

[0043] An action response output is generated based on the adjustment instructions, and the action response output is a device control signal.

[0044] Secondly, this application provides a YOLOv5-based system for recognizing the behavior of miners, the system comprising:

[0045] The acquisition module acquires a video frame sequence from the mine site, extracts an environmental interference vector from the video frame sequence, and optimizes the video frame sequence based on the environmental interference vector to obtain a clear frame sequence.

[0046] The analysis module extracts the coordinates of key human points and the posture vector from the clear frame sequence, analyzes the time series changes of the key point coordinates, and obtains preliminary motion features.

[0047] The optimization module calculates the similarity score between the preliminary action features and the preset standard template, and refines the preliminary action features based on the similarity score to obtain a refined action component vector.

[0048] The output module matches the refined action component vector from the preset action library to obtain the action recognition result, and generates the corresponding action response output based on the action recognition result.

[0049] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0050] This application optimizes video frame sequences by introducing environmental interference vectors, effectively addressing image blurring and feature loss caused by uneven lighting and dust interference in complex mining environments, ensuring clearer and more stable input data for subsequent action recognition. Secondly, in the human action feature extraction stage, by combining YOLOv5 target detection and pose estimation methods, it can quickly and accurately locate human regions and obtain key point coordinates in real-time video streams. Then, by analyzing time series changes through recurrent neural networks, it accurately characterizes the dynamic characteristics of actions, making it particularly suitable for analyzing the continuity and subtle differences of actions such as waving signals and crouching down for inspections by mining workers.

[0051] Furthermore, during the similarity calculation process, the weights of action features are adaptively adjusted using environmental interference vectors. This not only improves the accuracy of the similarity score but also avoids misjudgments and missed judgments in low-light and high-dust environments. Simultaneously, by refining the action component vectors, focusing on gesture differences and subtle posture changes using an attention mechanism, and combining this with dynamic gain coefficients to enhance local features, the recognition results are made more accurate. In the action recognition output stage, through pre-set action library matching and weight normalization, the system can dynamically update the recognition model, ensuring high-precision recognition in different work scenarios. It can also generate safety warning signals or production optimization instructions in a timely manner based on the recognition results. If abnormal actions are detected, reliable control instructions can be generated through dimensional fusion and stability verification, directly affecting equipment operation and achieving dual protection of safety and efficiency. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of an embodiment of the YOLOv5-based method for identifying the behavior of miners in this application.

[0054] Figure 2 This is a flowchart detailing the preliminary action features in the embodiments of this application;

[0055] Figure 3 This is a flowchart of the action response output in the embodiments of this application;

[0056] Figure 4 This is a schematic diagram of an embodiment of the YOLOv5-based mine personnel behavior recognition system in this application. Detailed Implementation

[0057] This application provides a method and system for identifying the behavior of miners based on YOLOv5. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0058] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the YOLOv5-based method for identifying the behavior of miners in this application includes:

[0059] Step S1: Obtain the video frame sequence of the mine site, extract the environmental interference vector from the video frame sequence, and optimize the video frame sequence based on the environmental interference vector to obtain a clear frame sequence.

[0060] Specifically, video surveillance at mine sites is primarily used for behavior recognition and safety monitoring. However, due to environmental interference, video quality is often low, especially in underground mines where uneven lighting and high dust concentrations severely affect image clarity and effective information extraction. Therefore, to ensure the accuracy of subsequent action recognition, video frame sequences must be preprocessed to reduce the impact of environmental interference. A fixed camera system with a resolution of 1920×1080 is deployed at the mine site. The cameras are mounted on the mine tunnel walls, and appropriate exposure settings are used to adapt to the low-light environment (exposure time set to 1 / 50 second). A video frame sequence of 30 frames per second is acquired. By analyzing each frame, the uniformity of light distribution and dust particle concentration are analyzed, the dynamic range of light and dust diffusion speed are calculated, and an environmental interference vector is generated. Based on the environmental interference vector, the light color temperature deviation and contrast of the video frames are adjusted. A convolutional neural network is used to denoise the video frames, enhancing image clarity. The denoising and contrast enhancement results are then fused to generate an optimized, clear frame sequence for subsequent action recognition.

[0061] Step S2: Extract the coordinates and pose vectors of human body key points from the clear frame sequence, analyze the time series changes of key point coordinates, and obtain preliminary motion features.

[0062] Specifically, the pre-processed clear frame sequence is processed frame by frame, and the coordinates of human key points are extracted using a pose estimation algorithm. Key points include the head, shoulders, elbows, wrists, hips, knees, and ankles. The coordinates are represented in the form of two-dimensional pixels or three-dimensional spatial coordinates, and a pose vector is formed by connecting them to describe the overall posture of the human body. Under dusty or uneven lighting conditions, the key point detection results are corrected by combining environmental interference vectors to ensure the accuracy of skeleton representation. Next, the obtained key point sequence is input into a recurrent neural network for analysis to capture the evolution of the action over time and identify significant actions. For example, in a waving action, the continuous raising and lowering of the arm spans multiple frames. The network can recognize this dynamic change rather than judging it based on the position of a single frame. To avoid misidentifying minute jitters or invalid movements as significant actions, correction is required by combining real-time acquisition frequency and preset quantization thresholds. This effectively filters out subtle disturbances caused by environmental noise, ensuring the reliability of feature descriptions. These filtered significant actions are then integrated into preliminary action features. These features not only include the temporal patterns output by the recurrent neural network but also quantified indicators such as velocity, displacement, and angle changes. This ensures that the overall dynamics are reflected while possessing good interpretability and anti-interference capabilities. In environments like mines, where lighting and dust interference are severe, this approach significantly improves the accuracy and robustness of action recognition, providing reliable input for subsequent similarity calculations and action database matching.

[0063] Step S3: Calculate the similarity score between the preliminary action features and the preset standard template, and refine the preliminary action features based on the similarity score to obtain the refined action component vector.

[0064] Specifically, at the mine site, a pre-set standard template is extracted from the mine action recognition database. This template contains common standard action features found in mines. The standard action features are compared with the collected preliminary action features, and a similarity score is calculated. If the similarity score is below a pre-set threshold (e.g., 0.7), it is considered that there is a significant difference between the preliminary action features and the template, requiring further refinement. During refinement, the weight coefficients of each action in the preliminary action features are dynamically adjusted based on the uniformity of illumination and dust particle concentration in the environmental interference vector. For example, when illumination is uneven and dust concentration is high, the reliance on gesture details is reduced, while the focus on overall posture (such as large-scale upper or lower limb movements) is increased. This dynamic adjustment helps avoid mismatches caused by local interference and improves recognition accuracy in complex environments. After weight adjustment, the similarity score is recalculated, and the final refined action component vector is derived based on the actual situation. This vector includes the temporal dynamic characteristics of the action, such as duration, speed, and displacement amplitude. These features describe the time span and amplitude changes of the action. The matching process of motion features is adaptively adjusted according to environmental changes. Even under complex conditions such as uneven lighting and high dust concentration, it can still ensure high-precision motion recognition and improve the robustness of mine safety monitoring.

[0065] Step S4: Match and refine the action component vectors from the preset action library to obtain the action recognition result, and generate the corresponding action response output based on the action recognition result.

[0066] Specifically, a preset action library is established, which stores standardized action patterns, such as "waving signal" and "squatting check". Each action pattern consists of multiple dimensions, such as time span, joint angle range, and trajectory direction. The extracted refined action component vectors are matched with the standard patterns in the action library, and the Euclidean distance is calculated. When the Euclidean distance is less than a preset threshold (e.g., 0.3), a matching relationship is considered to exist. An initial matching score is obtained based on the distance. After the matching is completed, the matching results need to be corrected by combining the update iteration cycle and weight normalization. The detailed correction process will be explained later. If the initial matching scores of the waving action and the squatting action are 0.6 and 0.7 respectively, after normalization, the probability of the waving action is 0.46 and the probability of the squatting action is 0.54. The squatting check can then be output as the final result. Finally, the recognition results are used to generate action response outputs. When the recognition result is a standard action, the system can generate corresponding safety warning signals or production efficiency optimization instructions. When the recognition result is an abnormal action, it will trigger equipment operation adjustment instructions. With the support of the standard action library, high-precision matching between actions and templates is achieved. Combined with environmental adaptability processing and iterative correction, the final recognition result is generated, ensuring the real-time performance and accuracy of action recognition in the mine.

[0067] In one specific embodiment, optimizing the video frame sequence based on environmental interference vectors includes the following steps:

[0068] Analyze the uniform distribution of illumination and the concentration of dust particles in each frame of the video frame sequence; calculate the dynamic range of illumination based on the uniform distribution of illumination and the dust diffusion rate based on the dust particle concentration; fuse the dynamic range of illumination and the dust diffusion rate to generate an environmental interference vector;

[0069] A convolutional neural network is used to process the video frame sequence, and the illumination color temperature deviation is adjusted according to the illumination dynamic range in the environmental interference vector.

[0070] By adjusting the contrast of the video frame based on the dust diffusion rate in the environmental interference vector, a clear frame sequence can be obtained.

[0071] Specifically, uniform illumination distribution is obtained by calculating the standard deviation of pixel brightness in each frame of the image. If the standard deviation exceeds a preset value (e.g., 0.5), the illumination is considered uneven. The dust particle concentration is estimated using a particle counting algorithm, with the unit being the number of particles per square centimeter. When the dust concentration exceeds 500 particles / square centimeter, it indicates strong environmental interference. The dynamic range of illumination is obtained by calculating the difference between the minimum and maximum brightness values ​​of each frame of the image. If the dynamic range is greater than 150 gray levels, it indicates strong illumination contrast, which may interfere with subsequent image processing. In this case, the interference is reduced by adjusting the illumination color temperature deviation, for example, adjusting the color temperature from 5000K to the standard 6500K to adapt to changes in ambient light. The displacement vector of dust particles in each frame of the image is obtained through optical flow estimation, thus determining the dust diffusion rate. If the diffusion rate exceeds 100 particles / second / square centimeter, it indicates rapid dust diffusion in the environment, requiring enhanced contrast processing to improve particle visibility. A convolutional neural network (CNN) is used to process the video frame sequence. Each frame image is input into the CNN, processed by convolutional kernels to capture local patterns and introduce nonlinear transformations, ultimately preserving key visual information in the image and ensuring that key points are not affected by lighting and dust interference. Furthermore, based on the uniformity of lighting distribution information in the environmental interference vector, the lighting color temperature deviation is adjusted. If the lighting is uneven, the color temperature is adjusted to remove the deviation, outputting a denoised frame. For scenes with high dust particle concentration, contrast enhancement methods are used to improve particle visibility. When the calculated dust concentration is high, histogram equalization is used to enhance the contrast of the video frames, making the dust particles in the image more prominent and ensuring the accuracy of key point extraction. Finally, the results of denoising and contrast enhancement are fused to obtain a clear frame sequence, which serves as input for subsequent behavior recognition. This process adapts to changes in different environments, optimizes video processing effects, and improves the safety and production efficiency of mining operations.

[0072] In one specific embodiment, adjusting the contrast of the video image based on the dust diffusion rate in the environmental interference vector specifically includes the following steps:

[0073] By applying an optical flow field estimation algorithm to multiple consecutive video images, the displacement vector of pixels in the dust region is obtained;

[0074] The direction and velocity distribution of dust movement are calculated based on the statistical results of the displacement vectors.

[0075] The contrast enhancement coefficient is adaptively set according to the dust density gradient, and the local area is subjected to regional histogram equalization to enhance the overall picture and highlight the edge features of dust particles.

[0076] Specifically, an optical flow field estimation algorithm is applied to multiple consecutive video frames. This algorithm is an image processing method used to calculate the motion vectors of pixels between adjacent frames. It extracts the displacement information of pixels in dusty areas and, combined with the displacement vectors of consecutive frames, statistically determines the directional and velocity distribution of dust in space, thus distinguishing whether the dust is spreading rapidly or floating slowly. Secondly, based on the statistically obtained dust velocity characteristics, an adaptive contrast enhancement coefficient is set: when the dust diffusion speed is detected to be fast, the enhancement coefficient is increased to highlight the boundary between dust particles and the background, avoiding edge blurring caused by rapid movement; when the diffusion speed is slow, the enhancement coefficient is decreased to maintain a smoother image effect and avoid artifacts introduced by excessive enhancement. Finally, to address the uneven distribution of dust, a regional histogram equalization method is adopted. This method divides the image into multiple local regions and adjusts the contrast of each region separately to avoid overexposure or underexposure caused by overall histogram equalization. In areas with high dust concentration and obvious local gradients, the contrast enhancement is stronger, thus highlighting the particle outlines; while in areas with clear backgrounds and less interference, a weaker enhancement is maintained to ensure the overall naturalness of the image. The above methods can effectively improve the visibility and detail recognition of video images in dusty mining environments, making subsequent human key point extraction and motion recognition more accurate.

[0077] In one specific embodiment, analyzing the time series changes of key point coordinates to obtain preliminary motion features specifically includes the following steps:

[0078] A target detection model based on YOLOv5 is established to detect and locate human body regions in clear frame sequences and extract target bounding boxes.

[0079] Input the target bounding box into the pose estimation network model to obtain the coordinates of human key points and pose vectors;

[0080] The continuous time series of key point coordinates is input into the recurrent neural network model to analyze the changes in displacement and velocity of the key points and output the dynamic characteristics of the action evolution over time.

[0081] Obtain the real-time acquisition frequency, multiply the dynamic features by the acquisition frequency, and obtain the weighted dynamic value;

[0082] If the weighted dynamic value is greater than the preset threshold, the action corresponding to the dynamic feature is marked as a significant action, and preliminary action features are generated.

[0083] Specifically, a target detection model is established based on YOLOv5 to detect and locate human body regions in clear frame sequences and extract the bounding boxes of human targets. YOLOv5 is a lightweight deep learning target detection algorithm that can achieve real-time detection while ensuring accuracy and is suitable for video stream processing in mining environments. Subsequently, the target bounding boxes are input into the pose estimation network model to obtain the coordinates of human key points and pose vectors. The pose vectors are the overall representation formed based on these key point coordinates and are used to describe the overall skeletal structure and pose of the human body. The continuous time series of key point coordinates is input into a recurrent neural network (RNN) model for time series analysis. RNNs are a type of neural network model capable of processing time series data. They record the dependencies between sequences through hidden states, capturing the displacement and velocity changes of human movements across different time frames. For example, when a worker waves their hand, the RNN can identify the continuous displacement trajectory of the arm's key points over dozens of frames, thereby extracting the dynamic features of the movement. The dynamic features output by the RNN are multiplied by the acquisition frequency to obtain a weighted dynamic value, which amplifies the magnitude of the movement's change over time. The real-time acquisition frequency refers to the number of frames captured per second by the camera, such as 30 frames per second. This parameter affects the resolution and sensitivity of the time series features. The weighted dynamic value is then compared with a quantitative threshold. If the threshold is set to 0.5 m / s, and the detected displacement velocity of the worker's arm is 0.6 m / s, the weighted dynamic value exceeds the threshold, and the system automatically marks the movement as significant. If it is below the threshold, it is judged as insignificant, thus avoiding misidentification due to minor perturbations or background interference. Through the above steps, the final generated preliminary motion features not only include the time span of the motion, but also dynamic parameters such as speed and displacement amplitude, which can accurately characterize the temporal dynamic characteristics of the motion.

[0084] In one specific embodiment, calculating the similarity score between preliminary action features and a preset standard template specifically includes the following steps:

[0085] A standardized set of posture vectors is extracted from the mine motion recognition database as a preset standard template, and the initial similarity between the preliminary motion features and the preset standard template is calculated.

[0086] If the initial similarity is lower than the preset threshold, the weight coefficient of each action component in the preliminary action feature is adjusted according to the uniform distribution of illumination and the concentration of dust particles in the environmental interference vector.

[0087] An adaptively adjusted similarity score is generated based on the updated weight coefficients.

[0088] Specifically, a preset standard template is extracted from the mine motion recognition database. This template consists of a standardized set of posture vectors, covering typical safety action patterns such as waving signals and crouching for inspection. Each template vector is obtained by standardizing the coordinates of key human body points and posture angles to ensure a unified benchmark. Subsequently, the preliminary motion features are compared with the preset standard template, and the initial similarity is calculated using a cosine similarity algorithm. Cosine similarity measures the similarity between two vectors in direction, with a value ranging from 0 to 1. The closer to 1, the higher the matching degree. For example, when the key point sequence of a worker's waving motion is compared with the template and the similarity is 0.82, it indicates that the motion is highly consistent with the template; if it is only 0.55, it indicates that there is a deviation. When the initial similarity is lower than a preset threshold (e.g., 0.7), an environmental interference vector is introduced for correction. The environmental interference vector consists of uniform illumination distribution and dust particle concentration, used to characterize the degree of interference of the external environment on feature extraction. Uniform illumination distribution is quantified by the variance of pixel brightness in video frames. The larger the variance value, the more uneven the illumination. Dust particle concentration is used to reflect the intensity of dust obstruction. Based on the above indicators, the weight coefficients of each action component in the initial action features are adjusted. These weight coefficients characterize the contribution of different action components to the overall similarity calculation. When the uniformity of illumination is low and the dust concentration is high, the weight of detailed gesture components (such as finger trajectories) is automatically reduced, while the weight of overall posture contour components (such as the overall arm swing or lower limb bending angle) is increased to avoid misjudgments caused by detail distortion. After weight adjustment, the adaptive similarity score is recalculated to obtain the final result. This score not only reflects the degree of matching between the action and the template but also incorporates environmental correction factors. For example, in a high-dust scene, the initial similarity may be only 0.55, which increases to 0.72 after weight adjustment, thus avoiding misjudgments caused by external interference and ensuring the reliability of the recognition results. Through these steps, the calculation of the similarity score achieves adaptive calibration for complex mining environments, ensuring that stable and accurate action matching results can still be generated under conditions of insufficient illumination and high dust concentration, providing a reliable basis for subsequent refinement and recognition.

[0089] In one specific embodiment, refining the initial action features based on similarity scores includes the following steps:

[0090] Determine if the similarity score is greater than the target threshold. If so, determine if there is a matching relationship between the preliminary action features and the preset template.

[0091] Otherwise, the initial motion features are decomposed into multiple motion components, each corresponding to a time segment in the motion sequence;

[0092] Extract key regions from the decomposed motion components. Key regions are the parts that can reflect differences in gestures or subtle changes in posture.

[0093] Weights are assigned to key regions using an attention mechanism to generate attention-weighted vectors;

[0094] The target threshold is adaptively adjusted based on the environmental interference vector;

[0095] The dynamic gain coefficient is calculated based on the similarity score. The key features in the attention weighted vector are corrected and amplified to obtain the refined motion component vector, which is the local level of velocity, angle and trajectory change features.

[0096] Specifically, such as Figure 2 The diagram shows a flowchart of the initial motion feature refinement process. When the similarity score is lower than the target threshold, it indicates that the overall features do not match the standard template well enough. In this case, the initial motion features are divided into several motion components, each corresponding to a time segment in the motion sequence. For example, in the process of recognizing a worker waving, if the overall similarity is 0.6, which is lower than the threshold of 0.7, the worker's waving motion is divided into three sub-segments, corresponding to the arm raising, horizontal swinging, and falling phases, respectively. In the decomposed motion components, an attention mechanism is further used to highlight key areas. The attention mechanism calculates the importance score of the input features (usually normalized using the softmax function) and assigns higher weights to salient areas. For example, when analyzing a waving motion, the attention mechanism may focus more on the wrist swing path and ignore the static parts of the head or torso. Building upon this foundation, an adaptive threshold adjustment mechanism is introduced. Since lighting and dust interference in the mining environment can cause feature shifts, a fixed threshold might lead to misjudgments. Therefore, this embodiment dynamically adjusts the threshold based on the environmental interference vector. For example, when dust concentration increases, the threshold is reduced from 0.7 to 0.6 to tolerate a certain degree of feature deviation, ensuring the system can still output effective results under low visibility conditions. Simultaneously, a dynamic coefficient gain mechanism is employed to correct and amplify the attention weighting vector. This dynamic coefficient gain is a proportional adjustment factor based on the similarity score, enhancing the impact of key features on the final recognition result. For instance, when the similarity score is only 0.55, the system automatically increases the gain coefficient, weighting and amplifying the speed change features of the arm's key points by 20% to ensure that the refined motion component vector accurately reflects the motion details. Finally, the adaptive adjustment result of the similarity threshold is combined with the correction result of the dynamic coefficient gain to obtain the refined motion component vector. This vector is a multi-dimensional representation, with each dimension corresponding to local motion features, such as speed, angle, or trajectory changes. For example, when a worker performs a waving signal, the refined motion component vector can not only represent the overall swing of the arm, but also describe the local curvature change of the wrist trajectory. Through the above processing, the refined motion component vector can effectively resist the interference of mine lighting and dust, and improve the accuracy and stability of motion recognition.

[0097] In one specific embodiment, calculating the dynamic gain coefficient based on the similarity score includes the following steps: inputting the similarity score into a combination model of a linear mapping function and an exponential smoothing function to obtain a gain factor; if the similarity score is less than a first threshold, automatically increasing the gain factor; if the similarity score is greater than the first threshold, decreasing the gain factor.

[0098] Specifically, the similarity score is normalized to a standard range (e.g., between 0 and 1) using a linear mapping function, and then combined with an exponential smoothing function to smooth fluctuations in the similarity value. The smoothed similarity value is compared with a preset first threshold. The system dynamically adjusts the gain factor according to the threshold. If the similarity score is lower than the first threshold (e.g., 0.5), the gain factor is automatically increased to enhance the influence of feature matching. If the similarity score is higher than the first threshold, the gain factor is decreased to avoid over-enhancing the influence of certain features and ensure more balanced action recognition.

[0099] In one specific embodiment, matching refined action component vectors from a preset action library to obtain action recognition results specifically includes the following steps:

[0100] A preset action library was built based on historical mine video data. The preset action library includes hand waving signals and crouching inspection action patterns.

[0101] The refined action component vectors are matched with a preset action library, and action recognition results are generated by combining the update iteration cycle and weight normalization processing.

[0102] Specifically, a preset action library is constructed by organizing historical mine video data. The preset action library contains standard action patterns commonly seen by workers on the job site. The refined action component vectors are matched with the preset action library, and Euclidean distance is used as the similarity calculation method, that is, the geometric distance between two vectors is calculated. The smaller the value, the closer the actions are. To improve the stability of the matching results, an update iteration cycle and a weight normalization mechanism are introduced. The update iteration cycle refers to recalculating the matching results at fixed time or frame intervals. For example, when the video capture frequency is 30 frames / second, an iteration update can be set every 150 frames (about 5 seconds) to reflect the dynamic changes of the action. The weight normalization process is used to balance the contribution of different action components in the similarity calculation. Specifically, the softmax function can be used to convert the scores of each component into a probability distribution so that the sum is 1. The action recognition result is generated based on the normalized score. If the comprehensive matching score of the waving action is 0.85 and remains stable above 0.8 for two consecutive iteration cycles, the output "Waving signal matching successful" is output, and the corresponding safety warning or operation command is triggered. In scenarios with high dust concentration, the iteration cycle can be shortened to 3 seconds to quickly confirm the validity of the action features. The recognition accuracy is improved by about 15% compared with traditional methods, and the probability of false alarms and false negatives is significantly reduced. By constructing a preset action library, matching based on refined action component vectors, adjusting dynamic thresholds, updating and iterating, and normalizing weights, high-precision identification of key actions of miners was achieved.

[0103] In one specific embodiment, generating the corresponding action response output based on the action recognition result specifically includes the following steps:

[0104] Generate safety warning signals or production efficiency optimization instructions based on motion recognition results;

[0105] If the action recognition result indicates an abnormal action, extract the multi-dimensional vector from the refined action component vector of the abnormal action, assign dimension coefficients to the multi-dimensional vectors, and perform weighted fusion of the multi-dimensional vectors based on the dimension coefficients to obtain the fused vector;

[0106] To verify the stability of the dimensional coefficients, the variance of each dimensional coefficient is calculated. If the variance is lower than a preset threshold, the dimensional coefficient is determined to be stable. Otherwise, the dimensional coefficients are iteratively adjusted until they are stable, and an instruction to trigger device operation adjustment is generated.

[0107] The action response output is generated based on the adjustment instructions, and the action response output is a device control signal.

[0108] Specifically, such as Figure 3The diagram shows a flowchart of the action response output. It identifies actions at the mine operation site. If a normal waving signal or squatting inspection is detected, the system generates production efficiency optimization instructions or safety warning signals based on the identification results. For example, if the system detects a waving signal, it generates an instruction to optimize equipment speed to improve work efficiency; if it detects a squatting inspection mode, the system issues a safety warning signal to alert surrounding personnel. If an abnormal action is detected (e.g., a worker exhibiting an abnormal squatting posture or excessive twisting), the system extracts detailed action component vectors related to the abnormal action. These vectors include information such as gesture differences and subtle posture changes, representing local changes in the action. Next, the gesture difference vector and the subtle posture change vector are weighted and summed to generate a fused vector to comprehensively represent the abnormal action. For example, the gesture difference vector contains three dimensions (x, y, z coordinate changes), while the subtle posture change vector contains two dimensions (angle and speed changes). These dimensions are merged to ensure that all action features are considered. The stability of the coefficients in the fused vector is verified by calculating the variance of each dimension. If the variance is greater than a preset threshold (e.g., 0.05), the coefficient is considered unstable and needs adjustment. Optimization algorithms such as least squares are used to make the variance less than the threshold, ensuring the stability of the action features. Based on the stabilized fused vector, instructions to trigger equipment adjustments are generated. For example, when the system detects an abnormal squatting action, it generates instructions to adjust the equipment operation based on the fused vector, such as reducing the equipment speed (e.g., from 100 rpm to 50 rpm). Finally, the generated instructions are converted into real-time equipment control signals and directly output to the equipment controller for immediate response. For example, safety warning signals can be converted into digital signals, such as alarm 1, and production efficiency optimization instructions can be converted into speed adjustment code 2. Through vector fusion and stability verification, not only is the accurate identification of abnormal actions of miners improved, but also the system's immediate control of equipment and safety warnings are ensured.

[0109] The above describes a YOLOv5-based method for identifying the behavior of miners in this application. The following describes a YOLOv5-based system for identifying the behavior of miners in this application. Please refer to [link to relevant documentation]. Figure 4 One embodiment of a YOLOv5-based mine worker behavior recognition system in this application includes:

[0110] The acquisition module acquires video frame sequences from the mine site, extracts environmental interference vectors from the video frame sequences, and optimizes the video frame sequences based on the environmental interference vectors to obtain clear frame sequences.

[0111] The analysis module extracts the coordinates of key human points and pose vectors from clear frame sequences, analyzes the temporal changes of key point coordinates, and obtains preliminary motion features.

[0112] The optimization module calculates the similarity score between the preliminary action features and the preset standard template, and refines the preliminary action features based on the similarity score to obtain the refined action component vector.

[0113] The output module matches and refines the motion component vectors from the preset motion library to obtain the motion recognition results, and generates the corresponding motion response output based on the motion recognition results.

[0114] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for identifying the behavior of miners based on YOLOv5, characterized in that, The method includes: Step S1: Obtain a video frame sequence of the mine site, extract an environmental interference vector from the video frame sequence, and optimize the video frame sequence based on the environmental interference vector to obtain a clear frame sequence; Step S2: Extract the coordinates of human key points and the pose vector from the clear frame sequence, analyze the time series changes of the key point coordinates, and obtain preliminary motion features; Step S3: Calculate the similarity score between the preliminary action features and the preset standard template, and refine the preliminary action features based on the similarity score to obtain a refined action component vector; The calculation of the similarity score between the preliminary action features and the preset standard template includes: extracting a standardized set of posture vectors from the mine action recognition database as the preset standard template, calculating the initial similarity between the preliminary action features and the preset standard template; if the initial similarity is lower than a preset threshold, adjusting the weight coefficient of each action component in the preliminary action features according to the uniform distribution of illumination and the concentration of dust particles in the environmental interference vector; and generating the adaptively adjusted similarity score based on the updated weight coefficients. The refinement of the preliminary action features based on the similarity score includes: determining whether the similarity score is greater than a target threshold; if so, determining that the preliminary action features match a preset template; otherwise, performing fine-grained decomposition on the preliminary action features, dividing them into multiple action components, each corresponding to a time segment in the action sequence; extracting key regions from the decomposed action components, the key regions being areas that can reflect gesture differences or subtle changes in posture; assigning weights to the key regions using an attention mechanism to generate an attention weighted vector; adaptively adjusting the target threshold in conjunction with an environmental interference vector; calculating a dynamic gain coefficient based on the similarity score, correcting and amplifying the key features in the attention weighted vector to obtain a refined action component vector, the refined action component vector being local-level speed, angle, and trajectory change features; Step S4: Match the refined action component vector from the preset action library to obtain the action recognition result, and generate the corresponding action response output based on the action recognition result.

2. The method according to claim 1, characterized in that, Optimization processing of the video frame sequence based on the environmental interference vector includes: Analyze the uniformity of illumination and dust particle concentration in each frame of the video frame sequence; The dynamic range of illumination is calculated based on the uniform distribution of illumination, and the dust diffusion rate is calculated based on the dust particle concentration. The environmental disturbance vector is generated by fusing the dynamic range of illumination and the dust diffusion rate. The video frame sequence is processed using a convolutional neural network, and the illumination color temperature deviation is adjusted according to the illumination dynamic range in the environmental interference vector. Based on the dust diffusion rate in the environmental interference vector, the contrast of the video image is adjusted to obtain a clear frame sequence.

3. The method according to claim 2, characterized in that, Adjusting the contrast of the video image based on the dust diffusion rate in the environmental interference vector includes: By applying an optical flow field estimation algorithm to multiple consecutive video images, the displacement vector of pixels in the dust region is obtained; The direction and velocity distribution of dust movement are calculated based on the statistical results of the displacement vectors. The contrast enhancement coefficient is adaptively set according to the dust density gradient, and the local area is subjected to regional histogram equalization to enhance the overall picture and highlight the edge features of dust particles.

4. According to the method of claim 1, analyzing the time series changes of the key point coordinates to obtain preliminary motion features includes: A target detection model based on YOLOv5 is established to detect and locate human body regions in the clear frame sequence and extract target bounding boxes. Input the target bounding box into the pose estimation network model to obtain the coordinates of human key points and the pose vector. The continuous time series of the key point coordinates is input into the recurrent neural network model to analyze the changes in the displacement and velocity of the key points and output the dynamic characteristics of the action evolution over time. Obtain the real-time acquisition frequency, and multiply the dynamic feature by the acquisition frequency to obtain a weighted dynamic value; If the weighted dynamic value is greater than a preset threshold, the action corresponding to the dynamic feature is marked as a significant action, and preliminary action features are generated.

5. The method according to claim 1, characterized in that, Calculating the dynamic gain coefficient based on the similarity score includes: The similarity score is input into a combined model of a linear mapping function and an exponential smoothing function to obtain a gain factor. If the similarity score is less than a first threshold, the gain factor is automatically increased; if the similarity score is greater than the first threshold, the gain factor is decreased.

6. The method according to claim 1, characterized in that, The action recognition results obtained by matching the refined action component vectors from the preset action library include: A preset action library is constructed based on historical mine video data, and the preset action library includes hand waving signals and crouching inspection action patterns. The refined action component vector is matched with the preset action library, and the action recognition result is generated by combining the update iteration cycle and weight normalization processing.

7. The method according to claim 1, characterized in that, The corresponding action response output generated based on the action recognition result includes: Based on the action recognition results, generate safety warning signals or production efficiency optimization instructions; If the action recognition result indicates an abnormal action, extract the multi-dimensional vector from the refined action component vector of the abnormal action, assign dimension coefficients to the multi-dimensional vector, and perform weighted fusion on the multi-dimensional vector based on the dimension coefficients to obtain a fused vector; The stability of the dimensional coefficients is verified by calculating the variance of each dimensional coefficient. If the variance is lower than a preset threshold, the dimensional coefficient is determined to be stable. Otherwise, the dimensional coefficients are iteratively adjusted until they are stable, and an instruction to trigger device operation adjustment is generated. An action response output is generated based on the adjustment instructions, and the action response output is a device control signal.

8. A YOLOv5-based mine personnel behavior recognition system, used to implement the YOLOv5-based mine personnel behavior recognition method as described in any one of claims 1-7, characterized in that, The system includes: The acquisition module acquires a video frame sequence from the mine site, extracts an environmental interference vector from the video frame sequence, and optimizes the video frame sequence based on the environmental interference vector to obtain a clear frame sequence. The analysis module extracts the coordinates of key human points and the posture vector from the clear frame sequence, analyzes the time series changes of the key point coordinates, and obtains preliminary motion features. The optimization module calculates the similarity score between the preliminary action features and the preset standard template, and refines the preliminary action features based on the similarity score to obtain a refined action component vector. Calculating the similarity score between the preliminary action features and the preset standard template includes: extracting a standardized pose vector set from a mine action recognition database as the preset standard template, and calculating the initial similarity between the preliminary action features and the preset standard template; if the initial similarity is lower than a preset threshold, adjusting the weight coefficient of each action component in the preliminary action features based on the uniform distribution of illumination and the concentration of dust particles in the environmental interference vector; generating an adaptively adjusted similarity score based on the updated weight coefficients; and refining the preliminary action features based on the similarity score includes: determining the similarity between the preliminary action features and the preset standard template. If the similarity score is greater than the target threshold, it is determined that the preliminary action feature matches the preset template; otherwise, the preliminary action feature is decomposed into multiple action components, each corresponding to a time segment in the action sequence. Key regions are extracted from the decomposed action components, which are regions that can reflect gesture differences or subtle changes in posture. Weights are assigned to the key regions through an attention mechanism to generate an attention weighted vector. The target threshold is adaptively adjusted in conjunction with the environmental interference vector. A dynamic gain coefficient is calculated based on the similarity score, and the key features in the attention weighted vector are corrected and amplified to obtain a refined action component vector, which represents the speed, angle, and trajectory change features at the local level. The output module matches the refined action component vector from the preset action library to obtain the action recognition result, and generates the corresponding action response output based on the action recognition result.

Images