Mining intelligent helmet safety early warning method based on image recognition

Abstract

The invention discloses a mining intelligent helmet safety early warning method based on image recognition, and relates to the technical field of mining safety, and the method comprises the steps: collecting underground environment data in real time, the underground environment data comprising dust concentration and illumination intensity; calculating a current exposure parameter according to the dust concentration and the illumination intensity based on a reinforcement learning algorithm, driving a camera to adjust the aperture and shutter speed, and outputting an optimized low-noise image; and according to the illegal operation probability and the personnel action sequence, triggering a grading alarm signal through a vibration motor arranged in the helmet, simultaneously superposing the dangerous area coordinates to an AR display interface, and generating a safe operation guidance text. According to the method, the space-time correlation features of multiple frames of images are extracted, a coupling relation model of dust scattering and illumination attenuation is established, an exposure parameter-assisted semantic segmentation feature enhancement mechanism is constructed, and the target recognition effect in a complex environment is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mine safety, and in particular to a mine intelligent helmet safety warning method based on image recognition. Background Art

[0002] At present, safety monitoring of mine operations has become a core link in the intelligent transformation of modern mines. Mine intelligent monitoring equipment, as a key device for ensuring the safety of underground personnel, the environmental perception accuracy and risk response speed directly determine the efficiency of safe production and the accident prevention and control ability. Therefore, there is an urgent need to construct a dynamic recognition and multi-modal warning mechanism for complex underground working conditions.

[0003] In the prior art, the parameter adjustment of traditional cameras depends on fixed thresholds or manual presets, and cannot dynamically adapt to the non-linear coupling relationship between dust concentration and light intensity, resulting in poor image noise suppression effect and affecting the subsequent semantic segmentation accuracy. The existing semantic segmentation models have high computational complexity, are difficult to achieve real-time processing on the embedded hardware of mine helmets, and lack the real-time interaction ability with dynamic knowledge graphs, resulting in a lag in the judgment of illegal operations. In addition, the problem of single warning mechanism is prominent. The existing vibration feedback and visual prompts do not achieve hierarchical linkage and cannot accurately match the operation scenarios of different risk levels. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a mine intelligent helmet safety warning method based on image recognition to solve the problems of insufficient optimization of dynamic images, lag in real-time risk recognition, and low accuracy of multi-modal warnings in complex mine environments.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a safety warning method for a mining intelligent helmet based on image recognition, which includes: real-time collecting underground environment data, where the underground environment data includes dust concentration and light intensity; based on a reinforcement learning algorithm, calculating the current exposure parameters according to the dust concentration and light intensity, and driving the camera to adjust the aperture and shutter speed to output an optimized low-noise image; inputting the low-noise image into a lightweight semantic segmentation model to identify the equipment area, personnel area, and dangerous area, generating a semantic mask, and extracting the equipment feature vector and the personnel action sequence; obtaining a knowledge graph from the mine server through a wireless Mesh network, where the knowledge graph contains equipment topological relationships, safety regulation entities, and historical violation records, and performing graph attention matching between the equipment feature vector and the equipment nodes in the knowledge graph to output the probability of illegal operation; according to the probability of illegal operation and the personnel action sequence, triggering a hierarchical warning signal through the vibration motor built in the helmet, and at the same time overlaying the coordinates of the dangerous area on the AR display interface and generating a safety operation guidance text.

[0007] As a preferred solution of the safety warning method for the mining intelligent helmet based on image recognition according to the present invention, among them: after real-time collecting the underground environment data, perform a moving average filtering process on the dust concentration data, perform a low-pass filtering process on the light intensity data, and map the two types of data to a unified numerical interval through a normalization operation.

[0008] As a preferred solution of the safety warning method for the mining intelligent helmet based on image recognition according to the present invention, among them: the reinforcement learning algorithm constructs a state space based on the real-time change trends of the dust concentration and light intensity, and generates exposure parameters by balancing image brightness optimization and noise suppression; The specific steps for generating the exposure parameters are as follows: Real-time collect the dust concentration, light intensity, and real-time image data feedback by the camera, and combine the current exposure parameters and processing delay to construct a multi-dimensional vector representing the environmental dynamics and equipment status; Input the multi-dimensional vector into a pre-trained deep reinforcement learning network, output adjustment instructions for the aperture, shutter, and gain, and generate actions through a dynamic strategy that balances brightness optimization and noise suppression; Convert the adjustment instructions into actual exposure parameters according to a preset step coefficient, write them into the camera register to trigger image acquisition, and synchronously update the image quality index and processing delay; Optimize the network parameters based on the new image index, and when detecting environmental anomalies or decision-making timeouts, switch to a physical model to calculate the basic parameters.

[0009] As a preferred solution of the image recognition-based mine intelligent helmet safety warning method of the present invention, wherein: the lightweight semantic segmentation model adopts a lightweight convolutional neural network, generates semantic masks for the equipment area, personnel area and dangerous area through multi-scale feature fusion, and extracts the equipment contour feature vector and the continuous action trajectory of personnel based on the masks.

[0010] As a preferred solution of the image recognition-based mine intelligent helmet safety warning method of the present invention, wherein: the knowledge graph obtains the equipment topology relationship and safety regulation entities in real time through a wireless network, matches the attention weights between the equipment feature vector and the equipment node in the graph, and outputs the violation probability associated with the historical violation records.

[0011] As a preferred solution of the image recognition-based mine intelligent helmet safety warning method of the present invention, wherein: the graph attention matching adopts multi-channel correlation calculation, screens out potential violation operation nodes and calculates the probability value by comparing the semantic similarity between the equipment feature vector and the knowledge graph node.

[0012] As a preferred solution of the image recognition-based mine intelligent helmet safety warning method of the present invention, wherein: the hierarchical alarm signal divides the threshold interval according to the violation probability, triggers different alarms through the intensity-frequency combination of the vibration motor, and at the same time superimposes the dangerous area contour box and safety guidance text on the AR interface.

[0013] As a preferred solution of the image recognition-based mine intelligent helmet safety warning method of the present invention, wherein: the dangerous area coordinate superposition adopts a three-dimensional coordinate conversion method to map the position of the dangerous area in the mine space coordinate system to the perspective projection layer of the AR display interface in real time.

[0014] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and wherein: when the computer program is executed by the processor, any step of the image recognition-based mine intelligent helmet safety warning method as described in the first aspect of the present invention is implemented.

[0015] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and wherein: when the computer program is executed by the processor, any step of the image recognition-based mine intelligent helmet safety warning method as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are as follows: By combining real-time environmental parameters such as dust concentration and light intensity with an optical transmission model, a physical constraint optimization space for exposure parameters is constructed based on a reinforcement learning algorithm, significantly improving the image quality in high-dynamic environments. By extracting the spatio-temporal correlation features of multiple frames of images, establishing a coupling relationship model between dust scattering and illuminance attenuation, and constructing a semantic segmentation feature enhancement mechanism assisted by exposure parameters, the target recognition effect in complex environments is effectively improved. In addition, by adopting physical constraint conditions based on safety regulations in knowledge graph reasoning and deeply integrating the device state features with the dynamic knowledge graph nodes, the physical interpretability of illegal operation determination is realized, significantly improving the response speed and decision-making reliability. Finally, through the collaborative optimization of the multi-modal alarm mechanism, the intelligent helmet demonstrates excellent real-time performance and environmental adaptability under complex underground working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0018] Figure 1 It is a flowchart of the safety warning method for a mine intelligent helmet based on image recognition in this embodiment.

[0019] Figure 2 It is a flowchart of sensor data processing and fault tolerance in this embodiment.

[0020] Figure 3 It is a flowchart of adjusting exposure parameters by reinforcement learning in this embodiment.

[0021] Figure 4 It is a flowchart of image processing and illegal operation determination in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings of the specification.

[0023] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0024] Secondly, the so-called "one embodiment" or "embodiment" herein refers to specific features, structures, or characteristics that may be included in at least one implementation manner of the present invention. The phrase "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an individual or alternative embodiment that is mutually exclusive with other embodiments.

[0025] In this embodiment, referring to Figures 1 to 4 , this embodiment provides a method for safety warning of a mine intelligent helmet based on image recognition, including the following steps: S1: Collect underground environment data in real time, where the underground environment data includes dust concentration and light intensity.

[0026] Specifically, it includes the following steps: S1.1: Collect underground environment data (dust concentration and light intensity) through a dust sensor and a light sensor, apply a moving average filter (window size is 5) to the dust data to suppress instantaneous noise (such as blasting interference); apply a low-pass filter to the light data to eliminate high-frequency power frequency interference (such as miner's lamp flickering).

[0027] If data anomalies (including but not limited to communication interruption, data jump, etc.) are detected continuously for multiple times, trigger the sensor self-check process and switch to the backup data source.

[0028] It should be noted that in the intelligent safety monitoring scheme for coal mines, the "backup data source" refers to an alternative data acquisition method automatically enabled when the main sensor is abnormal, including a three-level fault tolerance mechanism: for example, when the main dust sensor is abnormal, it immediately switches to the backup dust sensor deployed at the same location. The dust concentration can be estimated by combining the image features captured by the camera with the calibrated visibility parameters, and the light intensity can be deduced by image brightness analysis. The default values of the preset safety parameters are used as the final guarantee. The backup data is processed by dynamic weighted fusion, and abnormal events are recorded and uploaded to the monitoring center in real time to ensure continuous and stable monitoring. The visibility deduction method and image analysis technology adopted are both mature application schemes for current mining equipment.

[0029] S1.2: The microcontroller encapsulates the filtered dust concentration, light intensity, timestamp, and the status of the dust / light sensor into a fixed-format data frame and sends it to the main control (RK3566) through the SPI interface.

[0030] In the main control (RK3566), if the data processing times out (exceeds the preset response period), enable the linear prediction algorithm to complete the current data; when the sensor communication is interrupted, the main control automatically switches to the last measured value that passed the verification and superimposes the safety default value (such as medium dust concentration, low light intensity), and at the same time triggers a short vibration warning for the helmet.

[0031] It should be noted that exceeding the preset response period means that the main controller (RK3566) fails to complete data processing within the specified time. This response period is dynamically adjusted according to the sampling frequency of the dust / light sensor (1 - 5Hz) and is usually set to 50 - 100ms to ensure real-time performance. After timeout, linear prediction will be automatically enabled to complete the current frame based on the historical data trend.

[0032] The measured value refers to the valid result after strict verification of the underground environment data collected by the dust sensor and the light sensor. Specifically, the dust sensor measures the concentration data through the principle of laser scattering, and the light sensor measures the intensity data through photoelectric conversion. These underground environment data need to first check whether the signal is complete and whether the value is within a reasonable range (such as dust 0 - 2000μg / m³, light 0 - 65535Lux), and then undergo filtering processing to eliminate interference. Only the underground environment data that passes all these verification links will be recognized as valid "measured values". When it is necessary to switch the data source, the measurement result that passed all verifications most recently will be automatically called to ensure the authenticity and reliability of the data used.

[0033] S1.3: The main controller periodically sends the underground environment data to the camera digital signal processor (DSP) through the high-speed LVDS interface.

[0034] When high dust concentration or low light intensity is detected, the camera immediately pauses non-critical tasks (such as logging, background data synchronization) and preferentially performs exposure parameter adjustment.

[0035] Specifically, after receiving the original image data, the camera digital signal processor (DSP) extracts the luminance channel (YUV / Y component) through denoising and grayscale processing, calculates the global average luminance. When the global average luminance continuously drops below the preset luminance standard of the camera (for example, the Y-channel value is below 30 for a 50lux environment) and the proportion of dark area pixels exceeds 90%, a low light determination is triggered; at the same time, analyze the attenuation degree of the high-frequency details of the image. If the high-frequency energy of the current image drops by more than 30% compared to the reference data of the dust-free scene, it is directly determined that the dust concentration is abnormal; finally, align the image determination result with the data of the dust sensor and the photosensitive unit by force, preferentially adopt the readings of the hardware sensors as the final conclusion, and dynamically adapt to the environmental conditions through time or scene mode (such as automatically lowering the luminance determination benchmark at night) to ensure the uniqueness and certainty of the detection logic.

[0036] It should be noted that after the camera completes the exposure adjustment, it returns a confirmation signal to the main controller. If the confirmation signal is not received within the timeout period, the main controller resends the underground environment data to ensure the real-time performance of the control loop.

[0037] S2. Based on the reinforcement learning algorithm, calculate the current exposure parameters according to the dust concentration and light intensity, and drive the camera to adjust the aperture and shutter speed, and output an optimized low-noise image.

[0038] Specifically, it includes the following steps: S2.1: Read the dust concentration at the current moment and light intensity , and synchronously receive the global brightness mean , brightness standard deviation , high-frequency energy ratio of the previous frame image returned by the camera DSP, combine with the current exposure parameters and frame processing delay , and construct a 9-dimensional state vector : ; S2.2: Input the state vector into the pre-trained TD3-DDPG algorithm Actor network (network structure: 9→256→128→3), and output a normalized action vector .

[0039] Specifically, the normalized action vector is expressed as: ; In the formula, is the aperture increment, representing the relative change in the aperture opening size, is the shutter speed increment, representing the relative change in the sensor exposure time, is the gain increment, representing the relative change in the sensor analog / digital gain, is a three-dimensional cube space, indicating that the value range of each dimension is between and ; Among them, the action value (i.e., ) is mapped to the actual parameter increment through physical constraints, expressed as: ; In the formula, is the aperture value of the current frame (such as f / 2.0), is the aperture value of the previous frame , represents the aperture step coefficient (typical value 0.5, indicating that each unit corresponds to a 0.5-stop aperture change), is the shutter speed of the current frame (unit: millisecond), is the shutter speed of the previous frame The shutter speed, is the shutter step coefficient (typical value 10 ms, indicating a 10 ms change per unit corresponding change), is the gain value of the current frame (unit: dB), is the gain value of the previous frame ; is the gain step coefficient (typical value 5 dB, indicating a 5 dB change per unit corresponding change); S2.3: Write to the camera sensor registers via the I²C bus (such as the 0x3500 shutter register and 0x3508 gain register of OV5640), then trigger the camera to capture a frame of image, and calculate the and processing delay of the new image, and update the status to .

[0040] S2.4: Use the of the new image as the quantization index of the adjusted image quality, and calculate the immediate reward, expressed as: ; ; In the formula, represents the immediate reward at the current moment , represents the brightness approximation term, represents the average brightness of the adjusted image, represents the variance of the brightness of the adjusted image (quantized random noise), represents the dynamic brightness target, which is adaptively adjusted according to the light intensity , represents the highest brightness priority (accounting for 70% of the total reward), represents the noise suppression term, represents the adjusted high-frequency attenuation rate, represents the comprehensive index of quantized image noise and dust interference, represents the second-highest noise suppression weight (20%), represents the penalty weight of 5%, is the parameter out-of-bounds penalty term, represents the closed-loop response delay term; S2.5: Store the experience data in the priority replay buffer, and update the network according to the following rules, specifically including Critic network update, Actor network update, and network soft update.

[0041] Furthermore, update the Critic network: sample batch data and calculate the target value: ; In the formula, is the target value, which is the supervision signal for Critic network training. represents the reward discount factor, and represent the Critic network (dual network structure for stable training), represents the action at the next moment (generated by the Actor network), represents the action value function, and represent the parameters of the two Critic networks respectively; It should be noted that the parameters of the two Critic networks refer to the soft update (coefficient τ = 0.005) from the main Critic network and for delayed synchronization to stabilize value estimation; when calculating the target value, the minimum value of the outputs of the two is taken to suppress overestimation.

[0042] After processing every 2 frames of images, update the parameters of the Actor network once.

[0043] Specifically, adjust the Actor network (policy function) through the gradient ascent algorithm so that the actions (aperture, shutter, and gain increment) it outputs can maximize the long-term rewards evaluated by the Critic network.

[0044] The network soft update includes the Actor network update and the Critic network update.

[0045] Specifically, gradually synchronize the parameters of the current Actor network to the Actor network at a ratio of 0.5% to maintain policy stability; synchronize the parameters of the two main Critic networks to the corresponding Critic networks at a ratio of 0.5% respectively to prevent training oscillations.

[0046] S2.6: If the current light intensity or the dust concentration , start the multi-frame noise reduction process.

[0047] Specifically, continuously collect 3 frames of images that have been optimized for single-frame exposure but not denoised, calculate the weights based on the brightness fluctuations and dust concentration, and output the denoised image after weighted averaging; According to dynamically select ISP parameters: when the gain is high Enable the soft demosaicing algorithm at low shutter speeds Turn off the sharpening filter.

[0048] S2.7. If the reinforcement learning decision times out or the parameters exceed the bounds , it is determined as a timeout, and immediately switch to the physical model to calculate the exposure parameters (basic shutter speed and basic gain value): ; wherein represents the basic shutter speed (unit: millisecond), represents the current ambient light intensity, represents the calibration constant (fitted based on the camera optical characteristics and underground scene experiments), is an empirical value, representing the critical point when the dust transmittance is close to 0, represents the basic gain value (unit: dB), represents the normalization constant (calibrated based on the sensor noise model); Take the image after multi-frame noise reduction, environmental compensation, and reinforcement learning processing as the optimized low-noise image, and send the low-noise image and exposure parameters back to the main control (RK3566). If the quality verification fails (such as the average brightness deviates from the target value by ±15%), trigger sensor recalibration and record it in the optimization database.

[0049] S3: Input the low-noise image into the lightweight semantic segmentation model to identify the device area, personnel area, and dangerous area, generate a semantic mask, and extract the device feature vector and personnel action sequence.

[0050] Specifically, it includes the following steps: S3.1: Receive the low-noise image (YUV420 format), convert YUV420 to RGB format through bilinear interpolation, and perform normalization processing on each of the RGB three channels, linearly mapping the pixel value range from [0, 255] to [0, 1].

[0051] Unify the resolution of the low-noise image to 512×512 pixels, maintaining the original aspect ratio.

[0052] Among them, the insufficient area is filled with zeros (for example: the original image is 640×480, and after scaling, it is 512×384, and 64 pixel black areas are filled at the top and bottom); Apply contrast-limited adaptive histogram equalization (CLAHE) to the low-light area (for example: the global brightness mean is lower than 30), set the clipping limit to 2.0, and the block size to 8×8 to enhance the texture of the devices and the outlines of the personnel in the dark area; The exposure parameters Stitched into a 3D vector, it is input into the lightweight semantic segmentation model in parallel with the image data and participates in the segmentation decision as auxiliary features; S3.2: Use MobileViT-S as the backbone network. Input the preprocessed 512×512×3 image and the 3D exposure parameter vector. Extract multi-scale features through a series of convolutional layers (3×3 convolutional kernel, stride 2) and MobileViT blocks; Load the weights of the lightweight semantic segmentation model on the RK3566 chip, and the measured single-frame inference time ≤ 30ms; Example: When inputting a 512×512 image, the backbone network outputs 4 levels of feature maps with resolutions of 256×256, 128×128, 64×64, and 32×32 respectively.

[0053] Apply Atrous Spatial Pyramid Pooling (ASPP) to the feature map with a resolution of 128×128 output by the backbone network, set the atrous rates to 6, 12, and 18 respectively, fuse multi-scale context information, and output a binary mask of the device area; Add a Convolutional Block Attention Module (CBAM) to the feature map with a resolution of 64×64, strengthen the highlighted area of the reflective clothing through channel attention and spatial attention, and output a binary mask of the personnel area; Use heatmap supervision for the feature map with a resolution of 32×32, and label small targets through Gaussian kernel diffusion.

[0054] For example: The radius of the high-temperature equipment area is 5 pixels, and the radius of the landslide gravel area is 3 pixels, and output a probability map of the dangerous area; S3.3: Perform morphological closing operation (3×3 rectangular kernel, iterate 2 times) on the binary mask of the device area to fill the internal holes of the device structure.

[0055] Perform connected component analysis on the binary mask of the personnel area and filter out noise areas with an area less than 100 pixels.

[0056] Apply Non-Maximum Suppression (NMS) to the probability map of the dangerous area, merge adjacent areas with an overlap rate > 0.7, and generate the coordinates of the final dangerous area polygon.

[0057] Restore the three groups of masks (device, personnel, danger) to the original image resolution (for example: 640×480), crop and align them with the zero-padding area to ensure consistent coordinate mapping.

[0058] If the proportion of the effective area of any binary mask is lower than 5% (example: the effective pixels in the device binary mask < 5%), it is determined that the segmentation fails, and the exception handling process in step S3.5 is triggered.

[0059] S3.4: Crop the target circumscribed rectangle ROI within the binary mask of the device area (e.g., the ROI of the hydraulic support is 200×150 pixels), scale the ROI image to 224×224, input it into the pre-trained ResNet-18 network (removing the fully connected layer), and extract the 256-dimensional feature vector output by the global average pooling (GAP) layer.

[0060] Perform principal component analysis (PCA) on the 256-dimensional feature vector to reduce the dimension to 64 (retaining 95% of the variance) as the device feature vector.

[0061] For example, the feature vector of the hydraulic support includes material texture, shape moments, etc.

[0062] Based on the binary masks of the personnel area for 3 consecutive frames, calculate the Farneback optical flow (pyramid levels 3, window size 15) to generate a displacement field.

[0063] For the displacement field, calculate the mean and variance, and encode them into a 6-dimensional action vector.

[0064] For example: , update the personnel action sequence by sliding according to the time window.

[0065] S3.5: If the confidence levels of the device / personnel / hazardous area output by the lightweight semantic segmentation model are all <70% (judged by Softmax probability), immediately switch to the backup Fast-SCNN model, reduce the input resolution to 256×256 and perform re-inference.

[0066] If Fast-SCNN still fails (e.g., confidence <70%), re-adjust the exposure parameters (e.g., force the base gain value to be increased to 40 dB), re-acquire the image and execute steps 3.1 to 3.4.

[0067] S4: Obtain the knowledge graph from the mine server through the wireless Mesh network. The knowledge graph contains device topological relationships, safety regulation entities, and historical violation records, and perform graph attention matching between the device feature vector and the device nodes in the knowledge graph to output the probability of illegal operations.

[0068] Specifically, it includes the following steps: S4.1: According to the current underground working face location (e.g., the mining face number W-203), request the associated subgraph from the mine server through the wireless Mesh network.

[0069] Among them, the associated subgraph includes regional device topological relationships, safety regulation entities, and historical violation records; Adopt a differential update protocol (e.g., the BSDIFF algorithm) to only transmit the newly added or modified graph nodes and edges; Perform CRC check on the received associated subgraph. If the check fails, retransmit up to 3 times. If successful, update the local cache. S4.2: Calculate the cosine similarity between the device feature vector and the device node in the knowledge graph based on the 64-dimensional device feature vector. If the similarity > 0.85, it is determined as a successful match; otherwise, it is marked as an unknown device.

[0070] Example: The similarity between the characteristics of the hydraulic support and the graph node "ZY8800 hydraulic support" is 0.91, which is determined as a successful match; otherwise, it is marked as an unknown device.

[0071] For the unmatched devices, intercept the device nameplate area from the low-noise image, identify the text through Tesseract-OCR (such as "Model: ZY8800"), and forcibly associate the text with the graph node name.

[0072] S4.3: Combine the successfully matched device nodes, the personnel action sequence (the 6-dimensional action vector in step 3.4), and the dangerous area coordinates into the nodes and edges of the heterogeneous graph.

[0073] For example: Device node attributes: model, status, safety rules; Personnel node attributes: action type, location; Edge relationship: personnel-device operation relationship, device-dangerous area distance.

[0074] Input the constructed heterogeneous graph into a 2-layer GATv2 network (hidden layer 64 dimensions, number of attention heads 4), calculate the illegal association weight between nodes, and output the probability of illegal operation.

[0075] For example: When a person operates a shearer without wearing protective equipment, the probability of violation is 0.78.

[0076] S5: According to the probability of illegal operation and the personnel action sequence, trigger a graded warning signal through the vibration motor built into the helmet, and at the same time superimpose the dangerous area coordinates on the AR display interface and generate a safe operation guidance text.

[0077] Specifically, it includes the following steps: S5.1: Trigger vibration for the output probability of illegal operation and the personnel action sequence according to the following rules.

[0078] Specifically, if the probability of illegal operation < 0.3, it is determined as a low risk, trigger a single short vibration (pulse width 100ms, vibration intensity 50%), and a yellow semi-transparent device contour box is displayed on the AR interface.

[0079] If 0.3 ≤ the probability of illegal operation < 0.7, it is determined as a medium risk, trigger a double vibration (200ms pulse × 2, interval 300ms, intensity 70%), and a red flashing border and a safety distance line are superimposed on the AR.

[0080] If the probability of illegal operation is ≥0.7, it is judged as high risk and triggers continuous vibration (500ms pulse, 100% intensity). The AR interface is locked as a full-screen red curtain and the emergency stop icon is displayed.

[0081] Furthermore, through the personnel action sequence enhancement rule, if the displacement amplitude in the personnel action sequence is >15 pixels / frame and , it is determined to be strenuous exercise (such as running, falling), and the following enhancements are performed: Low / medium risk → vibration intensity is increased by 20% (such as the original 50% → 60%). High risk → AR interface additionally superimposes high-frequency flashing warning.

[0082] If the displacement amplitude is <2 pixels / frame for 5 seconds, turn off the vibration to avoid continuous interference.

[0083] During the vibration process, the helmet vibration motor is driven based on the PWM signal, with a frequency range of 50-200Hz, supporting millisecond-level response.

[0084] S5.2: Convert the danger zone polygon coordinates to the AR display coordinate system (such as ARKit's WorldSpace) and map them through the calibration matrix.

[0085] Example: Image coordinates (320, 240) → AR world coordinates (1.2m, -0.5m, 2.0m).

[0086] Use UnityEngine to draw dangerous areas, including high-temperature equipment areas and landslide and rubble areas.

[0087] Specifically, the high-temperature equipment area: red semi-transparent grid (Shader transparency 0.3). The landslide and rubble area: yellow flashing outline (frequency 2Hz).

[0088] S5.3: Generate safe operation guidance text by matching predefined templates based on violation cause entities (from the knowledge graph).

[0089] For example: Not wearing protective equipment → "Please put on insulating gloves and goggles immediately!" The equipment is not powered off → "Stop working! Turn off the power of the equipment and padlock it." Entering the danger zone → "Retreat to the safe area! No passage in this area." Set the safety operation guidance text based on the language parameters set by the helmet, call the pre-compiled text library, and render it in real time to the lower right corner of the AR interface.

[0090] It should be noted that when the wireless Mesh network is disconnected, the vibration mode is switched to high risk for continuous vibration. At the same time, "Network interruption, exercise caution during operation" is displayed on the AR interface, and the locally cached safety guidance text is used. If the current of the vibration motor is abnormal (such as >500 mA), the vibration is automatically turned off and the helmet LED red light is lit.

[0091] This embodiment also provides a computer device applicable to the case of the mine intelligent helmet safety warning method based on image recognition, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the mine intelligent helmet safety warning method based on image recognition as proposed in the above embodiment.

[0092] This computer device can be a terminal. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of this computer device is used to provide computing and control capabilities. The memory of this computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of this computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of this computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of this computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the outer shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0093] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for realizing the safety warning of a mine intelligent helmet based on image recognition proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, magnetic disk or optical disc.

[0094] In summary, by combining real-time environmental parameters such as dust concentration and light intensity with the optical transmission model, the present invention constructs a physical constraint optimization space for exposure parameters based on the reinforcement learning algorithm, significantly improving the image quality in high-dynamic environments. By extracting the spatio-temporal correlation features of multiple frames of images, establishing a coupling relationship model between dust scattering and illuminance attenuation, and constructing a semantic segmentation feature enhancement mechanism assisted by exposure parameters, the target recognition effect in complex environments is effectively improved. In addition, by adopting physical constraint conditions based on safety regulations in knowledge graph reasoning and deeply fusing the device state features with the dynamic knowledge graph nodes, the physical interpretability of illegal operation determination is realized, significantly improving the response speed and decision reliability. Finally, through the collaborative optimization of the multi-modal warning mechanism, the intelligent helmet demonstrates excellent real-time performance and environmental adaptability under complex underground working conditions.

[0095] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A safety warning method for a mining smart helmet based on image recognition, characterized in that: include, Collecting underground environment data in real time, including dust concentration and light intensity; Based on the reinforcement learning algorithm, the current exposure parameters are calculated according to the dust concentration and light intensity, and the camera is driven to adjust the aperture and shutter speed to output an optimized low-noise image; Inputting the low-noise image into a lightweight semantic segmentation model, identifying equipment areas, personnel areas, and dangerous areas, generating semantic masks, and extracting equipment feature vectors and personnel action sequences; Obtain a knowledge graph from a mine server through a wireless mesh network. The knowledge graph contains equipment topology relationships, safety regulations entities, and historical violation records. The equipment feature vector is matched with the equipment nodes in the knowledge graph by graph attention, and the probability of illegal operations is output. According to the probability of illegal operation and the sequence of personnel actions, a graded alarm signal is triggered by the vibration motor built into the helmet. At the same time, the coordinates of the dangerous area are superimposed on the AR display interface, and a safe operation guidance text is generated.

2. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 1, characterized in that: After the underground environment data is collected in real time, the dust concentration data is processed by sliding average filtering, the light intensity data is processed by low-pass filtering, and the two types of data are mapped to a unified numerical range through a normalization operation.

3. The safety warning method for a mining intelligent helmet based on image recognition according to claim 1 is characterized in that: The reinforcement learning algorithm constructs a state space based on the real-time changing trends of dust concentration and light intensity, and generates exposure parameters by balancing image brightness optimization and noise suppression; The specific steps of generating exposure parameters are as follows: Collect dust concentration, light intensity and real-time image data from the camera in real time, and build a multi-dimensional vector representing the environmental dynamics and equipment status by combining the current exposure parameters and processing delay; Input the multidimensional vector into a pre-trained deep reinforcement learning network, output aperture, shutter and gain adjustment instructions, and generate actions through a dynamic strategy that balances brightness optimization and noise suppression; Convert adjustment instructions into actual exposure parameters according to preset step coefficients, write to camera registers to trigger image acquisition, and synchronously update image quality indicators and processing delays; The network parameters are optimized based on the new image indicators. When an environmental anomaly or decision timeout is detected, the basic parameters are calculated by switching to the physical model.

4. The method for safety early warning of a mining intelligent helmet based on image recognition as claimed in claim 3 is characterized in that: The lightweight semantic segmentation model adopts a lightweight convolutional neural network to generate semantic masks of equipment areas, personnel areas and danger areas through multi-scale feature fusion, and extracts equipment contour feature vectors and personnel continuous motion trajectories based on the masks.

5. The method for safety early warning of a mining intelligent helmet based on image recognition as claimed in claim 4, characterized in that: The knowledge graph obtains device topology relationships and safety procedure entities in real time through wireless networks, matches device feature vectors with device nodes in the graph by attention weights, and outputs violation probabilities associated with historical violation records.

6. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 5, characterized in that: The graph attention matching adopts multi-channel association calculation, and by comparing the semantic similarity between the device feature vector and the knowledge graph node, it screens out potential illegal operation nodes and calculates the probability value.

7. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 6, characterized in that: The graded warning signal is divided into threshold intervals according to the probability of violation, and a difference warning is triggered through the intensity-frequency combination of the vibration motor, while a danger zone outline frame and safety guidance text are superimposed on the AR interface.

8. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 7, characterized in that: The coordinate superposition of the dangerous area adopts the three-dimensional coordinate conversion method to map the position of the dangerous area in the mine space coordinate system to the perspective projection layer of the AR display interface in real time.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the safety warning method for a mining smart helmet based on image recognition are implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the safety warning method for a mining smart helmet based on image recognition are implemented as described in any one of claims 1 to 8.

Images