A safety warning method for mining smart helmets based on image recognition

Abstract

The present invention discloses a safety warning method for a mining smart helmet based on image recognition, which relates to the field of mining safety technology. The method comprises the following steps: collecting underground environmental data in real time, wherein the underground environmental data includes dust concentration and light intensity; calculating the current exposure parameters according to the dust concentration and light intensity based on a reinforcement learning algorithm, driving the camera to adjust the aperture and shutter speed, and outputting an optimized low-noise image; triggering a graded alarm signal through a vibration motor built into the helmet according to the probability of illegal operation and the sequence of personnel actions, superimposing the coordinates of the dangerous area on the AR display interface, and generating a safe operation guidance text. The present invention extracts the spatiotemporal correlation features of multiple frames of images, establishes a coupled relationship model of dust scattering and illumination attenuation, and constructs a semantic segmentation feature enhancement mechanism assisted by exposure parameters, thereby effectively improving the target recognition effect in complex environments.

Description

Technical Field

[0001] The present invention relates to the field of mining safety technology, and in particular to a mining intelligent helmet safety early warning method based on image recognition. Background Art

[0002] Mine safety monitoring has become a core component of modern mines' intelligent transformation. Intelligent mining monitoring equipment, a key component for underground personnel safety, relies on environmental perception accuracy and risk response speed to directly impact production safety efficiency and accident prevention capabilities. Therefore, there is an urgent need for dynamic identification of complex underground operating conditions and the development of multimodal alarm mechanisms.

[0003] In existing technologies, traditional camera parameter adjustments rely on fixed thresholds or manual presets, and are unable to dynamically adapt to the nonlinear coupling relationship between dust concentration and light intensity. This results in poor image noise suppression and affects the accuracy of subsequent semantic segmentation. Existing semantic segmentation models have high computational complexity, making real-time processing difficult on the embedded hardware of mining helmets. They also lack the ability to interact with dynamic knowledge graphs in real time, resulting in delayed judgments of illegal operations. Furthermore, the alarm mechanism is largely unstructured, and existing vibration feedback and visual cues lack hierarchical linkage, making it impossible to accurately match operational scenarios with different risk levels. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a safety warning method for intelligent mining helmets based on image recognition to solve the problems of insufficient dynamic image optimization, delayed real-time risk identification and low multimodal alarm accuracy in complex mine environments.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, the present invention provides a safety warning method for a smart mining helmet based on image recognition, which includes real-time collection of underground environmental data, wherein the underground environmental data includes dust concentration and light intensity; based on a reinforcement learning algorithm, the current exposure parameters are calculated according to the dust concentration and light intensity, and the camera is driven to adjust the aperture and shutter speed to output an optimized low-noise image; the low-noise image is input into a lightweight semantic segmentation model to identify equipment areas, personnel areas and dangerous areas, generate semantic masks, and extract equipment feature vectors and personnel action sequences; a wireless Mesh network is used to obtain a knowledge graph from a mine server, calculate the equipment feature vectors and the equipment nodes of the knowledge graph, and calculate the violation association weights between nodes through graph attention matching, and output the probability of illegal operation; according to the illegal operation probability and the personnel action sequence, a graded alarm signal is triggered by the vibration motor built into the helmet, and the coordinates of the dangerous area are superimposed on the AR display interface, and a safe operation guidance text is generated.

[0008] As a preferred solution of the image recognition-based safety warning method for mining smart helmets described in the present invention, after the real-time collection of underground environmental data, the dust concentration data is subjected to sliding average filtering, the light intensity data is subjected to low-pass filtering, and the data is encapsulated into a fixed-format data frame and sent to the main control through the SPI interface.

[0009] As a preferred solution of the image recognition-based safety warning method for mining smart helmets described in the present invention, the reinforcement learning algorithm constructs a state space based on the real-time changing trends of dust concentration and light intensity, and generates exposure parameters by balancing image brightness optimization and noise suppression;

[0010] The specific steps of generating exposure parameters are as follows:

[0011] Real-time collection of dust concentration, light intensity, and real-time image data from the camera, combined with current exposure parameters and processing delays, to construct a multi-dimensional vector representing environmental dynamics and equipment status;

[0012] Inputting the multidimensional vector into a pre-trained deep reinforcement learning network, the network outputs aperture, shutter, and gain adjustment instructions, and generates actions through a dynamic strategy that balances brightness optimization and noise suppression;

[0013] Converts adjustment instructions into actual exposure parameters according to preset step coefficients, writes them into camera registers to trigger image acquisition, and simultaneously updates image quality indicators and processing delays;

[0014] The network parameters are optimized based on the new image metrics. When an environmental anomaly or decision timeout is detected, the basic parameters are calculated by switching to the physical model.

[0015] As a preferred solution of the image recognition-based safety warning method for mining smart helmets described in the present invention, the lightweight semantic segmentation model adopts a lightweight convolutional neural network to generate semantic masks of equipment areas, personnel areas and dangerous areas through multi-scale feature fusion, and extracts equipment contour feature vectors and personnel continuous motion trajectories based on the masks.

[0016] As a preferred solution of the image recognition-based safety warning method for smart mining helmets described in the present invention, the knowledge graph requests an associated subgraph from the mine server through a wireless Mesh network; the associated subgraph includes regional equipment topology relationships, safety regulations entities, and historical violation records.

[0017] As a preferred solution of the image recognition-based safety warning method for mining smart helmets described in the present invention, the graph attention matching constructs a heterogeneous graph containing device nodes, personnel action sequences and dangerous area coordinates, and uses a two-layer GATv2 network to calculate the violation association weights between nodes.

[0018] As a preferred solution of the image recognition-based safety warning method for mining smart helmets described in the present invention, the graded alarm signal is divided into threshold intervals according to the probability of violation, and the difference alarm is triggered by the intensity frequency combination of the vibration motor, while the danger zone outline box and safety guidance text are superimposed on the AR interface.

[0019] As an optimal solution of the image recognition-based safety warning method for smart mining helmets described in the present invention, the dangerous area coordinate superposition adopts a three-dimensional coordinate conversion method to map the dangerous area position in the mine space coordinate system to the perspective projection layer of the AR display interface in real time.

[0020] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the image recognition-based safety warning method for a mining smart helmet as described in the first aspect of the present invention is implemented.

[0021] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the image recognition-based safety warning method for a mining smart helmet as described in the first aspect of the present invention.

[0022] The beneficial effects of the present invention are as follows: by combining real-time environmental parameters such as dust concentration and light intensity with the optical transmission model, a physical constraint optimization space for exposure parameters is constructed based on a reinforcement learning algorithm, which significantly improves the image quality in a high dynamic environment. By extracting the spatiotemporal correlation features of multi-frame images, a coupling relationship model of dust scattering and illumination attenuation is established, and an exposure parameter-assisted semantic segmentation feature enhancement mechanism is constructed, which effectively improves the target recognition effect in a complex environment. In addition, by adopting physical constraints based on safety regulations in knowledge graph reasoning, the equipment status features are deeply integrated with the dynamic knowledge graph nodes, which realizes the physical explainability of illegal operation judgments and significantly improves the response speed and decision reliability. Finally, through the collaborative optimization of the multimodal alarm mechanism, the smart helmet exhibits excellent real-time performance and environmental adaptability under complex working conditions underground. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is a flow chart of the safety warning method for mining smart helmets based on image recognition in Example 1.

[0025] Figure 2 This is a flow chart of sensor data processing and fault tolerance in Example 1.

[0026] Figure 3 This is a flowchart of the reinforcement learning exposure parameter adjustment in Example 1.

[0027] Figure 4 This is a flowchart of image processing and violation determination in Example 1. DETAILED DESCRIPTION

[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0030] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0031] Example 1, with reference to Figures 1 to 4 This embodiment provides a safety warning method for a mining smart helmet based on image recognition, comprising the following steps:

[0032] S1: Collecting downhole environmental data in real time, wherein the downhole environmental data includes dust concentration and light intensity.

[0033] Specifically, the following steps are included:

[0034] S1.1: Collect underground environmental data (dust concentration and light intensity) through dust sensors and light sensors. Apply a sliding average filter (window size of 5) to the dust data to suppress transient noise (such as blasting interference). Apply a low-pass filter to the light data to eliminate high-frequency power frequency interference (such as flickering mining lamps).

[0035] If data anomalies (including but not limited to communication interruption, data jump, etc.) are detected multiple times in succession, the sensor self-check process is triggered and the backup data source is switched.

[0036] It should be noted that in the intelligent underground safety monitoring solution for coal mines, the "backup data source" refers to an alternative data acquisition method that is automatically activated when the main sensor malfunctions. This includes a three-level fault-tolerance mechanism: For example, if the main dust sensor malfunctions, it immediately switches to a backup dust sensor deployed in the same location. The camera captures image features combined with calibrated visibility parameters to estimate dust concentration. Image brightness analysis can infer light intensity, and preset safety parameter defaults serve as a final safeguard. The backup data undergoes dynamic weighted fusion processing, and abnormal events are recorded in real time and uploaded to the monitoring center to ensure continuous and stable monitoring. The visibility estimation method and image analysis technology used are both mature solutions currently used in mining equipment.

[0037] S1.2: The microcontroller encapsulates the filtered dust concentration, light intensity, timestamp, and dust / light sensor status into a fixed-format data frame and sends it to the master controller (RK3566) through the SPI interface.

[0038] If data processing times out (exceeds the preset response period) in the main control (RK3566), the linear prediction algorithm is used to complete the current data. When sensor communication is interrupted, the main control automatically switches to the last calibrated measurement value and superimposes safety default values ​​(such as medium dust concentration and low light intensity), while triggering a short vibration alarm on the helmet.

[0039] It should be noted that exceeding the preset response period refers to the situation where the master controller (RK3566) fails to complete data processing within the specified time. The response period is dynamically adjusted based on the dust / light sensor sampling frequency (1-5Hz) and is usually set to 50-100ms to ensure real-time performance. After the timeout, linear prediction is automatically enabled to complete the current frame based on historical data trends.

[0040] Measurements refer to valid results from rigorously verified downhole environmental data collected by dust and light sensors. Specifically, the dust sensor measures concentration using laser scattering, while the light sensor measures intensity through photoelectric conversion. These downhole environmental data are first checked for signal integrity and within a reasonable range (e.g., 0-2000 μg / m³ for dust and 0-65535 Lux for light), and then filtered to eliminate interference. Only downhole environmental data that passes all these verification steps is considered a valid "measurement." When switching data sources, the most recent measurement result that has passed all verifications is automatically recalled to ensure authenticity and reliability.

[0041] S1.3: The main control periodically sends the downhole environment data to the camera digital signal processor (DSP) through the high-speed LVDS interface.

[0042] When high dust concentration or low light intensity is detected, the camera immediately suspends non-critical tasks (such as logging and background data synchronization) and prioritizes exposure parameter adjustment.

[0043] Specifically, after the camera's digital signal processor (DSP) receives the raw image data, it extracts the brightness channel (YUV / Y component) through denoising and grayscale processing, and calculates the global average brightness. When the global average brightness continues to be lower than the camera's preset brightness standard (for example, a 50lux environment corresponds to a Y channel value below 30) and the proportion of dark area pixels exceeds 90%, low-light judgment is triggered; at the same time, the attenuation degree of the image's high-frequency details is analyzed. If the current image's high-frequency energy drops by more than 30% compared to the baseline data of a dust-free scene, it is directly judged as abnormal dust concentration; finally, the image judgment result is forced to be aligned with the data of the dust sensor and photosensitive unit, and the hardware sensor reading is preferentially adopted as the final conclusion. The environmental conditions are dynamically adapted through time or scene modes (such as automatically lowering the brightness judgment benchmark at night) to ensure the uniqueness and certainty of the detection logic.

[0044] It should be noted that after the camera completes the exposure adjustment, it returns a confirmation signal to the main control. If no confirmation signal is received within the time limit, the main control resends the downhole environment data to ensure the real-time control closed loop.

[0045] S2. Based on the reinforcement learning algorithm, the current exposure parameters are calculated according to the dust concentration and light intensity, and the camera is driven to adjust the aperture and shutter speed to output an optimized low-noise image.

[0046] Specifically, the following steps are included:

[0047] S2.1: Read the current dust concentration and light intensity , and synchronously receive the global brightness mean of the previous frame image returned by the camera DSP , brightness standard deviation , high-frequency energy ratio , combined with the current exposure parameters and frame processing delay , construct a 9-dimensional state vector :

[0048] ;

[0049] S2.2: The state vector Input the pre-trained TD3-DDPG algorithm Actor network (network structure: 9→256→128→3) and output the normalized action vector .

[0050] Specifically, the normalized motion vector , expressed as:

[0051] ;

[0052] Where, is the aperture increment, which indicates the relative change in the aperture opening size. is the shutter speed increment, which indicates the relative change in sensor exposure time. is the gain increment, which indicates the relative change of the sensor analog / digital gain. is a three-dimensional cube space, indicating that the value range of each dimension is arrive between;

[0053] Among them, the action value (i.e. ) is mapped to actual parameter increments through physical constraints, expressed as:

[0054] ;

[0055] Where, Current frame The aperture value (such as f / 2.0), Previous frame The aperture value, Indicates the aperture step coefficient (typical value 0.5, indicating per unit Corresponding to 0.5-stop aperture change), Current frame Shutter speed (in milliseconds), Previous frame shutter speed, The shutter step coefficient (typical value is 10ms, indicating that each unit corresponding to 10ms change), is the gain value of the current frame (unit: dB), Previous frame The gain value, is the gain step coefficient (typical value is 5dB, indicating per unit corresponding to a 5dB change);

[0056] S2.3: Use the I²C bus to Write to the camera sensor register (such as the shutter register 0x3500 and the gain register 0x3508 of OV5640), then trigger the camera to capture a frame of image and calculate the new image through DSP. and processing delays , update status to .

[0057] S2.4: The new image As a quantitative indicator of the adjusted image quality, the instant reward is calculated and expressed as:

[0058] ;

[0059] ;

[0060] Where, Indicates the current time Instant rewards, represents the brightness approximation term, represents the average brightness of the image after adjustment, represents the brightness variance after adjustment, Indicates a dynamic brightness target, which changes with the light intensity Adaptive adjustment, Indicates that brightness has the highest priority (accounting for 70% of the total rewards), represents the noise suppression term, represents the image brightness variance (quantized random noise), represents the adjusted high frequency attenuation rate, Represents a comprehensive index of quantitative image noise and dust interference, Indicates that the noise suppression weight is the second highest (20%), Indicates a penalty weight of 5%. is the parameter out-of-bounds penalty term, represents the closed-loop response delay term;

[0061] S2.5: Empirical data Store in the priority replay buffer and update the network according to the following rules, including critic network update, actor network update and network soft update.

[0062] Furthermore, the critic network is updated by sampling batch data and calculating the target Q value:

[0063] ;

[0064] Where, is the target Q value, the supervisory signal for Critic network training, represents the reward discount factor, and represents the Critic network (double Q network structure, used for stable training), Represents the action at the next moment (generated by the Actor network), represents the action-value function, and Represent the parameters of the two Critic networks respectively;

[0065] It should be noted that the parameters of the two critic networks are obtained by soft updating (coefficient τ = 0.005) from the main critic network. and Delayed synchronization is used to stabilize Q-value estimation; when calculating the target value, the minimum value of the two outputs is taken to suppress over-estimation.

[0066] After processing every two frames of images, the Actor network parameters are updated.

[0067] Specifically, the Actor network (strategy function) is adjusted through the gradient ascent algorithm so that its output actions (aperture, shutter, and gain increment) can maximize the long-term benefits evaluated by the Critic network.

[0068] Network soft updates include Actor network updates and Critic network updates.

[0069] Specifically, the parameters of the current Actor network are gradually synchronized to the Actor network at a ratio of 0.5% to maintain policy stability; the parameters of the two main Critic networks are synchronized to the corresponding Critic networks at a ratio of 0.5% respectively to prevent training oscillations.

[0070] S2.6: If the current light intensity or dust concentration , start the multi-frame noise reduction process.

[0071] Specifically, three frames of images that have been optimized for single-frame exposure but not subjected to noise reduction are continuously collected, and weights are calculated based on brightness fluctuations and dust concentration. The weighted average is then used to output the noise-reduced image.

[0072] according to Dynamically select ISP parameters: when high gain Enable soft demosaicing algorithm, slow shutter speed Turn off the sharpening filter.

[0073] S2.7. If the reinforcement learning decision times out Or the parameter is out of range , it is determined to be timed out and the system immediately switches to the physical model to calculate the exposure parameters (basic shutter speed and basic gain value):

[0074] ;

[0075] Where, Indicates the basic shutter speed (unit: milliseconds), Indicates the current ambient light intensity. represents the calibration constant (based on the optical characteristics of the camera and the experimental fitting of the downhole scene), It is an empirical value, indicating the critical point when the dust transmittance is close to 0. Indicates the basic gain value (unit: dB), represents the normalization constant (calibrated based on the sensor noise model);

[0076] The image processed by multi-frame noise reduction, environmental compensation and reinforcement learning is used as the optimized low-noise image. The low-noise image and exposure parameters are sent back to the main control (RK3566). If the quality verification fails (for example, the average brightness deviates from the target value by ±15%), the sensor recalibration is triggered and recorded in the optimization database.

[0077] S3: Input the low-noise image into a lightweight semantic segmentation model to identify the equipment area, personnel area and danger area, generate a semantic mask, and extract the equipment feature vector and personnel action sequence.

[0078] Specifically, the following steps are included:

[0079] S3.1: Receive a low-noise image (YUV420 format), convert the YUV420 to RGB format through bilinear interpolation, and normalize the three RGB channels separately to linearly map the pixel value range from [0, 255] to [0, 1].

[0080] The low-noise images were uniformly scaled to 512 × 512 pixels, preserving the original aspect ratio.

[0081] The insufficient areas are filled with zeros (for example, if the original image is 640×480 and is scaled to 512×384, 64 pixels of black area are filled above and below).

[0082] For low-light areas (e.g., where the global mean luminance is less than 30), contrast-limited adaptive histogram equalization (CLAHE) is applied, with a clipping limit of 2.0 and a block size of 8×8, to enhance device textures and human outlines in dark areas.

[0083] Exposure parameters The data are concatenated into a 3D vector and fed into a lightweight semantic segmentation model in parallel with the image data to participate in segmentation decision making as auxiliary features.

[0084] S3.2: MobileViT-S is used as the backbone network. It takes as input the preprocessed 512×512×3 image and the 3D exposure parameter vector. Multi-scale features are extracted by concatenating convolutional layers (3×3 kernels, stride 2) with MobileViT blocks.

[0085] Loading lightweight semantic segmentation model weights on the RK3566 chip, the measured single-frame inference time is ≤30ms

[0086] Example: When a 512×512 image is input, the backbone network outputs 4-level feature maps with resolutions of 256×256, 128×128, 64×64, and 32×32 respectively.

[0087] Atrous Spatial Pyramid Pooling (ASPP) is applied to the 128×128 resolution feature map output by the backbone network, with dilation rates set to 6, 12, and 18, respectively. Multi-scale context information is integrated to output a binary mask of the device region.

[0088] A convolutional block attention module (CBAM) is added to the 64×64 resolution feature map to enhance the highlight area of ​​the reflective clothing through channel attention and spatial attention, and output a binary mask of the person area;

[0089] Heat map supervision is used on the 32×32 resolution feature map, and small objects are annotated by Gaussian kernel diffusion.

[0090] For example, if the radius of the high-temperature equipment area is 5 pixels and the radius of the landslide and gravel area is 3 pixels, the probability map of the dangerous area will be output;

[0091] S3.3: Perform a morphological closing operation (3×3 rectangular kernel, 2 iterations) on the binary mask of the device area to fill the holes inside the device structure.

[0092] Perform connected domain analysis on the binary mask of the personnel area and filter out noise areas with an area less than 100 pixels.

[0093] Non-maximum suppression (NMS) was applied to the danger area probability map, and adjacent regions with an overlap ratio > 0.7 were merged to generate the final danger area polygon coordinates.

[0094] The three sets of masks (equipment, personnel, and hazards) are restored to the original image resolution (e.g., 640×480), cropped and aligned with the zero-filled area to ensure consistent coordinate mapping.

[0095] If the effective area of ​​any binary mask is less than 5% (for example, the effective pixels in the device binary mask are <5%), the segmentation is judged to be failed, triggering the exception handling process of step S3.5.

[0096] S3.4: Capture the target circumscribed rectangular ROI within the binary mask of the equipment area (for example, the ROI of the hydraulic support is 200×150 pixels). Scale the ROI image to 224×224 and input it into the pre-trained ResNet-18 network (remove the fully connected layer). Extract the 256-dimensional feature vector output by the global average pooling (GAP) layer.

[0097] Perform principal component analysis (PCA) on the 256-dimensional feature vector and reduce its dimension to 64 dimensions (retaining 95% variance), which is used as the device feature vector.

[0098] For example: the feature vector of a hydraulic support includes material texture, shape moment, etc.

[0099] Based on the binary mask of the personnel area in three consecutive frames, the Farneback optical flow is calculated (the number of pyramid layers is 3 and the window size is 15). Displacement field.

[0100] right The displacement field statistics mean and variance are encoded as a 6-dimensional motion vector.

[0101] For example: , update the personnel action sequence by sliding the time window.

[0102] S3.5: If the confidence levels of the equipment / personnel / hazardous area output by the lightweight semantic segmentation model are all less than 70% (determined by Softmax probability), immediately switch to the backup Fast-SCNN model, reduce the input resolution to 256×256, and re-inference.

[0103] If Fast-SCNN still fails (e.g., confidence level < 70%), readjust the exposure parameters (e.g., forcefully increase the base gain to 40 dB), reacquire the image, and perform steps 3.1 to 3.4.

[0104] S4: Use a wireless mesh network to obtain the knowledge graph from the mine server, calculate the device feature vector and the device nodes of the knowledge graph, and calculate the violation association weights between nodes through graph attention matching to output the probability of illegal operations.

[0105] Specifically, the following steps are included:

[0106] S4.1: Based on the current underground working face location (for example, mining working face number W-203), request the associated submap from the mine server through the wireless mesh network.

[0107] Among them, the associated subgraph includes regional equipment topology relationships, safety regulations entities, and historical violation records;

[0108] Use a differential update protocol (e.g., BSDIFF algorithm) to transmit only newly added or modified graph nodes and edges;

[0109] Perform CRC check on the received associated subgraph. If the check fails, retransmit it up to 3 times; if successful, update the local cache.

[0110] S4.2: Based on the 64-dimensional device feature vector, calculate the cosine similarity between the device feature vector and the device node in the knowledge graph. If the similarity is > 0.85, it is considered a successful match, otherwise it is marked as an unknown device.

[0111] Example: If the similarity between the hydraulic support feature and the atlas node "ZY8800 hydraulic support" is 0.91, it is considered a successful match; otherwise, it is marked as an unknown device.

[0112] For unmatched devices, the device nameplate area is captured from the low-noise image, and the text (such as "Model: ZY8800") is recognized through Tesseract-OCR, and the text is forcibly associated with the atlas node name.

[0113] S4.3: Combine the successfully matched device nodes, personnel action sequences (6-dimensional action vectors from step 3.4), and hazardous area coordinates into nodes and edges of a heterogeneous graph.

[0114] For example: equipment node attributes: model, status, safety rules; personnel node attributes: action type, location; edge relationships: personnel-equipment operation relationship, equipment-dangerous area distance.

[0115] Input the constructed heterogeneous graph into a 2-layer GATv2 network (64-dimensional hidden layer, 4 attention heads), calculate the illegal association weights between nodes, and output the probability of illegal operations.

[0116] For example, a person operating a coal mining machine does not wear protective equipment, and the probability of violation is 0.78.

[0117] S5: Based on the probability of illegal operation and the sequence of personnel actions, a graded alarm signal is triggered through the vibration motor built into the helmet, and the coordinates of the dangerous area are superimposed on the AR display interface, and a safe operation guidance text is generated.

[0118] Specifically, the following steps are included:

[0119] S5.1: Based on the outputted probability of illegal operation and the sequence of personnel actions, vibration is triggered according to the following rules.

[0120] Specifically, if the probability of illegal operation is less than 0.3, it is judged as low risk, triggering a single short vibration (100ms pulse width, vibration intensity 50%), and the AR interface displays a yellow translucent device outline frame.

[0121] If 0.3≤Probability of illegal operation<0.7, it is judged as medium risk and triggers double vibration (200ms pulse × 2, 300ms interval, 70% intensity), and AR superimposes a red flashing border and a safety distance line.

[0122] If the probability of illegal operation is ≥0.7, it is judged as high risk and triggers continuous vibration (500ms pulse, 100% intensity). The AR interface is locked as a full-screen red screen and the emergency stop icon is displayed.

[0123] Furthermore, through the personnel action sequence enhancement rule, if the displacement amplitude in the personnel action sequence is greater than 15 pixels / frame and If the user is deemed to be exercising vigorously (e.g., running or falling), the following enhancements will be implemented: Low / Medium risk → Vibration intensity increased by 20% (e.g., 50% to 60%). High risk → An additional high-frequency flashing warning will be added to the AR interface.

[0124] If the displacement amplitude is <2 pixels / frame for 5 seconds, turn off the vibration to avoid continuous interference.

[0125] During the vibration process, the helmet vibration motor is driven based on the PWM signal, with a frequency range of 50-200Hz, supporting millisecond-level response.

[0126] S5.2: Convert the danger zone polygon coordinates to the AR display coordinate system (such as ARKit's WorldSpace) and map them through the calibration matrix.

[0127] Example: Image coordinates (320, 240) → AR world coordinates (1.2m, -0.5m, 2.0m).

[0128] Use UnityEngine to draw dangerous areas, including high-temperature equipment areas and landslide and debris areas.

[0129] Specifically, high-temperature equipment areas are marked with a red semi-transparent grid (shader transparency 0.3), while landslide and debris areas are marked with a yellow flashing outline (frequency 2Hz).

[0130] S5.3: Generate safe operation guidance text based on the violation cause entity (from the knowledge graph) matching predefined templates.

[0131] For example: Not wearing protective gear → "Please put on insulating gloves and goggles immediately!" Operating equipment without powering off → "Stop working! Turn off the power to the equipment and padlock it." Entering a hazardous area → "Retreat to a safe area! No entry into this area."

[0132] Set the safety operation guidance text based on the language parameters set in the helmet, call the pre-compiled text library, and render it in real time to the lower right corner of the AR interface.

[0133] It should be noted that when the wireless Mesh network is disconnected, the vibration mode will be switched to high-risk for continuous vibration. At the same time, "Network interruption, operate with caution" will be displayed on the AR interface, and the locally cached safety guidance text will be used. If the vibration motor current is abnormal (such as >500mA), the vibration will be automatically turned off and the helmet LED red light will be turned on.

[0134] This embodiment also provides a computer device, which is suitable for the case of a safety warning method for a smart helmet for mining based on image recognition, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the safety warning method for a smart helmet for mining based on image recognition proposed in the above embodiment.

[0135] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0136] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for implementing a safety warning method for a mining smart helmet based on image recognition as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0137] In summary, the present invention combines real-time environmental parameters such as dust concentration and light intensity with the optical transmission model, and constructs a physical constraint optimization space for exposure parameters based on the reinforcement learning algorithm, which significantly improves the image quality in high dynamic environments. By extracting the spatiotemporal correlation features of multi-frame images, establishing a coupling relationship model between dust scattering and illumination attenuation, and constructing a semantic segmentation feature enhancement mechanism assisted by exposure parameters, the target recognition effect in complex environments is effectively improved. In addition, by adopting physical constraints based on safety regulations in knowledge graph reasoning, the equipment status features are deeply integrated with the dynamic knowledge graph nodes, achieving the physical explainability of illegal operation judgments, and significantly improving the response speed and decision reliability. Finally, through the collaborative optimization of the multimodal alarm mechanism, the smart helmet exhibits excellent real-time performance and environmental adaptability under complex working conditions underground.

[0138] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A safety warning method for a mining smart helmet based on image recognition, characterized by: include, Real-time collection of downhole environmental data, including dust concentration and light intensity; Based on a reinforcement learning algorithm, the current exposure parameters are calculated according to dust concentration and light intensity, and the camera is driven to adjust the aperture and shutter speed to output an optimized low-noise image. Inputting the low-noise image into a lightweight semantic segmentation model to identify equipment areas, personnel areas, and danger areas, generate semantic masks, and extract equipment feature vectors and personnel action sequences; A wireless mesh network is used to obtain a knowledge graph from a mine server. The cosine similarity between the device feature vector and the device nodes in the knowledge graph is calculated. The nodes and edges of a heterogeneous graph are constructed by matching the cosine similarities. The constructed heterogeneous graph is then fed into a two-layer GATv2 network, where the violation association weights between nodes are calculated and the probability of a violation operation is output. Based on the probability of illegal operations and the sequence of personnel actions, a graded alarm signal is triggered through the helmet's built-in vibration motor. At the same time, the coordinates of the dangerous area are superimposed on the AR display interface, and a safe operation guidance text is generated.

2. The method for safety early warning of a mining smart helmet based on image recognition according to claim 1, characterized in that: After the downhole environmental data is collected in real time, the dust concentration data is subjected to sliding average filtering, the light intensity data is subjected to low-pass filtering, and the data are packaged into fixed-format data frames and sent to the main control through the SPI interface.

3. The safety warning method for a mining smart helmet based on image recognition according to claim 1, characterized in that: The reinforcement learning algorithm constructs a state space based on the real-time changing trends of dust concentration and light intensity, and generates exposure parameters by balancing image brightness optimization and noise suppression; The specific steps of generating exposure parameters are as follows: Real-time collection of dust concentration, light intensity, and real-time image data from the camera, combined with current exposure parameters and processing delays, to construct a multi-dimensional vector representing environmental dynamics and equipment status; Inputting the multidimensional vector into a pre-trained deep reinforcement learning network, the network outputs aperture, shutter, and gain adjustment instructions, and generates actions through a dynamic strategy that balances brightness optimization and noise suppression; Converts adjustment instructions into actual exposure parameters according to preset step coefficients, writes them into camera registers to trigger image acquisition, and simultaneously updates image quality indicators and processing delays; The network parameters are optimized based on the new image metrics. When an environmental anomaly or decision timeout is detected, the basic parameters are calculated by switching to the physical model.

4. The method for safety early warning of a mining smart helmet based on image recognition according to claim 3, characterized in that: The lightweight semantic segmentation model adopts a lightweight convolutional neural network to generate semantic masks of equipment areas, personnel areas and danger areas through multi-scale feature fusion, and extracts equipment contour feature vectors and personnel continuous motion trajectories based on the masks.

5. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 4, characterized in that: The knowledge graph is an associated subgraph requested from the mine server through the wireless mesh network; the associated subgraph includes regional equipment topology relationships, safety regulations entities, and historical violation records.

6. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 5, characterized in that: The matching cosine similarity refers to combining successfully matched device nodes, personnel action sequences, and dangerous area coordinates into nodes and edges of a heterogeneous graph.

7. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 6, characterized in that: The graded alarm signal is divided into threshold intervals according to the probability of violation, and a difference alarm is triggered by the intensity frequency combination of the vibration motor. At the same time, a danger zone outline box and safety guidance text are superimposed on the AR interface.

8. The method for safety early warning of a mining intelligent helmet based on image recognition according to claim 7, characterized in that: The coordinate superposition of dangerous areas uses a three-dimensional coordinate conversion method to map the position of dangerous areas in the mine space coordinate system to the perspective projection layer of the AR display interface in real time.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the image recognition-based safety warning method for a mining smart helmet are implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image recognition-based safety warning method for a mining smart helmet are implemented.

Images