Coal cutter cutting interference prevention method and system based on infrared thermal imaging and 4D millimeter wave

Abstract

A coal cutter cutting interference prevention method and system based on infrared thermal imaging and 4D millimeter wave, the method comprises the following steps: a perception and calculation module is deployed and a coordinate system is calibrated; multi-source sensor data is synchronously collected; infrared image preprocessing and rib protection plate visual identification are performed; 4D millimeter wave radar point cloud processing and feature extraction are performed; multi-modal data space-time unification is performed; adaptive gating multi-modal fusion is performed; current safety distance is calculated in real time; interference risk prediction based on deep time sequence prediction is performed; risk level division and closed-loop risk control are performed; the system comprises the following steps: an infrared thermal imaging perception module is used for collecting infrared thermal radiation image data; a 4D millimeter wave radar perception module is used for collecting four-dimensional perception data; an operating parameter collection module is used for acquiring operating parameters; an edge calculation module is connected with the infrared thermal imaging perception module, the 4D millimeter wave radar perception module, the operating parameter collection module and a coal cutter controller. The present application can realize early warning and grading active risk avoidance for the cutting interference risk of the drum and the rib protection plate.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing and safety control technology in underground coal mines, specifically relating to a method and system for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave. Background Technology

[0002] With the continuous upgrading of intelligent coal mining technology, the safety and operational stability of the coal mining machine, as the core mining equipment in fully mechanized mining faces, are crucial to the efficient and safe production of the mine. In actual underground mining operations, due to complex working conditions such as coal seam undulations, uneven floor, hydraulic support posture deviation, incomplete extension and retraction of the side guards, and delayed response of the coal mining machine drum height adjustment, the cutting drum of the coal mining machine is prone to collision and interference with the hydraulic support side guards. This not only causes damage to equipment parts and shutdown of the working face, but also, in severe cases, can lead to underground safety accidents, greatly restricting the safe and stable operation of intelligent fully mechanized mining faces and the improvement of mining efficiency.

[0003] Currently, the industry's methods for preventing interference during coal mining machine cutting mainly rely on manual on-site observation, equipment operating parameter simulation, or detection by a single sensor. These methods suffer from drawbacks such as low sensing accuracy, poor real-time performance, and insufficient reliability, making it impossible to accurately identify the actual deployment state of the side guard plate and the dynamic approaching condition of the coal mining machine drum. Furthermore, the harsh underground environment, including low illumination, high dust levels, abundant water mist, coal dust adhesion to equipment surfaces, high-frequency mechanical vibration, and obstruction by complex metal structures, directly leads to the failure of visible light visual recognition technology, failing to meet the needs of underground sensing and detection.

[0004] While single-function lidar and millimeter-wave radar possess ranging, velocity measurement, and spatial positioning capabilities, making them adaptable to some harsh underground working conditions, they can only acquire basic distance and velocity information of the target. They cannot achieve semantic recognition of the structural morphology and extension / retraction status of the sidewall panel. Furthermore, the complex metallic environment underground easily generates multipath reflection interference, leading to a high false alarm rate and poor environmental adaptability. Consequently, it is difficult to accurately predict the interference risk between the coal mining machine and the sidewall panel, thus failing to achieve proactive safety protection. Infrared thermal imaging technology can achieve illumination-free imaging based on the target's own thermal radiation characteristics, stably and accurately extracting the thermal feature contours and temperature distribution of the sidewall panel, unaffected by lighting conditions. 4D millimeter-wave radar can stably acquire the target's spatial coordinates, radial velocity, and dispersion data even under extreme conditions such as dust, water mist, and obstruction. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method and system for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave technology. This method enables early warning and proactive avoidance of interference risks between the coal mining machine drum and side guard plate through online monitoring. Simultaneously, it does not significantly increase the complexity of the fully mechanized mining face equipment, facilitating engineering modifications and upgrades on existing fully mechanized mining faces. This system achieves early warning and graded proactive avoidance of interference risks between the drum and side guard plate under complex working conditions such as low illumination, high dust, and water mist.

[0006] To achieve the above objectives, this invention provides a method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter waves, comprising the following steps: Step 1: Deployment of sensing and computing modules and calibration of coordinate system; Install infrared thermal imaging sensing modules, 4D millimeter-wave radar sensing modules and edge computing modules at key locations of the coal mining machine, and establish their spatial installation relationship with the coordinate system of the coal mining machine. Step 2: Synchronous acquisition of multi-source sensor data; synchronous acquisition of infrared thermal radiation images, 4D millimeter-wave radar four-dimensional sensing data, and coal mining machine operating status parameters; Step 3: Infrared image preprocessing and side panel visual recognition; The original infrared image is enhanced by the edge computing module, and the structured infrared semantic information of the side panel is extracted by the lightweight target detection network to form infrared visual recognition data; Step 4: 4D millimeter-wave radar point cloud processing and feature extraction; The original point cloud of 4D millimeter-wave radar is processed by the edge computing module through multi-level filtering and clustering to remove false point clouds in the dense metal environment downhole and extract reliable point cloud feature data of key areas of the sidewall plate. Step 5: Spatiotemporal unification of multimodal data; through joint external parameter calibration and time synchronization mechanism, the infrared identification results and radar point cloud results are unified under the coordinate system of the coal mining machine body, and it is ensured that the two belong to the effective matching data of the same control cycle; Step 6: Adaptive gated multimodal fusion; The edge computing module dynamically adjusts the fusion weights based on the infrared recognition confidence, radar feature confidence, and time synchronization status, and performs element-wise weighted fusion of infrared semantic features and point cloud geometric features to generate fused perception data for the protective panel. Step 7: Real-time calculation of the current safe distance; The edge computing module constructs the dangerous envelope area of ​​the drum cutting based on the structural parameters and operating status of the coal mining machine, and generates a set of dangerous candidate points of the side guard plate from the fused perception data. The minimum distance between the two is calculated in real time as the current safe distance; When the current safe distance is less than or equal to zero, it is determined that the side guard plate has invaded the dangerous envelope area of ​​the drum, and the highest level of protection action is triggered immediately without waiting for the subsequent risk prediction process; Step 8: Interference risk prediction based on deep temporal prediction; The edge computing module uses the current safe distance, motion state parameters and fusion confidence as prediction inputs, and uses a bidirectional gated recurrent unit network to predict the minimum distance in multiple future control cycles, and conservatively corrects the prediction results based on the fusion confidence and prediction uncertainty; When the corrected predicted minimum distance is less than the expected safe distance, it is determined that there is a risk of truncation interference, and the risk classification and risk avoidance control process is triggered. Step 9: Risk level classification and closed-loop risk avoidance control; The edge computing module classifies the risk level according to the predicted minimum distance and generates a tiered risk avoidance control command by combining the current safe distance and movement trend; After the coal mining machine controller executes the corresponding action, it feeds back the actual status to the edge computing module; As a preferred option, the process of deploying the sensing and computing module and calibrating the coordinate system in step 1 is as follows: S11: Rigidly install an infrared thermal imaging sensing module and a 4D millimeter-wave radar sensing module at the rocker arm end of the coal mining machine to ensure that the field of view of both can cover the key movement area of ​​the hydraulic support side plate; install an edge computing module in the coal mining machine body or adjacent control box; S12: Obtain the external parameter matrix from the coordinate system of the infrared thermal imaging sensing module to the coordinate system of the coal mining machine body through mechanical design parameters and installation location calibration. ,in , These are the rotation matrix and translation vector from the infrared thermal imaging sensing module coordinate system to the coal mining machine body coordinate system, and the external parameter matrix from the 4D millimeter-wave radar coordinate system to the coal mining machine body coordinate system. ,in , These are the rotation matrix and translation vector from the 4D millimeter-wave radar coordinate system to the coal mining machine body coordinate system, respectively.

[0007] As a preferred embodiment, the infrared image preprocessing and side panel visual recognition process in step 3 is as follows: S31: Perform non-uniformity correction, blind pixel compensation, contrast enhancement and detail enhancement processing on each frame of infrared thermal radiation image to suppress interference while preserving the edge contour and end features of the protective plate. S32: Input the preprocessed image into the lightweight object detection network, and simultaneously output the target bounding box, pixel-level edge contour, end pixel position, unfolding state, recognition confidence, and image timestamp of the side panel, to obtain the first... Infrared visual recognition data of the protective panel at all times As shown in the following formula: ; In the formula, For the target frame of the protective board, For the set of points on the edge contour of the side panel, This refers to the end position of the side guard plate. With the side panels extended, For infrared recognition confidence level, This is the timestamp for infrared image acquisition.

[0008] As a preferred embodiment, the 4D millimeter-wave radar point cloud processing and feature extraction process in step 4 is as follows: S41: Perform multi-frame cumulative denoising, outlier removal, voxel downsampling, and density-based spatial clustering sequentially on the 4D millimeter-wave radar four-dimensional perception data, and use multi-dimensional information to remove outliers caused by dust reflection, multipath reflection, and metal obstruction. S42: Extract the point cloud spatial location, radial velocity distribution, and geometric features of key areas of the side panel from the clustering results, forming the first... Point cloud feature data of the protective board at any time As shown in the following formula: ; In the formula, For point cloud clustering identifiers, This is the set of three-dimensional spatial positions of the selected point cloud in the radar coordinate system. This is the set of radial velocities at corresponding points. The geometric feature vector extracted from the point cloud distribution. For radar feature confidence level, This is the timestamp of the radar data frame.

[0009] As a preferred embodiment, the spatiotemporal unification process of multimodal data in step 5 is as follows: S51: Based on the extrinsic parameter matrix obtained in step 1, transform the 4D millimeter-wave radar point cloud from the radar coordinate system to the coal mining machine coordinate system; for the first... Point cloud points Its coordinates in the body coordinate system are As shown in the following formula: ; S52: Utilizing the known intrinsic parameter matrix of the infrared thermal imaging sensing module Set the pixel coordinates of the end of the side panel The back projection is a normalized direction vector, and it is transformed into the body coordinate system via a rotation matrix from infrared to the body coordinate system. The unit direction vector of this end in the body coordinate system is obtained. As shown in the following formula: ; S53: Synchronize infrared visual recognition data with radar point cloud feature data in time; when the timestamps of both meet the following conditions... When both are determined to be valid fused data within the same control cycle, among which... The maximum allowable synchronization time difference is preset based on the control cycle and sensor delay.

[0010] As a preferred embodiment, the adaptive gated multimodal fusion process in step 6 is as follows: S61: Constructing fusion confidence As shown in the following formula: ; In the formula, , , These are the non-negative weight coefficients for infrared recognition, radar point cloud, and time synchronization, respectively. ; S62: Infrared semantic feature vectors extracted by the infrared visual perception network Radar point cloud geometric feature vectors The individual confidence scores and the fused confidence scores are concatenated and then used to generate gating weights through a lightweight gating network. As shown in the following formula: ; In the formula, It is the Sigmoid activation function. and For gating network parameters; S63: Adaptively fuse infrared semantic features and radar point cloud geometric features using gated weights to obtain a fused feature vector. As shown in the following formula: ; In the formula, and The feature mapping matrix; This represents element-wise multiplication; S64: Combining infrared structural information, radar spatial information, and fused features to form the first... Real-time protection board integrates sensing data As shown in the following formula: ; In the formula, The three-dimensional spatial point of the side guard plate end in the body coordinate system is obtained by combining the infrared end direction vector and radar depth information. This is the set of spatial locations of radar point clouds after coordinate transformation. To merge data timestamps.

[0011] As a preferred option, the real-time calculation process for the current safe distance in step 7 is as follows: S71: Utilizing the rocker arm angle, roller geometry, and preset radial and axial safety margins, a directional enclosing box danger envelope region of the roller in the machine coordinate system is constructed. As shown in the following formula: ; In the formula, The position of the roller center in the machine coordinate system is obtained from the forward kinematics of the rocker arm. Let be the three orthogonal axial unit vectors of the roller enclosure box, with one axis along the roller axis and the other two axes perpendicular to the axis; This is the half-dimension corresponding to the axial direction; S72: Generate a set of candidate hazardous points for the side panel from fused sensing data. ,in The set of spatial contour points is the infrared edge contour projected onto the body coordinate system using radar depth information. For radar point cloud spatial set, For the end point of the side panel; Define the shortest signed distance from the candidate point set to the dangerous envelope region as the current safe distance. The distance is positive when the point is outside the region, zero when it is on the surface, and negative when it is inside the region; when If the system determines that the side guard plate has entered the dangerous envelope area of ​​the roller, the highest level of protection will be triggered immediately.

[0012] As a preferred embodiment, in step 8, the process of interference risk prediction based on deep temporal prediction is as follows: S81: Constructing the... Input feature vector at time step As shown in the following formula: ; In the formula, This represents the rate of change of the current safe distance. This refers to the traction speed of the coal mining machine; This refers to the rate of change of the drum height. Quantification value of the unfolded state of the side panel; The Euclidean distance change of the geometric features of the radar point cloud between two consecutive frames. ; To integrate confidence levels; S82: Will continue The input features of each control cycle constitute a time-series prediction sequence. Input a pre-trained bidirectional gated recurrent unit network and output the future... Predicted distance sequence for each control cycle ,in This refers to the implicit state of the network. and These are the output layer parameters; From the predicted distance sequence Extract the minimum distance within the prediction time window ; S83: Apply a conservative correction to the predicted minimum distance to obtain the corrected predicted minimum distance. As shown in the following formula: ; In the formula, For predicting distance sequences Standard deviation; The fusion confidence compensation coefficient is a unit of length. This is a dimensionless compensation coefficient for prediction uncertainty. S84: When the corrected predicted minimum distance is less than the expected safe distance ,Right now If the time frame is reached, it is determined that there is a risk of cutting interference between the roller and the side plate within a future time window, triggering the risk response process.

[0013] As a preferred option, in step 9, the risk level classification and closed-loop risk avoidance control process is as follows: S91: Set the first distance threshold Second distance threshold And satisfy ,in Based on the corrected predicted minimum distance Risk levels are classified as follows: ; In the formula, This indicates a high-risk situation, requiring immediate triggering of mandatory protection. This indicates a medium-risk status, requiring preventative control measures. This indicates a low-risk status; normal mining and transportation operations should continue. S92: The edge computing module determines the risk level and the current safe distance. traction speed Based on the trend of drum height changes, specific risk avoidance control commands are generated: in case of high risk, an immediate stop or rapid height adjustment command is sent; in case of medium risk, a traction deceleration or preventative height adjustment command is sent; in case of low risk, the original operation command is maintained. After receiving the command, the coal mining machine controller executes actions such as drum height adjustment, traction deceleration, shutdown, or forced protection. It also feeds back the drum height, traction speed, drum speed, and equipment status in real time to the edge computing module through the operation parameter acquisition module, thereby achieving a fully closed-loop anti-collision safety control.

[0014] Compared with the prior art, the present invention has the following technical advantages: 1. Multimodal complementary sensing effectively overcomes the limitations of single-sensor sensing underground. An infrared thermal imaging sensing module and a 4D millimeter-wave radar sensing module are deployed simultaneously at the rocker arm end of the coal mining machine. Infrared thermal imaging can quickly acquire semantic information such as the target frame, edge contour, end position, and deployment status of the side protection plate. At the same time, 4D millimeter-wave radar can effectively penetrate dust and stably acquire the three-dimensional spatial coordinates, radial velocity, and radar cross section of key areas of the side protection plate. Based on this, the two sensing data are spatiotemporally registered and sent to the edge computing module, providing a more complete sensing foundation for subsequent fusion and discrimination than a single sensor. It can still maintain recognition stability under low illumination, high dust, water mist, and mechanical vibration conditions.

[0015] 2. The infrared image preprocessing and lightweight detection network effectively improve the quality of downhole infrared recognition. The infrared thermal radiation image undergoes sequential non-uniformity correction, blind pixel compensation, contrast enhancement, and detail enhancement, effectively suppressing non-uniform noise from the infrared focal plane array and interference from the downhole thermal background. This makes the difference between the thermal feature edges of the sidewall and the thermal background of the coal wall and support more obvious. Based on this, the lightweight target detection network simultaneously outputs the target bounding box, edge contour point set, end pixel coordinates, and deployment status of the sidewall. This fully utilizes the thermal radiation characteristics of the infrared image to accurately determine whether the sidewall has deployed and entered the danger zone.

[0016] 3. 4D millimeter-wave radar multidimensional filtering effectively eliminated abnormal point clouds in the mine. Addressing the issue that dense metal support structures and high dust concentrations in coal mines easily generate multipath reflection points, dust reflection points, and metal obstruction edge scattering points in 4D millimeter-wave radar, the four-dimensional sensing data underwent multi-frame cumulative denoising, outlier removal, voxel downsampling, and density-based spatial clustering sequentially. Multidimensional filtering was achieved using radial velocity and radar cross-section information, effectively eliminating abnormal point clouds not originating from the side support plate. This significantly improved the extraction purity of the true point cloud from the side support plate, providing a reliable data foundation for subsequent spatial location calculations and geometric feature extraction.

[0017] 4. Adaptive gating multimodal fusion improves robustness to environmental changes. Based on the joint extrinsic parameter calibration relationship between the infrared thermal imaging sensing module, the 4D millimeter-wave radar sensing module, and the coal mining machine body, the infrared recognition results and radar point cloud results are unified to the coal mining machine body coordinate system, and a timestamp synchronization mechanism ensures that both belong to the same valid data pair within the same control cycle. On this basis, the fusion confidence is dynamically calculated based on the infrared recognition confidence, radar feature confidence, and time synchronization status. A lightweight gating network generates gating weights, and the infrared semantic features and point cloud geometric features are adaptively weighted and fused. When the data quality of a certain modality deteriorates due to environmental changes, the gating mechanism automatically reduces the fusion weight of that modality and increases the contribution of another modality, thus forming robust fused sensing data for the protective plate against environmental changes.

[0018] 5. A dual-layer safety defense is constructed through real-time hard triggering and predictive soft decision-making; this invention effectively builds two complementary safety defenses. For the first defense: the minimum distance between the side guard and the dangerous envelope area of ​​the roller is calculated in real-time based on current fused sensing data. When this distance is less than or equal to zero, the highest level of protection action is triggered immediately without waiting for prediction, forming a real-time hard defense. The second defense: a bidirectional gating cycle unit network is used to predict multiple future control cycles in advance, issuing early warnings in medium- or low-risk stages, forming a predictive soft defense. The two defenses effectively complement each other in terms of response speed and predictive capability, overcoming the problems of delayed response due to reliance on real-time data or excessively high false alarm rates due to reliance on prediction alone.

[0019] 6. The conservative correction mechanism significantly improves safety robustness under complex operating conditions. In the risk prediction stage, this invention not only utilizes a bidirectional gated cyclic unit network to output the predicted distance sequence within the future control cycle and take the minimum value, but also introduces a dual conservative correction term. The first term is driven by the fusion confidence level; when the fusion confidence level of infrared and radar decreases due to adverse operating conditions, the deduction margin for the predicted distance is automatically increased. The second term is driven by the standard deviation of the predicted sequence; when the network output fluctuates greatly and the prediction uncertainty is high, the safety distance is automatically increased. This adaptive mechanism, which increases the safety margin with greater uncertainty, significantly reduces the risk of missed detections in the variable operating conditions of the coal mining machine.

[0020] 7. Spatial proximity was accurately quantified through directed bounding box modeling and multi-source candidate point sets. Based on the rocker arm angle, drum geometry, and preset radial and axial safety margins, the directed bounding box method was used to construct the drum cutting danger envelope region in the coal mining machine coordinate system. Simultaneously, spatial contour point sets, radar point cloud spatial sets, and three-dimensional points at the ends of the side guards were extracted from the fused perception data of the side guards to form a multi-source danger candidate point set, comprehensively covering the key parts where the side guards might first contact the drum. By calculating the shortest Euclidean distance from each point in the candidate point set to the directed bounding box as the current safety distance, the spatial proximity between the drum and the side guards could be accurately quantified.

[0021] 8. Active risk prevention is achieved based on risk level classification and closed-loop control. High, medium, and low risk levels are classified according to the corrected predicted minimum distance, first distance threshold, and second distance threshold. In cases of high risk, an immediate shutdown or rapid height adjustment command is generated; in cases of medium risk, a traction deceleration or preventative height adjustment command is generated; and in cases of low risk, the existing operating command is maintained. After execution by the coal mining machine controller, the actual drum height, traction speed, drum rotation speed, and equipment status are fed back to the edge computing module to update the hazard envelope modeling and risk prediction parameters for the next cycle. This forms a closed-loop prevention mechanism integrating infrared sensing, radar sensing, fusion discrimination, risk prediction, active control, and status feedback. This avoids the lag and misjudgment caused by relying solely on manual observation or single sensor judgment, significantly improving the safety, real-time performance, and stability of coal mining machine cutting operations under complex underground conditions.

[0022] This method utilizes multimodal complementary sensing of infrared thermal imaging and 4D millimeter-wave radar, adaptive gating fusion, a dual-layer safety defense line of real-time hard triggering and predictive soft decision, a conservative correction prediction mechanism, and closed-loop active control. Through online monitoring, it achieves early warning and proactive risk avoidance of interference risks between the coal mining machine drum and the side guard plate. At the same time, the sensing module is co-mounted at the rocker arm end, and the edge computing module is deployed nearby on the machine body, without significantly increasing the complexity of the fully mechanized mining face equipment, making it easy to carry out engineering transformation and upgrading on existing fully mechanized mining faces.

[0023] This invention also provides a coal mining machine cutting interference prevention system based on infrared thermal imaging and 4D millimeter waves, used to implement a coal mining machine cutting interference prevention method based on infrared thermal imaging and 4D millimeter waves, including: The infrared thermal imaging sensing module is installed at the rocker arm end of the coal mining machine to collect infrared thermal radiation image data of the side protection plate. The 4D millimeter-wave radar sensing module is installed at the rocker arm end of the coal mining machine and its relative position to the infrared thermal imaging sensing module is fixed. It is used to collect four-dimensional sensing data such as three-dimensional coordinates, radial velocity, radar cross section and received power data of key areas of the side protection plate. The operating parameter acquisition module is used to acquire the operating parameters of the coal mining machine; The edge computing module is connected to the infrared thermal imaging sensing module, the 4D millimeter-wave radar sensing module, and the operating parameter acquisition module, respectively. Internally, it includes: an infrared image preprocessing and visual recognition unit, used to extract the target frame, edge contour, end pixel coordinates, unfolded state, and infrared recognition confidence score of the side protection plate from the infrared thermal radiation image, forming infrared visual recognition data; a 4D millimeter-wave point cloud preprocessing and feature extraction unit, used to denoise, cluster, and remove outliers from the four-dimensional sensing data, extract the point cloud spatial position, radial velocity, and geometric features of key areas of the side protection plate, and generate radar feature confidence scores, forming side protection plate point cloud feature data; and a multimodal fusion unit, used to unify the infrared visual recognition data and the side protection plate point cloud feature data to the coal mining machine coordinate system, perform time synchronization, calculate the fusion confidence score based on the infrared recognition confidence score, radar feature confidence score, and time synchronization state, generate gating weights through a gating network, and weight the infrared semantic features and point cloud geometric features. The system integrates several mechanisms: a fusion unit to generate fused perception data for the sidewalls; a hazard envelope modeling unit to construct the hazard envelope region for drum cutting based on the rocker arm angle and drum geometry, extract a set of hazard candidate points from the fused perception data for the sidewalls, calculate the current safe distance, and directly output the highest-level protection trigger signal when the current safe distance is less than or equal to zero; a risk prediction unit to use the current safe distance, motion state parameters, and fusion confidence level from multiple consecutive control cycles as time-series inputs, predict the minimum distance within future control cycles using a bidirectional gated cyclic unit network, and perform conservative corrections based on the fusion confidence level and prediction uncertainty. When the corrected predicted minimum distance is less than the expected safe distance, a cutting interference risk is determined and a risk level is output; a hazard avoidance control command generation unit to generate corresponding hazard avoidance control commands based on the highest-level protection trigger signal or risk level; and a closed-loop update unit to receive feedback from the coal mining machine controller and update the parameters required for hazard envelope modeling and risk prediction. The coal mining machine controller is connected to the edge computing module to execute the risk avoidance control commands and feed back the executed operating status to the closed-loop update unit.

[0024] In this invention, an infrared thermal imaging sensing module and a 4D millimeter-wave radar sensing module are co-mounted at the rocker arm end of a coal mining machine. Infrared thermal radiation images are used to quickly acquire two-dimensional semantic information of the sidewall plate, while the 4D millimeter-wave radar penetrates dust and water mist to obtain the three-dimensional spatial coordinates, radial velocity, and scattering characteristics of the sidewall plate, forming a perception foundation with complementary physical mechanisms and naturally aligned fields of view. The infrared image preprocessing and visual recognition unit, through non-uniformity correction, blind pixel compensation, and contrast enhancement, combined with a lightweight target detection network, extracts the target frame, edge contour, end position, and deployment state of the sidewall plate in real time while suppressing underground thermal background interference. The 4D millimeter-wave point cloud preprocessing and feature extraction unit utilizes multi-frame denoising, outlier removal, and multi-dimensional filtering based on radial velocity and scattering cross-section to effectively remove anomalous point clouds such as multipath reflection and dust reflection, improving point cloud purity. The multimodal fusion unit achieves coordinate unification and time synchronization through pre-calibrated intrinsic and extrinsic parameter matrices. It dynamically generates gating weights based on infrared recognition confidence, radar feature confidence, and time synchronization status, and adaptively weights and fuses infrared semantic features with point cloud geometric features. It automatically adjusts weights when the quality of a single modality deteriorates, generating robust fused perception data to environmental changes. The hazard envelope modeling unit constructs a roller-cut hazard envelope region using directed bounding boxes. It extracts a multi-source hazard candidate point set from the fused data and calculates the current safe distance in real time. When the distance is less than or equal to zero, it directly triggers the highest-level protection signal without a prediction step, forming the shortest response link. The risk prediction unit uses a bidirectional gated cyclic unit network to perform time-series modeling of multi-cycle motion state parameters and fusion confidence, outputting the minimum predicted distance for the future. It performs dual conservative corrections based on fusion confidence and prediction standard deviation, automatically increasing the safety margin and reducing the risk of missed detections when uncertainty increases. The real-time hard triggering of the hazard envelope modeling unit and the predictive soft decision-making of the risk prediction unit constitute a two-layer safety defense, respectively achieving "response upon intrusion" and "early warning before intrusion," complementing each other in response speed and foresight. The hazard avoidance control command generation unit generates graded control commands based on the trigger source and risk level: immediate shutdown or rapid adjustment in high-risk situations, deceleration or preventative adjustment in medium-risk situations, and maintenance of operation in low-risk situations, balancing safety and efficiency. The closed-loop update unit uses the post-execution status feedback from the coal mining machine controller to update the hazard envelope and prediction parameters in real time, avoiding accumulated errors. The edge computing module, through lightweight model design and hardware acceleration, completes all data processing and decision-making locally on the coal mining machine, without relying on a ground server, resulting in low control latency and meeting the real-time requirements underground.

[0025] The system enables early warning and graded proactive risk avoidance of interference risks between the roller and the side guard plate during cutting, even under complex working conditions such as low light, high dust, and water mist. Attached Figure Description

[0026] Figure 1 This is a flowchart of the prediction method part of the present invention; Figure 2 This is a flowchart illustrating the process of obtaining fused sensing data of the protective panel based on infrared thermal radiation image data and 4D millimeter-wave radar four-dimensional sensing data according to the present invention. Figure 3 This is a flowchart of the risk level classification and closed-loop risk avoidance control based on prediction in this invention; Figure 4 This is a block diagram illustrating the principle of the prediction system in this invention. Detailed Implementation

[0027] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0028] like Figure 1 As shown, this invention provides a method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter waves, comprising the following steps: Step 1: Deployment of the sensing and computing modules and calibration of the coordinate system; Infrared thermal imaging sensing modules, 4D millimeter-wave radar sensing modules, and edge computing modules are installed at key locations of the coal mining machine, and their spatial installation relationship with the coordinate system of the coal mining machine is established to provide a unified spatial reference for subsequent multi-sensor data fusion. As a preferred option, the process of deploying the sensing and computing modules and calibrating the coordinate system is as follows: S11: An infrared thermal imaging sensing module and a 4D millimeter-wave radar sensing module are rigidly installed at the rocker arm end of the coal mining machine to ensure that the field of view of both can cover the key movement area of ​​the hydraulic support side plate; an edge computing module is installed in the coal mining machine body or adjacent control box, wherein the edge computing module has multi-channel data reception, real-time heterogeneous computing and communication capabilities; S12: Obtain the external parameter matrix from the coordinate system of the infrared thermal imaging sensing module to the coordinate system of the coal mining machine body through mechanical design parameters and installation location calibration. ,in , These are the rotation matrix and translation vector from the infrared thermal imaging sensing module coordinate system to the coal mining machine body coordinate system, and the external parameter matrix from the 4D millimeter-wave radar coordinate system to the coal mining machine body coordinate system. ,in , These are the rotation matrix and translation vector from the 4D millimeter-wave radar coordinate system to the coal mining machine body coordinate system, respectively.

[0029] In this technical solution, by rigidly mounting an infrared thermal imaging sensing module and a 4D millimeter-wave radar sensing module on a common frame at the rocker arm end of the coal mining machine, and ensuring that both modules' fields of view cover the key movement areas of the hydraulic support sidewall plate, the spatial consistency of the acquisition perspectives of the two heterogeneous sensors is guaranteed from a physical structure perspective, laying the foundation for the natural alignment of subsequent multimodal data in spatial dimensions. On the other hand, by pre-calibrating and obtaining the rotation matrices and translation vectors of the infrared thermal imaging sensing module and the 4D millimeter-wave radar sensing module to the coordinate system of the coal mining machine, a deterministic transformation relationship between the sensing data and a unified spatial reference is established. This allows the infrared recognition results in the image domain and the radar point cloud results in three-dimensional space to be unambiguously unified in the coordinate system of the coal mining machine, reducing the dependence of subsequent fusion processes on online calibration or iterative registration, and improving the calibration stability and computational efficiency of the system in underground vibration and dust environments.

[0030] Step 2: Synchronous acquisition of data from multiple sensor sources; Simultaneously acquire infrared thermal radiation images, 4D millimeter-wave radar four-dimensional perception data, and coal mining machine operating status parameters to provide raw input for side protection plate identification and anti-collision decision-making; As a preferred method, the process of synchronous data acquisition from multiple sensor sources is as follows: Infrared thermal radiation image data of the target area of ​​the sidewall is acquired at a fixed frame rate by an infrared thermal imaging sensing module, with each frame image accompanied by a timestamp. Four-dimensional sensing data such as three-dimensional spatial coordinates, radial velocity, radar cross section, and received power of key areas of the sidewall are acquired simultaneously by a 4D millimeter-wave radar sensing module, forming a point cloud data stream and marking the acquisition timestamp. Operating parameters are obtained in real time from the coal mining machine control system through an operating parameter acquisition module. The operating parameters include rocker arm angle, drum height, traction speed, running direction, and equipment operating status, providing a dynamic benchmark for subsequent drum hazard envelope construction and motion prediction. In this technical solution, the same target area of ​​the protective plate is synchronously acquired by the infrared thermal imaging sensing module and the 4D millimeter-wave radar sensing module. The acquisition timestamp is marked for each frame of infrared image and each frame of radar point cloud. At the same time, the operation parameter acquisition module acquires dynamic benchmark parameters such as the rocker arm angle, drum height, traction speed and operation status of the coal mining machine in real time. This enables the three heterogeneous data of infrared thermal radiation image, radar 4D point cloud and coal mining machine motion status to obtain a clear time correspondence within the same control cycle. This provides consistency in the time and motion dimensions for the subsequent data synchronization matching based on timestamp difference in step 5, multimodal spatiotemporal fusion in step 6, and accurate construction of the drum danger envelope based on real-time rocker arm posture in step 7. It avoids fusion misalignment and distance calculation errors caused by asynchronous acquisition time of each sensor or lag in motion parameters.

[0031] Step 3: Infrared image preprocessing and visual recognition of the side panel; The original infrared image is enhanced by the edge computing module, and the structured infrared semantic information of the side panel is extracted by the lightweight target detection network to form infrared visual recognition data. To ensure stable extraction of the target area of ​​the side protection plate under conditions of low illumination, high dust, water mist, coal dust adhesion, and mechanical vibration in underground mining, the process of infrared image preprocessing and visual recognition of the side protection plate is as follows: S31: Perform non-uniformity correction, blind pixel compensation, contrast enhancement and detail enhancement processing on each frame of infrared thermal radiation image to suppress interference such as high-temperature dust in the well while preserving the edge contour and end features of the side protection plate. S32: Input the preprocessed image into the lightweight object detection network, and simultaneously output the target bounding box, pixel-level edge contour, end pixel position, unfolding state, recognition confidence, and image timestamp of the side panel, to obtain the first... Infrared visual recognition data of the protective panel at all times As shown in the following formula: ; In the formula, For the target frame of the protective board, This is the set of points (two-dimensional coordinates of the image) representing the outline of the side panel. This refers to the end position of the side guard plate. The protective panels are in the unfolded state (e.g., folded, half-unfolded, fully unfolded). For infrared recognition confidence level, For infrared image acquisition timestamps; In this technical solution, by sequentially performing non-uniformity correction, blind pixel compensation, contrast enhancement, and detail enhancement on the infrared thermal radiation image, the interference of high-temperature dust, water mist, and thermal background on the infrared image is effectively suppressed, making the contrast between the thermal feature edges of the side protection plate and the surrounding environment more obvious. On this basis, a lightweight target detection network is used to simultaneously output the target box, pixel-level edge contour, end pixel coordinates, unfolding state, and infrared recognition confidence of the side protection plate. Image quality enhancement and multi-task visual recognition are integrated into the same processing link. On the edge computing platform, the recognition stability and real-time inference capability under low illumination and high dust conditions are taken into account, providing infrared semantic data with complete structure and quantifiable confidence for subsequent fusion.

[0032] Step 4: 4D millimeter-wave radar point cloud processing and feature extraction; The edge computing module performs multi-level filtering and clustering on the original point cloud of 4D millimeter-wave radar to remove false point clouds such as multipath reflection and dust reflection in the dense metal environment of the well, and extracts reliable point cloud feature data of the key area of ​​the side plate. To eliminate multipath reflection points, dust reflection points, and anomalous scattering points generated by 4D millimeter-wave radar in densely packed underground metal structures, the 4D millimeter-wave radar point cloud processing and feature extraction process is as follows: S41: The 4D millimeter-wave radar four-dimensional perception data is sequentially subjected to multi-frame cumulative denoising, outlier removal, voxel downsampling and density-based spatial clustering, and multi-dimensional information such as radial velocity and radar cross section is used to remove outliers caused by dust reflection, multipath reflection and metal obstruction. S42: Extract the point cloud spatial location, radial velocity distribution, and geometric features of key areas of the side panel from the clustering results, forming the first... Point cloud feature data of the protective board at any time As shown in the following formula: ; In the formula, For point cloud clustering identifiers, This is the set of three-dimensional spatial positions of the selected point cloud in the radar coordinate system. This is the set of radial velocities at corresponding points. These are geometric feature vectors (such as size and shape descriptors) extracted from the point cloud distribution. Radar feature confidence level reflects the quality and completeness of the point cloud. This is the timestamp of the radar data frame. This technical solution addresses the problem of multipath reflection points, dust reflection points, and abnormal scattering points caused by metal obstruction in 4D millimeter-wave radar due to the dense metal support structure in underground coal mines. It sequentially performs multi-frame cumulative denoising, outlier removal, voxel downsampling, and density-based spatial clustering on the four-dimensional sensing data. Furthermore, it utilizes radial velocity dimension and radar cross-section information for multi-dimensional joint filtering. This effectively identifies and removes false point clouds from non-target sources while preserving the true point cloud of the support plate, significantly improving the purity of the extracted point cloud data. Based on this, the spatial location, radial velocity distribution, and geometric features of key areas of the support plate are simultaneously output from the clustering results, and radar feature confidence is quantified. This provides high-quality, spatially reliable 3D point cloud feature data for subsequent multimodal fusion and hazard envelope modeling.

[0033] Step 5: Spatiotemporal unification of multimodal data; By combining external parameter calibration and time synchronization mechanisms, the infrared identification results and radar point cloud results are unified under the coordinate system of the coal mining machine, and it is ensured that the two belong to the same effective matching data of the same control cycle. As a preferred approach, the spatiotemporal unification process for multimodal data is as follows: S51: Based on the extrinsic parameter matrix obtained in step 1, transform the 4D millimeter-wave radar point cloud from the radar coordinate system to the coal mining machine coordinate system; for the first... Point cloud points Its coordinates in the body coordinate system are As shown in the following formula: ; S52: Utilizing the known intrinsic parameter matrix of the infrared thermal imaging sensing module Set the pixel coordinates of the end of the side panel The back projection is a normalized direction vector, and it is transformed into the body coordinate system via a rotation matrix from infrared to the body coordinate system. The unit direction vector of this end in the body coordinate system is obtained. As shown in the following formula: ; S53: Synchronize infrared visual recognition data with radar point cloud feature data in time; when the timestamps of both meet the following conditions... When both are determined to be valid fused data within the same control cycle, among which... The maximum allowable synchronization time difference is preset based on the control cycle and sensor delay.

[0034] In this technical solution, the 4D millimeter-wave radar point cloud is accurately transformed from the radar coordinate system to the coal mining machine coordinate system through a pre-calibrated extrinsic parameter matrix. At the same time, the intrinsic and extrinsic parameter rotation matrices of the infrared thermal imaging sensing module are used to back-project and normalize the pixel coordinates of the end of the side guard plate into a unit direction vector in the machine coordinate system, achieving unambiguous alignment of infrared two-dimensional semantic information and radar three-dimensional spatial information under the same spatial reference. On this basis, time synchronization is performed by judging the difference between the infrared image acquisition timestamp and the radar acquisition timestamp, retaining only valid data pairs with time deviations within the preset maximum allowable synchronization time difference. From both spatiotemporal dimensions, this ensures that the infrared visual recognition data and radar point cloud feature data input to the subsequent fusion stage belong to the same side guard plate target in the same control cycle, avoiding fusion misalignment or distortion of dangerous distance calculation due to spatiotemporal mismatch.

[0035] Step 6: Adaptive gated multimodal fusion; The edge computing module dynamically adjusts the fusion weights based on the confidence levels of infrared recognition, radar features, and time synchronization status, and performs element-wise weighted fusion of infrared semantic features and point cloud geometric features to generate highly reliable fused perception data for the protective panel. As a preferred option, the adaptive gated multimodal fusion process is as follows: S61: To comprehensively reflect the reliability of infrared recognition, radar point cloud, and time alignment, a fusion confidence level is constructed. As shown in the following formula: ; In the formula, , , These are the non-negative weight coefficients for infrared recognition, radar point cloud, and time synchronization, respectively. It can be pre-calibrated according to the characteristics of the sensor; S62: Infrared semantic feature vectors extracted by the infrared visual perception network Radar point cloud geometric feature vectors The individual confidence scores and the fused confidence scores are concatenated and then used to generate gating weights through a lightweight gating network. As shown in the following formula: ; In the formula, It is the Sigmoid activation function. and For gating network parameters; S63: Adaptively fuse infrared semantic features and radar point cloud geometric features using gated weights to obtain a fused feature vector. As shown in the following formula: ; In the formula, and This is the feature mapping matrix, used to project heterogeneous features onto a unified dimensional space; This represents element-wise multiplication; S64: Combining infrared structural information, radar spatial information, and fused features to form the first... Real-time protection board integrates sensing data As shown in the following formula: ; In the formula, The three-dimensional spatial point of the side guard plate end in the body coordinate system is obtained by combining the infrared end direction vector and radar depth information. This is the set of spatial locations of radar point clouds after coordinate transformation. To merge data timestamps.

[0036] In this technical solution, the confidence level is synthesized by weighting infrared recognition confidence, radar feature confidence, and time synchronization state terms. Infrared semantic features, radar point cloud geometric features, and confidence levels are input into a lightweight gating network. Element-by-element gating weights are dynamically generated in a data-driven manner, achieving adaptive weighted fusion of heterogeneous features in a unified dimensional space. When downhole dust, water mist, or vibration causes a decline in the quality of a certain modality's data, the gating mechanism can automatically suppress the contribution of low-confidence modalities and amplify the contribution of high-confidence modalities, avoiding perception degradation caused by the failure of a single sensor. At the same time, the fusion confidence level itself serves as a quantitative indicator of perception reliability and is transmitted to the subsequent risk prediction stage, providing a basis for conservative correction of the prediction model. The final generated fused perception data of the side protection plate possesses both semantic integrity and spatial positioning accuracy under complex time-varying conditions.

[0037] Step 7: Real-time calculation of the current safe distance; The edge computing module constructs the dangerous envelope area of ​​the drum cutting based on the structural parameters and operating status of the coal mining machine. At the same time, it generates a set of dangerous candidate points of the side guard plate from the fused perception data and calculates the minimum distance between the two in real time as the current safe distance. When the current safe distance is less than or equal to zero, it is determined that the side guard plate has invaded the dangerous envelope area of ​​the drum and immediately triggers the highest level of protection action without waiting for the subsequent risk prediction process. In order to predict the dynamic approach trend between the coal mining machine drum and the side guard plate in advance, the current real-time calculation process of the safe distance is as follows: S71: Utilizing the rocker arm angle, roller geometry (radius, width), and preset radial and axial safety margins, construct a directional bounding box danger envelope region of the roller in the machine coordinate system. As shown in the following formula: ; In the formula, The position of the roller center in the machine coordinate system is obtained from the forward kinematics of the rocker arm. Let be the three orthogonal axial unit vectors of the roller enclosure box, with one axis along the roller axis and the other two axes perpendicular to the axis; For the corresponding half dimension in the axial direction, the sum of the roller radius and the radial safety margin is taken for the two directions perpendicular to the axis, and the sum of half the roller width and the axial safety margin is taken for the axial direction. S72: Generate a set of candidate hazardous points for the side panel from fused sensing data. ,in The set of spatial contour points is the infrared edge contour projected onto the body coordinate system using radar depth information. For radar point cloud spatial set, For the end point of the side panel; Define the shortest signed distance from the candidate point set to the dangerous envelope region as the current safe distance. The distance is positive when the point is outside the region, zero when it is on the surface, and negative when it is inside the region; when If the system determines that the side guard plate has entered the dangerous envelope area of ​​the roller, the highest level of protection will be triggered immediately.

[0038] In this technical solution, a directional bounding box danger envelope region is constructed by integrating the roller radius, width, and preset radial and axial safety margins. At the same time, infrared edge contour back projection points, radar point cloud spatial points, and end three-dimensional points are extracted from the fused perception data of the side guard plate to form a multi-source danger candidate point set, which comprehensively covers the key parts of the side guard plate that may first contact the roller. The shortest Euclidean distance is used as the current safe distance to accurately quantify the spatial proximity between the roller and the side guard plate. When this distance is less than or equal to zero, the highest level of protection action is triggered directly without the prediction link, forming the shortest link from perception to response, ensuring zero-delay protection at the moment of intrusion. At the same time, the calculated current safe distance provides an accurate real-time benchmark for risk prediction in the subsequent step eight.

[0039] Step 8: Interference risk prediction based on deep time series prediction; The edge computing module uses the current safe distance, motion state parameters (including distance change rate, coal mining machine traction speed, drum height change trend, side guard plate deployment status, and point cloud geometric feature changes) and fusion confidence as prediction inputs. It uses a bidirectional gated cyclic unit network to predict the minimum distance over multiple future control cycles and conservatively corrects the prediction results based on the fusion confidence and prediction uncertainty. When the corrected predicted minimum distance is less than the expected safe distance, it determines that there is a risk of truncation interference and triggers the risk classification and risk avoidance control process. As a preferred approach, the process of interference risk prediction based on deep time series prediction is as follows: S81: Constructing the... Input feature vector at time step As shown in the following formula: ; In the formula, The rate of change of the current safe distance is calculated from the safe distance difference across several consecutive frames. This refers to the traction speed of the coal mining machine; The rate of change of the drum height is obtained by the difference of the drum height sequence. Quantification value of the unfolded state of the side panel; The Euclidean distance change of the geometric features of the radar point cloud between two consecutive frames. This reflects the degree of drastic change in the posture of the side guardrail; To integrate confidence levels; S82: Will continue The input features of each control cycle constitute a time-series prediction sequence. Input a pre-trained bidirectional gated recurrent unit network and output the future... Predicted distance sequence for each control cycle ,in This refers to the implicit state of the network. and These are the output layer parameters; From the predicted distance sequence Extract the minimum distance within the prediction time window ; S83: Considering the fusion of confidence level and prediction uncertainty, a conservative correction is made to the minimum prediction distance to obtain the corrected minimum prediction distance. As shown in the following formula: ; In the formula, For predicting distance sequences Standard deviation; The fusion confidence compensation coefficient is a unit of length. It is a dimensionless compensation coefficient for prediction uncertainty; the two compensations automatically increase the safety margin when the confidence level is low or the volatility is high.

[0040] S84: When the corrected predicted minimum distance is less than the expected safe distance ,Right now If the risk of interference between the roller and the side guard plate during cutting is determined within a future time window, the risk response process is triggered. In this technical solution, multi-dimensional time-series input features are constructed by using the current safe distance, distance change rate, coal mining machine traction speed, drum height change rate, side guard plate deployment status, point cloud geometric feature changes, and fusion confidence. A bidirectional gated cyclic unit network is used to extract the bidirectional contextual dependencies of historical data, enabling forward prediction of the minimum distance between the drum and side guard plate within multiple future control cycles. This extends safety assessment from the current moment to future trends, providing response time for early intervention. Based on this, a dual conservative correction is applied to the predicted minimum distance based on the fusion confidence and the standard deviation of the predicted distance sequence—automatically increasing the deduction margin when underground dust or vibration causes a decrease in fusion confidence, and automatically increasing the safe distance when the predicted sequence fluctuation increases. This makes the prediction model more conservative when sensing increased uncertainty, significantly reducing the risk of missed detections due to overly optimistic predictions. The comparison between the corrected predicted minimum distance and the expected safe distance serves as the intervention risk judgment condition, providing an accurate and reliable triggering basis for the risk level classification and graded risk avoidance control in the subsequent step nine.

[0041] Step 9: Risk level classification and closed-loop risk avoidance control; The edge computing module classifies risk levels based on the predicted minimum distance and generates tiered risk avoidance control commands by combining the current safe distance and movement trend. After the coal mining machine controller executes the corresponding actions (drum height adjustment, traction deceleration, shutdown or forced protection action), it feeds back the actual status to the edge computing module to form a closed-loop control. As a preferred approach, the process of risk level classification and closed-loop risk mitigation control is as follows: S91: Set the first distance threshold (Used to trigger strong risk avoidance control) and second distance threshold (Used to trigger preventative controls), and meets the following conditions: ,in Based on the corrected predicted minimum distance Risk levels are classified as follows: ; In the formula, This indicates a high-risk situation, requiring immediate triggering of mandatory protection. This indicates a medium-risk status, requiring preventative control measures. This indicates a low-risk status; normal mining and transportation operations should continue. S92: The edge computing module determines the risk level and the current safe distance. traction speed Based on the trend of drum height changes, specific risk avoidance control commands are generated: in case of high risk, an immediate stop or rapid height adjustment command is sent; in case of medium risk, a traction deceleration or preventative height adjustment command is sent; in case of low risk, the original operation command is maintained. After receiving the command, the coal mining machine controller executes actions such as drum height adjustment, traction deceleration, shutdown, or forced protection. It also feeds back the drum height, traction speed, drum speed, and equipment status in real time to the edge computing module through the operation parameter acquisition module, thereby achieving a fully closed-loop anti-collision safety control.

[0042] In this technical solution, by setting a first distance threshold and a second distance threshold, the corrected predicted minimum distance is quantified into three levels: high risk, medium risk, and low risk. This maps the cutting interference risk from continuous predicted distance values ​​to a clear control decision level, avoiding the limitation of binary decision-making caused by a single threshold trigger. Based on this, combined with the current safety distance, traction speed, and drum height change trends, differentiated risk avoidance control commands are generated for different risk levels. In case of high risk, the machine is immediately stopped or rapidly raised; in case of medium risk, the traction is slowed down or preventively raised; and in case of low risk, normal operation is maintained, minimizing the impact on mining and transportation efficiency while ensuring safety. At the same time, after the coal mining machine controller executes the command, it feeds back the actual drum height, traction speed, drum speed, and equipment status to the edge computing module in real time, forming a closed-loop control. This ensures that the hazard envelope modeling and risk prediction of the next control cycle are always based on the latest actual equipment status, avoiding the cumulative errors caused by unintended execution or external disturbances in open-loop control.

[0043] This invention also provides a coal mining machine cutting interference prevention system based on infrared thermal imaging and 4D millimeter waves, used to implement a coal mining machine cutting interference prevention method based on infrared thermal imaging and 4D millimeter waves, including: The infrared thermal imaging sensing module is rigidly installed at the rocker arm end of the coal mining machine. Its field of view covers the target area of ​​the hydraulic support side plate. It is used to collect infrared thermal radiation image data of the side plate at a fixed frame rate and send each frame of infrared image and its collection timestamp to the edge computing module. The 4D millimeter-wave radar sensing module is rigidly installed at the end of the coal mining machine rocker arm. Its relative position and pose with the infrared thermal imaging sensing module are fixed. Its detection field of view is directed towards the key area of ​​the hydraulic support side plate. It is used to simultaneously collect four-dimensional sensing data of the side plate and its surrounding environment, including three-dimensional spatial coordinates, radial velocity, radar cross section and receiving power, to form a point cloud data stream. The point cloud data and the collection timestamp are then sent to the edge computing module. The operation parameter acquisition module communicates with the coal mining machine control system to acquire the operation parameters of the coal mining machine in real time. The operation parameters include, but are not limited to, rocker arm angle, drum height, drum speed, traction speed, running direction and equipment operating status, and send the operation parameters to the edge computing module. The edge computing module is an industrial computer or embedded computing platform, which includes an infrared image preprocessing and visual recognition unit, a 4D millimeter-wave point cloud preprocessing and feature extraction unit, a multimodal fusion unit, a hazard envelope modeling unit, a risk prediction unit, a hazard avoidance control command generation unit, and a closed-loop update unit. The infrared image preprocessing and visual recognition unit is used to sequentially perform non-uniformity correction, blind pixel compensation, contrast enhancement and detail enhancement processing on the received infrared thermal radiation image. It uses a pre-set lightweight target detection network to extract the target box, edge contour point set, end pixel coordinates, unfolding state and infrared recognition confidence of the protective plate from the enhanced image, and forms the infrared visual recognition data of the protective plate together with the image timestamp. The 4D millimeter-wave point cloud preprocessing and feature extraction unit is used to sequentially perform multi-frame cumulative denoising, outlier removal, voxel downsampling, and density-based spatial clustering on the received four-dimensional sensing data. It uses radial velocity and radar cross section information to remove abnormal point clouds formed by dust reflection, multipath reflection, and metal obstruction. Then, it extracts the point cloud spatial location set, radial velocity set, and point cloud geometric features of key areas of the side panel from the clustering results, and generates radar feature confidence, which together with the radar acquisition timestamp forms the side panel point cloud feature data. The multimodal fusion unit incorporates pre-calibrated extrinsic matrixes from the infrared thermal imaging sensing module coordinate system to the coal mining machine coordinate system, extrinsic matrixes from the 4D millimeter-wave radar coordinate system to the coal mining machine coordinate system, and intrinsic matrixes of the infrared thermal imaging sensing module. These are used to perform the following operations: transforming the spatial position of the 4D millimeter-wave radar point cloud to the coal mining machine coordinate system using the extrinsic matrix; back-projecting and normalizing the pixel coordinates of the side guard plate end to a unit direction vector in the coal mining machine coordinate system using the intrinsic and extrinsic rotation matrices; and performing time synchronization based on the difference between the infrared image acquisition timestamp and the radar acquisition timestamp. Only valid data pairs that meet the maximum allowable synchronization time difference are retained; the fusion confidence is calculated based on infrared recognition confidence, radar feature confidence, and time synchronization status; gating weights are dynamically generated through a lightweight gating network, and the infrared semantic feature vector and radar point cloud geometric feature vector are fused element-wise using the gating weights to generate a fused feature vector; finally, by combining infrared structural information, radar spatial information, and fused features, fused perception data of the protective panel is generated, including the target box of the protective panel, edge contour, end 3D points, key area point cloud set, unfolded status, fused features, fused confidence, and fused timestamp. The hazard envelope modeling unit is used to construct a hazard envelope region of the drum section described by a directed bounding box in the coordinate system of the coal mining machine, based on the current rocker arm angle, drum geometry, and preset radial and axial safety margins. Simultaneously, it extracts a set of spatial contour points projected from the back projection of infrared edge contours, a set of radar point clouds, and three-dimensional points at the end of the backing plate from the fused perception data of the side guard plate, forming a set of hazard candidate points for the side guard plate. It calculates the shortest Euclidean distance from each point in the candidate point set to the hazard envelope region in real time, takes the minimum value as the current safe distance, and when the current safe distance is less than or equal to zero, it sends the highest level protection trigger signal directly to the hazard avoidance control command generation unit without waiting for the output of the risk prediction unit. The risk prediction unit is used to construct a time-series input sequence based on the current safe distance, the rate of change of safe distance, the traction speed of the coal mining machine, the rate of change of drum height, the deployment status of the side guard plate, the change in point cloud geometric features, and the fusion confidence level for multiple consecutive control cycles. It then uses a pre-trained bidirectional gated recurrent unit network to output a predicted distance sequence for a preset number of future control cycles, and takes the minimum value as the predicted minimum distance. Based on the fusion confidence level and the standard deviation of the predicted distance sequence, the predicted minimum distance is conservatively corrected to obtain the corrected predicted minimum distance. When the corrected predicted minimum distance is less than the expected safe distance, a truncation interference risk is determined, and the risk level is classified into high risk, medium risk, and low risk according to the relationship between the corrected predicted minimum distance and the first and second distance thresholds. The risk level is then sent to the risk avoidance control command generation unit. The risk avoidance control command generation unit is used to receive the highest-level protection trigger signal from the hazard envelope modeling unit or the risk level from the risk prediction unit, and generate corresponding risk avoidance control commands. Specifically, when the highest-level protection trigger signal or a high-risk level is received, an immediate shutdown or rapid increase command is generated; when the risk level is medium, a traction deceleration or preventive increase command is generated; when the risk level is low, the existing operation command is maintained; and the generated command is sent to the coal mining machine controller. The closed-loop update unit is used to receive the drum height, traction speed, drum speed and equipment status fed back by the coal mining machine controller after the action is executed, and to use the feedback data to update the drum center position in the danger envelope modeling unit and the motion parameters required for subsequent construction of input feature vectors, thus forming closed-loop control.

[0044] The coal mining machine controller communicates with the edge computing module to receive risk avoidance control commands and execute actions such as drum height adjustment, traction deceleration, shutdown, or forced protection accordingly. At the same time, it feeds back the executed drum height, traction speed, drum rotation speed, and equipment status to the closed-loop update unit of the edge computing module in real time.

[0045] In this invention, an infrared thermal imaging sensing module is installed at the rocker arm end of a coal mining machine. This allows for the rapid acquisition of two-dimensional semantic information about the sidewall, including its target frame, edge contour, end position, and deployment state, through thermal radiation images. A 4D millimeter-wave radar sensing module is also installed at the rocker arm end, with its relative pose fixed to the infrared thermal imaging sensing module. This allows it to penetrate dust and water mist, stably acquiring four-dimensional sensing data on the key areas of the sidewall, including three-dimensional spatial coordinates, radial velocity, radar cross section, and received power. Thus, the two sensing modules are physically complementary, and their shared mounting and overlapping fields of view provide a spatially aligned sensing data source for subsequent multimodal fusion. This ensures stable collaborative operation even under low illumination, high dust, water mist, and mechanical vibration conditions. Infrared image preprocessing and a visual recognition unit sequentially perform non-uniformity correction, blind pixel compensation, contrast enhancement, and detail enhancement on the infrared thermal radiation image. This effectively suppresses non-uniform noise from the infrared focal plane array and interference from high-temperature dust and thermal background in the mine, making the difference between the thermal feature edges of the sidewall and the thermal background of the coal wall and hydraulic supports more pronounced. Building upon this foundation, a pre-built lightweight target detection network is used to simultaneously extract the target bounding box, edge contour point set, end pixel coordinates, and unfolded state of the side panel from the enhanced image. It then outputs the infrared recognition confidence score and image timestamp, forming structured infrared visual recognition data. Integrating image quality enhancement and multi-task visual recognition into the same processing link reduces the interference of the complex underground thermal environment on infrared recognition while ensuring real-time inference capabilities on the edge computing platform using a lightweight network. Based on the 4D millimeter-wave point cloud preprocessing and feature extraction unit, the four-dimensional perception data undergoes multi-frame cumulative denoising, outlier removal, voxel downsampling, and density-based spatial clustering. Multi-dimensional filtering is performed using radial velocity and radar cross-section information, effectively identifying and removing multipath reflection points, dust reflection points, and metal-obstructed edge scattering points generated in the dense environment of underground metal support structures in coal mines. This significantly improves the extraction purity of the true point cloud of the side panel, providing a reliable data foundation for subsequent spatial location calculations and geometric feature extraction. The multimodal fusion unit incorporates pre-calibrated extrinsic matrixes from the infrared thermal imaging sensing module coordinate system to the coal mining machine coordinate system, extrinsic matrixes from the 4D millimeter-wave radar coordinate system to the coal mining machine coordinate system, and intrinsic matrixes of the infrared thermal imaging sensing module. These calibration parameters unify the infrared recognition results and radar point cloud results to the same spatial reference. A timestamp synchronization mechanism ensures that both belong to the same valid data pair within the same control cycle. Based on this, the fusion confidence level is dynamically calculated according to the infrared recognition confidence level, radar feature confidence level, and time synchronization status. A lightweight gating network generates gating weights, and adaptively weighted fusion is performed on the infrared semantic feature vector and the radar point cloud geometric feature vector.When the data quality of a certain modality deteriorates due to environmental changes, the fusion weight of that modality is automatically reduced while the contribution of another modality is increased, thereby generating robust fusion perception data for the sidewall panel that is resistant to environmental changes. Based on the hazard envelope modeling unit, the current rocker arm angle and drum geometry are used to construct a hazard envelope region for drum truncation described by a directed bounding box in the coal mining machine coordinate system. Simultaneously, the unit extracts the spatial contour point set of infrared edge contour back projection, the spatial set of radar point clouds, and the three-dimensional points at the end of the sidewall panel from the fusion perception data, forming a multi-source hazard candidate point set that comprehensively covers the critical parts where the sidewall panel may first contact the drum. Furthermore, the unit calculates the shortest Euclidean distance from the candidate point set to the hazard envelope region in real time as the current safe distance. When the current safe distance is less than or equal to zero, it sends the highest-level protection trigger signal directly to the hazard avoidance control command generation unit without waiting for the output of the risk prediction unit, forming the shortest response link from perception to triggering, ensuring the highest level of safety at the physical level. The risk prediction unit uses the current safe distance, motion state parameters, and fusion confidence scores from multiple consecutive control cycles to form a time-series input sequence. A pre-trained bidirectional gated recurrent unit network extracts the bidirectional contextual dependencies of historical data, outputting a predicted distance sequence for future preset control cycles. The minimum value of this sequence is taken as the predicted minimum distance. Furthermore, a dual conservative correction is applied to the predicted minimum distance based on the fusion confidence score and the standard deviation of the predicted distance sequence. When the fusion confidence score decreases, the deduction margin is automatically increased; when prediction fluctuations increase, the safe distance is automatically widened. This significantly reduces the risk of missed detections caused by sensing uncertainty or prediction fluctuations in the variable operating conditions of the coal mining machine. Simultaneously, the real-time hard-triggered protection provided by the hazard envelope modeling unit and the predictive soft-decision protection provided by the risk prediction unit work together to form a two-layer safety defense line for this system. The hazard envelope modeling unit, based on current fused sensing data, achieves zero-delay protection with immediate response upon intrusion; the risk prediction unit, based on forward-looking time-series prediction, achieves proactive prevention with early warning before intrusion. The two systems complement each other in terms of response speed and foresight, overcoming the problems of delayed response caused by relying solely on real-time criteria or excessively high false alarm rates caused by relying solely on predictive criteria. The risk avoidance control command generation unit receives the highest-level protection trigger signal from the hazard envelope modeling unit or the risk level from the risk prediction unit, and generates differentiated risk avoidance control commands based on different trigger sources and risk levels. When the highest-level protection trigger signal or a high-risk level is received, an immediate shutdown or rapid increase command is generated to ensure that the approaching danger is cut off as soon as possible; when the risk is medium, a traction deceleration or preventative increase command is generated to reduce risk without interrupting operations; when the risk is low, the existing operating commands are maintained to avoid unnecessary downtime losses. This hierarchical control strategy balances safety and production efficiency.The closed-loop update unit receives feedback from the coal mining machine controller after executing avoidance actions, including drum height, traction speed, drum rotation speed, and equipment status. It then uses this feedback data to update the drum center position parameters in the hazard envelope modeling unit and the motion parameters required by the risk prediction unit to construct the input feature vector. This unit enables the system to continuously perceive changes in the coal mining machine's own state, avoiding accumulated errors caused by unintended command execution or external disturbances in open-loop control. It ensures that hazard envelope modeling and risk prediction in the next control cycle are always based on the latest actual equipment status. The system also enables direct communication between the coal mining machine controller and the edge computing module. After receiving avoidance control commands, the controller executes drum height adjustment, traction deceleration, shutdown, or forced protection actions, and feeds back the executed operating status to the closed-loop update unit in real time. All data processing, fusion, risk prediction, and command generation by the edge computing module are completed on the local industrial computer or embedded computing platform of the coal mining machine, eliminating the need to upload large amounts of raw data to a ground server. This collaborative architecture of local computing and local execution significantly reduces the bandwidth dependence on the roadway communication network, keeping the control response latency at the millisecond level of the local computing node, thus meeting the stringent real-time requirements of dynamic coal mining machine operations.

[0046] The system achieves early warning and graded active risk avoidance of interference risks between the coal mining machine drum and the side guard plate in complex underground environments such as low illumination, high dust and water mist by co-architectural complementary deployment of infrared thermal imaging sensing module and 4D millimeter-wave radar sensing module, combined with adaptive gated weighted fusion of multimodal fusion unit, real-time hard trigger protection of dangerous envelope modeling unit and forward conservative prediction of risk prediction unit.

[0047] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave, characterized in that, Includes the following steps: Step 1: Deployment of sensing and computing modules and calibration of coordinate system; Install infrared thermal imaging sensing modules, 4D millimeter-wave radar sensing modules and edge computing modules at key locations of the coal mining machine, and establish spatial installation relationships; Step 2: Synchronous acquisition of multi-source sensor data; synchronous acquisition of infrared thermal radiation images, 4D millimeter-wave radar four-dimensional sensing data, and coal mining machine operating status parameters; Step 3: Infrared image preprocessing and side panel visual recognition; The original infrared image is enhanced and the structured infrared semantic information of the side panel is extracted to form infrared visual recognition data; Step 4: 4D millimeter-wave radar point cloud processing and feature extraction; perform multi-level filtering and clustering on the original 4D millimeter-wave radar point cloud, remove false point clouds, and extract point cloud feature data of key areas of the side panel. Step 5: Spatiotemporal unification of multimodal data; through joint external parameter calibration and time synchronization mechanism, the infrared identification results and radar point cloud results are unified under the coordinate system of the coal mining machine body, and it is ensured that the two belong to the effective matching data of the same control cycle; Step 6: Adaptive gated multimodal fusion; The fusion weights are dynamically adjusted based on the confidence level of infrared recognition, the confidence level of radar features, and the time synchronization status. The infrared semantic features and point cloud geometric features are then fused element by element to generate fused perception data for the protective panel. Step 7: Real-time calculation of the current safe distance; Construct the dangerous envelope area of ​​the drum cutting based on the structural parameters and operating status of the coal mining machine, and generate a set of dangerous candidate points of the side guard plate from the fused sensing data, and calculate the current safe distance in real time; When the current safe distance is less than or equal to zero, the highest level of protection action is immediately triggered; Step 8: Interference risk prediction based on deep time series prediction; Using the current safe distance, motion state parameters, and fusion confidence as prediction inputs, a bidirectional gated cyclic unit network is used to predict the minimum distance over multiple future control cycles, and the prediction results are conservatively corrected. When the corrected predicted minimum distance is less than the expected safe distance, the risk classification and risk avoidance control process is triggered. Step 9: Risk Level Classification and Closed-Loop Risk Avoidance Control; Based on the predicted minimum distance, risk levels are classified, and a tiered risk avoidance control command is generated by combining the current safe distance and movement trend; After the coal mining machine controller executes the corresponding action, it provides feedback on the actual status.

2. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 1, the deployment of the sensing and computing module and the calibration of the coordinate system are as follows: S11: Rigidly install an infrared thermal imaging sensing module and a 4D millimeter-wave radar sensing module at the rocker arm end of the coal mining machine to ensure that the field of view of both can cover the key movement area of ​​the hydraulic support side plate; install an edge computing module in the coal mining machine body or adjacent control box; S12: Obtain the external parameter matrix from the coordinate system of the infrared thermal imaging sensing module to the coordinate system of the coal mining machine body through mechanical design parameters and installation location calibration. ,in , These are the rotation matrix and translation vector from the infrared thermal imaging sensing module coordinate system to the coal mining machine body coordinate system, and the external parameter matrix from the 4D millimeter-wave radar coordinate system to the coal mining machine body coordinate system. ,in , These are the rotation matrix and translation vector from the 4D millimeter-wave radar coordinate system to the coal mining machine body coordinate system, respectively.

3. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 3, the infrared image preprocessing and visual recognition of the side panel are as follows: S31: Perform non-uniformity correction, blind pixel compensation, contrast enhancement and detail enhancement on each frame of infrared thermal radiation image to suppress interference while preserving the edge contour and end features of the protective panel. S32: Input the preprocessed image into the lightweight object detection network, and simultaneously output the target bounding box, pixel-level edge contour, end pixel position, unfolding state, recognition confidence, and image timestamp of the side panel, to obtain the first... Infrared visual recognition data of the protective panel at all times As shown in the following formula: ； In the formula, For the target frame of the protective board, For the set of points on the edge contour of the side panel, This refers to the end position of the side guard plate. With the side panels extended, For infrared recognition confidence level, This is the timestamp for infrared image acquisition.

4. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 4, the 4D millimeter-wave radar point cloud processing and feature extraction process is as follows: S41: Perform multi-frame cumulative denoising, outlier removal, voxel downsampling, and density-based spatial clustering sequentially on the 4D millimeter-wave radar four-dimensional perception data, and use multi-dimensional information to remove outliers caused by dust reflection, multipath reflection, and metal obstruction. S42: Extract the point cloud spatial location, radial velocity distribution, and geometric features of key areas of the side panel from the clustering results, forming the first... Point cloud feature data of the protective board at any time As shown in the following formula: ； In the formula, For point cloud clustering identifiers, This is the set of three-dimensional spatial positions of the selected point cloud in the radar coordinate system. This is the set of radial velocities at corresponding points. The geometric feature vector extracted from the point cloud distribution. For radar feature confidence level, This is the timestamp of the radar data frame.

5. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 5, the spatiotemporal unification process of multimodal data is as follows: S51: Based on the extrinsic parameter matrix obtained in step 1, transform the 4D millimeter-wave radar point cloud from the radar coordinate system to the coal mining machine coordinate system; for the first... Point cloud points Its coordinates in the body coordinate system are As shown in the following formula: ； S52: Utilizing the known intrinsic parameter matrix of the infrared thermal imaging sensing module Set the pixel coordinates of the end of the side panel The back projection is a normalized direction vector, and it is transformed into the body coordinate system via a rotation matrix from infrared to the body coordinate system. The unit direction vector of this end in the body coordinate system is obtained. As shown in the following formula: ； S53: Synchronize infrared visual recognition data with radar point cloud feature data in time; when the timestamps of both meet the following conditions... When both are determined to be valid fused data within the same control cycle, among which... The maximum allowable synchronization time difference is preset based on the control cycle and sensor delay.

6. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 6, the adaptive gated multimodal fusion process is as follows: S61: Constructing Fusion Confidence As shown in the following formula: ； In the formula, , , These are the non-negative weight coefficients for infrared recognition, radar point cloud, and time synchronization, respectively. ; S62: Infrared semantic feature vectors extracted by the infrared visual perception network Radar point cloud geometric feature vectors The individual confidence scores and the fused confidence scores are concatenated and then used to generate gating weights through a lightweight gating network. As shown in the following formula: ； In the formula, It is the Sigmoid activation function. and For gating network parameters; S63: Adaptively fuse infrared semantic features and radar point cloud geometric features using gated weights to obtain a fused feature vector. As shown in the following formula: ； In the formula, and The feature mapping matrix; This represents element-wise multiplication; S64: Combining infrared structural information, radar spatial information, and fused features to form the first... Real-time protection board integrates sensing data As shown in the following formula: ； In the formula, The three-dimensional spatial point of the side guard plate end in the body coordinate system is obtained by combining the infrared end direction vector and radar depth information. This is the set of spatial locations of radar point clouds after coordinate transformation. To merge data timestamps.

7. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 7, the real-time calculation process for the current safe distance is as follows: S71: Utilizing the rocker arm angle, roller geometry, and preset radial and axial safety margins, a directional enclosing box danger envelope region of the roller in the machine coordinate system is constructed. As shown in the following formula: ； In the formula, The position of the roller center in the machine coordinate system is obtained from the forward kinematics of the rocker arm. Let be the three orthogonal axial unit vectors of the roller enclosure box, with one axis along the roller axis and the other two axes perpendicular to the axis; This is the half-dimension corresponding to the axial direction; S72: Generate a set of candidate hazardous points for the side panel from fused sensing data. ,in The set of spatial contour points is the infrared edge contour projected onto the body coordinate system using radar depth information. For radar point cloud spatial set, For the end point of the side panel; Define the shortest signed distance from the candidate point set to the dangerous envelope region as the current safe distance. The distance is positive when the point is outside the region, zero when it is on the surface, and negative when it is inside the region; when If the system determines that the side guard plate has entered the dangerous envelope area of ​​the roller, the highest level of protection will be triggered immediately.

8. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 8, the process of interference risk prediction based on deep temporal prediction is as follows: S81: Constructing the... Input feature vector at time step As shown in the following formula: ； In the formula, This represents the rate of change of the current safe distance. This refers to the traction speed of the coal mining machine; The rate of change of the roller height; Quantification value of the unfolded state of the side panel; The Euclidean distance change of the geometric features of the radar point cloud between two consecutive frames. ; To integrate confidence levels; S82: Will continue The input features of each control cycle constitute a time-series prediction sequence. Input a pre-trained bidirectional gated recurrent unit network and output the future... Predicted distance sequence for each control cycle ,in This refers to the implicit state of the network. and For output layer parameters; From the predicted distance sequence Extract the minimum distance within the prediction time window ; S83: Apply a conservative correction to the predicted minimum distance to obtain the corrected predicted minimum distance. As shown in the following formula: ； In the formula, For predicting distance sequences Standard deviation; The fusion confidence compensation coefficient is for the length dimension. This is a dimensionless compensation coefficient for prediction uncertainty. S84: When the corrected predicted minimum distance is less than the expected safe distance ,Right now If the time frame is reached, it is determined that there is a risk of cutting interference between the roller and the side plate within a future time window, triggering the risk response process.

9. The method for preventing interference during coal mining machine cutting based on infrared thermal imaging and 4D millimeter wave as described in claim 1, characterized in that, In step 9, the risk level classification and closed-loop risk avoidance control process is as follows: S91: Set the first distance threshold Second distance threshold And satisfy ,in Based on the corrected predicted minimum distance Risk levels are classified as follows: ； In the formula, This indicates a high-risk situation, requiring immediate triggering of mandatory protection. This indicates a medium-risk status, requiring preventative control measures. This indicates a low-risk status; normal mining and transportation operations should continue. S92: The edge computing module determines the risk level and the current safe distance. traction speed Based on the trend of drum height changes, specific risk avoidance control commands are generated: in case of high risk, an immediate stop or rapid height adjustment command is sent; in case of medium risk, a traction deceleration or preventative height adjustment command is sent; in case of low risk, the original operation command is maintained. After receiving the command, the coal mining machine controller executes actions such as drum height adjustment, traction deceleration, shutdown, or forced protection. It also feeds back the drum height, traction speed, drum speed, and equipment status in real time to the edge computing module through the operation parameter acquisition module, thereby achieving a fully closed-loop anti-collision safety control.

10. A coal mining machine cutting interference prevention system based on infrared thermal imaging and 4D millimeter wave, used to implement the coal mining machine cutting interference prevention method based on infrared thermal imaging and 4D millimeter wave as described in any one of claims 1 to 9, characterized in that, include: The infrared thermal imaging sensing module is installed at the rocker arm end of the coal mining machine to collect infrared thermal radiation image data of the side protection plate. The 4D millimeter-wave radar sensing module is installed at the rocker arm end of the coal mining machine and its relative position to the infrared thermal imaging sensing module is fixed. It is used to collect four-dimensional sensing data such as three-dimensional coordinates, radial velocity, radar cross section and received power data of key areas of the side protection plate. The operating parameter acquisition module is used to acquire the operating parameters of the coal mining machine; The edge computing module is connected to the infrared thermal imaging sensing module, the 4D millimeter-wave radar sensing module, and the operating parameter acquisition module, respectively. Internally, it includes: an infrared image preprocessing and visual recognition unit, used to extract the target frame, edge contour, end pixel coordinates, unfolded state, and infrared recognition confidence score of the side protection plate from the infrared thermal radiation image, forming infrared visual recognition data; a 4D millimeter-wave point cloud preprocessing and feature extraction unit, used to denoise, cluster, and remove outliers from the four-dimensional sensing data, extract the point cloud spatial position, radial velocity, and geometric features of key areas of the side protection plate, and generate radar feature confidence scores, forming side protection plate point cloud feature data; and a multimodal fusion unit, used to unify the infrared visual recognition data and the side protection plate point cloud feature data to the coal mining machine coordinate system, perform time synchronization, calculate the fusion confidence score based on the infrared recognition confidence score, radar feature confidence score, and time synchronization state, generate gating weights through a gating network, and weight the infrared semantic features and point cloud geometric features. The system integrates several mechanisms: a fusion unit to generate fused perception data for the sidewalls; a hazard envelope modeling unit to construct the hazard envelope region for drum cutting based on the rocker arm angle and drum geometry, extract a set of hazard candidate points from the fused perception data for the sidewalls, calculate the current safe distance, and directly output the highest-level protection trigger signal when the current safe distance is less than or equal to zero; a risk prediction unit to use the current safe distance, motion state parameters, and fusion confidence level from multiple consecutive control cycles as time-series inputs, predict the minimum distance within future control cycles using a bidirectional gated cyclic unit network, and perform conservative corrections based on the fusion confidence level and prediction uncertainty. When the corrected predicted minimum distance is less than the expected safe distance, a cutting interference risk is determined and a risk level is output; a hazard avoidance control command generation unit to generate corresponding hazard avoidance control commands based on the highest-level protection trigger signal or risk level; and a closed-loop update unit to receive feedback from the coal mining machine controller and update the parameters required for hazard envelope modeling and risk prediction. The coal mining machine controller is connected to the edge computing module to execute the risk avoidance control commands and feed back the executed operating status to the closed-loop update unit.

Images