An intelligent identification system and early warning device for hidden dangers in coal mines with AI video analysis

Abstract

The present application relates to the technical field of coal mine safety production monitoring, and specifically discloses a coal mine hidden danger intelligent identification system and early warning device fusing AI video analysis. The system comprises a video acquisition module, an AI visual analysis and calculation module, a three-dimensional situation generation module, an early warning decision and path planning module, a device control module and a feedback adjustment module. The video acquisition module acquires coal flow surface images in real time, the AI visual analysis and calculation module adopts an optical flow method to track particle motion characteristics and outputs arching probability indexes, the three-dimensional situation generation module converts two-dimensional coordinates into arching point three-dimensional coordinates, the early warning decision and path planning module generates a collision-free motion trajectory, the device control module drives the early warning device to perform an arch breaking action, and the feedback adjustment module dynamically adjusts arch breaking parameters according to updated arching probability indexes after arch breaking. The present application realizes early identification, accurate positioning and adaptive disposal of coal bunker blockage hidden dangers in coal mines, and significantly improves the safety level of underground production.

Description

Technical Field

[0001] This invention relates to the field of coal mine safety production monitoring technology, specifically to an intelligent identification system and early warning device for coal mine hidden dangers that integrates AI video analysis. Background Technology

[0002] A coal mine is an area where humans extract coal resources from coal-rich geological formations. They are generally divided into underground coal mines and open-pit coal mines. When the coal seam is far from the surface, underground tunnels are typically dug to extract the coal; this is an underground coal mine. When the coal seam is very close to the surface, the surface soil is typically stripped away to extract the coal; this is an open-pit coal mine. The vast majority of coal mines are underground mines. A coal mine encompasses a large area, including both above-ground and underground sections, as well as related facilities. A coal mine is a rationally excavated space created by humans when mining coal-rich geological strata, typically including tunnels, shafts, and working faces.

[0003] For example, application number "CN201910658059.2" discloses a mine safety production management system based on a local area network, which addresses the problem that existing mine safety production management systems lack the ability to monitor the environment of mine shafts and cannot provide early warnings to help people escape quickly before a mine collapse occurs. However, in the current coal mine production process, coal bunkers, as a key hub in the coal transportation system, are a major hidden danger that seriously affects production efficiency and safety. Currently, video monitoring systems are commonly used in underground coal mines to monitor the operation of coal bunkers, but traditional monitoring methods can only provide visual images and rely on manual observation to determine whether there is a risk of blockage. They cannot quantify and analyze the subtle signs before a blockage occurs. Existing anti-blockage measures mostly involve installing air cannons or mechanical loosening devices. These devices usually operate in a timed triggering or manual activation mode, passively clearing blockages after they occur. However, due to the dark environment and high dust concentration inside the coal bunker, early abnormal changes on the surface of the coal flow are difficult to detect with the naked eye. They are often only noticed when blockages have already formed or even seriously affect production. Fixed air cannons or mechanical devices can only spray or vibrate at fixed points and cannot accurately intervene in dynamically changing blockage locations. This not only results in low unblocking efficiency, but also the high-pressure airflow or mechanical vibration may induce gas exceedances or coal dust explosions, posing serious secondary safety risks. Summary of the Invention

[0004] This invention provides an intelligent identification system and early warning device for coal mine hazards that integrates AI video analysis, solving the problems mentioned in the background art above.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an intelligent early warning device for coal mine hazards integrating AI video analysis, comprising: Explosion-proof pan-tilt camera unit, used to capture video images of the coal flow surface; The multi-degree-of-freedom arch-breaking execution unit includes an explosion-proof multi-joint robotic arm and an arch-breaking head installed at the end of the robotic arm, which is used to move to the target position to perform arch-breaking operations according to the received instructions; The environmental perception and positioning unit, including an angle encoder and an attitude sensor, is used to monitor the angles of each joint of the robotic arm and its overall attitude in real time. The explosion-proof main control and communication unit is connected to the explosion-proof pan-tilt camera unit, the multi-degree-of-freedom arch-breaking execution unit and the environmental perception and positioning unit, respectively, and is used to receive and process video data, generate control commands, coordinate the work of each unit and communicate data with the well surface monitoring center. The power and energy unit is connected to the multi-degree-of-freedom arch-breaking actuator and is used to provide the power required for the arch-breaking action.

[0006] A coal mine hazard identification system integrating AI video analysis includes: The video acquisition module is deployed in the coal bunker monitoring area underground in the coal mine to collect continuous video image data of the coal flow surface in real time. The AI ​​visual analysis and calculation module is communicatively connected to the video acquisition module. It is used to receive the continuous video image data and use the optical flow method to track and analyze the motion characteristics of the particles on the coal flow surface, and calculate the arching probability index that characterizes the motion state of the coal flow. The three-dimensional situation generation module is connected to the AI ​​visual analysis and calculation module. It is used to receive the arching probability index and its coordinates in the two-dimensional image. Combined with the pre-stored three-dimensional structure model of the coal bunker, the module calculates the precise three-dimensional coordinates of the arching point in the bunker through a spatial coordinate transformation algorithm. The early warning decision and path planning module is communicatively connected to the three-dimensional situation generation module. It is used to receive the precise three-dimensional coordinates of the arch point, and combine them with the preset kinematic model of the early warning device and the obstacle distribution data in the warehouse to calculate and generate a collision-free motion trajectory from the current posture of the early warning device to the target arch breaking point. The device control module is communicatively connected to the early warning decision and path planning module and the early warning device, respectively, and is used to receive the collision-free motion trajectory and convert it into driving commands to control the early warning device to perform the arch-breaking action. The feedback adjustment module is communicatively connected to the device control module and the AI ​​visual analysis and calculation module, respectively. It is used to obtain the updated arching probability index in real time after the early warning device performs the arch-breaking action, and generate an adjustment command based on the comparison result of the updated arching probability index and the preset safety threshold, and send it to the device control module to dynamically adjust the arch-breaking parameters of the early warning device.

[0007] Furthermore, the video acquisition module includes explosion-proof high-definition cameras, supplementary lights, and lens self-cleaning units deployed on the top and side walls of the coal bunker. The lens of the explosion-proof high-definition camera faces the surface area of ​​the coal flow and is used to acquire clear continuous video image data in the dim and dusty underground environment. The supplementary lights are used to automatically turn on to supplement lighting when there is insufficient light. The lens self-cleaning unit is used to remove coal dust adhering to the lens surface at regular intervals or according to instructions to maintain image clarity. The video acquisition module transmits the acquired continuous video image data to the AI ​​visual analysis and computing module in real time through the mine communication interface.

[0008] Furthermore, the AI ​​visual analysis and calculation module includes a motion feature extraction unit and a probability calculation unit. The motion feature extraction unit uses optical flow or particle image velocimetry to perform pixel-by-pixel analysis on the received continuous video image frames, tracking the instantaneous displacement vector of each identifiable particle on the coal flow surface. The probability calculation unit calculates the average displacement velocity and residence time of all particles in a specified area based on the instantaneous displacement vector, compares the average displacement velocity with a preset historical average velocity threshold, and compares the residence time with a preset time threshold. When the average displacement velocity is lower than 30% of the historical average velocity threshold and the residence time exceeds 5 seconds, it is determined that there is a hidden arching trend in the area and the corresponding arching probability index is output. The arching probability index is a value between 0 and 1, with a larger value indicating a higher probability of arching.

[0009] Furthermore, the three-dimensional situation generation module includes a coordinate transformation unit and a model matching unit. The coordinate transformation unit receives the arching probability index and its corresponding two-dimensional pixel coordinates output by the AI ​​visual analysis and calculation module. Using a binocular vision stereo matching algorithm or a depth estimation technique based on monocular vision, and combining the intrinsic and extrinsic parameter matrices of the camera in the video acquisition module, the three-dimensional coordinates of the arching point in the camera coordinate system are calculated using the triangulation principle. The model matching unit transforms the three-dimensional coordinates in the camera coordinate system to the world coordinate system of the pre-stored three-dimensional structure model of the coal bunker, obtaining the accurate three-dimensional coordinates of the arching point within the bunker, and outputs the accurate three-dimensional coordinates to the early warning decision and path planning module.

[0010] Furthermore, the early warning decision and path planning module includes a reachability analysis unit and a path optimization unit. The reachability analysis unit receives the precise three-dimensional coordinates of the arch point output by the three-dimensional situation generation module, and calculates whether the end effector of the robotic arm can reach the target position by combining the preset kinematic model of the early warning device. If it is determined to be reachable, the path optimization unit is triggered to work. The path optimization unit searches for a collision-free, smooth joint movement and the lowest energy consumption motion trajectory from the current posture of the early warning device to the target arch breaking point in the configuration space based on the preset obstacle distribution data in the warehouse and the motion constraints of each joint of the early warning device using a fast extended random tree algorithm. The motion trajectory is encoded into time series joint angle data and sent to the device control module.

[0011] Furthermore, the device control module includes an instruction parsing unit and a drive control unit. The instruction parsing unit receives motion trajectory data sent by the early warning decision and path planning module, and parses it into joint angle, angular velocity, and angular acceleration commands arranged in a time sequence. The drive control unit generates corresponding pulse width modulation signals or analog voltage signals according to the parsed commands and sends them to the servo drivers of each joint of the early warning device to control the motor or hydraulic valve to move along a predetermined trajectory. At the same time, the drive control unit receives the actual position data fed back by the angle encoder and attitude sensor installed on the early warning device in real time, performs a closed-loop comparison with the theoretical commands, and corrects the output signal in real time to ensure motion accuracy.

[0012] Furthermore, the feedback adjustment module includes an effectiveness evaluation unit and a parameter adaptation unit. After the early warning device performs the arch-breaking action, the effectiveness evaluation unit obtains the updated arch-forming probability index from the AI ​​visual analysis and calculation module, compares the updated arch-forming probability index with a preset safety threshold, and if the updated arch-forming probability index is higher than the safety threshold, it determines that the current arch-breaking effect is insufficient and triggers the parameter adaptation unit to work. The parameter adaptation unit dynamically calculates and generates the adjustment amount of the arch-breaking parameters based on the difference between the updated arch-forming probability index and the safety threshold using a proportional-integral-derivative control algorithm or a fuzzy control algorithm. The arch-breaking parameters include the blowing pressure of the arch-breaking head, the pulse frequency, and the slight angle correction value of the robotic arm end, and sends the adjustment command containing the above adjustment amount to the device control module.

[0013] Furthermore, the parameter adaptive unit includes a pressure adjustment subunit, a frequency adjustment subunit, and an angle fine-tuning subunit. The pressure adjustment subunit calculates the required high-pressure air injection pressure adjustment value based on the rate of change of the arching probability index. When the arching probability index decreases slowly, the pressure value is increased until a preset upper limit is reached. The frequency adjustment subunit calculates the pulse injection frequency adjustment value based on the speed at which the coal flow surface particles recover their flow. When the coal flow speed recovers quickly, the injection frequency is reduced to save energy. The angle fine-tuning subunit calculates the small offset of the robotic arm end on the original trajectory based on the spatial distribution characteristics of the arching points and the current actual position of the arch-breaking head, so as to impact the arching area from different angles until the coal flow returns to normal flow. The calculation results of all subunits are combined to generate a complete adjustment command.

[0014] Furthermore, the system also includes a data storage and traceability module, which is connected to the video acquisition module, AI visual analysis and calculation module, 3D situation generation module, early warning decision and path planning module, device control module, and feedback adjustment module, respectively. This module is used to store the original video image data, the historical curve of the arching probability index, the 3D coordinate change trajectory of the arching point, the planned arch-breaking movement trajectory, the actual executed driving commands, and the feedback adjustment records after each arch-breaking action in the order of timestamps. When a traceability query command is received, the system retrieves and replays the complete data chain at the corresponding time according to the specified time period or event identifier to support accident cause analysis and system parameter optimization.

[0015] This invention provides an intelligent identification system and early warning device for coal mine hazards that integrates AI video analysis. It has the following beneficial effects: (I) This intelligent identification system and early warning device for coal mine hidden dangers, which integrates AI video analysis, achieves precise quantitative monitoring of particle movement on the surface of coal flow through AI visual analysis technology. It successfully transforms conventional monitoring videos into effective data sources that can identify hidden arching trends, fundamentally solving the problem that traditional manual visual observation cannot detect early blockage hazards in coal bunkers. Specifically, the video acquisition module is deployed on the top or side wall of the coal bunker, continuously acquiring images of the coal flow surface in the dark and dusty environment underground. The AI ​​visual analysis and calculation module uses optical flow to analyze the continuous video frames pixel by pixel, accurately tracking the movement trajectory of each tiny particle. By calculating the average displacement velocity and residence time of all particles in a specified area, when the average displacement velocity is lower than 30% of the historical average velocity threshold and the residence time exceeds 5 seconds, the system can determine that there is a hidden arching trend in the area and output the corresponding arching probability index. This process transforms the subtle changes in coal flow that are originally difficult to detect with the naked eye into quantifiable numerical indicators, enabling the detection of early signs of arching seconds or even minutes before coal bunker blockage occurs. This provides a reliable data foundation for subsequent precise positioning and proactive intervention, completely changing the backward situation in traditional coal mine safety monitoring that can only rely on passive post-event processing or manual experience judgment. It significantly improves the early warning capability and intelligence level of coal mine safety production, effectively avoiding production interruptions and safety accidents caused by coal bunker blockage.

[0016] (II) This intelligent coal mine hazard identification system and early warning device integrating AI video analysis accurately converts the two-dimensional image information output from the first layer into the three-dimensional spatial coordinates of the arching point through three-dimensional spatial transformation and path planning algorithms, and dynamically plans the optimal arch-breaking intervention path. This solves the fatal flaw of traditional fixed arch-breaking devices that cannot accurately locate and effectively intervene in dynamically changing arching points. Specifically, the three-dimensional situation generation module receives the arching probability index and its corresponding two-dimensional pixel coordinates output by the AI ​​visual analysis and calculation module. Using a binocular visual stereo matching algorithm combined with the camera's intrinsic and extrinsic parameter matrices, it accurately calculates the three-dimensional coordinates of the arching point in the camera coordinate system through the principle of triangulation. Then, it converts these coordinates to the pre-stored three-dimensional coal bunker structure. In the world coordinate system of the structural model, the precise three-dimensional coordinates of the arching point within the silo are obtained. Based on these precise coordinates, the early warning decision and path planning module, combined with the kinematic model of the early warning device and the obstacle distribution data within the silo, uses a fast expanding random tree algorithm to search for a collision-free, smooth joint movement trajectory with the lowest energy consumption from the current posture of the early warning device to the target arch-breaking point in the configuration space. This enables the early warning device to accurately reach blind spots that traditional fixed nozzles cannot cover, and to precisely break the arch at the specific point of occurrence of each arching event. This completely avoids ineffective operations and energy waste caused by inaccurate positioning, significantly improves the targeting and efficiency of arch-breaking operations, and reduces secondary safety risks such as gas over-limits or coal dust explosions that may be caused by blind arch breaking.

[0017] (III) In this intelligent identification system and early warning device for coal mine hazards integrating AI video analysis, after the device control module drives the early warning device to perform the first arch-breaking action, the efficiency evaluation unit in the feedback adjustment module immediately obtains the updated arch-forming probability index from the AI ​​visual analysis and calculation module and compares it with the preset safety threshold. If the updated arch-forming probability index is still higher than the safety threshold, the parameter adaptive unit dynamically calculates and generates the adjustment amount of the arch-breaking parameters based on the difference between the index and the safety threshold, including the adjustment value of the high-pressure air injection pressure, the adjustment value of the pulse injection frequency, and the adjustment value of the robotic arm end effector on the original trajectory. The system detects a small offset in the airflow and then sends an instruction containing the aforementioned adjustment amount to the device control module, driving the early warning device to perform a secondary arch-breaking action. This cycle continues until the arching probability index drops below the safety threshold, enabling the early warning device to adaptively adjust the arch-breaking strategy based on the actual recovery of the coal flow. When the arching is relatively stubborn, it automatically increases the injection pressure and frequency and repeatedly impacts from different angles. When the coal flow has begun to recover, it appropriately reduces the intervention intensity to save energy. This maximizes the arch-breaking effect while avoiding excessive intervention, forming a complete intelligent closed loop that significantly improves the safety and reliability of underground arch-breaking operations in coal mines. Attached Figure Description

[0018] Figure 1This is a schematic diagram of the overall workflow of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] First embodiment: as follows Figure 1 As shown, the present invention provides a technical solution: an intelligent early warning device for coal mine hazards integrating AI video analysis, comprising: Explosion-proof pan-tilt camera unit, used to capture video images of the coal flow surface; The multi-degree-of-freedom arch-breaking execution unit includes an explosion-proof multi-joint robotic arm and an arch-breaking head installed at the end of the robotic arm, which is used to move to the target position to perform arch-breaking operations according to the received instructions; The environmental perception and positioning unit, including an angle encoder and an attitude sensor, is used to monitor the angles of each joint of the robotic arm and its overall attitude in real time. The explosion-proof main control and communication unit is connected to the explosion-proof pan-tilt camera unit, the multi-degree-of-freedom arch-breaking execution unit and the environmental perception and positioning unit, respectively. It is used to receive and process video data, generate control commands, coordinate the work of each unit and communicate data with the well surface monitoring center. The power and energy unit, connected to the multi-degree-of-freedom arch-breaking actuator, is used to provide the power required for the arch-breaking action.

[0021] A coal mine hazard identification system integrating AI video analysis includes: The video acquisition module is deployed in the coal bunker monitoring area underground in the coal mine to collect continuous video image data of the coal flow surface in real time. The AI ​​visual analysis and calculation module is connected to the video acquisition module to receive continuous video image data and use optical flow method to track and analyze the motion characteristics of particles on the coal flow surface, and calculate the arching probability index that characterizes the motion state of the coal flow. The three-dimensional situation generation module communicates with the AI ​​visual analysis and calculation module to receive the arching probability index and its coordinates in the two-dimensional image. Combined with the pre-stored three-dimensional structure model of the coal bunker, the module calculates the precise three-dimensional coordinates of the arching point in the bunker through a spatial coordinate transformation algorithm. The early warning decision and path planning module communicates with the three-dimensional situation generation module to receive the precise three-dimensional coordinates of the arch point. Combined with the preset kinematic model of the early warning device and the obstacle distribution data in the warehouse, it calculates and generates a collision-free motion trajectory from the current attitude of the early warning device to the target arch breaking point. The device control module is connected to the early warning decision and path planning module and the early warning device respectively. It is used to receive the collision-free motion trajectory and convert it into driving commands to control the early warning device to perform the arch-breaking action. The feedback adjustment module is connected to the device control module and the AI ​​visual analysis and calculation module respectively. It is used to obtain the updated arching probability index in real time after the early warning device performs the arch-breaking action, and generate an adjustment command based on the comparison result of the updated arching probability index and the preset safety threshold, and send it to the device control module to dynamically adjust the arch-breaking parameters of the early warning device.

[0022] The video acquisition module includes explosion-proof high-definition cameras, supplementary lights, and lens self-cleaning units deployed on the top and side walls of the coal bunker. The lenses of the explosion-proof high-definition cameras face the surface area of ​​the coal flow, and are used to acquire clear continuous video image data in the dim and dusty underground environment. The supplementary lights are used to automatically turn on to supplement lighting when the light is insufficient. The lens self-cleaning unit is used to remove coal dust attached to the lens surface at regular intervals or according to instructions to maintain image clarity. The video acquisition module transmits the acquired continuous video image data to the AI ​​visual analysis and computing module in real time through the mine communication interface.

[0023] In actual underground coal bunker monitoring scenarios, explosion-proof high-definition cameras deployed on the top or side walls of the coal bunker collect continuous video image data of the coal flow surface in real time. This data is then transmitted to the AI ​​visual analysis and computing module via a mine communication interface. Next, the motion feature extraction unit in the AI ​​visual analysis and computing module uses optical flow to perform pixel-by-pixel analysis of the continuous video image frames, accurately tracking the instantaneous displacement vector of each identifiable particle on the coal flow surface. Then, the probability calculation unit calculates the average displacement velocity and residence time of all particles within a specified area based on these displacement vectors, comparing the average displacement velocity with a preset historical average velocity threshold, and simultaneously comparing the residence time with a preset historical average velocity threshold. By comparing the set time thresholds, when the average displacement velocity is lower than 30% of the historical average velocity threshold and the dwell time exceeds 5 seconds, the system determines that there is a hidden arching trend in the area and outputs an arching probability index between 0 and 1. The larger the value, the higher the probability of arching. The beneficial effect of this embodiment is that by converting conventional monitoring video into quantifiable particle motion characteristic parameters, it is possible to capture the initial signs of arching seconds or even minutes before coal bunker blockage occurs. This solves the problem that traditional manual visual observation cannot detect early hidden dangers, and provides a reliable data foundation for subsequent accurate positioning and active intervention, thereby significantly improving the early warning capability of coal mine safety production.

[0024] Second embodiment: as follows Figure 1As shown, the AI ​​visual analysis and calculation module includes a motion feature extraction unit and a probability calculation unit. The motion feature extraction unit uses optical flow or particle image velocimetry to perform pixel-by-pixel analysis on the received continuous video image frames, tracking the instantaneous displacement vector of each identifiable particle on the coal flow surface. The probability calculation unit calculates the average displacement velocity and residence time of all particles in the specified area based on the instantaneous displacement vector, compares the average displacement velocity with a preset historical average velocity threshold, and compares the residence time with a preset time threshold. When the average displacement velocity is less than 30% of the historical average velocity threshold and the residence time exceeds 5 seconds, it is determined that there is a hidden arching trend in the area and the corresponding arching probability index is output. The arching probability index is a value between 0 and 1, and the larger the value, the higher the probability of arching.

[0025] The 3D situation generation module includes a coordinate transformation unit and a model matching unit. The coordinate transformation unit receives the arching probability index and its corresponding 2D pixel coordinates output by the AI ​​visual analysis and calculation module. It uses a binocular vision stereo matching algorithm or a depth estimation technique based on monocular vision, combined with the intrinsic and extrinsic parameter matrices of the camera in the video acquisition module, to calculate the 3D coordinates of the arching point in the camera coordinate system through the principle of triangulation. The model matching unit transforms the 3D coordinates in the camera coordinate system to the world coordinate system of the pre-stored 3D structure model of the coal bunker, obtaining the accurate 3D coordinates of the arching point in the bunker, and outputs the accurate 3D coordinates to the early warning decision and path planning module.

[0026] Next, based on Example 1, this example further describes how the system utilizes the arching probability index and its two-dimensional image coordinates output by the first layer, through three-dimensional spatial transformation and path planning algorithms, to accurately calculate the spatial location of the arching point and generate the optimal arch-breaking intervention path. Specifically, the coordinate transformation unit in the three-dimensional situation generation module receives the arching probability index and its corresponding two-dimensional pixel coordinates output by the AI ​​visual analysis and calculation module, and uses a binocular vision stereo matching algorithm combined with the camera's intrinsic and extrinsic parameter matrices to calculate the three-dimensional coordinates of the arching point in the camera coordinate system through the principle of triangulation. Then, the model matching unit transforms the three-dimensional coordinates in the camera coordinate system to the world coordinate system of the pre-stored three-dimensional structure model of the coal bunker, obtaining the accurate three-dimensional coordinates of the arching point within the bunker and sending them to the early warning decision and path planning module; the early warning decision and path planning module can... The reachability analysis unit first determines whether the robotic arm end effector can reach the target position based on the precise three-dimensional coordinates and the preset kinematic model of the early warning device. If it is determined to be reachable, the path optimization unit searches for a collision-free, smooth joint movement, and energy-efficient motion trajectory from the current posture of the early warning device to the target arch-breaking point in the configuration space using the fast extended random tree algorithm based on the obstacle distribution data in the chamber and the motion constraints of each joint of the early warning device. The motion trajectory is then encoded into time-series joint angle data and sent to the device control module. The beneficial effect of this embodiment is that by converting two-dimensional image information into three-dimensional spatial coordinates and dynamically planning the optimal path, it solves the defect of traditional fixed arch-breaking devices that cannot accurately locate and effectively intervene in dynamically changing arch points, avoids ineffective operations and energy waste, and greatly improves the targeting and efficiency of arch-breaking operations.

[0027] Third embodiment: as follows Figure 1 As shown, the early warning decision and path planning module includes a reachability analysis unit and a path optimization unit. The reachability analysis unit receives the precise three-dimensional coordinates of the arch point output by the three-dimensional situation generation module, and calculates whether the end effector of the robotic arm can reach the target position by combining the preset kinematic model of the early warning device. If it is determined to be reachable, the path optimization unit is triggered to work. The path optimization unit searches for a collision-free, smooth joint movement and the lowest energy consumption motion trajectory from the current posture of the early warning device to the target arch breaking point in the configuration space based on the preset obstacle distribution data in the warehouse and the motion constraints of each joint of the early warning device using the fast extended random tree algorithm. The motion trajectory is encoded into time series joint angle data and sent to the device control module.

[0028] The device control module includes an instruction parsing unit and a drive control unit. The instruction parsing unit receives motion trajectory data sent by the early warning decision and path planning module, and parses it into joint angle, angular velocity and angular acceleration commands arranged in time sequence. The drive control unit generates corresponding pulse width modulation signals or analog voltage signals according to the parsed commands and sends them to the servo drivers of each joint of the early warning device to control the motor or hydraulic valve to move along the predetermined trajectory. At the same time, the drive control unit receives the actual position data fed back by the angle encoder and attitude sensor installed on the early warning device in real time, performs a closed-loop comparison with the theoretical commands, and corrects the output signal in real time to ensure motion accuracy.

[0029] The feedback adjustment module includes an effectiveness evaluation unit and a parameter adaptation unit. After the early warning device performs the arch-breaking action, the effectiveness evaluation unit obtains the updated arch-forming probability index from the AI ​​visual analysis and calculation module, compares the updated arch-forming probability index with a preset safety threshold, and if the updated arch-forming probability index is higher than the safety threshold, it determines that the current arch-breaking effect is insufficient and triggers the parameter adaptation unit to work. The parameter adaptation unit dynamically calculates and generates the adjustment amount of the arch-breaking parameters based on the difference between the updated arch-forming probability index and the safety threshold, using a proportional-integral-derivative control algorithm or a fuzzy control algorithm. The arch-breaking parameters include the blowing pressure of the arch-breaking head, the pulse frequency, and the slight angle correction value of the robotic arm end, and sends the adjustment command containing the above adjustment amount to the device control module.

[0030] The parameter adaptive unit includes a pressure regulation subunit, a frequency regulation subunit, and an angle fine-tuning subunit. The pressure regulation subunit calculates the required high-pressure air injection pressure adjustment value based on the rate of change of the arching probability index. When the arching probability index decreases slowly, the pressure value is increased until the preset upper limit is reached. The frequency regulation subunit calculates the pulse injection frequency adjustment value based on the speed at which the coal flow surface particles recover their flow. When the coal flow speed recovers quickly, the injection frequency is reduced to save energy. The angle fine-tuning subunit calculates the small offset of the robotic arm end on the original trajectory based on the spatial distribution characteristics of the arching points and the current actual position of the arch-breaking head, so as to impact the arching area from different angles until the coal flow returns to normal flow. The calculation results of all subunits are combined to generate a complete adjustment command.

[0031] The system also includes a data storage and traceability module, which is connected to the video acquisition module, AI visual analysis and calculation module, 3D situation generation module, early warning decision and path planning module, device control module, and feedback adjustment module. This module stores the original video image data, the historical curve of the arching probability index, the 3D coordinate change trajectory of the arching point, the planned arch-breaking movement trajectory, the actual executed drive commands, and the feedback adjustment records after each arch-breaking action in the order of timestamps. When a traceability query command is received, the system retrieves and replays the complete data chain at the corresponding time according to the specified time period or event identifier to support accident cause analysis and system parameter optimization.

[0032] Finally, based on Example 2, this example further describes how the system dynamically adjusts execution parameters according to real-time feedback after arch breaking. When the device control module converts the motion trajectory into a drive command and controls the early warning device to perform the first arch breaking action, the efficiency evaluation unit in the feedback adjustment module immediately obtains the updated arching probability index from the AI ​​visual analysis and calculation module, and compares the updated arching probability index with a preset safety threshold. If the updated arching probability index is still higher than the safety threshold, it is determined that the current arch breaking effect is insufficient, and the parameter adaptive unit is triggered. This unit dynamically calculates and generates the adjustment amount of the arch breaking parameters according to the difference between the updated arching probability index and the safety threshold using a proportional-integral-derivative control algorithm. Specifically, the pressure adjustment subunit calculates the adjustment value of the high-pressure air injection pressure according to the rate of change of the arching probability index, and increases the pressure value until the preset upper limit is reached when the arching probability index decreases slowly; the frequency adjustment subunit adjusts the pressure according to the coal flow... The adjustment value of the pulse injection frequency is calculated based on the velocity of surface particles recovering flow. When the coal flow velocity recovers quickly, the injection frequency is appropriately reduced to save energy. The angle fine-tuning subunit calculates the small offset of the robotic arm end on the original trajectory based on the spatial distribution characteristics of the arching point and the current actual position of the arch-breaking head, so as to impact the arching area from different angles. Finally, the parameter adaptive unit sends the adjustment command containing the above pressure, frequency and angle correction values ​​to the device control module, driving the early warning device to perform a secondary arch-breaking action. This cycle continues until the arching probability index drops below the safety threshold. The beneficial effect of this embodiment is that by constructing a complete mechanism of identification-positioning-execution-feedback-adjustment, the early warning device can adaptively adjust the arch-breaking strategy according to the actual recovery of the coal flow, realizing an intelligent leap from passive fixed-point injection to active tracking arch breaking. This maximizes the arch-breaking effect while avoiding excessive intervention and significantly reduces the secondary safety risks caused by arch-breaking operations.

[0033] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0034] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A coal mine hazard intelligent early warning device integrating AI video analysis, characterized in that: include: Explosion-proof pan-tilt camera unit, used to capture video images of the coal flow surface; The multi-degree-of-freedom arch-breaking execution unit includes an explosion-proof multi-joint robotic arm and an arch-breaking head installed at the end of the robotic arm, which is used to move to the target position to perform arch-breaking operations according to the received instructions; The environmental perception and positioning unit, including an angle encoder and an attitude sensor, is used to monitor the angles of each joint of the robotic arm and its overall attitude in real time. The explosion-proof main control and communication unit is connected to the explosion-proof pan-tilt camera unit, the multi-degree-of-freedom arch-breaking execution unit and the environmental perception and positioning unit, respectively, and is used to receive and process video data, generate control commands, coordinate the work of each unit and communicate data with the well surface monitoring center. The power and energy unit is connected to the multi-degree-of-freedom arch-breaking actuator and is used to provide the power required for the arch-breaking action.

2. A coal mine hazard identification system integrating AI video analysis, using the device described in claim 1, characterized in that: include: The video acquisition module is deployed in the coal bunker monitoring area underground in the coal mine to collect continuous video image data of the coal flow surface in real time. The AI ​​visual analysis and calculation module is communicatively connected to the video acquisition module. It is used to receive the continuous video image data and use the optical flow method to track and analyze the motion characteristics of the particles on the coal flow surface, and calculate the arching probability index that characterizes the motion state of the coal flow. The three-dimensional situation generation module is connected to the AI ​​visual analysis and calculation module. It is used to receive the arching probability index and its coordinates in the two-dimensional image. Combined with the pre-stored three-dimensional structure model of the coal bunker, the module calculates the precise three-dimensional coordinates of the arching point in the bunker through a spatial coordinate transformation algorithm. The early warning decision and path planning module is communicatively connected to the three-dimensional situation generation module. It is used to receive the precise three-dimensional coordinates of the arch point, and combine them with the preset kinematic model of the early warning device and the obstacle distribution data in the warehouse to calculate and generate a collision-free motion trajectory from the current posture of the early warning device to the target arch breaking point. The device control module is communicatively connected to the early warning decision and path planning module and the early warning device, respectively, and is used to receive the collision-free motion trajectory and convert it into driving commands to control the early warning device to perform the arch-breaking action. The feedback adjustment module is communicatively connected to the device control module and the AI ​​visual analysis and calculation module, respectively. It is used to obtain the updated arching probability index in real time after the early warning device performs the arch-breaking action, and generate an adjustment command based on the comparison result of the updated arching probability index and the preset safety threshold, and send it to the device control module to dynamically adjust the arch-breaking parameters of the early warning device.

3. The intelligent coal mine hazard identification system integrating AI video analysis according to claim 2, characterized in that: The video acquisition module includes explosion-proof high-definition cameras, supplementary lights, and lens self-cleaning units deployed on the top and side walls of the coal bunker. The lenses of the explosion-proof high-definition cameras face the surface area of ​​the coal flow, and are used to acquire clear continuous video image data in the dim and dusty underground environment. The supplementary lights are used to automatically turn on to supplement lighting when there is insufficient light. The lens self-cleaning unit is used to remove coal dust adhering to the lens surface at regular intervals or according to instructions to maintain image clarity. The video acquisition module transmits the acquired continuous video image data to the AI ​​visual analysis and computing module in real time through the mine communication interface.

4. The intelligent identification system for coal mine hidden dangers integrating AI video analysis according to claim 3, characterized in that: The AI ​​visual analysis and calculation module includes a motion feature extraction unit and a probability calculation unit. The motion feature extraction unit uses optical flow or particle image velocimetry to perform pixel-by-pixel analysis on the received continuous video image frames, tracking the instantaneous displacement vector of each identifiable particle on the coal flow surface. The probability calculation unit calculates the average displacement velocity and residence time of all particles in a specified area based on the instantaneous displacement vector, compares the average displacement velocity with a preset historical average velocity threshold, and compares the residence time with a preset time threshold. When the average displacement velocity is less than 30% of the historical average velocity threshold and the residence time exceeds 5 seconds, it is determined that there is a hidden arching trend in the area and the corresponding arching probability index is output. The arching probability index is a value between 0 and 1, with a larger value indicating a higher probability of arching.

5. The intelligent identification system for coal mine hidden dangers integrating AI video analysis according to claim 4, characterized in that: The three-dimensional situation generation module includes a coordinate transformation unit and a model matching unit. The coordinate transformation unit receives the arching probability index and its corresponding two-dimensional pixel coordinates output by the AI ​​visual analysis and calculation module. It uses a binocular vision stereo matching algorithm or a depth estimation technique based on monocular vision, combined with the intrinsic and extrinsic parameter matrices of the camera in the video acquisition module, to calculate the three-dimensional coordinates of the arching point in the camera coordinate system through the principle of triangulation. The model matching unit transforms the three-dimensional coordinates in the camera coordinate system to the world coordinate system of the pre-stored three-dimensional structure model of the coal bunker, to obtain the accurate three-dimensional coordinates of the arching point in the bunker, and outputs the accurate three-dimensional coordinates to the early warning decision and path planning module.

6. The intelligent identification system for coal mine hidden dangers integrating AI video analysis according to claim 5, characterized in that: The early warning decision and path planning module includes a reachability analysis unit and a path optimization unit. The reachability analysis unit receives the precise three-dimensional coordinates of the arch point output by the three-dimensional situation generation module, and calculates whether the end effector of the robotic arm can reach the target position by combining the preset kinematic model of the early warning device. If it is determined to be reachable, the path optimization unit is triggered to work. The path optimization unit searches for a collision-free, smooth joint movement and the lowest energy consumption motion trajectory from the current posture of the early warning device to the target arch breaking point in the configuration space based on the preset obstacle distribution data in the chamber and the motion constraints of each joint of the early warning device using a fast extended random tree algorithm. The motion trajectory is encoded into time series joint angle data and sent to the device control module.

7. The intelligent identification system for coal mine hidden dangers integrating AI video analysis according to claim 6, characterized in that: The device control module includes an instruction parsing unit and a drive control unit. The instruction parsing unit receives motion trajectory data sent by the early warning decision and path planning module, and parses it into joint angle, angular velocity, and angular acceleration instructions arranged in a time sequence. The drive control unit generates corresponding pulse width modulation signals or analog voltage signals according to the parsed instructions and sends them to the servo drivers of each joint of the early warning device to control the motor or hydraulic valve to move along a predetermined trajectory. At the same time, the drive control unit receives the actual position data fed back by the angle encoder and attitude sensor installed on the early warning device in real time, performs a closed-loop comparison with the theoretical instructions, and corrects the output signal in real time to ensure motion accuracy.

8. The intelligent identification system for coal mine hidden dangers integrating AI video analysis according to claim 7, characterized in that: The feedback adjustment module includes an efficiency evaluation unit and a parameter adaptation unit. After the early warning device performs the arch-breaking action, the efficiency evaluation unit obtains the updated arch-forming probability index from the AI ​​visual analysis and calculation module, compares the updated arch-forming probability index with a preset safety threshold, and if the updated arch-forming probability index is higher than the safety threshold, it determines that the current arch-breaking effect is insufficient and triggers the parameter adaptation unit to work. The parameter adaptation unit dynamically calculates and generates the adjustment amount of the arch-breaking parameters based on the difference between the updated arch-forming probability index and the safety threshold, using a proportional-integral-derivative control algorithm or a fuzzy control algorithm. The arch-breaking parameters include the blowing pressure of the arch-breaking head, the pulse frequency, and the slight angle correction value of the robotic arm end, and sends the adjustment command containing the above adjustment amount to the device control module.

9. A coal mine hazard identification system integrating AI video analysis according to claim 8, characterized in that: The parameter adaptive unit includes a pressure regulation subunit, a frequency regulation subunit, and an angle fine-tuning subunit. The pressure regulation subunit calculates the required high-pressure air injection pressure adjustment value based on the rate of change of the arching probability index. When the arching probability index decreases slowly, the pressure value is increased until a preset upper limit is reached. The frequency regulation subunit calculates the pulse injection frequency adjustment value based on the speed at which the coal flow surface particles recover their flow. When the coal flow speed recovers quickly, the injection frequency is reduced to save energy. The angle fine-tuning subunit calculates the small offset of the robotic arm end on the original trajectory based on the spatial distribution characteristics of the arching points and the current actual position of the arch-breaking head, so as to impact the arching area from different angles until the coal flow returns to normal flow. The calculation results of all subunits are combined to generate a complete adjustment command.

10. A coal mine hazard identification system integrating AI video analysis according to claim 9, characterized in that: The system also includes a data storage and traceability module, which is connected to the video acquisition module, AI visual analysis and calculation module, 3D situation generation module, early warning decision and path planning module, device control module, and feedback adjustment module. This module stores the original video image data, the historical curve of the arching probability index, the 3D coordinate change trajectory of the arching point, the planned arch-breaking movement trajectory, the actual executed driving commands, and the feedback adjustment records after each arch-breaking action in timestamp order. When a traceability query command is received, the system retrieves and replays the complete data chain at the corresponding time according to the specified time period or event identifier to support accident cause analysis and system parameter optimization.

Images