A method and system for constructing a robot swarm brain based on artificial intelligence

Abstract

The application relates to a robot group brain construction method and system based on artificial intelligence, and belongs to the technical field of intelligent mines. The method comprises the following steps: a group brain comprising a perception layer, a cognition layer, a decision layer and an execution layer is established; state data of a plurality of source heterogeneous robots in a mine is received through the perception layer; a digital twin map of the mine is constructed in the cognition layer, and the heterogeneous robots are mapped in real time, a safety risk multi-model is constructed in the cognition layer for situation analysis, and risk probability prediction in multiple time steps is realized; task allocation, path planning, conflict detection and dynamic threshold adjustment are performed in the decision layer; and group-level cooperation is realized through path and task instruction issuing in the execution layer. Through global cooperation of the group brain and adaptive scheduling of multiple intelligent agents, the overall operation efficiency and survival ability of the underground robot group are improved, group-level robustness and autonomy are realized, future situation analysis of the underground environment can be realized, and accurate real-time risk early warning is provided.

Description

Technical Field

[0001] This invention belongs to the field of smart mining technology and relates to a method and system for constructing a robot swarm brain based on artificial intelligence. Background Technology

[0002] Currently, coal mining remains a high-risk industry. The environment of fully mechanized coal mining faces is extremely complex. Most fully mechanized coal mining operations involve very confined working spaces and are affected by various natural factors such as geological conditions, gas, water, fire, and rock bursts. Furthermore, during construction, accidents such as gas and coal dust explosions or water inrushes can lead to casualties. In coal mining geology, soft, low-permeability, and high-gas coal seams account for approximately 60%, making them extremely difficult to drain. Therefore, gas hazards endanger most mining areas. Gas hazards are the most threatening and destructive major disaster in deep coal mining, including various types such as coal and gas outbursts, gas ejections, gas explosions, gas combustion, and gas asphyxiation. As shallow coal resources gradually deplete and mining depth and intensity continue to increase, coal deposit conditions become more complex, and factors such as ground stress, gas pressure, and ground temperature increase, leading to increasingly severe and complex gas hazards. Because coal mines contain a wide variety of harmful and toxic gases, and because the causes of disasters in coal mines are not limited to gas, existing monitoring methods have low accuracy in predicting disasters in coal mines, insufficient prevention and control capabilities, and cannot meet the needs of predicting the future development trend of gas disasters, thus failing to achieve early prevention.

[0003] To prevent gas disasters, the traditional approach is to manually inspect the longwall mining face at regular intervals. This not only consumes a large amount of manpower but also threatens the personal safety of the inspection personnel. With the development of robots and their control technologies, using robots for inspection has become a more ideal choice. Currently, the operation of mobile equipment in underground coal mines suffers from problems such as opaque environmental information, poor ability to cope with changes in roadway geological conditions, and low levels of integration and systematization. Intelligentization is still at the stage of robotization of individual pieces of equipment, and a comprehensive system-wide management and control of an intelligent robot swarm has not yet been formed.

[0004] Furthermore, current technologies for controlling sensors, robots, and other equipment in coal mines are decentralized, meaning that separate control modules are used for sensors and separate control modules for robots. This management approach is costly and does not create a coupling between the two, which is not conducive to achieving more unmanned and intelligent coal mine safety construction. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method and system for constructing a robot swarm brain based on artificial intelligence. By treating underground sensors, mobile robots, and other components as various heterogeneous robots, a swarm brain with a hierarchical architecture of perception, cognition, decision-making, and execution is constructed. This enables multi-source information fusion, risk prediction, dynamic scheduling of group tasks, and conflict coordination among heterogeneous robots, forming an autonomous mine integrated management and control system with collective intelligence.

[0006] To achieve the above objectives, the present invention provides a method for constructing a robot swarm brain based on artificial intelligence, the method comprising:

[0007] The sensors and mobile robots deployed underground are considered as heterogeneous robots. Each heterogeneous robot is assigned a unique ID, and each heterogeneous robot establishes a communication connection with the ground server through wireless communication. The so-called group brain includes a perception layer, a cognition layer, a decision-making layer, and an execution layer. The perception layer is used to collect the status data of multi-source heterogeneous robots. The cognition layer conducts situational analysis by constructing a digital twin map of the mine and multiple safety risk models. The decision-making layer performs task allocation, conflict detection, and dynamic threshold adjustment. The execution layer issues path and task instructions to achieve group-level collaboration.

[0008] A geometric model of the mine is constructed at the cognitive level, and a digital twin map of the mine is built based on this geometric model;

[0009] The sensors deployed underground in the mine are mapped onto the constructed digital twin map to establish a correspondence between the sensors and their physical locations; the mobile robots deployed underground in the mine are also mapped onto the constructed digital twin map, and the position changes of the mobile robots are simultaneously displayed in the digital twin map.

[0010] When a sensor issues an over-limit alarm while a neighboring sensor of the same type does not, the sensor that issued the alarm is verified using Kalman filtering. If the sensor is found to be abnormal or has a false alarm, the nearest mobile robot to the sensor is called and a path is planned to reach that location for on-site verification.

[0011] At the decision-making level, a coupled detection model and prediction model for downhole safety risks are constructed. The prediction model is used to calculate the future predicted values ​​of sensor data related to safety risks, and the detection model combines the current monitoring value of the sensor and the predicted value at the future time to calculate the risk probability at the current time, and at the same time calculate the risk probability at the future time.

[0012] The decision-making level determines whether to issue an alarm message based on the risk probability at the current moment and the future moment output by the detection model, using a dynamic threshold adjustment strategy.

[0013] Furthermore, the data verification includes: taking the historical values ​​of the sensor within a certain time window in the perception layer as the initial state, and updating the current time through Kalman filtering. t Posterior state estimation Compare the actual measured values ​​of the sensor. and Calculate the residual If residual Exceeding the preset threshold If the sensor is marked as a suspected anomaly in the digital twin map, a mobile robot will be triggered to verify it.

[0014] Furthermore, when performing path planning for the mobile robot at the execution layer, the system first determines the mobile robot closest to the sensor location based on the Euclidean distance. i ;

[0015] Construct a graph structure in the digital twin map that includes alleyways, dynamic obstacles, and intersections. ; The nodes in the graph structure include the locations of dynamic obstacles and lane intersections; For the graph structure, each edge represents a path connecting nodes. Related attributes include tunnel length and minimum width of the alley , , M The number of sides;

[0016] With mobile robots i Starting from the current position and using the location of this sensor as the destination point, the robot is guided by the A* algorithm. i Plan the shortest path from the starting point to the destination;

[0017] Update neighbor nodes;

[0018] In mobile robots i During the process of traveling to this area according to the planned route, the calculation of the next node is performed in each control cycle. Direction vector and expected speed .

[0019] Furthermore, when the execution layer controls the mobile robot to move along the planned path, the swarm brain performs real-time multi-robot conflict detection through a digital twin map, including node conflicts and edge conflicts.

[0020] When the robot is detected i Other higher priority robots j When a conflict occurs, the robot i Adopt an avoidance strategy:

[0021] If the robot i , j If there is a node conflict between them, then the robot... i Perform deceleration or stop and wait;

[0022] If the robot i , j If there is an edge conflict between them, then the robot i Perform deceleration or stop waiting, or calculate the robot's speed. i Stay away from robots j Safety offset direction Then modify the robot i The speed, add a pointer The amount of avoidance;

[0023] If the robot i If a deadlock occurs, the robot will... j Treating them as static obstacles and using the A* algorithm for the robot i Plan temporary paths and generate bypass routes to avoid the robot. j And from the robot i The robot reaches the next node in its global path from its current position via an alternative short path segment. After reaching the next node using this alternative short path segment, the robot switches back to the global path.

[0024] Furthermore, the cognitive layer includes various safety risk prediction models, including a gas accumulation prediction model, a fire prediction model, and a roof crack prediction model. These models predict the future values ​​and image feature evolution of various risk-related sensors based on historical time-series data and image features, thereby enabling a forward-looking assessment of the risk situation at multiple time steps in the future.

[0025] Furthermore, the cognitive layer also includes a variety of safety risk detection models, including a gas accumulation detection model, a fire detection model, and a roof crack detection model. These models are based on the joint input of real-time sensor data and prediction data, respectively. They extract temporal and image multimodal features through an attention fusion mechanism, calculate the risk probability of the current and future multiple time steps, and realize joint detection of risks in both time domains.

[0026] The gas accumulation detection model calculates the risk probability of gas accumulation at the current moment based on the current monitoring value of the gas accumulation-related sensors and the predicted value at the future moment, and also calculates the risk probability of gas accumulation at the future moment.

[0027] The fire detection model calculates the probability of a fire at the current moment based on the current monitoring values ​​of fire-related sensors and the predicted values ​​at future moments, and also calculates the probability of a fire at future moments.

[0028] The roof crack detection model calculates the risk probability of roof cracks at the current moment based on the current monitoring values ​​of roof crack-related sensors and the predicted values ​​at future moments, and also calculates the risk probability of roof cracks at future moments.

[0029] Furthermore, the gas accumulation detection model, fire detection model, and roof crack detection model all employ the same network structure, comprising:

[0030] The input layer is used to preprocess the sensor's current monitoring values ​​and future predicted values, transforming them into high-dimensional vectors.

[0031] The time series data branch extracts the time series features of both the current time-series data and the predicted data to obtain the time series features of the current time-series data. and the time series characteristics of the predicted data ;

[0032] In the image data branch, for both the current time-step data and the prediction data, image features are extracted from the input image data to obtain the image features of the current time-step data. and predicted data image features ;

[0033] The attention layer performs weighted fusion of temporal features and image features from the current time-series data to obtain the fused features. Simultaneously, the temporal features and image features from the prediction data are weighted and fused to obtain the fused features. ;

[0034] Dynamic fusion module, for and Perform weighted fusion: , , For dynamic fusion weights;

[0035] The output layer is activated using the sigmoid function to obtain the current time step. t Risk probability .

[0036] In this process, each detection model outputs fused features at the attention layer when calculating the risk probability at future time moments. Then it directly enters the output layer to obtain t + j Risk probability at any moment .

[0037] Furthermore, the output of the prediction model undergoes uncertainty assessment and confidence calibration before being input into the detection model. This assessment includes: using Kalman filtering to perform posterior correction on the predicted value and obtain the posterior estimated covariance; calculating the prediction confidence based on the posterior estimated covariance; and replacing the predicted value with a safe default value when the confidence is below a preset threshold to reduce the interference of low-confidence predictions on the detection model. The actual value input to the detection model is:

[0038]

[0039]

[0040] In the formula, For sensors exist t + j Predicted value at time, For the corresponding confidence level, The minimum confidence threshold, As a safe default value, Scaling factor for t + j The posterior estimate of the covariance at time t.

[0041] Furthermore, based on the multi-time-step risk probabilities output by each detection model, the decision-making level calculates the trend indicators for each security risk and dynamically adjusts the early warning thresholds for each security risk, including: for a certain security risk, calculating the future... k The average risk probability corresponding to each time step is used as a trend indicator. The alarm threshold is adaptively adjusted based on the trend indicator and the current risk probability to achieve dynamic threshold control that changes with the risk trend.

[0042] Specifically, based on the future output of a certain detection model k Risk probability at time 1 , ..., Assess future risk trends and set trend indicators. Defined as the future k The average risk probability at each moment. Set dynamic alarm thresholds Dynamically adjust based on the current risk probability and trend indicators. : , Basic alarm threshold, This is for adjusting the coefficient.

[0043] On the other hand, the present invention provides an artificial intelligence-based robot swarm brain system, the system comprising:

[0044] The underlying physical layer includes sensors and various types of mobile robots deployed underground in the mine. The sensors include sensors for methane, carbon monoxide, carbon dioxide, oxygen, temperature, wind speed, stress, and image processing. The mobile robots include autonomous inspection robots and task execution robots.

[0045] Intermediate layer functional modules: forming the core of the group brain, including:

[0046] The robot swarm management module is used to collect the status information of various heterogeneous robots and encode it uniformly, upload data to the swarm brain and receive task and path instructions;

[0047] The prediction module includes sub-modules for gas accumulation prediction, fire prediction, and roof crack prediction, which are used to predict the evolution of sensor values ​​and image features in the future at multiple time steps based on historical multimodal time series data.

[0048] The detection module includes sub-modules for gas accumulation detection, fire detection, and roof crack detection, which calculate the risk probability at multiple time steps based on real-time monitoring values ​​and predicted values.

[0049] The data verification module verifies single-point over-limit sensor data by combining Kalman filtering with neighborhood sensor information and triggers on-site robot verification.

[0050] The planning module dynamically plans paths for multiple robots based on a digital twin map and executes node / edge conflict detection, hierarchical avoidance, and deadlock resolution strategies.

[0051] The dynamic threshold adjustment module adaptively adjusts the warning thresholds for each risk based on risk probability and trend indicators.

[0052] The edge computing module provides real-time computing power support for each functional sub-module;

[0053] The storage module stores multi-source sensor data, robot operating parameters, and model weights.

[0054] Top-level application layer: includes a digital twin module, an early warning module, and a display module. The digital twin module constructs a digital twin map of the mine based on the mine's geometric model and real-time status and maps the positions of robots and sensors. The early warning module outputs early warning information based on the risk probability adjusted by dynamic thresholds. The display module is used to visualize the mine's operating status and risk distribution.

[0055] The beneficial effects of this invention are as follows:

[0056] (1) This invention targets distributed sensors and mobile robots in underground mines. By treating them as heterogeneous robots and managing them comprehensively through a server, a robot swarm brain for the underground environment is constructed. Based on the robot swarm brain, communication and interaction are established with the heterogeneous robots to achieve group management of the heterogeneous robots, laying a technical foundation for unmanned and intelligent coal mines. Through the global coordination and multi-agent adaptive scheduling of the swarm brain, the overall operating efficiency and survivability of the underground robot swarm are improved, achieving group-level robustness and autonomy.

[0057] (2) To address the problem of difficulty in predicting downhole safety risks due to the coupling of multiple factors, this invention constructs a dedicated detection model for each safety risk and a dedicated prediction model for the corresponding sensor data of each safety risk. The probability of safety risks is calculated through the dedicated detection models. At the same time, a dual-channel input of real-time data and prediction data is designed, and a fusion module is used to perform weighted fusion to calculate the risk probability, which can make risk prediction more accurate. In addition, based on the risk probabilities output by each detection model, this invention also proposes a dynamic threshold adjustment strategy, which dynamically adjusts the alarm threshold based on the current risk probability and future risk trend, and can provide more accurate real-time risk warnings.

[0058] (3) To address the problem of false alarms from single-point sensors, this invention proposes a data verification mechanism based on Kalman filtering. The false alarm sensor is verified by the monitoring values ​​of adjacent sensors of the same type, which avoids frequent manual re-inspection operations and can greatly save time and costs.

[0059] (4) In response to the problems of dangerous manual verification, low efficiency of single robot operation, and frequent conflicts in alleyways, this invention proposes to dynamically schedule the nearest robot verification false alarm based on digital twin map, and designs a conflict detection model (node / edge conflict) and a graded avoidance strategy (deceleration / path replanning), which can greatly reduce the dependence on human resources, avoid manual risks, improve the verification response speed, ensure efficient collaboration of multiple robots in narrow alleys through the conflict resolution mechanism, and enhance equipment utilization through the deadlock resolution mechanism.

[0060] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0062] Figure 1A schematic diagram illustrating a method for constructing a robot swarm brain based on artificial intelligence, as provided in an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of the group brain architecture shown in an embodiment of the present invention;

[0064] Figure 3 This is a block diagram of an artificial intelligence-based robot swarm brain system provided for an embodiment of the present invention. Detailed Implementation

[0065] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0066] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0067] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0068] like Figure 1 As shown, this invention provides a method for constructing a robot swarm brain based on artificial intelligence. The method includes:

[0069] 1. Treat the sensors and mobile robots deployed underground as heterogeneous robots, assign a unique ID to each heterogeneous robot, and establish a communication connection with the ground server via wireless communication. For example... Figure 2As shown, the group brain includes a perception layer, a cognition layer, a decision-making layer, and an execution layer. The perception layer is used to aggregate the state data of multi-source heterogeneous robots. The cognition layer performs situational analysis by constructing a digital twin map of the mine and a multi-model of safety risks. The decision-making layer performs task allocation, conflict detection, and dynamic threshold adjustment. The execution layer issues path and task instructions to achieve group-level collaboration.

[0070] The perception layer receives and processes the data uploaded by each heterogeneous robot. The data uploaded by the heterogeneous robots are divided into the following categories according to the robot type: 1) Sensor-type heterogeneous robots upload their collected sensor data, location information, and ID codes, etc.; 2) Mobile robot-type heterogeneous robots upload their location information, motion information (such as moving speed, azimuth angle, etc.), and status information (such as battery level, ID code, stationary or moving state, whether deadlocked, etc.).

[0071] The decision-making layer plans a path for the mobile robot according to the task requirements, and the execution layer issues the planned path and task instructions to the corresponding mobile robot, which then executes the task according to the planned path through the processor on the mobile robot.

[0072] 2. Construct a geometric model of the mine at the cognitive level, and then construct a digital twin map of the mine based on this geometric model.

[0073] The geometric model includes the mine's ventilation network, roadways, longwall mining faces, intake shafts, and return shafts. A digital twin map of the mine is constructed by reconstructing and rendering the geometric model.

[0074] 3. Map the sensors deployed underground to the constructed digital twin map to establish the correspondence between the sensors and their physical locations; map the mobile robots deployed underground to the constructed digital twin map, and synchronously present the position changes of the mobile robots in the digital twin map.

[0075] In the main areas of the mine, such as the fully mechanized mining face, there are methane sensors, carbon monoxide sensors, temperature sensors, ethylene sensors, stress sensors, wind speed sensors, and image sensors. In the roadways and chambers, there are also methane sensors, wind speed sensors, temperature sensors, ethylene sensors, stress sensors, and image sensors.

[0076] The monitoring data uploaded by each sensor is collected and displayed in the form of a heat map in the digital twin map to show the status of each area downhole; specifically, a heat map is drawn for the location of each sensor based on the threshold of each monitoring item.

[0077] 4. Construct a tightly coupled detection and prediction model for downhole safety risks at the cognitive layer.

[0078] 1) The safety risks mentioned include gas accumulation, fire and roof cracks.

[0079] 2) The prediction models include a gas accumulation prediction model, a fire prediction model, and a roof crack prediction model, which are specifically designed to predict the future output of sensor data related to each safety risk. For example, the gas accumulation prediction model takes methane, carbon monoxide, carbon dioxide, oxygen, wind speed sensor data, and image sensor data as input and predicts the future evolution of these sensor data and related image features.

[0080] Correspondingly, the safety risk detection models include a gas accumulation detection model, a fire detection model, and a roof crack detection model, which are specifically designed to detect safety risks such as underground gas accumulation, fire, and roof cracks. Each detection model selects appropriate sensor data and the output of the corresponding prediction model as input based on the specific task, and then predicts the probability of the safety risk. For example, the gas accumulation detection model takes data from methane, carbon monoxide, carbon dioxide, oxygen, wind speed sensors, and image sensor data, as well as the future evolution of these sensors as output from the gas accumulation prediction model, and then predicts the probability of gas accumulation at the current moment and the probability of gas accumulation at future moments.

[0081] 3) For the gas accumulation prediction model, it is used to predict future numerical time-series data from methane, carbon monoxide, carbon dioxide, oxygen, and wind speed sensors, as well as image features related to gas accumulation (e.g., predicting anomalous patterns in future image frames using a lightweight ConvLSTM or GAN). Specifically, the inputs to the gas accumulation prediction model include historical methane, carbon monoxide, carbon dioxide, oxygen, and wind speed sensor data, along with relevant image data, and its output is the future numerical time-series data from these sensors. n The predicted values ​​at each time step, as well as the predictions of gas accumulation-related features in future images (such as coal dust and smoke during gas outbursts).

[0082] For fire prediction models, the future time-series data of temperature, carbon monoxide, ethylene, and oxygen sensors, as well as the future evolution of visual features such as smoke and flames in images, are used to predict these parameters. Specifically, the inputs to the fire prediction model include historical temperature, carbon monoxide, ethylene, and oxygen sensor data, as well as image data; the output is the future data of these sensors. n Predicted values ​​at each time step, and predictions of fire-related features in future images.

[0083] The roof crack prediction model is used to predict future numerical time-series data from stress sensors, as well as the future evolution of visual features such as cracks and deformations in images. Specifically, the inputs to the roof crack prediction model include historical stress sensor data and image data, and its output is the future numerical time-series data from stress sensors. n The predicted values ​​at each time step, and the predicted features related to the roof cracks in future images.

[0084] By designing a prediction model specific to each security risk, we can more professionally learn and capture the temporal patterns of sensor data related to its specific risk, thereby improving prediction accuracy, avoiding predictions of irrelevant sensor data, and reducing the overall computational load.

[0085] 4) For the gas accumulation detection model, its inputs include real-time numerical time-series data from sensors such as methane sensor, carbon monoxide sensor, carbon dioxide sensor, oxygen sensor, and wind speed sensor, real-time image data from image sensor, and prediction output from the gas accumulation prediction model.

[0086] Before being input into the gas accumulation detection model, the predicted outputs of the gas accumulation prediction model need to undergo uncertainty assessment and confidence-based calibration / filtering to improve robustness, as detailed below:

[0087] A. Predictive Output Integration with Kalman Filtering: Assuming the gas accumulation prediction model is effective for the sensor... S i In the future k time steps ( ... The gas concentration was predicted. These predictions were treated as "prior estimates" of the Kalman filter, and the uncertainty was quantified by the covariance update of the Kalman filter.

[0088] Define system state For sensors exist t The actual gas concentration value at that moment. For the prediction model... Each future prediction value output The Kalman filter iterative process is as follows:

[0089] State prediction: , For sensors exist t + j Prior state estimation at time 10:00 F Let be the state transition matrix, describing the state transition from... t + j -1 to t + j The evolution, For sensors exist t + j Posterior state estimation at time -1;

[0090] Covariance prediction: , For sensors exist t + j The prior estimate of covariance at time t, For sensors exist t + j The posterior estimate of the covariance at time t. Q For process noise covariance;

[0091] Update Kalman gain: , For Kalman gain, H For the observation matrix, R To observe the noise covariance;

[0092] Status Update: , For sensors exist t + j The posterior state estimate at time 1, i.e., the gas concentration prediction after Kalman filtering correction;

[0093] Covariance update: , For sensors exist t + j The posterior estimate of the covariance at time t.

[0094] B. Uncertainty assessment and confidence level calculation:

[0095] Will As a sensor exist t + j A measure of the uncertainty of the predicted value at any given time. Define the prediction confidence level. , This is a positive scaling factor used to adjust the sensitivity of the confidence level to the impact of uncertainty. The larger the value, the greater the penalty to the confidence level due to uncertainty. A specific value can be set based on expert experience.

[0096] C. Confidence-based input calibration / filtering: This involves adjusting the sensor... exist t + j Predicted value at time ( ) and its corresponding confidence level Multiply, and use as the actual value input to the gas accumulation detection model. , When the confidence level of the predicted value is low, its impact on the gas accumulation detection model will be weakened. Furthermore, a minimum confidence threshold can be set. ,if Then the predicted value is considered to be... Too unreliable, use a safe default value for it. Alternative input detection model.

[0097]

[0098] in, You can choose the historical average value of gas concentration or the safe lower limit value of gas concentration, etc.

[0099] The above describes the uncertainty assessment and confidence-based calibration / filtering process for the gas accumulation prediction model based on gas sensor data. Similarly, the data from other sensors related to gas accumulation undergo the same uncertainty assessment and confidence-based calibration / filtering process. Likewise, the fire prediction model and roof crack prediction model also employ the above process to assess the uncertainty and perform confidence-based calibration / filtering on the predicted values ​​of the corresponding sensor data before inputting the processed predicted values ​​into the corresponding detection models.

[0100] The gas accumulation detection model simultaneously judges the gas accumulation at the current moment and predicts the gas accumulation risk at future moments based on real-time sensor data and processed predicted values. It dynamically fuses predicted values ​​and real-time values ​​and dynamically adjusts the sensitivity of the early warning based on the fusion result.

[0101] The gas accumulation detection model includes an input layer, a time-series data branch, an image data branch, an attention layer, a dynamic fusion module, and an output layer.

[0102] The input layer preprocesses the input real-time numerical time-series data and image data and transforms them into high-dimensional vectors. The numerical time-series data is encoded as vectors. Image data is encoded into vectors. The predicted numerical time-series data and image data are processed in the same way and encoded as follows: , .

[0103] The time-series data branch uses an LSTM network to capture the long-term dependencies and trends of numerical time-series data such as gas concentration and wind speed. Then, the LSTM output is further processed through a one-dimensional convolutional layer (CNN) to extract local time-series features, enhancing the feature representation capability. The process is as follows:

[0104] ,

[0105] ,

[0106] in, , For the output of the LSTM network, , This is the temporal feature vector output by a one-dimensional convolutional layer.

[0107] The image data branch uses a multi-layer CNN network (such as a lightweight version of ResNet or InceptionNet) to extract image feature vectors. , It is used to help identify visual anomalies that may be caused by gas accumulation.

[0108] The attention layer is implemented based on Transformer, which excels at capturing complex relationships between data from different modalities. The attention layer uses the Transformer Encoder structure to... and Weighted fusion is performed. Details are as follows:

[0109] First, concatenate the feature vectors: Then generate the Query (Q), Key (K), Value (V) matrix: , , ,in , , This is a learnable weight matrix. Then, the attention weights are calculated: , For matrix K The dimension; finally, the feature vectors are fused: Similarly, the temporal features and image features of the predicted values ​​are obtained through weighted fusion using an attention layer. .

[0110] The dynamic fusion module will and Perform weighted fusion again: , , These are the dynamic fusion weights, Among them, before the integration First, we need to reduce the dimensionality to... Maintain the same dimensions.

[0111] The output layer uses the sigmoid function activation to obtain the current time step. t The probability of gas accumulation .

[0112] The gas accumulation detection model also calculates and outputs future time-based prediction data. t + j The probability of gas accumulation It is important to note that when calculating future moments...t + j When calculating the probability of gas accumulation risk, weighted fusion is not required through a dynamic fusion module; the fused features are output at the attention layer. Then it directly enters the output layer to obtain .

[0113] 5) For fire detection models, the input data includes data from temperature sensors, carbon monoxide sensors, ethylene sensors, oxygen sensors, image sensors (used to identify visual features such as smoke and flames), and the predicted values ​​of these sensor data from the fire prediction model.

[0114] The fire detection model has the same structure as the gas accumulation detection model. The difference lies in that the GRU network is used to process numerical time-series data such as temperature and CO in the time-series data branch, while EfficientNet is used to process image data in the image data branch to extract visual features such as smoke and flames. The data processing procedure of the fire detection model is the same as that of the gas accumulation detection model.

[0115] 6) For the roof crack detection model, the input data includes stress sensor data, image sensor data (used to identify visual features such as roof cracks and support deformation), and the fire prediction model's prediction values ​​of these sensor data.

[0116] The structure of the roof crack detection model is consistent with that of the gas accumulation detection model. The difference lies in that the TCN network is used to process the stress numerical time-series data in the time-series data branch, and UNet is used to extract image features in the image data branch, thereby better capturing the details and contextual information in the images. The data processing procedure of the roof crack detection model is the same as that of the gas accumulation detection model.

[0117] It should be noted that in downhole applications, if the output probability value of a certain detection model remains close to 50%, it indicates that the detection model lacks confidence in its risk assessment and needs to be retrained.

[0118] 5. The decision-making layer receives the risk probabilities for the current and future times from each detection model and decides whether to output alarm information through a dynamic threshold adjustment strategy.

[0119] For example, regarding gas accumulation, a dynamic threshold adjustment strategy is used to determine whether to output a gas accumulation alarm message, as described below:

[0120] First, based on the future k Risk probability at time 1 , ..., Assess future risk trends and set a trend indicator. Defined as the future kThe average risk probability at each moment. ;

[0121] Then, set a dynamic alarm threshold. Dynamically adjust based on the current risk probability and trend indicators. : , The basic alarm threshold for gas accumulation (set based on experience). An adjustment coefficient between 0 and 1. This is achieved through a dynamic alarm threshold, when... At higher levels, Larger, making Reducing the warning threshold allows for earlier alerts. Conversely, if the future risk trend is stable or declining, the dynamic threshold will either increase or remain unchanged to avoid excessive warnings.

[0122] For fires and roof cracks, a dynamic threshold adjustment strategy is used to determine whether to output the corresponding alarm information.

[0123] 6. When a sensor issues an over-limit alarm while adjacent sensors of the same type do not, Kalman filtering is used to verify the data of the alarming sensor. If the sensor is found to be abnormal or has a false alarm, it is highlighted as a suspected anomaly on the digital twin map. The nearest mobile robot to the sensor's location is then activated, and a path is planned to reach that location for on-site verification. Specifically, the sensors mounted on the mobile robot are used to verify whether the alarm is false.

[0124] 1) The data verification process based on Kalman filtering includes:

[0125] Initial state estimation Specifically, the average value of the sensors that should have issued an alarm within a certain time window is taken as the initial state. Initial covariance matrix. Take a smaller diagonal matrix.

[0126] Prior state estimation: , B To control the input matrix; For control vectors, representing t External control input at any given time, i.e., adjacent similar sensors at... t Real-time sensor data.

[0127] Covariance prediction: .

[0128] Update Kalman gain: .

[0129] Status Update: ,in, for t Posterior state estimation at time 1.

[0130] Covariance update: .

[0131] Compare the actual measured values ​​of the sensor and Calculate the residual If the residual Exceeding the preset threshold If the threshold value is not met, the sensor's data is considered abnormal or a false alarm. It can be determined based on historical data, sensor accuracy, and the mine's sensitivity to gas concentration.

[0132] 2) Path planning methods for mobile robots include:

[0133] ① First, determine the mobile robot closest to the sensor's location. Based on the sensor's coordinates, calculate the Euclidean distance between the current coordinates of all mobile robots and the sensor's coordinates. The mobile robot with the smallest distance is selected. i As the mobile robot closest to the sensor.

[0134] ②Then, for the robot i Plan the path to the sensor:

[0135] A. Construct a graph structure based on the digital twin map. ,in, The nodes in the graph structure include the locations of dynamic obstacles and alleyway intersections. Let be the edges of the graph structure, representing the pathways connecting the nodes. Each edge... Related attributes include tunnel length and minimum width of the alley , .

[0136] B. Using robots i Starting from the current position and using the location of this sensor as the destination point, the robot is guided by the A* algorithm. i The shortest path from the starting point to the destination is calculated, and the cost of the path is expressed as:

[0137]

[0138] in, From the starting point to the node v The actual cost is generally the cumulative distance or time. This is a heuristic function used to estimate the number of nodes.v The cost of reaching the destination.

[0139] node v neighboring nodes u The update method is represented as:

[0140]

[0141] in, For the edge The cost, i.e., the node v and nodes u The length of the alleyway between them.

[0142] C. In the robot i During the process of traveling to this area according to the planned route, the calculation of the next node is performed in each control cycle. Direction vector and expected speed :

[0143]

[0144] in, To control the cycle, For robots i Maximum moving speed. This speed is tracked by a PID controller. .

[0145] The control cycle is for each robot update. i The period of the remaining global path.

[0146] D. In the robot i During the process of the robot traveling along the planned path to the location of the sensor, the robot is detected. i Other higher priority robots j Potential conflicts between nodes include node conflicts and edge conflicts.

[0147] Among them, node conflict is a robot i , j Expected in the future T Arrive at the same node within a time limit Edge conflict occurs when two robots are on the same edge. Moving towards each other, and expected in the future T Distance within time is less than , , Robots i , j The effective collision radius of the robot. ,in, r The physical radius of the robot. To account for safety margins in positioning and control errors.

[0148] Set up a robot i Arrival at the point of conflict The estimated time is ,robot j Arrival at the point of conflict The estimated time is Then the robot i , j The conditions for a conflict are:

[0149]

[0150] in, For safety time margin.

[0151] E. When a robot is detected i Other higher priority robots j When a conflict occurs, the robot i Adopt an avoidance strategy.

[0152] If the robot i , j If there is a node conflict between them, then the robot... i Perform a slowdown or stop and wait.

[0153] If the robot i , j There is an edge conflict between them, and Then the robot i Perform deceleration or stop and wait; if Computational robots i Stay away from robots j Safety offset direction Then modify the robot i The speed, add a pointer The avoidance component, the modified robot velocity is expressed as:

[0154]

[0155] in, To avoid the intensity coefficient, the final speed parameters input to the PID controller are:

[0156]

[0157] If the robot i If a deadlock occurs, the robot will... j Treating them as static obstacles and using the A* algorithm for the robot i Plan temporary paths and generate bypass routes to avoid the robot. j And from the roboti The robot reaches the next node in its global path from its current position via an alternative short path segment. After reaching the next node using this alternative short path segment, the robot switches back to the global path.

[0158] like Figure 3 As shown, another embodiment of the present invention provides an artificial intelligence-based swarm brain system, which adopts a hierarchical architecture and includes:

[0159] The underlying physical layer includes sensors and various types of mobile robots deployed underground in the mine. The sensors include sensors for methane, carbon monoxide, carbon dioxide, oxygen, temperature, wind speed, stress, and image processing. The mobile robots include autonomous inspection robots and task execution robots.

[0160] Intermediate layer functional modules: forming the core of the group brain, including:

[0161] The robot swarm management module is used to collect the status information of various heterogeneous robots and encode it uniformly, upload data to the swarm brain and receive task and path instructions;

[0162] The prediction module includes sub-modules for gas accumulation prediction, fire prediction, and roof crack prediction, which are used to predict the evolution of sensor values ​​and image features in the future at multiple time steps based on historical multimodal time series data.

[0163] The detection module includes sub-modules for gas accumulation detection, fire detection, and roof crack detection, which calculate the risk probability at multiple time steps based on real-time monitoring values ​​and predicted values.

[0164] The data verification module verifies single-point over-limit sensor data by combining Kalman filtering with neighborhood sensor information and triggers on-site robot verification.

[0165] The planning module dynamically plans paths for multiple robots based on a digital twin map and executes node / edge conflict detection, hierarchical avoidance, and deadlock resolution strategies.

[0166] The dynamic threshold adjustment module adaptively adjusts the warning thresholds for each risk based on risk probability and trend indicators.

[0167] The edge computing module provides real-time computing power support for each functional sub-module;

[0168] The storage module stores multi-source sensor data, robot operating parameters, and model weights.

[0169] Top-level application layer: includes a digital twin module, an early warning module, and a display module. The digital twin module constructs a digital twin map of the mine based on the mine's geometric model and real-time status and maps the positions of robots and sensors. The early warning module outputs early warning information based on the risk probability adjusted by dynamic thresholds. The display module is used to visualize the mine's operating status and risk distribution.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for constructing a robot swarm brain based on artificial intelligence, characterized in that, The method includes: The sensors and mobile robots deployed underground are considered as heterogeneous robots. Each heterogeneous robot is assigned a unique ID, and each heterogeneous robot establishes a communication connection with the ground server through wireless communication. The group brain includes a perception layer, a cognition layer, a decision-making layer, and an execution layer. The perception layer is used to collect the status data of multi-source heterogeneous robots. The cognition layer conducts situational analysis by constructing a digital twin map of the mine and multiple safety risk models. The decision-making layer performs task allocation, conflict detection, and dynamic threshold adjustment. The execution layer issues path and task instructions to achieve group-level collaboration. A geometric model of the mine is constructed at the cognitive level, and a digital twin map of the mine is built based on this geometric model; The sensors deployed underground in the mine are mapped onto the constructed digital twin map to establish a correspondence between the sensors and their physical locations; the mobile robots deployed underground in the mine are also mapped onto the constructed digital twin map, and the position changes of the mobile robots are simultaneously displayed in the digital twin map. When a sensor issues an over-limit alarm while a neighboring sensor of the same type does not, the sensor that issued the alarm is verified using Kalman filtering. If the sensor is found to be abnormal or has a false alarm, the nearest mobile robot to the sensor is called and a path is planned to reach that location for on-site verification. In the cognitive layer, a coupled detection model and prediction model for downhole safety risks are constructed. The prediction model is used to calculate the future predicted values ​​of sensor data related to safety risks, and the detection model combines the current monitoring value of the sensor and the predicted value at the future time to calculate the risk probability at the current time, and at the same time calculate the risk probability at the future time. The decision-making level determines whether to issue an alert based on the risk probabilities at the current and future times output by the detection model, using a dynamic threshold adjustment strategy. This includes: based on future... k Risk probability at time 1 , ..., Assess future risk trends and set trend indicators. Defined as the future k The average risk probability at each moment. Set dynamic alarm thresholds Dynamically adjust based on the current risk probability and trend indicators. : , Basic alarm threshold, This is for adjusting the coefficient.

2. The method according to claim 1, characterized in that, The data verification includes: taking the historical values ​​of the sensor within a certain time window in the perception layer as the initial state, and updating the current time through Kalman filtering. t Posterior state estimation Compare the actual measured values ​​of the sensor. and Calculate the residual If residual Exceeding the preset threshold If the sensor is marked as a suspected anomaly in the digital twin map, a mobile robot will be triggered to verify it.

3. The method according to claim 2, characterized in that, When the decision-making level performs path planning for the mobile robot, it first determines the mobile robot closest to the sensor location based on Euclidean distance. i ; Construct a graph structure in the digital twin map that includes alleyways, dynamic obstacles, and intersections. ; The nodes in the graph structure include the locations of dynamic obstacles and lane intersections; For the graph structure, each edge represents a path connecting nodes. Related attributes include tunnel length and minimum width of the alley , , M The number of sides; With mobile robots i Starting from the current position and using the location of this sensor as the destination point, the robot is guided by the A* algorithm. i The shortest path from the starting point to the destination is calculated as follows: In the formula, From the starting point to the node v The actual cost, This is a heuristic function used to estimate the number of nodes. v The cost of reaching the destination; node v neighboring nodes u The update method is represented as: In the formula, For the edge The cost; In mobile robots i During the process of traveling to the sensor location along the planned path, the calculation of the next node is performed in each control cycle. Direction vector and expected speed : In the formula, To control the cycle, For robots i Maximum moving speed, tracked by a PID controller. The control cycle is for each update of the mobile robot. i The period of the remaining global path.

4. The method according to claim 3, characterized in that, As the mobile robot travels along the planned path, the decision-making layer uses a digital twin map to perform real-time multi-robot conflict detection, including node conflicts and edge conflicts. Node conflicts for robots i , j Expected in the future T Arrive at the same node within a time limit Edge conflict occurs when two robots are on the same edge. Moving towards each other, and expected in the future T Distance within time is less than , , Robots i , j The effective collision radius of the robot; the effective collision radius of the robot , r The physical radius of the robot. To account for safety margins in positioning and control errors; Set up a robot i Arrival at the point of conflict The estimated time is ,robot j Arrival at the point of conflict The estimated time is Then the robot i , j The conditions for a conflict are: in, For safety time margin; When the robot is detected i Other higher priority robots j When a conflict occurs, the robot i Adopt an avoidance strategy: If the robot i , j If there is a node conflict between them, then the robot... i Perform deceleration or stop and wait; If the robot i , j There is an edge conflict between them, and Then the robot i Perform deceleration or stop and wait; if Computational robots i Stay away from robots j Safety offset direction Then modify the robot i The speed, add a pointer The amount of avoidance; If the robot i If a deadlock occurs, the robot will... j Treating them as static obstacles and using the A* algorithm for the robot i Plan temporary paths and generate bypass routes to avoid the robot. j And from the robot i The robot reaches the next node in its global path from its current position via an alternative short path segment; after reaching the next node via this alternative short path segment, the robot switches back to the global path.

5. The method according to claim 1, characterized in that, The cognitive layer includes various safety risk prediction models, including gas accumulation prediction models, fire prediction models, and roof crack prediction models. These models predict the future values ​​and image feature evolution of various risk-related sensors based on historical time series data and image features, enabling a forward-looking assessment of the risk situation at multiple time steps in the future.

6. The method according to claim 5, characterized in that, The cognitive layer also includes various safety risk detection models, including a gas accumulation detection model, a fire detection model, and a roof crack detection model. These models are based on the joint input of real-time sensor data and prediction data, respectively. They extract temporal and image multimodal features through an attention fusion mechanism, calculate the risk probability at the current and future time steps, and achieve joint risk detection in both time domains. Among them, the gas accumulation detection model calculates the risk probability of gas accumulation at the current time based on the current monitoring value of gas accumulation-related sensors and the prediction value at the future time, and also calculates the risk probability of gas accumulation at the future time. The fire detection model calculates the risk probability of a fire at the current moment based on the current monitoring values ​​of fire-related sensors and the predicted values ​​at future moments, and also calculates the risk probability of a fire at future moments. The roof crack detection model calculates the risk probability of roof cracks at the current moment based on the current monitoring values ​​of roof crack-related sensors and the predicted values ​​at future moments, and also calculates the risk probability of roof cracks at future moments.

7. The method according to claim 6, characterized in that, The gas accumulation detection model, fire detection model, and roof crack detection model all use the same network structure, including: The input layer is used to preprocess the sensor's current monitoring values ​​and future predicted values, transforming them into high-dimensional vectors. The time series data branch extracts the time series features of both the current time-series data and the predicted data to obtain the time series features of the current time-series data. and the time series characteristics of the predicted data ; In the image data branch, for both the current time-step data and the prediction data, image features are extracted from the input image data to obtain the image features of the current time-step data. and predicted data image features ; The attention layer performs weighted fusion of temporal features and image features from the current time-series data to obtain the fused features. Simultaneously, the temporal features and image features from the prediction data are weighted and fused to obtain the fused features. ; Dynamic fusion module, for and Perform weighted fusion: , , For dynamic fusion weights; The output layer is activated using the sigmoid function to obtain the current time step. t Risk probability ; In this process, each detection model outputs fused features at the attention layer when calculating the risk probability at future time moments. Then it directly enters the output layer to obtain t + j Risk probability at any moment .

8. The method according to claim 7, characterized in that, The output of the prediction model undergoes uncertainty assessment and confidence calibration before being input into the detection model. The assessment includes: using Kalman filtering to perform posterior correction on the predicted value and obtain the posterior estimated covariance; calculating the prediction confidence based on the posterior estimated covariance; and replacing the predicted value with a safe default value when the confidence is below a preset threshold to reduce the interference of low-confidence predictions on the detection model. For a certain prediction model, the sensor In the future t + j Predicted value , k The predicted time step is obtained through the Kalman filter update process. t + j Posterior estimated covariance at time ,Will As a sensor exist t + j A measure of the uncertainty of the predicted value at any given time, defining the prediction confidence level. , The scaling factor; the sensor exist t + j Predicted value at time With the corresponding confidence level Multiply, and use as the actual value of the detection model corresponding to the prediction model as input. : in, The minimum confidence threshold, This is the default value for safety.

9. A robot swarm brain system based on artificial intelligence, characterized in that, The system includes: The underlying physical layer includes sensors and various types of mobile robots deployed underground in the mine. The sensors include sensors for methane, carbon monoxide, carbon dioxide, oxygen, temperature, wind speed, stress, and image processing. The mobile robots include autonomous inspection robots and task execution robots. Intermediate layer functional modules: forming the core of the group brain, including: The robot swarm management module is used to collect the status information of various heterogeneous robots and encode it uniformly, upload data to the swarm brain and receive task and path instructions; The prediction module includes sub-modules for gas accumulation prediction, fire prediction, and roof crack prediction, which are used to predict the evolution of sensor values ​​and image features in the future at multiple time steps based on historical multimodal time series data. The detection module includes sub-modules for gas accumulation detection, fire detection, and roof crack detection, which calculate the risk probability at multiple time steps based on real-time monitoring values ​​and predicted values. The data verification module verifies single-point over-limit sensor data by combining Kalman filtering with neighborhood sensor information and triggers on-site robot verification. The planning module dynamically plans paths for multiple robots based on a digital twin map and executes node / edge conflict detection, hierarchical avoidance, and deadlock resolution strategies. The dynamic threshold adjustment module adaptively adjusts the warning thresholds for each risk based on risk probability and trend indicators, and based on future... k Risk probability at time 1 , ..., Assess future risk trends and set trend indicators. Defined as the future k The average risk probability at each moment. Set dynamic alarm thresholds Dynamically adjust based on the current risk probability and trend indicators. : , Basic alarm threshold, For adjustment coefficients; The edge computing module provides real-time computing power support for each functional sub-module; The storage module stores multi-source sensor data, robot operating parameters, and model weights. Top-level application layer: includes a digital twin module, an early warning module, and a display module. The digital twin module constructs a digital twin map of the mine based on the mine's geometric model and real-time status and maps the positions of robots and sensors. The early warning module outputs early warning information based on the risk probability adjusted by dynamic thresholds. The display module is used to visualize the mine's operating status and risk distribution.

Images