Self-localization method for visual-inertial-radar fusion based on self-supervised neural network

Abstract

The present invention discloses a visual inertial radar fusion self-localization method based on a self-supervised neural network, which relates to the technical field of artificial intelligence and sensor fusion localization, and includes the following steps: constructing a meta-learner based on a long short-term memory network; in the present invention, dynamic topology adaptation is achieved by constructing an LSTM-based meta-learner, which can respond to environmental changes and complete topology adjustment within about 2 minutes and accurately capture environmental features; by using the knowledge memory module and related algorithms of the attention mechanism, efficient knowledge transfer and inheritance are realized, and new and old knowledge can be accurately fused to ensure tunneling safety; through joint training and optimization, multiple modules cooperate to optimize parameters, and the NSGA-II algorithm is used for multi-objective balance optimization, taking into account performance, efficiency and knowledge utilization under different geological conditions, and improving the overall level of tunneling operations.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and sensor fusion positioning technology, and in particular to a visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network. Background Art

[0002] In the underground coal mine excavation operation scenario, the application of self-supervised neural network technology faces many challenges. Traditional self-supervised neural network architectures are usually trained and run based on fixed network topology structures. However, the underground coal mine excavation environment is extremely complex and dynamic. As the excavation work progresses, the geological conditions are constantly changing, and the hardness, structure and geological structure of the rock vary significantly. For example, during the excavation process, faults, folds, and alternations of soft and hard rocks may suddenly be encountered. These changes make the topological structure of the underground environment in dynamic change, and traditional neural networks are difficult to adapt to such dynamic changes in real time.

[0003] When encountering changes in geological conditions, traditional neural networks cannot automatically adjust their internal connections and processing procedures, and it is difficult to effectively capture new environmental features and positioning clues. In coal mines, this may cause deviations in the excavation direction, affect excavation efficiency, and even cause safety accidents. For example, when encountering a fault, traditional neural networks cannot perceive and adjust the recognition of geological structures in a timely manner, causing excavation equipment to deviate from the predetermined track, resulting in waste of resources and safety hazards. This shows that it is urgent to develop a new self-supervised neural network architecture that can automatically adjust the network topology according to the dynamic changes in the underground coal mine excavation environment. However, existing technologies are difficult to meet this demand.

[0004] When trying to develop a new self-supervised neural network architecture to adapt to the dynamic changes in the underground coal mine excavation environment, the problem of knowledge migration and inheritance between different topological structures during the network topology adjustment process has arisen. Different network topological structures are suitable for different geological conditions and excavation task requirements. When the network is adjusted from one topological structure to another, how to effectively retain and utilize the useful knowledge learned in the previous topological structure to avoid knowledge forgetting, while quickly adapting to the environmental characteristics and task requirements under the new topological structure, has become a very challenging problem.

[0005] During the tunneling process in coal mines, when the network topology is adjusted due to changes in geological conditions, the new topology may not be able to effectively utilize the previously learned knowledge about rock structure, equipment operation status, etc. For example, when tunneling from an area with stable geological conditions to a fault area, the previously learned knowledge of normal rock structure characteristics cannot be effectively utilized by the new topology, resulting in deviations in the identification of rock structures in the fault area, which in turn affects the accurate judgment of the tunneling direction and reduces the safety and efficiency of tunneling. Existing neural network theories and methods are almost blank in dealing with this cross-topology knowledge transfer and inheritance.

[0006] In view of this, a visual inertial radar fusion self-localization method based on a self-supervised neural network is provided, which is specifically aimed at the tunneling environment in coal mines to overcome the above problems. Summary of the Invention

[0007] The purpose of the present invention is to provide a visual inertial radar fusion self-localization method based on a self-supervised neural network to solve the problems raised in the above background technology.

[0008] To solve the above technical problems, the visual inertial radar fusion self-localization method based on a self-supervised neural network provided by the present invention includes the following steps:

[0009] Construct a meta-learner based on a long short-term memory network: The input layer of the meta-learner receives the processed environmental feature vector, which is obtained after the convolutional neural network feature extraction of the underground roadway image data collected by the visual camera, the normalization processing of the millimeter-wave radar data, and the calibration and preprocessing of the inertial device data; the middle hidden layer consists of multiple LSTM units, and the gating mechanism is used to control the inflow, outflow, and memory of information; the output layer is based on the output of the hidden layer, and the network topology structure adjustment parameters for the current tunneling environment state are calculated through the fully connected layer. The parameters include the increase and decrease coefficients of neuron connections, the adjustment factors of the inter-layer connection weights, and the adjustment values of the number of hidden layer neurons;

[0010] When the tunneling environment state in the coal mine changes, the new environmental state information is collected in real time and input into the trained meta-learner; the meta-learner calculates and outputs the topology structure adjustment parameters according to the learned mapping relationship, and the neural network quickly adjusts its own topology structure according to these parameters, including adjusting the connection weights between neurons, and increasing or decreasing the number of hidden layer neurons related to the motion feature processing according to the attitude change of the tunneling equipment and the geological conditions;

[0011] Use the attention mechanism to analyze the feature maps of the middle layer of the network, extract the important knowledge feature vectors related to the rock structure in the coal mine and the motion state of the tunneling equipment, calculate the attention weights of each knowledge feature vector, and store them in the knowledge memory module in the form of a weighted sum;

[0012] After the network topology structure is adjusted, the new topology structure generates a query vector according to the current tunneling task requirements and the environmental features under the current geological conditions; calculate the similarity between the query vector and the knowledge feature vectors stored in the memory module, select the knowledge feature vectors with high similarity, and fuse them with the knowledge feature vectors learned under the new topology structure in a weighted fusion manner. The fusion coefficient is dynamically adjusted according to the current tunneling environment and task requirements;

[0013] Jointly train the meta-learner, the neural network body, and the knowledge memory module as a whole; input the environmental data of different driving areas and different geological conditions in the coal mine into the meta-learner, the meta-learner generates topology adjustment parameters, the neural network calculates the task loss through forward propagation according to the adjusted topology, and at the same time calculates the storage and retrieval losses of the knowledge memory module, and updates the parameters of each module through the total loss;

[0014] Set the objective function, including the driving direction positioning accuracy of the network under different geological conditions, the time overhead of topology adjustment, and the contribution to the improvement of driving performance after knowledge fusion; use the multi-objective optimization method based on the non-dominated sorting genetic algorithm to search and optimize the network parameters and structure.

[0015] Furthermore, the convolutional neural network uses pre-trained convolutional kernels, including but not limited to VGG16, ResNet series, to convert the image data of the underground roadway collected by the visual camera into a feature vector with semantic information.

[0016] Furthermore, the normalization process of the millimeter wave radar data is to map numerical values such as distance and speed to the interval [0,1].

[0017] Furthermore, the state update formula of the LSTM hidden layer is:

[0018] ,

[0019] ,

[0020] ,

[0021] ,

[0022] ,

[0023] ,

[0024] Among them, is the input gate vector, is the forget gate vector, is the output gate vector, is the candidate memory unit, is the memory unit, is the hidden layer output vector. 、 、 、 are the weight matrices from the input layer to the corresponding gates and memory units, 、 、 、 is the weight matrix from the hidden layer to the corresponding gates and memory units, , , , are the corresponding bias vectors, is the Sigmoid activation function, is the hyperbolic tangent activation function, represents element-wise multiplication, represents the state of the memory unit at the previous time step.

[0025] Furthermore, the formula for calculating the topological structure adjustment parameters of the output obtained by the output layer is:

[0026] ,

[0027] where, is the weight matrix from the hidden layer to the output layer, is the bias vector of the output layer.

[0028] Furthermore, the formula for calculating the attention weights of each knowledge feature vector in the attention mechanism is:

[0029] ,

[0030] where, is the query vector, used to measure the importance of each knowledge feature vector. By performing dot product operations with the knowledge feature vectors and normalizing through the Softmax function, the attention weights of each feature vector are obtained.

[0031] Furthermore, the cosine similarity formula is used to calculate the similarity between the query vector and the knowledge feature vectors stored in the memory module:

[0032] .

[0033] Furthermore, knowledge fusion adopts a weighted fusion method, and the formula is:

[0034] ,

[0035] where, is the knowledge feature vector learned under the new topological structure, is the set of selected relevant knowledge feature vectors, is the fusion coefficient, which is dynamically adjusted according to the current environment and task requirements.

[0036] Furthermore, for the target recognition task, the cross-entropy loss function is used to calculate the task loss:

[0037] ,

[0038] Among them, is the number of samples, is the number of categories, is the sample belonging to the category true label, is the probability that the model predicts the sample belongs to the category.

[0039] Furthermore, the mean squared error loss is used to calculate the storage and retrieval losses of the knowledge memory module:

[0040] ,

[0041] Among them, is the number of samples when calculating the loss, is the knowledge feature vector, is the knowledge feature vector required by the actual demand.

[0042] Furthermore, the total loss calculation formula is:

[0043] ,

[0044] Among them is the weight coefficient, used to balance the task loss and the memory loss.

[0045] Furthermore, the objective function is expressed as:

[0046] ,

[0047] Among them, , , are weight coefficients.

[0048] Furthermore, in the multi-objective optimization method, in each generation of evolution, non-dominated sorting is performed on the individuals in the population, and the individuals are divided into different ranks. Individuals with higher ranks are preferentially selected for genetic operations, including but not limited to crossover and mutation, to generate a new population, and crowding degree calculation is used to maintain the diversity of the population.

[0049] Furthermore, the environmental data includes environmental data in different tunneling areas and different geological conditions, and corresponding task labels are marked, including but not limited to rock type, tunneling direction deviation, equipment failure type, and gas concentration level.

[0050] Furthermore, the adjustment of the neural network topology structure includes adding or deleting neuron connections, enhancing or weakening the connection strength between layers, and dynamically increasing or decreasing the number of hidden layer neurons.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] The dynamic topology adaptation effect is remarkable

[0053] Quickly respond to environmental changes: Traditional self-supervised neural networks are based on a fixed topology structure and it is difficult to adjust in real time when facing complex and changeable geological conditions in coal mines. For example, when encountering sudden changes in rock hardness, faults, etc., it may take a long time for traditional networks to adapt to the new environment, seriously affecting the tunneling progress. However, the meta-learner based on LSTM constructed in the present invention can receive the information of the underground tunneling environment state in real time and use the gating mechanism to learn the dynamic change law. When the environment changes, the topology structure can be adjusted within about 2 minutes, greatly shortening the adaptation time, quickly responding to geological environment changes, and ensuring the continuity of tunneling work.

[0054] Precisely capture environmental features: Traditional networks cannot automatically adjust internal connections and processing processes to capture new environmental features and positioning clues. The topology structure adjustment parameters output by the meta-learner of the present invention can enable the neural network to increase neuron connections when encountering different rock structures, such as hard rocks, enhancing the ability to capture rock structure features; when encountering abnormal inertial navigation data due to geological structure changes, changing the inter-layer connection weights, making the neural network pay more attention to effective information and ignore interference information, effectively improving the adaptability in complex dynamic tunneling environments and the positioning accuracy of the tunneling direction.

[0055] The knowledge transfer and inheritance ability is outstanding

[0056] Efficiently store key knowledge: Existing neural network theories and methods are almost blank in cross-topology structure knowledge transfer and inheritance. The knowledge memory module using the attention mechanism in the present invention can intelligently screen and store key knowledge under different topology structures in the form of weighted sums, such as the texture characteristics of different rocks, the relationship between hardness and tunneling speed, and the energy consumption law of equipment under different geological conditions, improving the efficiency of knowledge storage and management and laying a foundation for knowledge transfer and inheritance.

[0057] Effectively integrate new and old knowledge: After the network topology structure is adjusted, through cosine similarity calculation and dynamic weighted fusion algorithm, the new topology structure can accurately retrieve relevant knowledge and dynamically weight and fuse it with new knowledge. In coal mines, when tunneling encounters high-hardness rocks or gas layers, the new topology structure can use the previously stored knowledge, combine with new knowledge under the current geological conditions, accurately judge the rock properties, equipment operation status, etc., adjust the tunneling strategy, avoid safety accidents caused by misjudgment, and enhance the safety and reliability of tunneling operations in coal mines.

[0058] Joint training and optimization to improve comprehensive performance

[0059] Multi-module collaborative optimization: The meta-learner, the main body of the neural network, and the knowledge memory module are jointly trained, and the parameters of each module are collaboratively optimized through the backpropagation of the total loss. In traditional methods, each module is independent and cannot achieve collaboration. In the present invention, the task loss backpropagation optimizes the task execution ability of the neural network, and the memory module loss backpropagation optimizes the effectiveness of the knowledge memory module in storing and retrieving knowledge, comprehensively improving the network performance, enabling the network to better serve the tunneling operation under different geological conditions.

[0060] Multi-objective balanced optimization: The multi-objective optimization method based on NSGA-II is adopted, and the objective function including the tunneling direction positioning accuracy, the time cost of topological structure adjustment, and the contribution to the improvement of tunneling performance after knowledge fusion is set. Traditional methods mostly focus on a single objective. In the present invention, by continuously iteratively searching for the optimal network parameters and structure, the best balance among performance, efficiency, and knowledge utilization is achieved under different geological conditions. In coal mine underground tunneling, while ensuring a high tunneling direction positioning accuracy, it can quickly adapt to geological changes for topological structure adjustment and fully utilize knowledge transfer and inheritance to improve tunneling efficiency. Brief Description of the Drawings

[0061] Figure 1 It is the schematic diagram of the visual inertial radar fusion self-positioning method based on the self-supervised neural network of the present invention. Detailed Embodiments

[0062] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0063] Please refer to Figure 1 , the present invention provides a technical solution:

[0064] Refer to Figure 1 shown, an embodiment of the visual inertial radar fusion self-positioning method based on the self-supervised neural network:

[0065] (I) Topological structure adjustment strategy based on meta-learning

[0066] 1. Construction of the meta-learner

[0067] In the coal mine underground tunneling operation scenario, a visual camera, an inertial navigation device, and a millimeter-wave radar are equipped on the tunneling equipment. The visual camera is responsible for collecting the image data of the underground roadway, the inertial navigation device obtains key information such as the attitude and acceleration of the tunneling equipment, and the millimeter-wave radar is used to monitor data such as the distance and speed of the objects in front of the tunneling. These data are transmitted to the meta-learner in real time.

[0068] Use a Convolutional Neural Network (CNN) to extract features from visual image data. With the help of pre-trained convolutional kernels, such as VGG16 or ResNet series, the image is transformed into a feature vector with semantic information. At the same time, normalize the millimeter-wave radar data, map numerical values such as distance and speed to the [0, 1] interval, and eliminate the influence of dimension. In addition, calibrate and preprocess the data of attitude, acceleration, etc. collected by inertial navigation equipment to ensure that it is on the same scale as other data for subsequent processing.

[0069] Adopt a meta-learner structure based on Long Short-Term Memory Network (LSTM). As a Recurrent Neural Network (RNN), LSTM can effectively process time series data and solve the problem of long-term dependence. Its input layer receives the preprocessed environmental feature vector. The middle hidden layer consists of multiple LSTM units. Through the gating mechanism (input gate, forget gate, and output gate), it controls the inflow, outflow, and memory of information, so as to learn the dynamic change law of the underground tunneling environment state. The output layer, based on the output result of the hidden layer, calculates and generates the network topology adjustment parameters for the current tunneling environment state through the fully connected layer. These parameters include the increase and decrease coefficients of neuron connections, which are used to determine the establishment of new connections or the deletion of old connections; the adjustment factors of inter-layer connection weights, which can perform multiplication operations on the existing connection weights to enhance or weaken the connection strength; and the adjustment values of the number of hidden layer neurons, which dynamically increase or decrease the number of neurons according to the complexity of the tunneling environment.

[0070] LSTM has powerful time series modeling capabilities and can capture the change trends and dependency relationships of the environmental state over time. By training on a large number of samples with different tunneling environment scenarios and topological structure changes, the meta-learner can learn the complex mapping relationship between the tunneling environment state and the appropriate topological structure, and provide accurate topological adjustment parameters for the neural network.

[0071] Let the input environmental state feature vector be, and the state update formula of the LSTM hidden layer at time is:

[0072] ,

[0073] ,

[0074] ,

[0075] ,

[0076] ,

[0077] ,

[0078] Among them, is the input gate vector, is the forget gate vector, is the output gate vector, is the candidate memory cell, is the memory cell, is the hidden layer output vector. , , , are the weight matrices from the input layer to the corresponding gates and memory cells, , , , are the weight matrices from the hidden layer to the corresponding gates and memory cells, , , , are the corresponding bias vectors, is the Sigmoid activation function, is the hyperbolic tangent activation function, represents element-wise multiplication, represents the memory cell state at the previous time step.

[0079] The topological structure adjustment parameters of the output are calculated by the output layer:

[0080] ,

[0081] where, is the weight matrix from the hidden layer to the output layer, is the bias vector of the output layer.

[0082] The meta-learner deeply learns the internal relationship between the dynamic changes of the tunneling environment and the topological structure through the above mechanism, and provides accurate and highly targeted topological structure adjustment parameters for the neural network. This enables the neural network to quickly adjust its own topological structure in the face of complex and changing tunneling environments underground in coal mines, such as changes in rock structures and geological structure differences, etc., effectively capture new environmental features and positioning clues, and significantly improve the adaptability and real-time response ability of the network to the tunneling environment.

[0083] It should be noted that:

[0084] Advantages of the meta-learner structure: The meta-learner adopts an LSTM-based structure, which can effectively process time series data and solve the problem of long-term dependencies. In the tunneling scenario underground in coal mines, the tunneling environment changes dynamically over time. For example, there are differences in the hardness and structure of different rock strata, as well as geological changes such as faults and folds encountered during tunneling. LSTM precisely controls the inflow, outflow, and memory of information through its gating mechanism, thereby learning the dynamic change law of the tunneling environment state over time, providing strong technical support for in-depth learning of the internal relationship between environmental dynamic changes and topological structures, enabling the meta-learner to capture the subtle features of environmental changes and the correlation between these changes and topological structure adjustments.

[0085] Effectiveness of the training process: The meta-learner is trained on a large number of samples with different tunneling environment scenarios and topological structure changes. Through learning from massive data, the meta-learner can fully understand the characteristics of various tunneling environment states and the corresponding appropriate topological structures. Tunneling underground in coal mines covers different geological conditions, such as soft rock, hard rock, fault areas, and other scenarios. By learning from these rich samples, the meta-learner summarizes general rules, thereby providing accurate and highly targeted topological structure adjustment parameters for the neural network. When facing specific dynamic environmental changes, based on the knowledge learned previously, the meta-learner can quickly generate adjustment parameters suitable for the current environment.

[0086] Role in the neural network: The topological structure adjustment parameters generated by the meta-learner directly act on the neural network. When facing different rock structures, such as encountering hard rock, the parameters generated by the meta-learner can make the neural network increase neuron connections to enhance the ability to capture rock structure features; when encountering abnormal inertial navigation data due to geological structure changes, the adjustment parameters can change the inter-layer connection weights, making the neural network pay more attention to effective information and ignore interference information, thereby quickly adjusting its own topological structure. Through these methods, the neural network can effectively capture new environmental features and positioning clues, significantly improving its adaptability and real-time response ability in complex dynamic tunneling environments.

[0087] 2. Fast adaptation mechanism

[0088] During the tunneling process underground in coal mines, when encountering geological condition changes, such as sudden changes in rock hardness, the occurrence of faults, the presence of groundwater, etc., new environmental state information, including changes in rock image features, abnormal equipment postures, changes in the state of objects detected by millimeter-wave radar in front, etc., is collected in real time and input into the pre-trained meta-learner.

[0089] Based on the learned mapping relationship between environmental states and topological structure adjustment parameters, the meta-learner quickly processes the input new environmental information, calculates, and outputs the corresponding topological structure adjustment parameters.

[0090] Based on the adjustment parameters output by the meta-learner, the neural network quickly adjusts its own topological structure. For example, by adjusting the connection weights between neurons, it enhances the ability to extract image features of rock structure changes; according to the attitude changes of the tunneling equipment and geological conditions, it increases or decreases the number of hidden layer neurons related to motion feature processing, optimizes the perception processing flow of tunneling direction and environmental changes, and ensures that the tunneling equipment can promptly adapt to new geological conditions and maintain the correct tunneling direction.

[0091] The training of the meta-learner on a large number of tunneling environment samples enables it to establish a stable and accurate mapping relationship. Therefore, when facing new environmental changes, it can quickly respond based on existing learning experience and generate effective adjustment parameters.

[0092] To intuitively demonstrate the advantages of the fast adaptation mechanism based on meta-learning, comprehensive actual tests were carried out in different tunneling roadways of multiple coal mines, comparing the traditional neural network with a fixed topological structure and the neural network with a dynamic topological structure based on meta-learning. The results are as follows in the table:

[0093] Comparison Items Traditional Fixed Topology Neural Network Meta-Learning Based Dynamic Topology Neural Network Time to Adapt to New Geological Environment Approximately 30 minutes Approximately 2 minutes Accuracy of Boring Direction Positioning under Complex Geological Conditions 60% 85% Number of Boring Direction Errors Caused by Geological Changes (per hour) 15 times 3 times Number of Target Detections when Encountering Faults (per hour) 50 80 False Judgment Rate when Facing Complex Geological Structures (such as folds) 20% 5% Comparison Items Traditional Fixed Topology Neural Network Meta-Learning Based Dynamic Topology Neural Network

[0094] From the data, it can be seen that the traditional neural network with a fixed topological structure takes about 30 minutes to adapt to the new geological environment, while the neural network with a dynamic topological structure based on meta-learning only takes about 2 minutes, with a significant reduction in time and the ability to quickly respond to geological environment changes. The positioning accuracy of the tunneling direction under complex geological conditions has been improved from 60% to 85%, enabling a more accurate determination of the tunneling direction. The number of tunneling direction errors caused by geological changes has been reduced from 15 times per hour to 3 times, effectively avoiding tunneling direction errors. When encountering a fault, the number of target detections has increased from 50 per hour to 80; in the face of complex geological structures such as folds, the misjudgment rate has been reduced from 20% to 5%.

[0095] Compared with the traditional neural network with a fixed topological structure, the adjustment method based on meta-learning in the present invention does not require large-scale retraining. In a scenario with rapidly changing environments such as underground coal mines, it can complete the topological structure adjustment in a short time, ensure that the network continuously and accurately captures environmental information, significantly improve the adaptability and response speed of the network, and provide a solid guarantee for the safe and efficient operation of underground coal mine tunneling.

[0096] It should be further supplemented and explained that:

[0097] In the field of complex environment monitoring and positioning, the adaptability and accuracy of technologies are directly related to the efficiency and safety of operations. At present, the combined application of traditional complex environment monitoring and positioning technologies mostly adopts fixed network structures or simple adaptive adjustment methods, which have obvious limitations when dealing with complex and changing environments.

[0098] Taking the tunneling scenario of a certain coal mine as an example, the geological conditions of this coal mine are extremely complex, with multiple ore layers having huge differences in hardness and a large number of complex geological structures distributed, such as frequently occurring faults, folds, etc. In such an environment, traditional technologies encounter many difficulties in practical applications.

[0099] In traditional technologies, it is common to adjust network parameters based on a fixed threshold judgment method. This approach is too mechanical and only operates according to a preset fixed threshold, unable to deeply understand the internal relationship between environmental changes and network structure. Once the geological conditions change slightly, this fixed threshold adjustment method is difficult to accurately adapt, and it is even more powerless in the face of complex situations such as sudden changes in ore hardness and frequent changes in geological structures in the coal mine. This directly leads to a low positioning accuracy rate and a significant reduction in tunneling efficiency.

[0100] In the face of areas with large changes in ore hardness, the defects of traditional technologies are particularly prominent. Due to the inability to quickly adapt to hardness changes, it is difficult to accurately complete the attitude adjustment of tunneling equipment. The inaccuracy of the tunneling equipment attitude seriously affects the positioning accuracy, resulting in frequent deviations in the tunneling direction. To accurately obtain the actual performance data of traditional technologies in the tunneling of this coal mine, 5 tunneling roadways using traditional technologies in this coal mine were continuously monitored within one month. The results showed that the average number of tunneling direction errors caused by positioning deviations reached 12 times per hour. Through the precise measurement and analysis of the tunneling trajectory, it was found that the tunneling path of traditional technologies in this area increased by an average of 35% compared to the precise tunneling path. This not only significantly increased the time cost of tunneling. After statistics, in the case of the same tunneling task volume, traditional technologies consumed nearly 40% more time than the theoretically efficient tunneling time.

[0101] In contrast, this method shows significant advantages. This method introduces a meta-learner based on the long short-term memory network (LSTM). With its unique gating mechanism, it can effectively process time series data and deeply learn the dynamic change laws of the coal mine tunneling environment state. Once the geological conditions change, the meta-learner can quickly respond and rapidly adjust the topological structure of the neural network.

[0102] Meanwhile, combined with the attention mechanism, this method can accurately screen and store knowledge under different geological conditions. When the network faces new geological conditions, it can quickly retrieve and utilize relevant knowledge, avoiding the waste and forgetting of knowledge. At the tunneling site of this coal mine, 5 tunneling roadways with geological conditions similar to those during the monitoring of traditional technologies were selected, and this method was applied for tunneling operations, and strict monitoring was carried out for half a year. During the monitoring process, high-precision positioning equipment was used to record the position information of the tunneling equipment in real time. By comparing with the preset standard tunneling path, the positioning accuracy rate was calculated; a professional efficiency monitoring software was used to statistically analyze the tunneling efficiency in combination with the equipment operation data and tunneling progress; the attitude adjustment time of the tunneling equipment when facing the ore hardness change area was recorded through sensors. The specific data is as follows:

[0103] Comparison Items Traditional Technology This Method Basis for Data Acquisition Positioning Accuracy 60% (Based on the statistics of multiple positioning data per hour within one month for 5 boring roadways. If the positioning error exceeds the set range, it is determined as a positioning error, and the proportion of the number of errors in the total number of positionings is calculated.) Increased to 90% (The increase amplitude reaches 30%. During the six-month monitoring period for 5 boring roadways applying this method, with the same positioning error determination standard, the proportion of the number of positioning errors in the total number of positionings is statistically calculated.) Use high-precision positioning equipment to record the position information of the boring equipment in real time, compare it with the preset standard boring path, and calculate the positioning error rate Boring Efficiency Bore 5 meters per hour (Calculated based on the ratio of the total boring length to the total working time of 5 boring roadways within one month.) Increase by 25% to reach 6.25 meters per hour (During the six-month monitoring period, the ratio of the total boring length to the total working time of 5 boring roadways applying this method is statistically calculated.) Use professional efficiency monitoring software to combine equipment operation data and boring progress to statistically calculate the boring efficiency Time for Boring Equipment Attitude Adjustment when Facing Areas with Ore Hardness Changes The average adjustment time exceeds 10 minutes (During the one-month monitoring period, record the equipment attitude adjustment process when encountering areas with ore hardness changes in 5 boring roadways, statistically calculate the time of each adjustment, and calculate the average value.) Can complete rapid adjustment within 2 minutes (During the six-month monitoring period, for 5 boring roadways applying this method, record the time of equipment attitude adjustment when encountering areas with ore hardness changes and calculate the average value.) Record the attitude adjustment time through sensors installed on the boring equipment Number of Boring Direction Errors Caused by Positioning Deviation (per hour) 12 times (During the one-month monitoring period, statistically calculate the boring direction deviation situation per hour for 5 boring roadways, record the number of direction errors, and calculate the average value per hour.) Reduce to 3 times (Reduce by 75%. During the six-month monitoring period, statistically calculate the boring direction deviation situation per hour for 5 boring roadways applying this method, and calculate the average number of errors per hour.) Statistically calculate the number of boring direction errors by observing the deviation of the boring trajectory from the preset direction Increase Proportion of Boring Path Increase by 35% compared to the precise boring path (Based on the comparison and measurement of the actual boring trajectory and the preset precise boring path of 5 boring roadways within one month, calculate the proportion of the extra path length to the precise path length.) Only increase by 5% (During the six-month monitoring period, compare and measure the actual boring trajectory and the preset precise boring path of 5 boring roadways applying this method, and calculate the proportion of the extra path length to the precise path length.) Measure the actual tunneling trajectory and the preset precise tunneling path using high-precision measurement equipment, and calculate the increased proportion of the path by comparison Proportion of time cost reduction - Reduced by about 30% (calculate the proportion of the time difference to the total time of the traditional technology by comparing the total time required for the traditional technology and this method to complete the task under the same tunneling workload) Statistically calculate the total working time of the traditional technology and this method under the same tunneling workload, and calculate the proportion of time cost reduction

[0104] It can be clearly seen from the above detailed and well-founded data comparison that the performance of this method under complex geological conditions is very excellent. Whether it is the substantial improvement in the positioning accuracy rate, the significant increase in the tunneling efficiency, or the rapid response ability in the face of complex geological changes, it strongly proves that this method can effectively solve the problems of traditional technologies in complex environment monitoring and positioning, providing a more efficient, accurate and reliable solution for this field.

[0105] (2) Knowledge transfer and inheritance based on the memory mechanism

[0106] 1. Knowledge memory module

[0107] During the tunneling process in the coal mine underground, when the tunneling equipment is operating normally, the network topology focuses on learning knowledge such as the rock structure characteristics and the operating state of the tunneling equipment during the roadway tunneling process, such as the texture characteristics of different rocks, the relationship between hardness and tunneling speed, and the energy consumption law of the equipment under different geological conditions. These knowledge needs to be stored efficiently so as to continue to play a role after the topology is adjusted.

[0108] When the network is learning under the normal tunneling topology, the attention mechanism is used to analyze the feature map of the middle layer of the network, and important knowledge feature vectors related to the rock structure and the motion state of the tunneling equipment are extracted.

[0109] Attention calculation: The attention mechanism assigns an attention weight to each knowledge feature vector by calculating its importance . The specific calculation method is as follows:

[0110] ,

[0111] where, is the query vector, which is used to measure the importance of each knowledge feature vector. It performs a dot product operation with the knowledge feature vector to evaluate the importance of the knowledge feature vector for the current task or environment. In the coal mine roadway driving scenario, the query vector is generated according to the current driving task requirements and the environmental characteristics under the current geological conditions, and it represents the current network's requirements and focus on knowledge. For example, when encountering geological conditions of high-hardness rocks, the query vector will contain information related to the characteristics of high-hardness rocks, so as to filter out the knowledge feature vectors for judging rock properties and adjusting the driving strategy.

[0112] and are both knowledge feature vectors. represents the knowledge feature vector for which the attention weight is currently being calculated. represents each vector in the set of all knowledge feature vectors stored in the memory module. These knowledge feature vectors are obtained by analyzing and extracting the feature maps of the middle layer of the network and are related to the rock structure underground in the coal mine and the motion state of the driving equipment. For example, the texture features of different rocks, the relationship between hardness and driving speed, and the energy consumption law of the equipment under different geological conditions will all exist in the form of knowledge feature vectors in the set. When calculating the attention weight, and perform a dot product operation, and the in the denominator is the sum of the exponentiations of the dot product operation results of all knowledge feature vectors and the query vector . Its function is to normalize the result of the numerator so that the sum of the attention weights of all knowledge feature vectors is 1, so that the importance weights of different knowledge feature vectors can be reasonably distributed.

[0113] is the attention weight of each knowledge feature vector obtained through the above operations. This weight reflects the relative importance of the corresponding knowledge feature vector in the current task and environment, which helps the network to efficiently utilize key knowledge. For example, when facing the current geological conditions, the knowledge feature vectors closely related to the geological conditions will obtain high attention weights, thus playing a more important role in the decision-making process and helping the network to more accurately judge and respond to complex situations.

[0114] Knowledge storage: Knowledge is stored in the memory module in the form of a weighted sum, that is:

[0115] ,

[0116] where is the number of knowledge feature vectors. This storage method can highlight important knowledge, effectively retain the key information of knowledge, and facilitate subsequent retrieval and use.

[0117] It should be noted here that the attention mechanism can focus on important knowledge content, enabling the memory module to store more precisely the knowledge that plays a key role in improving network performance. Through weighted storage, the most valuable knowledge information can be retained within limited storage space, improving the efficiency of knowledge storage and management.

[0118] It should be supplemented that through the above mechanism, the knowledge memory module can effectively store the important knowledge learned under the normal tunneling topology structure, laying a solid foundation for the knowledge transfer and inheritance after the subsequent topology structure adjustment. When the network topology structure changes, these stored knowledge can be quickly retrieved and utilized, avoiding the loss of knowledge, ensuring that the network can make full use of the existing learning achievements under different geological conditions, and improving the overall performance.

[0119] 2. Knowledge Retrieval and Fusion Mechanism

[0120] When tunneling encounters high-hardness rocks, gas layers, etc., the network topology structure is adjusted to a structure adapted to the new geological conditions. At this time, it is necessary to retrieve the relevant knowledge learned before from the knowledge memory module and fuse it with the new knowledge learned under the new geological conditions to improve the judgment ability of various situations during tunneling and ensure tunneling safety and efficiency.

[0121] The new topology structure generates a query vector according to the current tunneling task requirements and environmental characteristics under the current geological conditions, such as abnormal rock structures, sudden changes in equipment posture, changes in gas concentration, etc.

[0122] Calculate the similarity between the query vector and the knowledge feature vectors stored in the memory module, using the cosine similarity formula:

[0123] ,

[0124] where is the newly generated query vector (generated by the new network topology structure according to the current environmental characteristics and task requirements), is the knowledge feature vector stored in the memory module (representing the key knowledge learned historically), the dot product measures the consistency of the two vectors in direction, and the larger the value, the closer the direction; the product of the vector norms normalizes the vector length to eliminate the influence of dimensions;

[0125] Through cosine similarity calculation, the similarity between the query vector and each knowledge feature vector can be measured, so as to screen out the knowledge related to the current geological conditions and tunneling tasks.

[0126] Knowledge selection: According to the similarity calculation results, select the knowledge feature vectors with high similarity as the relevant knowledge.

[0127] Knowledge fusion: Design a knowledge fusion algorithm and adopt the weighted fusion method:

[0128] ,

[0129] where is the knowledge feature vector learned under the new topological structure, is the set of selected relevant knowledge feature vectors, is the fusion coefficient, which is dynamically adjusted according to the current environment and task requirements. By dynamically adjusting the fusion coefficient, the fusion ratio of new knowledge and old knowledge can be flexibly balanced, ensuring that the fused knowledge can not only adapt to the new environment but also make full use of the existing knowledge and experience.

[0130] It should be noted here that: Through similarity calculation, the knowledge related to the current geological conditions and tunneling tasks can be accurately retrieved from the memory module. The weighted fusion method can perform reasonable fusion according to the relevance and importance of knowledge, and the dynamic adjustment of the fusion coefficient further enhances the flexibility and adaptability of the fusion, effectively avoiding knowledge forgetting, enabling the new topological structure to quickly adapt to the tunneling environment characteristics and task requirements under the current geological conditions.

[0131] It should be further noted that: After the topological structure is adjusted, the new topological structure can make full use of the existing knowledge in the memory module with the help of the knowledge retrieval and fusion mechanism, and organically fuse it with the new knowledge learned under the new geological conditions. This significantly improves the network's recognition ability of the current geological conditions and effectively reduces the misjudgment and missed judgment of abnormal situations during tunneling. For example, when encountering high-hardness rocks, it can use the previously learned knowledge of rock properties, combined with the current rock images and radar data, to accurately judge the rock properties and adjust the tunneling strategy; when monitoring the change of gas concentration, it can avoid safety accidents caused by misjudgment, thus greatly improving the safety and reliability of underground coal mine tunneling operations and ensuring the normal progress of tunneling operations.

[0132] It needs to be further supplemented and explained that:

[0133] In the field of complex environmental monitoring and positioning, the knowledge transfer and inheritance ability of technology play a crucial role in its performance. However, most traditional technologies have obvious shortcomings in this regard, lacking effective knowledge storage and retrieval mechanisms, and it is difficult to make full use of existing knowledge and experience under different environmental conditions. Taking traditional positioning technology as an example, once the environment changes, the knowledge learned in the past is difficult to reuse, and each time a new situation is faced, it is necessary to learn again, which greatly reduces work efficiency. To more intuitively and powerfully compare the differences between this method and traditional technologies in terms of knowledge transfer and inheritance, the following takes a coal mine tunneling scenario as an example and presents specific data through a table.

[0134] The geological conditions of a certain coal mine are extremely complex, and the ore properties and geological structures in different regions vary greatly. In such an environment, the problems of traditional technologies in knowledge utilization are fully exposed. Due to the inability to accurately distinguish and utilize knowledge under different geological conditions, when encountering a new ore type, traditional technologies are difficult to quickly judge its characteristics, which in turn leads to unreasonable parameter settings of tunneling equipment, seriously affecting the tunneling efficiency and quality.

[0135] To deeply understand the actual performance of traditional technologies in this coal mine, a three-month monitoring was carried out. During this period, 5 tunneling roadways using traditional technologies were selected, and the situations when they encountered new ore types were recorded in detail. Through the monitoring and analysis of the parameters of tunneling equipment, it was found that due to the inability to accurately judge the ore characteristics, the matching degree between the parameters of tunneling equipment and the actual requirements was only 40%. This resulted in frequent equipment jams and unstable tunneling speeds during the tunneling process, leading to an average tunneling efficiency of only 4 meters per hour. At the same time, by statistically analyzing the positioning data, it was found that the positioning accuracy was only 70%. A large number of positioning deviations made it difficult to accurately control the tunneling direction, not only increasing the additional tunneling workload but also posing safety hazards.

[0136] However, this method shows significant advantages in terms of knowledge transfer and inheritance due to its unique design. Through a knowledge memory module based on the attention mechanism, this method can intelligently screen and store the key knowledge learned under different topological structures. When facing new geological conditions, it uses cosine similarity calculation and dynamic weighted fusion algorithms for knowledge retrieval and fusion, enabling the new topological structure to quickly adapt to the tunneling environmental characteristics and task requirements under the current geological conditions.

[0137] In this coal mine, 5 driving headings with geological conditions similar to those in the traditional technology monitoring were also selected, and this method was applied for driving operations, and strict monitoring was carried out for half a year. During the monitoring process, high-precision ore property detection equipment was used to analyze the ore properties in real time. By comparing with the standard ore property database, the judgment accuracy rate of the properties of new ore types was calculated; a professional efficiency monitoring software was used to combine the equipment operation data and the driving progress to count the driving efficiency; high-precision positioning equipment was used to record the position information of the driving equipment in real time. By comparing with the preset standard driving path, the positioning accuracy rate was calculated. The specific data is as follows:

[0138] Comparison items Traditional technology This method Basis for data acquisition Accuracy rate of judging the characteristics of new ore types 40% (obtained based on the proportion of the number of accurate judgments of the characteristics of new ore types to the total number of judgments when 5 tunneling headings using traditional technology encountered new ore types within three months) Increased to 80% (the increase amplitude is 40%. During the six-month monitoring period, when 5 tunneling headings applying this method encountered new ore types, statistically calculate the proportion of the number of accurate judgments of their characteristics to the total number of judgments) Use high-precision ore characteristic detection equipment to analyze ore characteristics in real time, compare with the standard ore characteristic database, and statistically calculate the proportion of accurate judgments Tunneling efficiency Tunnel 4 meters per hour (calculated based on the ratio of the total tunneling length to the total working time of 5 tunneling headings using traditional technology within three months) Increase by 18% to reach 4.72 meters per hour (during the six-month monitoring period, statistically calculate the ratio of the total tunneling length to the total working time of 5 tunneling headings applying this method) Use professional efficiency monitoring software, combined with equipment operation data and tunneling progress, to statistically calculate tunneling efficiency Positioning accuracy rate 70% (obtained based on the statistics of multiple positioning data per hour of 5 tunneling headings using traditional technology within three months. If the positioning error exceeds the set range, it is determined as a positioning error, and calculate the proportion of the number of errors to the total number of positionings) Increase to 90% (the increase amplitude is 20%. During the six-month monitoring period, for 5 tunneling headings applying this method, use the same positioning error judgment standard, and statistically calculate the proportion of the number of positioning errors to the total number of positionings) Use high-precision positioning equipment to record the position information of tunneling equipment in real time, compare with the preset standard tunneling path, and calculate the positioning error rate Number of misjudgments and missed judgments caused by insufficient knowledge utilization (per week) 10 times (during the three-month monitoring period, statistically record the misjudgment and missed judgment situations of 5 tunneling headings using traditional technology per week, record the number of misjudgments and missed judgments, and calculate the weekly average value) Reduce to 3 times (during the six-month monitoring period, statistically calculate the weekly average number of misjudgments and missed judgments of 5 tunneling headings applying this method) By observing the misjudgments of ore characteristics, positioning deviations, etc. during the tunneling process, statistically calculate the number of misjudgments and missed judgments

[0139] As can be clearly seen from the above table, this method performs excellently in knowledge transfer and inheritance under complex geological conditions. Whether it is the significant improvement in the judgment accuracy rate of the properties of new ore types, the remarkable increase in driving efficiency, or the growth of positioning accuracy rate, it strongly proves that this method can effectively solve the problem of insufficient knowledge utilization of traditional technologies in complex environment monitoring and positioning, and provides more efficient, accurate, safe and reliable technical support for coal mine driving operations.

[0140] (III) Joint Training and Optimization

[0141] 1. Overall Training Framework

[0142] In the training stage of the coal mine underground driving scenario, a large amount of environmental data under different driving areas and different geological conditions is collected, and corresponding task labels are marked, such as rock type, driving direction deviation, equipment failure type, gas concentration level, etc.

[0143] Specific steps:

[0144] Module integration: The meta-learner, the main body of the neural network, and the knowledge memory module are jointly trained as a closely collaborating whole.

[0145] Data input and parameter generation: Input the environmental data into the meta-learner, and the meta-learner generates topology adjustment parameters according to the environmental information.

[0146] Forward propagation and task loss calculation: The neural network performs forward propagation according to the adjusted topology, processes the input data, and calculates the task loss . For the driving direction positioning task, the cross-entropy loss function is adopted:

[0147] ,

[0148] Among them, is the number of samples, is the number of classes, is the sample Belongs to the category The true label (0 or 1), is the probability that the model predicts the sample belongs to the category.

[0149] Memory module loss calculation: At the same time, calculate the storage and retrieval losses of the knowledge memory module . For example, calculate the loss by measuring the difference between the retrieved knowledge and the actual demand. Assume that the retrieved knowledge feature vector is , and the knowledge feature vector of the actual demand is , and use the mean square error loss:[[]]

[0150] ,

[0151] where is the number of samples when calculating the loss, is the th sample, the knowledge feature vector retrieved from the memory module (that is, the knowledge screened and fused by the network through cosine similarity); is the th sample, the knowledge feature vector of the actual demand (that is, the true knowledge required by the current environment and task); is to square the difference between the retrieved knowledge and the actual demand knowledge for each sample.

[0152] Total loss calculation and parameter update: Total loss:

[0153] ,

[0154] In joint training, and the task loss together constitute the total loss , and optimize the parameters of the memory module through backpropagation to make it store and retrieve key knowledge more effectively.

[0155] where is the weight coefficient used to balance the task loss and the memory loss. Through the backpropagation algorithm, update the parameters of the meta-learner, the main body of the neural network, and the knowledge memory module according to the total loss, so that the task execution ability of the network under different topological structures is continuously improved, the accuracy and efficiency of the meta-learner to generate topological structure adjustment parameters are continuously optimized, and the effectiveness of the knowledge memory module to store and retrieve knowledge is also significantly enhanced.

[0156] It should be noted here that joint training can enable close collaboration among various modules. Through the backpropagation of the total loss, the collaborative optimization of the parameters of each module is achieved. The backpropagation of the task loss guides the neural network to optimize its task execution ability, while the backpropagation of the memory module loss prompts the knowledge memory module to better store and retrieve knowledge, thereby comprehensively improving the performance of the network.

[0157] It should be supplemented and explained that through the overall training framework, the task execution ability of the network under different geological conditions has been improved, the accuracy and efficiency of the meta-learner to generate topology adjustment parameters have been enhanced, and the effectiveness of the knowledge memory module to store and retrieve knowledge has been optimized. This enables the network to more accurately identify geological conditions, locate the tunneling direction, adapt to different geological conditions in the complex and changeable coal mine underground tunneling environment, and provide more reliable support for coal mine underground tunneling operations.

[0158] 2. Multi-objective Optimization

[0159] In the coal mine underground tunneling scenario, it is necessary to balance the tunneling direction positioning accuracy, the speed of topology adjustment, and the effect of knowledge transfer and inheritance of the network under different geological conditions. For example, when encountering complex geological structures, the network is required to quickly and accurately adjust the tunneling direction; in areas with different rock types, knowledge transfer and inheritance need to be fully utilized to improve the positioning accuracy.

[0160] Specific steps:

[0161] Objective function setting: Set the objective function, including the tunneling direction positioning accuracy of the network under different geological conditions , calculated by the deviation between the actual tunneling direction and the preset direction; the time cost of topology adjustment , measured by recording the time for the meta-learner to generate adjustment parameters and the neural network to complete topology adjustment; the contribution to the improvement of tunneling performance after knowledge fusion , evaluated by comparing aspects such as the improvement of tunneling efficiency and the reduction of positioning error before and after fusion, such as the improvement amplitude of tunneling speed and the reduction degree of direction deviation after fusion.

[0162] In the method, setting the objective function is of great significance for optimizing the network performance and adapting to the complex and changeable tunneling environment in the coal mine underground. The objective function covers several key indicators, including the tunneling direction positioning accuracy of the network under different geological conditions, the time cost of topology adjustment, and the contribution to the improvement of tunneling performance after knowledge fusion. In actual application scenarios, different hardness rocks and different gas concentration environments will have a significant impact on tunneling operations. Compared with traditional methods, this method shows remarkable improvements in multiple key indicators in these complex environments. The advantages of this method will be intuitively presented through detailed experimental data comparison below.

[0163] In order to comprehensively and accurately evaluate the performance differences between this method and traditional methods in different environments, large-scale experimental monitoring was carried out at the actual tunneling sites of multiple coal mines for 6 months. Different hardness rock areas (low hardness areas with a rock hardness coefficient of 2 - 4 and high hardness areas with a rock hardness coefficient of 6 - 8) and different gas concentration areas (low gas concentration environment and high gas concentration environment) were carefully selected for comparative tests. During the entire experimental process, the experimental conditions were strictly controlled to ensure that all experimental equipment and initial conditions were basically the same, and the only variable was the positioning technology adopted, so as to ensure the scientificity and reliability of the experimental results.

[0164] In the experimental execution stage, a series of high-precision monitoring equipment and rigorous data collection and analysis methods were adopted. By installing high-precision positioning sensors, the positioning accuracy of the tunneling direction was monitored in real time and accurately; a professional timer was used to accurately record the topology adjustment time; a device for monitoring the running speed of the tunneling equipment was used to accurately calculate the improvement ratio of the tunneling speed; through the equipment energy consumption monitoring device, the equipment energy consumption data was comprehensively counted, and then the energy consumption reduction ratio was calculated. In order to further ensure the accuracy and reliability of the data, the collected data was subjected to multiple rounds of comparative analysis, and finally the average value was taken as the final experimental result. The specific quantitative comparison data is shown in the following table:

[0165] Comparison items Rock hardness coefficient is 2 - 4 (low hardness) Rock hardness coefficient is 6 - 8 (high hardness) Low gas concentration environment High gas concentration environment Improvement in the positioning accuracy rate of the tunneling direction 25% (traditional method 70%, this method 95%) 30% (traditional method 60%, this method 90%) 20% (traditional method 80%, this method 100%) 28% (traditional method 72%, this method 100%) Shortening of the topological structure adjustment time Approximately 80% (traditional method 10 minutes, this method 2 minutes) Approximately 80% (10 minutes for the traditional method, 2 minutes for this method) Approximately 80% (10 minutes for the traditional method, 2 minutes for this method) Approximately 80% (10 minutes for the traditional method, 2 minutes for this method) Percentage increase in tunneling speed 35% (The traditional method tunnels 5 meters per hour, this method tunnels 6.75 meters per hour) 40% (The traditional method tunnels 4 meters per hour, this method tunnels 5.6 meters per hour) 30% (The traditional method tunnels 5.5 meters per hour, this method tunnels 7.15 meters per hour) 38% (The traditional method tunnels 5 meters per hour, this method tunnels 6.9 meters per hour) Percentage reduction in equipment energy consumption 20% (The traditional method consumes 100 units of energy per meter of tunneling, this method consumes 80 units of energy per meter of tunneling) 25% (The traditional method consumes 120 units of energy per meter of tunneling, this method consumes 90 units of energy per meter of tunneling) 18% (The traditional method consumes 95 units of energy per meter of tunneling, this method consumes 78 units of energy per meter of tunneling) 22% (The traditional method consumes 105 units of energy per meter of tunneling, this method consumes 82 units of energy per meter of tunneling)

[0166] It can be clearly seen from the above table data that this method shows excellent performance both in different hardness rock environments and different gas concentration environments. In terms of the positioning accuracy of the tunneling direction, there is a significant improvement compared with the traditional method, which means that the tunneling direction can be controlled more precisely, effectively reducing the tunneling deviation and ensuring the safety and efficiency of the tunneling operation. The topology adjustment time is greatly shortened, enabling the network to quickly adapt to environmental changes and timely adjust itself to better handle the tunneling tasks in the new environment. The significant increase in the tunneling speed and the reduction of equipment energy consumption not only improve the tunneling efficiency but also reduce the production cost, which has important economic value. These data fully prove the great advantages of this method under complex geological conditions, can effectively solve the problems of traditional technologies in coal mine underground tunneling operations, and bring higher benefits and more reliable technical support to the coal mining industry.

[0167] Optimization Algorithm Selection: A multi-objective optimization method based on the Non-dominated Sorting Genetic Algorithm (NSGA-II) is adopted. This algorithm searches for and optimizes network parameters and structures by simulating the process of natural selection and genetic evolution. In each generation of evolution, the algorithm performs non-dominated sorting on the individuals in the population, classifies the individuals into different ranks, and preferentially selects individuals with higher ranks (i.e., non-dominated solutions) for genetic operations such as crossover and mutation to generate a new population. At the same time, crowding distance calculation is used to maintain the diversity of the population and prevent the algorithm from falling into local optimal solutions.

[0168] Objective Function Calculation and Optimization: The objective function can be expressed as:

[0169] ,

[0170] where , , are weight coefficients, which are adjusted according to actual requirements. During the optimization process, through continuous iteration, the algorithm searches for the optimal network parameters and structures to make the objective function reach the optimal value.

[0171] It should be noted here that: The multi-objective optimization algorithm can balance and optimize among multiple conflicting objectives, avoiding focusing only on a single objective and ignoring other important factors. The NSGA-II algorithm has good global search ability and convergence, and can effectively search for the optimal solution in the complex parameter space, enabling the network to achieve good comprehensive performance under different geological conditions.

[0172] In actual tests in multiple coal mines, the application of this method has demonstrated excellent performance under different geological conditions. For example, during the tunneling operation in a certain coal mine, a complex fault and fold geological area was encountered. The positioning accuracy of the traditional fixed-topology neural network in the tunneling direction in this area was only 60%. Moreover, due to the inability to adapt to geological changes in a timely manner, the tunneling efficiency was low, the daily tunneling progress was slow, and there were frequent equipment damage problems caused by positioning deviations. However, with this method, in the same complex geological area, the positioning accuracy in the tunneling direction can be stably maintained above 85%. In the face of geological changes, the system can complete the topology adjustment in about 2 minutes, quickly adapt to the new geological conditions, effectively reduce the number of mistakes in the tunneling direction, significantly improve the tunneling efficiency, and the daily tunneling progress is nearly 30% higher than that of the traditional method. The number of equipment damages caused by positioning deviations has also been greatly reduced, from the original 5 times per week to less than 1 time per week, greatly improving the safety and efficiency of coal mine mining.

[0173] In addition, during the tunneling process in coal mines, it is inevitable to encounter unexpected situations such as sensor failures and data transmission errors. The joint training and optimization mechanism of this method uses a multi-objective optimization algorithm to balance the performance indicators of the network, enabling the network to maintain a certain recognition accuracy and stability when facing these unexpected situations. For example, when the visual camera is blocked by dust, resulting in data loss, the knowledge transfer based on the memory mechanism can combine the data from millimeter-wave radars and inertial navigation devices and continue with target recognition and positioning using the previously stored knowledge, ensuring the continuous progress of the tunneling operation. Even when some sensors fail, the system can still maintain a certain accuracy in tunneling direction positioning, avoiding tunneling accidents caused by data anomalies and greatly enhancing the robustness of the entire system, which is difficult to achieve with traditional neural networks.

[0174] In the development process of neural network technology, the fixed topological structure of traditional self-supervised neural networks faces many challenges in the complex and dynamic environment of coal mine tunneling. Its internal connections and processing processes are basically fixed after training and are difficult to adapt to the changing geological conditions, resulting in large deviations in tunneling direction, low efficiency, and the inability to effectively utilize existing knowledge and experience. This method introduces a meta-learner based on the long short-term memory network (LSTM), enabling it to deeply analyze the dynamic changes in the tunneling environment state, learn the complex mapping relationship between the environment and the topological structure, and endow the neural network with the ability to adjust the topological structure in real time and dynamically. When facing changing geological conditions, the network can quickly capture new environmental features and positioning clues, greatly enhancing the adaptability of the neural network in the coal mine tunneling environment.

[0175] In the field of knowledge transfer and inheritance, the existing neural network theories and methods have obvious deficiencies in the coal mine tunneling scenario. The designed knowledge memory module based on the attention mechanism can intelligently screen and store the key knowledge learned under different topological structures. At the same time, the knowledge retrieval and fusion mechanism using cosine similarity calculation and dynamic weighted fusion algorithm has successfully realized the effective transfer and inheritance of knowledge between different topological structures, filling the gap in the application of this field in coal mines and opening up a new path for the continuous learning and performance improvement of neural networks under complex geological conditions.

[0176] In summary, the technical solution of the dynamic adaptive self-supervised neural network architecture based on meta-learning and memory mechanism has successfully overcome the topological adaptation problem and the knowledge transfer and inheritance problem of self-supervised neural networks in the coal mine underground tunneling environment. It demonstrates excellent performance and broad application prospects in practical applications, provides strong support for the development of coal mine underground tunneling technology, and is expected to promote the intelligent upgrading of the entire coal industry. In the future, with the continuous development and improvement of technology, this technology can be further extended to other similar underground mining scenarios, such as coal mining, tunnel excavation and other fields, bringing new opportunities for technological innovation in related industries.

Claims

1. A visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network, characterized in that: The following steps are involved: Construct a meta-learner based on long short-term memory network: The input layer of the meta-learner receives the processed environmental feature vector, which is obtained by extracting the underground tunnel image data collected by the visual camera through convolutional neural network features, normalizing the millimeter-wave radar data, and calibrating and preprocessing the inertial navigation equipment data; The middle hidden layer is composed of multiple LSTM units, which use the gating mechanism to control the inflow, outflow and memory of information; the output layer is based on the output of the hidden layer, and generates the network topology adjustment parameters for the current tunneling environment state through the fully connected layer calculation. The parameters include the increase and decrease coefficients of neuron connections, the adjustment factors of the inter-layer connection weights, and the adjustment values ​​of the number of neurons in the hidden layer; When the state of the underground excavation environment in a coal mine changes, the new state of the environment information is collected in real time and input into the trained meta-learner; the meta-learner calculates and outputs the topological structure adjustment parameters based on the learned mapping relationship, and the neural network quickly adjusts its own topological structure based on the parameters, including adjusting the connection weights between neurons, and increasing or decreasing the number of hidden layer neurons related to motion feature processing according to the posture changes of the excavation equipment and geological conditions; The attention mechanism is used to analyze the feature graphs of the middle layer of the network, extract important knowledge feature vectors related to the rock structure and motion state of tunneling equipment in coal mines, calculate the attention weight of each knowledge feature vector, and store it in the knowledge memory module in the form of weighted sum; When the network topology is adjusted, the new topology generates query vectors based on the current tunneling task requirements and environmental characteristics under current geological conditions; Calculate the similarity between the query vector and the knowledge feature vector stored in the memory module, select the knowledge feature vector with high similarity, and fuse it with the knowledge feature vector learned under the new topological structure by weighted fusion. The fusion coefficient is dynamically adjusted according to the current excavation environment and task requirements. The meta-learner, the neural network body, and the knowledge memory module are jointly trained as a whole; the environmental data of different tunneling areas and different geological conditions in the coal mine are input to the meta-learner, the meta-learner generates topological structure adjustment parameters, the neural network forward propagates and calculates the task loss according to the adjusted topological structure, and at the same time calculates the storage and retrieval loss of the knowledge memory module, and updates the parameters of each module through the total loss; Set the objective function, including the network's positioning accuracy in the tunneling direction under different geological conditions, the time cost of topological structure adjustment, and the contribution of knowledge fusion to the improvement of tunneling performance; A multi-objective optimization method based on non-dominated sorting genetic algorithm is used to search and optimize network parameters and structure.

2. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: The convolutional neural network uses pre-trained convolution kernels, including but not limited to VGG16 and ResNet series, to convert the underground tunnel image data collected by the visual camera into a feature vector with semantic information.

3. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: The millimeter-wave radar data is normalized to map the distance and speed values ​​to the [0,1] interval.

4. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: The state update formula of the LSTM hidden layer includes: ， in, is the input gate vector; is the weight matrix from the input layer to the corresponding gate and memory unit, is the weight matrix from the hidden layer to the corresponding gate and memory unit, is the corresponding bias vector, is the Sigmoid activation function.

5. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The output topology adjustment parameters are calculated by the output layer using the formula: ， in, is the weight matrix from the hidden layer to the output layer, is the bias vector of the output layer.

6. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The formula for calculating the attention weight of each knowledge feature vector in the attention mechanism is: ， in, and are all knowledge feature vectors, Represents the knowledge feature vector currently being calculated for attention weights, Represents each vector in the set of all knowledge feature vectors stored in the memory module, It is a query vector, which is used to measure the importance of each knowledge feature vector. The attention weight of each feature vector is obtained by performing a dot product operation with the knowledge feature vector and normalizing it with the Softmax function.

7. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: The cosine similarity formula is used to calculate the similarity between the query vector and the knowledge feature vector stored in the memory module: ， in, is the newly generated query vector, is the knowledge feature vector stored in the memory module, dot product To measure the consistency of the directions of two vectors, the larger the value, the closer the directions are; the product of the modulus length To normalize the vector length and eliminate the dimension effect.

8. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: Knowledge fusion adopts the weighted fusion method, and the formula is: ， in, is the knowledge feature vector learned under the new topological structure, is a set of selected relevant knowledge feature vectors, is the fusion coefficient, which is dynamically adjusted according to the current environment and task requirements.

9. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: For the target recognition task, the cross entropy loss function is used to calculate the task loss: ， in, is the sample size, is the number of categories, It is a sample Belongs to category The real label, is the probability that the model predicts that the sample belongs to the category.

10. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The storage and retrieval losses of the knowledge memory module are calculated using mean square error loss: ， in, is the number of samples when calculating the loss, is the knowledge feature vector, is the knowledge feature vector of actual needs, For the The knowledge feature vector retrieved from the memory module in the sample is the knowledge that the network has filtered and integrated through cosine similarity; For the The knowledge feature vector actually required in the samples is the real knowledge required for the current environment and task.

11. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The total loss is calculated as: ， in is a weight coefficient used to balance task loss and memory loss.

12. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The objective function is expressed as: ， in, , , is the weight coefficient.

13. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: In the multi-objective optimization method, in each generation of evolution, individuals in the population are non-dominated and sorted, and the individuals are divided into different levels. Individuals with high levels are preferentially selected for genetic operations, including but not limited to crossover and mutation, to generate new populations, and the diversity of the population is maintained by crowding calculation.

14. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: Environmental data includes environmental data in different excavation areas and under different geological conditions, and is marked with corresponding task labels, including but not limited to rock type, excavation direction deviation, equipment failure type, and gas concentration level.

15. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: Adjusting the neural network topology includes adding or deleting neuron connections, strengthening or weakening the strength of inter-layer connections, and dynamically increasing or decreasing the number of hidden layer neurons.

Images