Visual inertial navigation radar fusion self-positioning method based on self-supervised neural network

Abstract

The invention discloses a visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network, and relates to the technical field of artificial intelligence and sensor fusion positioning, and the method comprises the following steps: constructing a meta-learner based on a long short-term memory network; according to the method, dynamic topology adaptation is realized by constructing the LSTM-based meta learning device, and topology adjustment can be completed and environment characteristics can be accurately captured in response to environment change within about 2 minutes; efficient knowledge migration and inheritance are realized by utilizing a knowledge memory module of an attention mechanism and a related algorithm, and new and old knowledge can be accurately fused to guarantee tunneling safety; through joint training and optimization and multi-module collaborative optimization of parameters, multi-target balance optimization is performed by adopting an NSGA-II algorithm, performance, efficiency and knowledge utilization are considered under different geological conditions, and the overall level of tunneling operation is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and sensor fusion positioning technology, and in particular to a visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network. Background Art

[0002] In the underground coal mine excavation operation scenario, the application of self-supervised neural network technology faces many challenges. Traditional self-supervised neural network architectures are usually trained and run based on fixed network topology structures. However, the underground coal mine excavation environment is extremely complex and dynamic. As the excavation work progresses, the geological conditions are constantly changing, and the hardness, structure and geological structure of the rock vary significantly. For example, during the excavation process, faults, folds, and alternations of soft and hard rocks may suddenly be encountered. These changes make the topological structure of the underground environment in dynamic change, and traditional neural networks are difficult to adapt to such dynamic changes in real time.

[0003] When encountering changes in geological conditions, traditional neural networks cannot automatically adjust their internal connections and processing procedures, and it is difficult to effectively capture new environmental features and positioning clues. In coal mines, this may cause deviations in the excavation direction, affect excavation efficiency, and even cause safety accidents. For example, when encountering a fault, traditional neural networks cannot perceive and adjust the recognition of geological structures in a timely manner, causing excavation equipment to deviate from the predetermined track, resulting in waste of resources and safety hazards. This shows that it is urgent to develop a new self-supervised neural network architecture that can automatically adjust the network topology according to the dynamic changes in the underground coal mine excavation environment. However, existing technologies are difficult to meet this demand.

[0004] When trying to develop a new self-supervised neural network architecture to adapt to the dynamic changes in the underground coal mine excavation environment, the problem of knowledge migration and inheritance between different topological structures during the network topology adjustment process has arisen. Different network topological structures are suitable for different geological conditions and excavation task requirements. When the network is adjusted from one topological structure to another, how to effectively retain and utilize the useful knowledge learned in the previous topological structure to avoid knowledge forgetting, while quickly adapting to the environmental characteristics and task requirements under the new topological structure, has become a very challenging problem.

[0005] During the tunneling process in coal mines, when the network topology is adjusted due to changes in geological conditions, the new topology may not be able to effectively utilize the previously learned knowledge about rock structure, equipment operation status, etc. For example, when tunneling from an area with stable geological conditions to a fault area, the previously learned knowledge of normal rock structure characteristics cannot be effectively utilized by the new topology, resulting in deviations in the identification of rock structures in the fault area, which in turn affects the accurate judgment of the tunneling direction and reduces the safety and efficiency of tunneling. Existing neural network theories and methods are almost blank in dealing with this cross-topology knowledge transfer and inheritance.

[0006] In view of this, a visual inertial navigation radar fusion self-positioning method based on self-supervised neural network is provided, which is specifically aimed at the underground excavation environment of coal mines to overcome the above problems. Summary of the invention

[0007] The purpose of the present invention is to provide a visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network to solve the problems raised in the above background technology.

[0008] In order to solve the above technical problems, the present invention provides a visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network, comprising the following steps:

[0009] Construct a meta-learner based on long short-term memory network: the input layer of the meta-learner receives the processed environmental feature vector, which is obtained by extracting the underground tunnel image data collected by the visual camera through convolutional neural network features, normalizing the millimeter-wave radar data, and calibrating and preprocessing the inertial navigation equipment data; the middle hidden layer is composed of multiple LSTM units, which use the gating mechanism to control the inflow, outflow and memory of information; the output layer is based on the hidden layer output, and the network topology structure adjustment parameters for the current excavation environment state are calculated through the fully connected layer. The parameters include the increase and decrease coefficients of neuron connections, the adjustment factors of inter-layer connection weights, and the adjustment values ​​of the number of neurons in the hidden layer;

[0010] When the state of the underground excavation environment in a coal mine changes, the new state of the environment information is collected in real time and input into the trained meta-learner; the meta-learner calculates and outputs the topological structure adjustment parameters based on the learned mapping relationship, and the neural network quickly adjusts its own topological structure based on the parameters, including adjusting the connection weights between neurons, and increasing or decreasing the number of hidden layer neurons related to motion feature processing according to the posture changes of the excavation equipment and geological conditions;

[0011] The attention mechanism is used to analyze the feature graphs of the middle layer of the network, extract important knowledge feature vectors related to the rock structure and motion state of tunneling equipment in coal mines, calculate the attention weight of each knowledge feature vector, and store it in the knowledge memory module in the form of weighted sum;

[0012] When the network topology is adjusted, the new topology generates a query vector according to the current excavation task requirements and the environmental characteristics under the current geological conditions; the similarity between the query vector and the knowledge feature vector stored in the memory module is calculated, and the knowledge feature vector with high similarity is selected and fused with the knowledge feature vector learned under the new topology using a weighted fusion method. The fusion coefficient is dynamically adjusted according to the current excavation environment and task requirements;

[0013] The meta-learner, the neural network body, and the knowledge memory module are jointly trained as a whole; the environmental data of different tunneling areas and different geological conditions in the coal mine are input to the meta-learner, the meta-learner generates topological structure adjustment parameters, the neural network forward propagates and calculates the task loss according to the adjusted topological structure, and at the same time calculates the storage and retrieval loss of the knowledge memory module, and updates the parameters of each module through the total loss;

[0014] The objective function is set, including the network's positioning accuracy in the excavation direction under different geological conditions, the time cost of topological structure adjustment, and the contribution of knowledge fusion to the improvement of excavation performance; a multi-objective optimization method based on a non-dominated sorting genetic algorithm is used to search and optimize the network parameters and structure.

[0015] Furthermore, the convolutional neural network uses pre-trained convolution kernels, including but not limited to VGG16 and ResNet series, to convert the underground tunnel image data collected by the visual camera into a feature vector with semantic information.

[0016] Furthermore, the millimeter-wave radar data is normalized to map values ​​such as distance and speed to the [0,1] interval.

[0017] Furthermore, the state update formula of the LSTM hidden layer is:

[0018] ,

[0019] ,

[0020] ,

[0021] ,

[0022] ,

[0023] ,

[0024] in, is the input gate vector, is the forget gate vector, is the output gate vector, is a candidate memory unit, For the memory unit, Output vector for the hidden layer. , , , is the weight matrix from the input layer to the corresponding gate and memory unit, , , , is the weight matrix from the hidden layer to the corresponding gate and memory unit, , , , is the corresponding bias vector, is the Sigmoid activation function, is the hyperbolic tangent activation function, represents element-wise multiplication, Represents the state of the memory cell at the previous time step.

[0025] Furthermore, the formula for output topological structure adjustment parameters calculated by the output layer is:

[0026] ,

[0027] in, is the weight matrix from the hidden layer to the output layer, is the bias vector of the output layer.

[0028] Furthermore, the formula for calculating the attention weight of each knowledge feature vector in the attention mechanism is:

[0029] ,

[0030] in, It is a query vector, which is used to measure the importance of each knowledge feature vector. The attention weight of each feature vector is obtained by performing a dot product operation with the knowledge feature vector and normalizing it with the Softmax function.

[0031] Furthermore, the cosine similarity formula is used to calculate the similarity between the query vector and the knowledge feature vector stored in the memory module:

[0032] .

[0033] Furthermore, knowledge fusion adopts the weighted fusion method, and the formula is:

[0034] ,

[0035] in, is the knowledge feature vector learned under the new topological structure, is a set of selected relevant knowledge feature vectors, is the fusion coefficient, which is dynamically adjusted according to the current environment and task requirements.

[0036] Furthermore, for the target recognition task, the cross entropy loss function is used to calculate the task loss:

[0037] ,

[0038] in, is the sample size, is the number of categories, It is a sample Belongs to category The real label, is the probability that the model predicts that the sample belongs to the category.

[0039] Furthermore, the storage and retrieval losses of the knowledge memory module are calculated using the mean square error loss:

[0040] ,

[0041] in, is the number of samples when calculating the loss, is the knowledge feature vector, is the knowledge feature vector of actual needs.

[0042] Furthermore, the total loss calculation formula is:

[0043] ,

[0044] in is a weight coefficient used to balance task loss and memory loss.

[0045] Furthermore, the objective function is expressed as:

[0046] ,

[0047] in, , , is the weight coefficient.

[0048] Furthermore, in the multi-objective optimization method, in each generation of evolution, individuals in the population are non-dominated and sorted, and the individuals are divided into different levels. Individuals with high levels are preferentially selected for genetic operations, including but not limited to crossover and mutation, to generate new populations, and the diversity of the population is maintained through crowding calculation.

[0049] Furthermore, the environmental data includes environmental data in different excavation areas and under different geological conditions, and is marked with corresponding task tags, including but not limited to rock type, excavation direction deviation, equipment failure type, and gas concentration level.

[0050] Furthermore, neural network topology adjustments include adding or deleting neuron connections, increasing or decreasing the strength of inter-layer connections, and dynamically increasing or decreasing the number of hidden layer neurons.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] Dynamic topology adaptation effect is remarkable

[0053] Rapid response to environmental changes: Traditional self-supervised neural networks are based on fixed topological structures, and are difficult to adjust in real time when faced with complex and changeable geological conditions underground in coal mines. For example, when encountering sudden changes in rock hardness or the appearance of faults, it may take a long time for traditional networks to adapt to the new environment, seriously affecting the progress of excavation. The present invention constructs a meta-learner based on LSTM, which can receive real-time information on the state of the underground excavation environment and use a gating mechanism to learn the laws of dynamic changes. When the environment changes, the topological structure adjustment can be completed in about 2 minutes, which greatly shortens the adaptation time, can quickly respond to changes in the geological environment, and ensure the continuity of excavation work.

[0054] Accurately capture environmental features: Traditional networks cannot automatically adjust internal connections and processing flows to capture new environmental features and positioning clues. The topological structure adjustment parameters output by the meta-learner of the present invention can increase neuronal connections when the neural network encounters different rock structures, such as hard rocks, and enhance its ability to capture rock structure features; when encountering geological structural changes that cause abnormal inertial navigation data, the inter-layer connection weights are changed, so that the neural network pays more attention to effective information and ignores interference information, effectively improving its adaptability in complex dynamic excavation environments and the accuracy of positioning the excavation direction.

[0055] Outstanding knowledge transfer and inheritance capabilities

[0056] Efficient storage of key knowledge: Existing neural network theories and methods are almost blank in cross-topological knowledge transfer and inheritance. The present invention uses the knowledge memory module of the attention mechanism to intelligently screen and store key knowledge under different topological structures in the form of weighted sum, such as the texture characteristics of different rocks, the relationship between hardness and excavation speed, and the energy consumption law of equipment under different geological conditions, etc., to improve the efficiency of knowledge storage and management and lay the foundation for knowledge transfer and inheritance.

[0057] Effective integration of new and old knowledge: After the network topology is adjusted, the new topology can accurately retrieve relevant knowledge and dynamically weighted integrate it with new knowledge through cosine similarity calculation and dynamic weighted fusion algorithm. In underground coal mines, when encountering high-hardness rock or gas layers, the new topology can use the previously stored knowledge and combine it with the new knowledge under the current geological conditions to accurately judge the rock properties, equipment operating status, etc., adjust the excavation strategy, avoid safety accidents caused by misjudgment, and enhance the safety and reliability of underground coal mine excavation operations.

[0058] Joint training and optimization improve overall performance

[0059] Multi-module collaborative optimization: The meta-learner, neural network body and knowledge memory module are jointly trained, and the parameters of each module are collaboratively optimized through total loss back propagation. In traditional methods, each module is independent and cannot be coordinated. In the present invention, task loss back propagation optimizes the task execution capability of the neural network, and memory module loss back propagation optimizes the effectiveness of knowledge memory module storage and retrieval, which comprehensively improves network performance and enables the network to better serve tunneling operations under different geological conditions.

[0060] Multi-objective balanced optimization: A multi-objective optimization method based on NSGA-II is used to set an objective function that includes the accuracy of positioning in the excavation direction, the time overhead of topological structure adjustment, and the contribution of knowledge fusion to the improvement of excavation performance. Traditional methods focus on a single goal. The present invention achieves the best balance of performance, efficiency, and knowledge utilization under different geological conditions by continuously iteratively searching for the optimal network parameters and structure. In underground coal mine excavation, it can quickly adapt to geological changes to adjust the topological structure while ensuring high positioning accuracy in the excavation direction, and make full use of knowledge transfer inheritance to improve excavation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a schematic diagram of the visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] See also Figure 1 , the present invention provides a technical solution:

[0064] See also Figure 1 As shown, an embodiment of a visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network:

[0065] 1. Topology Adjustment Strategy Based on Meta-learning

[0066] 1. Construction of Meta-Learner

[0067] In the underground tunneling operation scenario of coal mines, the tunneling equipment is equipped with visual cameras, inertial navigation equipment and millimeter-wave radar. The visual camera is responsible for collecting image data of the underground tunnel, the inertial navigation equipment obtains key information such as the attitude and acceleration of the tunneling equipment, and the millimeter-wave radar is used to monitor the distance, speed and other data of the objects in front of the tunneling, and these data are transmitted to the meta-learner in real time.

[0068] Convolutional neural networks (CNNs) are used to extract features from visual image data. Pre-trained convolution kernels, such as VGG16 or ResNet series, are used to convert images into feature vectors with semantic information. At the same time, millimeter-wave radar data is normalized to map values ​​such as distance and speed to the [0,1] interval to eliminate the dimension effect. In addition, the attitude, acceleration and other data collected by the inertial navigation equipment are calibrated and pre-processed to ensure that they are at the same scale as other data for subsequent processing.

[0069] A meta-learner structure based on the long short-term memory network (LSTM) is adopted. As a recurrent neural network (RNN), LSTM can effectively process time series data and solve the problem of long-term dependence. Its input layer receives the preprocessed environmental feature vector, and the middle hidden layer is composed of multiple LSTM units. The inflow, outflow and memory of information are controlled through the gating mechanism (input gate, forget gate and output gate), so as to learn the dynamic change law of the underground excavation environment state. Based on the output results of the hidden layer, the output layer calculates and generates the network topology adjustment parameters for the current excavation environment state through the fully connected layer. These parameters include the increase and decrease coefficients of neuron connections, which are used to determine the establishment of new connections or the deletion of old connections; the adjustment factor of the inter-layer connection weight, which can multiply the existing connection weight to increase or decrease the connection strength; and the adjustment value of the number of neurons in the hidden layer, which dynamically increases or decreases neurons according to the complexity of the excavation environment.

[0070] LSTM has a powerful time series modeling capability and can capture the changing trends and dependencies of environmental states over time. By training on a large number of samples with different tunneling environment scenarios and topological structure changes, the meta-learner can learn the complex mapping relationship between tunneling environment states and appropriate topological structures, providing accurate topological adjustment parameters for the neural network.

[0071] Assume that the input environment state feature vector is, and the state update formula of the LSTM hidden layer at time is:

[0072] ,

[0073] ,

[0074] ,

[0075] ,

[0076] ,

[0077] ,

[0078] in, is the input gate vector, is the forget gate vector, is the output gate vector, is a candidate memory unit, For the memory unit, Output vector for the hidden layer. , , , is the weight matrix from the input layer to the corresponding gate and memory unit, , , , is the weight matrix from the hidden layer to the corresponding gate and memory unit, , , , is the corresponding bias vector, is the Sigmoid activation function, is the hyperbolic tangent activation function, represents element-wise multiplication, Represents the state of the memory cell at the previous time step.

[0079] The output topology adjustment parameters are calculated by the output layer:

[0080] ,

[0081] in, is the weight matrix from the hidden layer to the output layer, is the bias vector of the output layer.

[0082] Through the above mechanism, the meta-learner deeply learns the intrinsic connection between the dynamic changes of the tunneling environment and the topological structure, and provides the neural network with accurate and targeted topological structure adjustment parameters. This enables the neural network to quickly adjust its own topological structure when facing the complex and changeable tunneling environment in coal mines, such as changes in rock structure and differences in geological structures, effectively capture new environmental features and positioning clues, and significantly improve the network's adaptability to the tunneling environment and real-time response capabilities.

[0083] It is necessary to add that:

[0084] Meta-learner structural advantages: The meta-learner adopts an LSTM-based structure, which can effectively process time series data and solve long-term dependency problems. In the underground coal mine excavation scenario, the excavation environment changes dynamically over time, such as the hardness and structural differences of different rock layers, as well as geological changes such as faults and folds encountered during the excavation process. LSTM accurately controls the inflow, outflow and memory of information through its gating mechanism, thereby learning the dynamic changes of the excavation environment state over time, providing strong technical support for in-depth learning of the intrinsic connection between the dynamic changes of the environment and the topological structure, enabling the meta-learner to capture the subtle characteristics of environmental changes and the relationship between these changes and topological structure adjustments.

[0085] Effectiveness of the training process: The meta-learner is trained on a large number of samples of different excavation environment scenarios and topological structure changes. By learning from massive data, the meta-learner can fully understand the characteristics of various excavation environment states and the corresponding appropriate topological structures. Coal mine underground excavation covers different geological conditions, such as soft rock, hard rock, fault areas and other scenarios. By learning from these rich samples, the meta-learner summarizes general rules, thereby providing accurate and targeted topological structure adjustment parameters for the neural network. When faced with specific dynamic environmental changes, the meta-learner can quickly generate adjustment parameters suitable for the current environment based on previously learned knowledge.

[0086] Effect on neural network: The topology adjustment parameters generated by the meta-learner directly act on the neural network. When faced with different rock structures, such as hard rock, the parameters generated by the meta-learner can enable the neural network to increase neuron connections to enhance its ability to capture rock structure characteristics; when encountering geological structural changes that lead to abnormal inertial navigation data, adjusting the parameters can change the inter-layer connection weights, allowing the neural network to pay more attention to effective information and ignore interference information, thereby quickly adjusting its own topology. In these ways, the neural network can effectively capture new environmental features and positioning clues, significantly improving its adaptability and real-time response capabilities in complex and dynamic tunneling environments.

[0087] 2. Rapid Adaptation Mechanism

[0088] During underground coal mine excavation, when encountering changes in geological conditions, such as sudden changes in rock hardness, the appearance of faults, encountering groundwater, etc., new environmental status information, including changes in rock image features, abnormal equipment posture, changes in the state of objects ahead detected by millimeter-wave radar, etc., is collected in real time and input into the trained meta-learner.

[0089] Based on the learned mapping relationship between environmental states and topological structure adjustment parameters, the meta-learner quickly processes the input new environmental information, calculates and outputs the corresponding topological structure adjustment parameters.

[0090] The neural network quickly adjusts its own topological structure according to the adjustment parameters output by the meta-learner. For example, by adjusting the connection weights between neurons, the ability to extract image features of rock structure changes is enhanced; according to the posture changes of the tunneling equipment and geological conditions, the number of hidden layer neurons related to motion feature processing is increased or decreased, and the perception and processing process of tunneling direction and environmental changes is optimized to ensure that the tunneling equipment can adapt to new geological conditions in a timely manner and maintain the correct tunneling direction.

[0091] The meta-learner is trained on a large number of excavation environment samples, enabling it to establish a stable and accurate mapping relationship. Therefore, when faced with new environmental changes, it can respond quickly based on existing learning experience and generate effective adjustment parameters.

[0092] In order to intuitively demonstrate the advantages of the rapid adaptation mechanism based on meta-learning, comprehensive actual tests were carried out in different tunnels of multiple coal mines to compare the traditional fixed topology neural network with the dynamic topology neural network based on meta-learning. The results are shown in the following table: Compare Projects Traditional fixed topology neural network Dynamic topology neural network based on meta-learning Time to adapt to the new geological environment About 30 minutes About 2 minutes Tunneling direction positioning accuracy under complex geological conditions 60% 85% Number of errors in excavation direction due to geological changes (per hour) 15 times 3 times Number of target detections when encountering faults (per hour) 50 80 Misjudgment rate when facing complex geological structures (such as folds) 20% 5% Compare Projects Traditional fixed topology neural network Dynamic topology neural network based on meta-learning

[0093] From the data, we can see that it takes about 30 minutes for the traditional fixed topology neural network to adapt to the new geological environment, while the dynamic topology neural network based on meta-learning only takes about 2 minutes, which greatly shortens the time and can quickly respond to changes in the geological environment. The accuracy of positioning the excavation direction under complex geological conditions has increased from 60% to 85%, and the excavation direction can be determined more accurately. The number of excavation direction errors caused by geological changes has been reduced from 15 times per hour to 3 times, effectively avoiding errors in the excavation direction. When encountering faults, the number of target detections increased from 50 to 80 per hour; when faced with complex geological structures such as folds, the misjudgment rate was reduced from 20% to 5%.

[0094] Compared with traditional neural networks with fixed topology structures, the adjustment method based on meta-learning in the present invention does not require large-scale re-training. In scenarios where the environment changes rapidly underground in coal mines, the topology structure adjustment can be completed in a short time, ensuring that the network continues to accurately capture environmental information, significantly improving the network's adaptability and response speed, and providing a solid guarantee for the safe and efficient excavation operations in underground coal mines.

[0095] Further explanation is needed:

[0096] In the field of complex environment monitoring and positioning, the adaptability and accuracy of technology are directly related to the efficiency and safety of operations. At present, the combination of traditional complex environment monitoring and positioning technologies mostly adopts fixed network structures or simple adaptive adjustment methods, which have obvious limitations when dealing with complex and changing environments.

[0097] Take a coal mine excavation scenario as an example. The geological conditions of the coal mine are extremely complex, with a variety of ore layers with huge differences in hardness, and a large number of complex geological structures, such as frequent faults and folds. In such an environment, traditional technologies encounter many difficulties in practical applications.

[0098] In traditional technology, it is common to adjust network parameters based on fixed threshold judgment methods. This method is too mechanical and only operates based on pre-set fixed thresholds, which cannot deeply understand the intrinsic connection between environmental changes and network structures. Once the geological conditions change slightly, this fixed threshold adjustment method is difficult to accurately adapt, and it is even more powerless when faced with complex situations such as sudden changes in ore hardness and frequent changes in geological structures in coal mines. This directly leads to low positioning accuracy and greatly reduced excavation efficiency.

[0099] The defects of traditional technology are particularly prominent when facing areas with large changes in ore hardness. Due to the inability to quickly adapt to changes in hardness, the posture adjustment of the tunneling equipment is difficult to complete accurately. The inaccurate posture of the tunneling equipment seriously affects the positioning accuracy, resulting in frequent deviations in the tunneling direction. In order to accurately obtain the actual performance data of traditional technology in tunneling in this coal mine, the five tunneling tunnels using traditional technology in the coal mine were continuously monitored within one month. The results showed that the number of tunneling direction errors caused by positioning deviations reached an average of 12 times per hour. Through precise measurement and analysis of the tunneling trajectory, it was found that the tunneling path of traditional technology in this area increased by an average of 35% compared to the precise tunneling path. This not only greatly increased the time cost of tunneling, but also, according to statistics, under the same tunneling task volume, the traditional technology consumed nearly 40% more time than the theoretically efficient tunneling time.

[0100] In contrast, this method shows significant advantages. This method introduces a meta-learner based on the long short-term memory network (LSTM). With its unique gating mechanism, it can effectively process time series data and deeply learn the dynamic changes of the state of the coal mine excavation environment. Once the geological conditions change, the meta-learner can respond quickly and quickly adjust the topological structure of the neural network.

[0101] At the same time, combined with the attention mechanism, this method can accurately screen and store knowledge under different geological conditions. When the network faces new geological conditions, it can quickly retrieve and utilize relevant knowledge to avoid knowledge waste and forgetting. At the coal mine excavation site, 5 excavation tunnels with geological conditions similar to those when monitoring traditional technologies were selected. This method was applied to excavation operations and was strictly monitored for half a year. During the monitoring process, high-precision positioning equipment was used to record the location information of the excavation equipment in real time. The positioning accuracy was calculated by comparing it with the preset standard excavation path; professional efficiency monitoring software was used to combine equipment operation data and excavation progress to calculate the excavation efficiency; sensors were used to record the posture adjustment time of the excavation equipment when facing areas with ore hardness changes. The specific data are as follows: Compare Projects Traditional Technology This method Data acquisition basis Positioning accuracy 60% (based on the statistics of multiple positioning data per hour in 5 tunnels within one month. Positioning errors exceeding the set range are considered positioning errors, and the ratio of the number of errors to the total number of positioning times is calculated) Increased to 90% (the increase was 30%, and the same positioning error judgment standard was used to calculate the proportion of positioning errors to the total positioning times during the half-year monitoring period in 5 tunnels where this method was applied) Use high-precision positioning equipment to record the location information of the tunneling equipment in real time, compare it with the preset standard tunneling path, and calculate the positioning error rate Excavation efficiency 5 meters of excavation per hour (calculated based on the ratio of the total excavation length of 5 excavation tunnels to the total operation time in one month) Increased by 25% to 6.25 meters per hour (the ratio of the total excavation length to the total operation time of the five excavation tunnels using this method was calculated during the six-month monitoring period) Use professional efficiency monitoring software to combine equipment operation data and excavation progress to calculate excavation efficiency Time required to adjust the attitude of the tunneling equipment when facing areas with varying ore hardness The average adjustment time is more than 10 minutes (during the one-month monitoring period, the equipment posture adjustment process in the 5 tunneling tunnels when encountering the area with ore hardness changes was recorded, and the time of each adjustment was counted and the average value was calculated) Ability to complete quick adjustment within 2 minutes (During the half-year monitoring period, for 5 tunneling tunnels where this method was applied, the time for equipment posture adjustment when encountering areas with ore hardness changes was recorded and the average value was calculated) Recording of attitude adjustment time by sensors installed on the excavation equipment Number of errors in excavation direction due to positioning deviation (per hour) 12 times (During the one-month monitoring period, the excavation direction deviation of the five excavation tunnels was counted every hour, the number of direction errors was recorded and the hourly average was calculated) Reduced to 3 times (reduced by 75%. During the half-year monitoring period, the deviation of the driving direction of the 5 tunnels using this method was counted every hour, and the average number of errors per hour was calculated) By observing the deviation of the excavation trajectory from the preset direction, the number of errors in the excavation direction is counted Increase ratio of excavation path 35% increase compared to the precise excavation path (based on the comparison of the actual excavation trajectory of 5 excavation tunnels with the preset precise excavation path within one month, and the ratio of the extra path length to the precise path length was calculated) Only 5% increase (during the half-year monitoring period, the actual excavation trajectories of the five tunneling tunnels using this method were compared with the preset precise excavation paths, and the ratio of the extra path length to the precise path length was calculated) Use high-precision measuring equipment to measure the actual excavation trajectory and the preset precise excavation path, and compare the calculated path increase ratio Time cost reduction ratio - Reduced by about 30% (comparing the total time required to complete the task with the same excavation task volume, the difference in calculation time accounts for the proportion of the total time of the traditional technology) Statistics on the total operation time of traditional technology and this method under the same excavation task volume, and calculate the time cost reduction ratio

[0102] It is clear from the above detailed and well-founded data comparison that this method performs very well under complex geological conditions. Whether it is the significant improvement in positioning accuracy, the significant improvement in excavation efficiency, or the ability to respond quickly to complex geological changes, it has been strongly proved that this method can effectively solve the problems of traditional technologies in complex environment monitoring and positioning, and provide a more efficient, accurate and reliable solution for this field.

[0103] 2. Knowledge transfer and inheritance based on memory mechanism

[0104] 1. Knowledge memory module

[0105] During the tunneling process in coal mines, when the tunneling equipment is operating normally, the network topology focuses on learning the rock structure characteristics and tunneling equipment operating status during tunneling, such as the texture characteristics of different rocks, the relationship between hardness and tunneling speed, and the energy consumption law of equipment under different geological conditions. This knowledge needs to be stored efficiently so that it can continue to play a role after the topology structure is adjusted.

[0106] When the network is learning under the normal tunneling topology, the attention mechanism is used to analyze the feature graphs of the middle layer of the network to extract important knowledge feature vectors related to the rock structure and the motion state of the tunneling equipment.

[0107] Attention calculation: The attention mechanism calculates the importance of each knowledge feature vector and assigns it an attention weight. The specific calculation method is:

[0108] ,

[0109] in, is the query vector, which is used to measure the importance of each knowledge feature vector. It is dot-producted with the knowledge feature vector to evaluate the importance of the knowledge feature vector for the current task or environment. In the coal mine excavation scenario, the query vector It is generated based on the current excavation task requirements and the environmental characteristics under the current geological conditions. It represents the current network's knowledge needs and focus. For example, when encountering geological conditions of high-hardness rock, the query vector It will contain information related to the characteristics of high-hardness rocks, so as to filter out knowledge feature vectors for judging rock properties and adjusting excavation strategies.

[0110] and are all knowledge feature vectors, Represents the knowledge feature vector currently being calculated for attention weights, Represents each vector in the set of all knowledge feature vectors stored in the memory module. These knowledge feature vectors are extracted by analyzing the feature graphs of the middle layer of the network and are related to the rock structure and the motion state of the tunneling equipment in the coal mine. For example, the texture characteristics of different rocks, the relationship between hardness and tunneling speed, and the energy consumption law of equipment under different geological conditions are all stored in the form of knowledge feature vectors. In the calculation of attention weights, and Perform a dot product operation, and the denominator is the feature vector of all knowledge With query vector The summation of the exponential results of the dot product operation is used to normalize the results of the numerator so that the sum of the attention weights of all knowledge feature vectors is 1, so that the importance weights of different knowledge feature vectors can be reasonably allocated.

[0111] is the attention weight of each knowledge feature vector obtained through the above operation. This weight reflects the relative importance of the corresponding knowledge feature vector in the current task and environment. It helps the network to efficiently use key knowledge. For example, when facing the current geological conditions, the knowledge feature vectors closely related to the geological conditions will obtain high attention weights, thus playing a more important role in the decision-making process and helping the network to judge and deal with complex situations more accurately.

[0112] Knowledge storage: Knowledge is stored in the memory module in the form of weighted sum, namely:

[0113] ,

[0114] in is the number of knowledge feature vectors. This storage method can highlight important knowledge, effectively retain key information of knowledge, and facilitate subsequent retrieval and use.

[0115] It should be noted here that the attention mechanism can focus on important knowledge content, allowing the memory module to more accurately store knowledge that is key to improving network performance. Through weighted storage, the most valuable knowledge information can be retained within a limited storage space, improving the efficiency of knowledge storage and management.

[0116] It should be noted that the knowledge memory module can effectively store important knowledge learned under normal excavation topology through the above mechanism, laying a solid foundation for knowledge migration and inheritance after subsequent topology adjustment. When the network topology changes, the stored knowledge can be quickly retrieved and used, avoiding the loss of knowledge and ensuring that the network can fully utilize the existing learning results under different geological conditions to improve overall performance.

[0117] 2. Knowledge retrieval and fusion mechanism

[0118] When encountering high-hardness rock, gas layer, etc. during excavation, the network topology is adjusted to adapt to the new geological conditions. At this time, it is necessary to retrieve the previously learned relevant knowledge from the knowledge memory module and integrate it with the new knowledge learned under the new geological conditions to improve the ability to judge various situations during the excavation process and ensure the safety and efficiency of excavation.

[0119] The new topological structure generates a query vector based on the current excavation task requirements and the environmental characteristics under the current geological conditions, such as abnormal rock structure, sudden changes in equipment posture, and changes in gas concentration.

[0120] Calculate the similarity between the query vector and the knowledge feature vector stored in the memory module using the cosine similarity formula:

[0121] ,

[0122] in, is the newly generated query vector (generated by the new network topology according to the current environment characteristics and task requirements), is the knowledge feature vector stored in the memory module (representing the key knowledge learned historically), dot product To measure the consistency of the directions of two vectors, the larger the value, the closer the directions are; the product of the modulus length To normalize the vector length and eliminate the dimension effect;

[0123] Through cosine similarity calculation, the similarity between the query vector and each knowledge feature vector can be measured, thereby screening out knowledge related to the current geological conditions and excavation tasks.

[0124] Knowledge selection: Based on the similarity calculation results, knowledge feature vectors with high similarity are selected as relevant knowledge.

[0125] Knowledge fusion: Design a knowledge fusion algorithm and use a weighted fusion approach:

[0126] ,

[0127] in, is the knowledge feature vector learned under the new topological structure, is a set of selected relevant knowledge feature vectors, is the fusion coefficient, which is dynamically adjusted according to the current environment and task requirements. By dynamically adjusting the fusion coefficient, the fusion ratio of new knowledge and old knowledge can be flexibly balanced to ensure that the fused knowledge can adapt to the new environment and make full use of existing knowledge and experience.

[0128] It should be noted here that the knowledge related to the current geological conditions and excavation tasks can be accurately retrieved from the memory module through similarity calculation. The weighted fusion method can reasonably integrate the knowledge according to its relevance and importance, and the dynamic adjustment of the fusion coefficient further enhances the flexibility and adaptability of the fusion, effectively avoiding knowledge forgetting, and enabling the new topological structure to quickly adapt to the excavation environment characteristics and task requirements under the current geological conditions.

[0129] It should be noted that after the topology is adjusted, the new topology can make full use of the existing knowledge in the memory module with the help of knowledge retrieval and fusion mechanisms, and organically integrate it with the new knowledge learned under new geological conditions. This significantly improves the network's ability to identify current geological conditions and effectively reduces the misjudgment and omission of abnormal conditions during tunneling. For example, when encountering high-hardness rocks, it can use the previously learned rock property knowledge, combined with the current rock image and radar data, to accurately judge the rock properties and adjust the tunneling strategy; when monitoring changes in gas concentration, it avoids safety accidents caused by misjudgment, thereby greatly improving the safety and reliability of underground tunneling operations in coal mines and ensuring the normal progress of tunneling operations.

[0130] Further explanation is needed:

[0131] In the field of complex environment monitoring and positioning, the knowledge transfer and inheritance capabilities of technology play a key role in its performance. However, most traditional technologies have obvious shortcomings in this regard. They lack effective knowledge storage and retrieval mechanisms and find it difficult to fully utilize existing knowledge and experience under different environmental conditions. Taking traditional positioning technology as an example, once the environment changes, the knowledge learned in the past becomes difficult to reuse, and each time a new situation is faced, it needs to be relearned, which greatly reduces work efficiency. In order to more intuitively and effectively compare the differences between this method and traditional technologies in terms of knowledge transfer and inheritance, the following takes a coal mine excavation scene as an example and presents specific data in a table.

[0132] The geological conditions of a coal mine are extremely complex, and the ore properties and geological structures in different areas vary greatly. In such an environment, the problems of traditional technology in knowledge utilization are exposed. Due to the inability to accurately distinguish and utilize the knowledge under different geological conditions, when encountering new ore types, traditional technology is difficult to quickly determine its characteristics, which leads to unreasonable parameter settings of tunneling equipment, seriously affecting tunneling efficiency and quality.

[0133] In order to gain a deeper understanding of the actual performance of traditional technology in the coal mine, a three-month monitoring was carried out. During this period, five tunneling tunnels using traditional technology were selected, and detailed records were made of their situations when encountering new ore types. Through the monitoring and analysis of tunneling equipment parameters, it was found that due to the inability to accurately judge the characteristics of the ore, the matching degree between the parameters of the tunneling equipment and the actual needs was only 40%. This caused frequent problems such as equipment jamming and unstable tunneling speed during the tunneling process, resulting in an average tunneling efficiency of only 4 meters per hour. At the same time, statistics on the positioning data showed that the positioning accuracy was only 70%, and a large number of positioning deviations made it difficult to accurately control the tunneling direction, which not only increased the additional tunneling workload, but also posed a safety hazard.

[0134] This method, with its unique design, shows significant advantages in knowledge transfer and inheritance. Through the knowledge memory module based on the attention mechanism, this method can intelligently filter and store key knowledge learned under different topological structures. When faced with new geological conditions, cosine similarity calculation and dynamic weighted fusion algorithm are used for knowledge retrieval and fusion, so that the new topological structure can quickly adapt to the excavation environment characteristics and task requirements under the current geological conditions.

[0135] In this coal mine, five tunneling tunnels with geological conditions similar to those when monitoring traditional technologies were also selected, and this method was applied to tunneling operations, and strict monitoring was carried out for half a year. During the monitoring process, high-precision ore property detection equipment was used to analyze ore properties in real time, and the accuracy of the judgment of the characteristics of new ore types was calculated by comparing with the standard ore property database; professional efficiency monitoring software was used to combine equipment operation data and tunneling progress to calculate tunneling efficiency; high-precision positioning equipment was used to record the location information of tunneling equipment in real time, and the positioning accuracy was calculated by comparing with the preset standard tunneling path. The specific data is as follows: Compare Projects Traditional Technology This method Data acquisition basis Accuracy in determining the characteristics of new ore types 40% (based on the percentage of correct judgments of new ore types encountered in five tunneling tunnels using conventional technology over a three-month period) Increased to 80% (the increase was 40%. During the six-month monitoring period, when the five tunneling tunnels using this method encountered new ore types, the number of accurate judgments on their characteristics accounted for the proportion of the total number of judgments) Use high-precision ore property detection equipment to analyze ore properties in real time, compare with the standard ore property database, and statistically determine the accurate proportion Excavation efficiency 4 meters of excavation per hour (calculated based on the ratio of total excavation length to total operation time of 5 tunnels using traditional technology over a period of three months) Improved by 18% to 4.72 meters per hour (the ratio of the total excavation length to the total operation time of the five excavation tunnels using this method was calculated during the six-month monitoring period) Use professional efficiency monitoring software to combine equipment operation data and excavation progress to calculate excavation efficiency Positioning accuracy 70% (based on the statistics of multiple positioning data per hour in 5 tunnels using traditional technology within three months. Positioning errors exceeding the set range are considered positioning errors, and the ratio of the number of errors to the total number of positioning times is calculated) Increased to 90% (the increase was 20%. During the six-month monitoring period, the same positioning error judgment criteria were used in 5 tunneling tunnels that applied this method, and the proportion of positioning errors to the total positioning times was calculated) Use high-precision positioning equipment to record the location information of the tunneling equipment in real time, compare it with the preset standard tunneling path, and calculate the positioning error rate Number of misjudgments and missed judgments due to insufficient knowledge utilization (per week) 10 times (During the three-month monitoring period, the weekly misjudgment and missed judgment of 5 tunnels using traditional technology were counted, the number of misjudgments and missed judgments was recorded, and the weekly average was calculated) Reduced to 3 times (during the half-year monitoring period, the weekly misjudgment and missed judgment of the 5 tunnels using this method were counted, and the average number of times per week was calculated) By observing the misjudgment of ore characteristics and positioning deviation during excavation, the number of misjudgments and missed judgments is counted

[0136] It can be clearly seen from the above table that this method performs well in knowledge transfer and inheritance under complex geological conditions. Whether it is the significant improvement in the accuracy of judging the characteristics of new ore types, or the significant improvement in excavation efficiency and the increase in positioning accuracy, it has been strongly proved that this method can effectively solve the problem of insufficient knowledge utilization of traditional technologies in complex environment monitoring and positioning, and provide more efficient, accurate, safe and reliable technical support for coal mine excavation operations.

[0137] 3. Joint training and optimization

[0138] 1. Overall training framework

[0139] During the training phase of the underground coal mine excavation scenario, a large amount of environmental data from different excavation areas and geological conditions is collected, and the corresponding task labels are marked, such as rock type, excavation direction deviation, equipment failure type, gas concentration level, etc.

[0140] Specific steps:

[0141] Module integration: The meta-learner, neural network body, and knowledge memory module are jointly trained as a closely coordinated whole.

[0142] Data input and parameter generation: Input environmental data into the meta-learner, and the meta-learner generates topological structure adjustment parameters based on environmental information.

[0143] Forward propagation and task loss calculation: The neural network performs forward propagation according to the adjusted topology, processes the input data, and calculates the task loss. For the excavation direction positioning task, the cross entropy loss function is used:

[0144] ,

[0145] in, is the sample size, is the number of categories, It is a sample Belongs to category The true label (0 or 1), is the probability that the model predicts that the sample belongs to the category.

[0146] Memory module loss calculation: At the same time, the storage and retrieval losses of the knowledge memory module are calculated For example, the loss is calculated by measuring the difference between the retrieved knowledge and the actual demand. Assume that the retrieved knowledge feature vector is , the knowledge feature vector of actual demand is , using mean square error loss:

[0147] ,

[0148] in, is the number of samples when calculating the loss, For the The knowledge feature vector retrieved from the memory module in the sample (i.e., the knowledge filtered and integrated by the network through cosine similarity); For the The knowledge feature vector of actual requirements in samples (i.e., the real knowledge required for the current environment and task); Calculate the square of the difference between the retrieved knowledge and the actual required knowledge for each sample.

[0149] Total loss calculation and parameter update: Total loss:

[0150] ,

[0151] In joint training, With mission loss The total loss , the parameters of the memory module are optimized through back-propagation, making it more efficient in storing and retrieving key knowledge.

[0152] in is a weight coefficient used to balance task loss and memory loss. Through the back-propagation algorithm, the parameters of the meta-learner, neural network body, and knowledge memory module are updated according to the total loss, so that the network's task execution ability under different topological structures is continuously improved, the accuracy and efficiency of the meta-learner's topological structure adjustment parameters are continuously optimized, and the effectiveness of the knowledge memory module in storing and retrieving knowledge is also significantly enhanced.

[0153] It should be noted here that joint training can form a close cooperative relationship between the modules, and realize the coordinated optimization of the parameters of each module through the back propagation of the total loss. The back propagation of the task loss guides the neural network to optimize the task execution ability, and the back propagation of the memory module loss prompts the knowledge memory module to better store and retrieve knowledge, thereby comprehensively improving the performance of the network.

[0154] It should be noted that the overall training framework improves the network's ability to perform tasks under different geological conditions, enhances the accuracy and efficiency of the meta-learner in generating topological structure adjustment parameters, and optimizes the effectiveness of knowledge storage and retrieval in the knowledge memory module. This enables the network to more accurately identify geological conditions, locate excavation directions, and adapt to different geological conditions in the complex and changeable coal mine underground excavation environment, providing more reliable support for coal mine underground excavation operations.

[0155] 2. Multi-objective optimization

[0156] In the underground coal mine excavation scenario, it is necessary to balance the network's excavation direction positioning accuracy, topology adjustment speed, and knowledge transfer inheritance effect under different geological conditions. For example, when encountering complex geological structures, the network is required to quickly and accurately adjust the excavation direction; in areas with different rock types, it is necessary to make full use of knowledge transfer inheritance to improve positioning accuracy.

[0157] Specific steps:

[0158] Objective function setting: Setting the objective function, including the network’s positioning accuracy in the direction of excavation under different geological conditions , calculated by the deviation between the actual excavation direction and the preset direction; the time cost of topological structure adjustment , measured by recording the time it takes for the meta-learner to generate adjustment parameters and the time it takes for the neural network to complete topological structure adjustment; the contribution of knowledge fusion to the improvement of tunneling performance , by comparing the improvement in excavation efficiency and the reduction in positioning error before and after fusion, the evaluation is carried out, such as the increase in excavation speed and the reduction in directional deviation after fusion.

[0159] In the method, setting the objective function is of great significance for optimizing network performance and adapting to the complex and changeable tunneling environment in coal mines. The objective function covers several key indicators, including the accuracy of the network's tunneling direction positioning under different geological conditions, the time cost of topological structure adjustment, and the contribution of knowledge fusion to the improvement of tunneling performance. In actual application scenarios, rocks of different hardness and different gas concentration environments will have a significant impact on tunneling operations. Compared with traditional methods, this method shows remarkable improvements in multiple key indicators under these complex environments. The following is a detailed comparison of experimental data to intuitively present the advantages of this method.

[0160] In order to comprehensively and accurately evaluate the performance differences between this method and traditional methods in different environments, a large-scale experimental monitoring was carried out for 6 months in multiple coal mines and actual coal mine excavation sites. The experiment carefully selected rock areas with different hardness (low hardness areas with rock hardness coefficients of 2-4 and high hardness areas with rock hardness coefficients of 6-8) and areas with different gas concentrations (low gas concentration environment and high gas concentration environment) for comparative testing. Throughout the experiment, the experimental conditions were strictly controlled to ensure that all experimental equipment and initial conditions were basically the same. The only variable was the positioning technology used, so as to ensure the scientificity and reliability of the experimental results.

[0161] During the experimental execution phase, a series of high-precision monitoring equipment and rigorous data collection and analysis methods were used. By installing high-precision positioning sensors, the positioning accuracy of the excavation direction is monitored in real time and accurately; using professional timers, the topology adjustment time is accurately recorded; with the help of devices that monitor the operating speed of excavation equipment, the excavation speed increase ratio is accurately calculated; through equipment energy consumption monitoring devices, the equipment energy consumption data is comprehensively counted, and then the energy consumption reduction ratio is calculated. In order to further ensure the accuracy and reliability of the data, multiple rounds of comparative analysis were carried out on the collected data, and the average value was finally taken as the final experimental result. The specific quantitative comparison data is shown in the following table: Compare Projects The rock hardness coefficient is 2-4 (low hardness) The rock hardness coefficient is 6-8 (high hardness) Low gas concentration environment High gas concentration environment Improved positioning accuracy in excavation direction 25% (70% for traditional method, 95% for this method) 30% (60% for traditional method, 90% for this method) 20% (80% for traditional method, 100% for this method) 28% (72% for traditional method, 100% for this method) Shortened topology adjustment time About 80% (10 minutes for traditional method, 2 minutes for this method) About 80% (10 minutes for traditional method, 2 minutes for this method) About 80% (10 minutes for traditional method, 2 minutes for this method) About 80% (10 minutes for traditional method, 2 minutes for this method) Excavation speed increase ratio 35% (the traditional method excavates 5 meters per hour, while this method excavates 6.75 meters per hour) 40% (the traditional method excavates 4 meters per hour, while this method excavates 5.6 meters per hour) 30% (the traditional method excavates 5.5 meters per hour, while this method excavates 7.15 meters per hour) 38% (the traditional method excavates 5 meters per hour, while this method excavates 6.9 meters per hour) Equipment energy consumption reduction ratio 20% (the traditional method consumes 100 units of energy per meter of excavation, while this method consumes 80 units of energy per meter of excavation) 25% (the traditional method consumes 120 units of energy per meter of excavation, while this method consumes 90 units of energy per meter of excavation) 18% (the traditional method consumes 95 units of energy per meter of excavation, while this method consumes 78 units of energy per meter of excavation) 22% (the traditional method consumes 105 units of energy per meter of excavation, while this method consumes 82 units of energy per meter of excavation)

[0162] It can be clearly seen from the above table data that this method has demonstrated excellent performance in both rock environments with different hardness and gas concentrations. In terms of the accuracy of positioning the excavation direction, it has been significantly improved compared to traditional methods, which means that the excavation direction can be controlled more accurately, effectively reducing excavation deviations and ensuring the safety and efficiency of excavation operations. The topology adjustment time is greatly shortened, allowing the network to quickly adapt to environmental changes and adjust itself in time to better handle excavation tasks in new environments. The significant increase in excavation speed and the reduction in equipment energy consumption not only improve excavation efficiency, but also reduce production costs, which has important economic value. These data fully demonstrate the great advantages of this method under complex geological conditions, and can effectively solve the problems of traditional technologies in underground coal mine excavation operations, bringing higher benefits and more reliable technical support to the coal mining industry.

[0163] Optimization algorithm selection: A multi-objective optimization method based on the non-dominated sorting genetic algorithm (NSGA-II) is used. The algorithm searches and optimizes network parameters and structures by simulating natural selection and genetic evolution processes. In each generation of evolution, the algorithm performs non-dominated sorting on individuals in the population, divides individuals into different levels, and preferentially selects individuals with high levels (i.e., non-dominated solutions) for genetic operations such as crossover and mutation to generate new populations. At the same time, the diversity of the population is maintained through crowding calculation to prevent the algorithm from falling into a local optimal solution.

[0164] Objective function calculation and optimization: The objective function can be expressed as:

[0165] ,

[0166] in, , , is the weight coefficient, which is adjusted according to actual needs. In the optimization process, through continuous iteration, the algorithm searches for the optimal network parameters and structure so that the objective function reach the optimal value.

[0167] It should be noted here that the multi-objective optimization algorithm can balance and optimize between multiple conflicting objectives, avoiding focusing on a single objective and ignoring other important factors. The NSGA-II algorithm has good global search capabilities and convergence, and can effectively search for the optimal solution in a complex parameter space, so that the network can achieve good comprehensive performance under different geological conditions.

[0168] In actual tests of multiple coal mines, the application of this method has shown excellent performance under different geological conditions. For example, in the excavation operation of a coal mine, a complex fault and fold geological area was encountered. The positioning accuracy of the excavation direction of the traditional fixed topology neural network in this area was only 60%, and because it could not adapt to geological changes in time, the excavation efficiency was low, the daily excavation progress was slow, and the equipment damage caused by positioning deviation occurred frequently. However, in the same complex geological area, the positioning accuracy of the excavation direction of this method can be stably maintained at more than 85%. In the face of geological changes, the system can complete the topological structure adjustment in about 2 minutes, quickly adapt to new geological conditions, effectively reduce the number of errors in the excavation direction, and significantly improve the excavation efficiency. The daily excavation progress is nearly 30% higher than that of the traditional method, and the number of equipment damages caused by positioning deviations is also greatly reduced, from 5 times a week to less than 1 time a week, which greatly improves the safety and efficiency of coal mining.

[0169] In addition, during the tunneling process in coal mines, unexpected situations such as sensor failure and data transmission errors are difficult to avoid. The joint training and optimization mechanism of this method uses a multi-objective optimization algorithm to balance the various performance indicators of the network, so that the network can still maintain a certain recognition accuracy and stability when facing these accidents. For example, when the visual camera is blocked by dust and data is lost, the knowledge transfer based on the memory mechanism can combine the data of the millimeter-wave radar and inertial navigation equipment, and use the previously stored knowledge to continue target identification and positioning, ensuring the continuation of tunneling operations. Even in the event of failure of some sensors, the system can still maintain a certain accuracy in positioning the tunneling direction, avoiding tunneling accidents caused by data anomalies, and greatly improving the robustness of the entire system, which is difficult for traditional neural networks to achieve.

[0170] In the development history of neural network technology, the fixed topological structure of traditional self-supervised neural networks faces many challenges in the complex dynamic environment of underground coal mine excavation. Its internal connections and processing procedures are basically fixed after training, which makes it difficult to adapt to the changing geological conditions, resulting in large deviations in the excavation direction, low efficiency, and inability to effectively utilize existing knowledge and experience. This method introduces a meta-learner based on a long short-term memory network (LSTM), which enables it to deeply analyze the dynamic changes in the excavation environment state, learn the complex mapping relationship between the environment and the topological structure, and give the neural network the ability to dynamically adjust the topological structure in real time, so that the network can quickly capture new environmental features and positioning clues when facing changing geological conditions, greatly improving the adaptability of neural networks in the underground coal mine excavation environment.

[0171] In the field of knowledge transfer and inheritance, existing neural network theories and methods have obvious deficiencies in underground coal mine excavation scenarios. The designed knowledge memory module based on the attention mechanism can intelligently filter and store key knowledge learned under different topological structures. At the same time, the knowledge retrieval and fusion mechanism of cosine similarity calculation and dynamic weighted fusion algorithm is used to successfully realize the effective transfer and inheritance of knowledge between different topological structures, filling the gap in the application of this field in underground coal mines, and opening up a new path for the continuous learning and performance improvement of neural networks under complex geological conditions.

[0172] In summary, this dynamic adaptive self-supervised neural network architecture technology solution based on meta-learning and memory mechanism has successfully overcome the topological adaptation problem of self-supervised neural networks in underground coal mine excavation environments, as well as the problems of knowledge transfer and inheritance. It has demonstrated excellent performance and broad application prospects in practical applications, providing strong support for the development of underground coal mine excavation technology, and is expected to promote the intelligent upgrade of the entire coal industry. In the future, with the continuous development and improvement of technology, this technology can be further expanded and applied to other similar underground mining scenarios, such as coal mining, tunnel excavation and other fields, bringing new opportunities for technological innovation in related industries.

Claims

1. A visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network, characterized in that: The following steps are involved: Construct a meta-learner based on long short-term memory network: The input layer of the meta-learner receives the processed environmental feature vector, which is obtained by extracting the underground tunnel image data collected by the visual camera through convolutional neural network features, normalizing the millimeter-wave radar data, and calibrating and preprocessing the inertial navigation equipment data; The middle hidden layer is composed of multiple LSTM units, which use the gating mechanism to control the inflow, outflow and memory of information; the output layer is based on the output of the hidden layer, and generates the network topology adjustment parameters for the current tunneling environment state through the fully connected layer calculation. The parameters include the increase and decrease coefficients of neuron connections, the adjustment factors of the inter-layer connection weights, and the adjustment values ​​of the number of neurons in the hidden layer; When the state of the underground excavation environment in a coal mine changes, the new state of the environment information is collected in real time and input into the trained meta-learner; the meta-learner calculates and outputs the topological structure adjustment parameters based on the learned mapping relationship, and the neural network quickly adjusts its own topological structure based on the parameters, including adjusting the connection weights between neurons, and increasing or decreasing the number of hidden layer neurons related to motion feature processing according to the posture changes of the excavation equipment and geological conditions; The attention mechanism is used to analyze the feature graphs of the middle layer of the network, extract important knowledge feature vectors related to the rock structure and motion state of tunneling equipment in coal mines, calculate the attention weight of each knowledge feature vector, and store it in the knowledge memory module in the form of weighted sum; When the network topology is adjusted, the new topology generates query vectors based on the current tunneling task requirements and environmental characteristics under current geological conditions; Calculate the similarity between the query vector and the knowledge feature vector stored in the memory module, select the knowledge feature vector with high similarity, and fuse it with the knowledge feature vector learned under the new topological structure by weighted fusion. The fusion coefficient is dynamically adjusted according to the current excavation environment and task requirements. The meta-learner, the neural network body, and the knowledge memory module are jointly trained as a whole; the environmental data of different tunneling areas and different geological conditions in the coal mine are input to the meta-learner, the meta-learner generates topological structure adjustment parameters, the neural network forward propagates and calculates the task loss according to the adjusted topological structure, and at the same time calculates the storage and retrieval loss of the knowledge memory module, and updates the parameters of each module through the total loss; Set the objective function, including the network's positioning accuracy in the tunneling direction under different geological conditions, the time cost of topological structure adjustment, and the contribution of knowledge fusion to the improvement of tunneling performance; A multi-objective optimization method based on non-dominated sorting genetic algorithm is used to search and optimize network parameters and structure.

2. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: The convolutional neural network uses pre-trained convolution kernels, including but not limited to VGG16 and ResNet series, to convert the underground tunnel image data collected by the visual camera into a feature vector with semantic information.

3. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: Millimeter-wave radar data normalization is to map values ​​such as distance and speed to the [0,1] interval.

4. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: The state update formula of the LSTM hidden layer includes: ， in, is the input gate vector; is the weight matrix from the input layer to the corresponding gate and memory unit, is the weight matrix from the hidden layer to the corresponding gate and memory unit, is the corresponding bias vector, is the Sigmoid activation function.

5. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The output topology adjustment parameters are calculated by the output layer using the formula: ， in, is the weight matrix from the hidden layer to the output layer, is the bias vector of the output layer.

6. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The formula for calculating the attention weight of each knowledge feature vector in the attention mechanism is: ， in, and are all knowledge feature vectors, Represents the knowledge feature vector currently being calculated for attention weights, Represents each vector in the set of all knowledge feature vectors stored in the memory module, It is a query vector, which is used to measure the importance of each knowledge feature vector. The attention weight of each feature vector is obtained by performing a dot product operation with the knowledge feature vector and normalizing it with the Softmax function.

7. The visual inertial navigation radar fusion self-positioning method based on a self-supervised neural network as claimed in claim 1, characterized in that: The cosine similarity formula is used to calculate the similarity between the query vector and the knowledge feature vector stored in the memory module: ， in, is the newly generated query vector, is the knowledge feature vector stored in the memory module, dot product To measure the consistency of the directions of two vectors, the larger the value, the closer the directions are; the product of the modulus length To normalize the vector length and eliminate the dimension effect.

8. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: Knowledge fusion adopts the weighted fusion method, and the formula is: ， in, is the knowledge feature vector learned under the new topological structure, is a set of selected relevant knowledge feature vectors, is the fusion coefficient, which is dynamically adjusted according to the current environment and task requirements.

9. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: For the target recognition task, the cross entropy loss function is used to calculate the task loss: ， in, is the sample size, is the number of categories, It is a sample Belongs to category The real label, is the probability that the model predicts that the sample belongs to the category.

10. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The storage and retrieval losses of the knowledge memory module are calculated using mean square error loss: ， in, is the number of samples when calculating the loss, is the knowledge feature vector, is the knowledge feature vector of actual needs, For the The knowledge feature vector retrieved from the memory module in the sample is the knowledge that the network has filtered and integrated through cosine similarity; For the The knowledge feature vector actually required in the samples is the real knowledge required for the current environment and task.

11. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The total loss is calculated as: ， in is a weight coefficient used to balance task loss and memory loss.

12. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: The objective function is expressed as: ， in, , , is the weight coefficient.

13. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: In the multi-objective optimization method, in each generation of evolution, individuals in the population are non-dominated and sorted, and the individuals are divided into different levels. Individuals with high levels are preferentially selected for genetic operations, including but not limited to crossover and mutation, to generate new populations, and the diversity of the population is maintained by crowding calculation.

14. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: Environmental data includes environmental data in different excavation areas and under different geological conditions, and is marked with corresponding task labels, including but not limited to rock type, excavation direction deviation, equipment failure type, and gas concentration level.

15. The visual inertial navigation radar fusion self-positioning method based on self-supervised neural network as claimed in claim 1, characterized in that: Adjusting the neural network topology includes adding or deleting neuron connections, strengthening or weakening the strength of inter-layer connections, and dynamically increasing or decreasing the number of hidden layer neurons.

Images