Rapid tunneling system state real-time three-dimensional visualization and fault prediction method

Abstract

The invention belongs to the field of mine equipment digital twinborn visualization and artificial intelligence, and provides a rapid tunneling system state real-time three-dimensional visualization and fault prediction method in order to solve the problem of lack of oneness of real-time reflection and fault prediction of a rapid tunneling system state at present. Then real-time operation data are collected to update the digital twinborn model in real time, so that an operator can visually see the operation state of the rapid tunneling system and find potential problems in time, meanwhile, a fault prediction model is constructed to analyze operation state information generated by the digital twinborn model, and a fault prediction result of the rapid tunneling system is obtained; according to the method, the potential risk points of the rapid tunneling system can be identified, fault early warning is realized, sudden faults are reduced, and the maintenance period of equipment is optimized.

Description

Technical Field

[0001] The present invention belongs to the field of digital twin visualization and artificial intelligence of mine equipment, and specifically relates to a method for real-time three-dimensional visualization and fault prediction of the status of a rapid tunneling system. Background Art

[0002] The tunneling face is a crucial area for excavation, support, and transportation in underground coal mines. Rapid tunneling systems are the primary production tool for these areas, and their efficiency and safety are crucial to the progress and quality of the entire project. However, the tunneling face operates in a complex environment characterized by high dust concentrations, poor ventilation, and a diverse and complex distribution of equipment. These factors make it difficult for traditional manual inspection and monitoring methods to fully and timely capture the dynamic conditions at the worksite.

[0003] In recent years, the development of digital twins, 3D visualization, and artificial intelligence technologies has provided new insights into the condition monitoring and fault management of tunneling systems. Digital twins combine physical systems with virtual models to map the system's operating status in real time, providing a panoramic, dynamic 3D visualization. Furthermore, deep learning-based fault prediction technologies can uncover underlying patterns in historical data, enabling early diagnosis and early warning of system failures.

[0004] Existing visualization methods for rapid tunneling systems are usually limited to modeling the rapid tunneling system and lack real-time reflection of the rapid tunneling system status; while fault prediction methods are usually limited to single-dimensional data collection, and are mostly presented in the form of two-dimensional charts or text reports, lacking an intuitive display and comprehensive perception of the mine's overall environment and potential hazards. Summary of the Invention

[0005] In order to solve at least one of the above-mentioned technical problems existing in the prior art, the present invention provides a method for real-time three-dimensional visualization and fault prediction of a rapid tunneling system state.

[0006] The present invention is implemented by the following technical solution: a method for real-time three-dimensional visualization and fault prediction of a rapid tunneling system state, comprising the following steps:

[0007] S1: Acquire historical data of the rapid tunneling system during operation, wherein the historical data at least includes dimensional parameters of the rapid tunneling system, historical working video data, historical operation data, and historical environmental data;

[0008] S2: Based on the historical data and using motion capture technology to extract the operating motions of the rapid tunneling system, a digital twin model of the rapid tunneling system is constructed;

[0009] S3: Acquire real-time operating data of the rapid tunneling system, transmit the real-time operating data to the digital twin model, and obtain operating status information of the rapid tunneling system;

[0010] S4: Use convolutional neural networks and long short-term memory networks to build an initial fault prediction model, and use the historical data from step S1 to train the model to obtain a trained fault prediction model;

[0011] S5: Inputting the operating status information into the fault prediction model to obtain a fault prediction result of the rapid tunneling system.

[0012] Preferably, the step S2 includes:

[0013] Preprocessing the historical working video data, the preprocessing at least including video stabilization, denoising and contrast adjustment;

[0014] Extract key frames from pre-processed historical working video data, and then use motion capture technology to determine the operating movements of the rapid tunneling system;

[0015] A digital twin model is constructed by combining the dimensional parameters, historical operation data, historical environmental data and operation actions of the rapid tunneling system.

[0016] Preferably, the historical operating data includes at least one of temperature, vibration frequency, load, operating efficiency and mechanical loss; the historical environmental data includes at least one of mine humidity, temperature and rock hardness.

[0017] Preferably, in step S4, the fault prediction model includes:

[0018] 1D convolutional layer, used to extract channel features of multi-dimensional data;

[0019] The LSTM module is used to mine the temporal dependency relationship based on the channel characteristics and output the fault prediction result.

[0020] Preferably, the LSTM module implements dynamic modeling of temporal features through a forget gate, an input gate, a memory unit, and an output gate, specifically including:

[0021] The forget gate is used to determine the running data that needs to be retained;

[0022] The input gate is used to determine the input value of the memory unit;

[0023] The memory unit is used to update the memory unit state according to the operating data and the input value;

[0024] The output gate is used to generate the hidden state of the current time step.

[0025] Preferably, step S5 further includes:

[0026] The fault prediction result is compared with a preset safety threshold range. If the fault prediction result is not within the preset safety threshold range, an audible and visual alarm is triggered, or a maintenance suggestion is generated.

[0027] Preferably, step S2 further includes:

[0028] The three-dimensional visualization display is performed through the digital twin model, including at least the real-time motion trajectory, real-time operating status and dynamic rendering of environmental adaptability parameters of the rapid tunneling system.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] This invention uses a digital twin model to monitor all components of the rapid tunneling system in real time. It also presents the system's operating status through 3D visualization, enabling operators to visually visualize the system's operational status and promptly identify potential problems. By utilizing convolutional neural networks and long-short-term memory networks to construct a fault prediction model, the system analyzes operational data and historical failure patterns, predicting potential failures in advance. Using artificial intelligence algorithms, the system can identify potential risk points, provide early warnings, reduce the occurrence of sudden failures, and optimize equipment maintenance cycles. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 It is a flow chart of a method for real-time three-dimensional visualization and fault prediction of a rapid tunneling system state provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present invention are clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other implementations derived by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts are within the scope of protection of the present invention.

[0034] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions under which the present invention can be implemented. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should fall within the scope of the technical content disclosed in the present invention without affecting the efficacy and purpose that can be achieved by the present invention. It should be noted that in this specification, relational terms such as first and second are only used to distinguish one entity from several other entities, and do not necessarily require or imply any actual relationship or order between these entities.

[0035] Existing visualization methods for rapid tunneling systems are usually limited to modeling the rapid tunneling system and lack real-time reflection of the rapid tunneling system status; while fault prediction methods are usually limited to single-dimensional data collection, and are mostly presented in the form of two-dimensional charts or text reports, lacking an intuitive display and comprehensive perception of the mine's overall environment and potential hazards.

[0036] Therefore, in order to solve the above problems, the present invention provides a real-time three-dimensional visualization and fault prediction method for the status of a rapid tunneling system. By constructing a digital twin model, the working status of the rapid tunneling system can be accurately simulated and displayed in real time. At the same time, through the analysis of historical data and the prediction of deep learning models, potential faults or abnormal working conditions can be effectively warned.

[0037] In order to more clearly introduce the above-mentioned objects, features and advantages of the present invention, further detailed description is given below with reference to the accompanying drawings and specific embodiments.

[0038] like Figure 1 As shown, the present invention provides a flowchart of a method for real-time three-dimensional visualization and fault prediction of a rapid tunneling system state, including the following steps:

[0039] S1: Acquire historical data of the rapid tunneling system during operation, wherein the historical data at least includes dimensional parameters of the rapid tunneling system, historical working video data, historical operation data, and historical environmental data.

[0040] In this embodiment, multi-dimensional data related to the rapid tunneling system in a mine is acquired, including dimensional parameters, historical operating video data, historical operational data, and historical environmental data. Specifically, the dimensional parameters of the rapid tunneling system may include length, width, height, and weight; historical operating video data can be captured using a 360-degree panoramic camera to record the equipment in operation within the mine; historical operational data includes temperature, vibration frequency, load, etc. Historical operational data includes not only the input variables of the rapid tunneling system but also the actual output results of the rapid tunneling system, such as the rapid tunneling system's operating efficiency or mechanical losses; and historical environmental data includes the mine's humidity, temperature, rock hardness, etc.

[0041] S2: Based on the historical data and using motion capture technology to extract the operating actions of the rapid tunneling system, a digital twin model of the rapid tunneling system is constructed.

[0042] Optionally, it includes: preprocessing the historical work video data, the preprocessing including at least video stabilization, denoising and contrast adjustment; extracting key frames from the preprocessed historical work video data, and then using motion capture technology to determine the operating actions of the rapid tunneling system; combining the dimensional parameters, historical operating data, historical environmental data and operating actions of the rapid tunneling system to construct a digital twin model.

[0043] In this example, historical operational video data is preprocessed, including video stabilization, noise removal, and contrast and brightness adjustments, to improve the accuracy of subsequent analysis. Key frames of the rapid tunneling system's operation are then extracted from the continuous historical operational video data. Motion capture technology is used to identify and track the equipment's motion trajectory within these key frames, determining the system's operational actions. Finally, 3D modeling software is used to combine the system's dimensional parameters, historical operational data, historical environmental data, and the captured operational actions to construct a realistically reproduced digital twin model. In practical applications, the 3D modeling software used can be Blender, Maya, or other similar software.

[0044] S3: Acquire real-time operating data of the rapid tunneling system, transmit the real-time operating data to the digital twin model, and obtain operating status information of the rapid tunneling system.

[0045] Optionally, a three-dimensional visualization display is performed through the digital twin model, including at least dynamic rendering of the real-time motion trajectory, real-time operating status and environmental adaptability parameters of the rapid tunneling system.

[0046] In this embodiment, multiple sensors installed on the rapid tunneling system collect real-time data during operation and feed it into the digital twin model. This dynamically updates the digital twin model, enabling it to accurately reflect the current operating status of the rapid tunneling system and provide an accurate 3D visualization. The digital twin model dynamically reflects the actual operating status of the rapid tunneling system in the mine environment, including the equipment's real-time motion trajectory, operating status, and environmental adaptability.

[0047] S4: Use convolutional neural networks and long short-term memory networks to build an initial fault prediction model, and use the historical data from step S1 to train the model to obtain a trained fault prediction model.

[0048] Optionally, the fault prediction model includes: a 1D convolution layer for extracting channel features of multi-dimensional data; and an LSTM module for mining temporal dependencies based on the channel features and outputting fault prediction results.

[0049] Optionally, the LSTM module implements dynamic modeling of timing features through a forget gate, an input gate, a memory unit and an output gate, specifically including: the forget gate is used to determine the running data that needs to be retained; the input gate is used to determine the input value of the memory unit; the memory unit is used to update the memory unit state according to the running data and the input value; and the output gate is used to generate the hidden state of the current time step.

[0050] In this example, a 1D convolutional feature extractor is constructed to extract channel features from the input data. The feature vector is then fed into an LSTM module to obtain deeper temporal features. By combining convolutional neural networks (CNNs) with long short-term memory (LSTM) networks, the dynamic characteristics and temporal relationships of the rapid tunneling system are extracted, further improving fault prediction accuracy. This stacking of models forms the feature extractor for the entire deep neural network.

[0051] The input data for each indicator is batch normalized and then fed into the fault prediction model. The fault prediction model is trained using mean square error and cross entropy to optimize weights and biases. This training improves the stability of the fault prediction model and reduces overfitting. The specific steps are as follows:

[0052] S41: Calculate the forget gate: First, calculate the output of the forget gate based on the current input and the hidden state of the previous time step to determine which information in the current memory unit needs to be forgotten.

[0053] S42: Calculate input gate: Then calculate the value of the input gate and the candidate memory unit value to determine which of the current input information should be added to the memory unit.

[0054] S43: Update the memory cell state: Add the results of the forget gate and the input gate to update the memory cell state. This step ensures that the memory cell retains useful information while also introducing new information.

[0055] S44: Calculate the output gate: Finally, based on the current input and memory cell state, calculate the value of the output gate to obtain the new hidden state. This hidden state will be passed as the output to the next time step and will also serve as the output of the current time step.

[0056] In this embodiment, the forget gate is used to control which information in the current memory unit needs to be forgotten. Its calculation method is:

[0057] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0058] Where, f t is the output of the forget gate, W f is the weight matrix of the forget gate, h t-1 is the hidden state of the previous time step, x t is the input of the current time step, b f is the bias vector of the forget gate, and σ is the sigmoid activation function.

[0059] In this embodiment, the input gate determines the extent to which the current input information is written into the memory cell. The calculation process consists of two parts. First, the write ratio is generated:

[0060] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0061] Where i t is the value of the input gate, W i is the weight matrix of the input gate, b i is the bias vector of the input gate.

[0062] Then generate a new candidate value and activate it through the tanh function:

[0063]

[0064] Where, is the candidate memory cell value, Wc is the weight matrix of the memory unit, b c is the bias vector of the memory cell.

[0065] Then, the value of the input gate is multiplied by the candidate memory cell value to obtain the updated input value of the memory cell.

[0066] In this embodiment, the memory unit state update refers to combining the results of the forget gate and the input gate to update the memory unit state. The calculation process is:

[0067]

[0068] Where C t-1 is the state value of the memory cell at the previous time step.

[0069] In this embodiment, the output gate determines the output information of the current time step, and its calculation process is:

[0070] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0071] In the formula, o t is the value of the input gate, W o is the weight matrix of the input gate, b o is the bias vector of the input gate.

[0072] The memory cell state is then processed through the tanh activation function and multiplied by the value of the output gate to obtain the final hidden state output:

[0073] h t =o t tanh(C t )

[0074] Where h t is the hidden state at the current time step.

[0075] In this embodiment, a minimization loss function is used to train the fault prediction model and adjust the parameters of the fault prediction model. In this embodiment, a gradient descent method is used.

[0076] S5: Inputting the operating status information into the fault prediction model to obtain a fault prediction result of the rapid tunneling system.

[0077] Optionally, the method further includes comparing the fault prediction result with a preset safety threshold range, and if the fault prediction result is not within the preset safety threshold range, triggering an audible and visual alarm, or generating a maintenance suggestion.

[0078] In this embodiment, the numerical values of various indicators are input into a trained fault prediction model to predict the values of various indicators of the rapid tunneling system at the next moment. If an abnormality is likely, a warning is issued. Specifically, after the data is input into the fault prediction model, the equipment status is predicted through forward propagation.

[0079] This invention uses a digital twin model to monitor all components of the rapid tunneling system in real time. It also presents the system's operating status through 3D visualization, enabling operators to visually visualize the system's operational status and promptly identify potential problems. By utilizing convolutional neural networks and long-short-term memory networks to construct a fault prediction model, the system analyzes operational data and historical failure patterns, predicting potential failures in advance. Using artificial intelligence algorithms, the system can identify potential risk points, provide early warnings, reduce the occurrence of sudden failures, and optimize equipment maintenance cycles.

[0080] The foregoing description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed herein should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for real-time 3D visualization and fault prediction of a rapid tunneling system, characterized in that: The steps include: S1: Acquire historical data of the rapid tunneling system during operation, wherein the historical data at least includes dimensional parameters of the rapid tunneling system, historical working video data, historical operation data, and historical environmental data; S2: Based on the historical data and using motion capture technology to extract the operating motions of the rapid tunneling system, a digital twin model of the rapid tunneling system is constructed; S3: Acquire real-time operating data of the rapid tunneling system, transmit the real-time operating data to the digital twin model, and obtain operating status information of the rapid tunneling system; S4: Use convolutional neural networks and long short-term memory networks to build an initial fault prediction model, and use the historical data from step S1 to train the model to obtain a trained fault prediction model; S5: Inputting the operating status information into the fault prediction model to obtain a fault prediction result of the rapid tunneling system.

2. A method for real-time three-dimensional visualization and fault prediction of a rapid tunneling system according to claim 1, characterized in that: Said S2 comprises: Preprocessing the historical working video data, the preprocessing at least including video stabilization, denoising and contrast adjustment; Extract key frames from pre-processed historical working video data, and then use motion capture technology to determine the operating movements of the rapid tunneling system; A digital twin model is constructed by combining the dimensional parameters, historical operation data, historical environmental data and operation actions of the rapid tunneling system.

3. The method for real-time 3D visualization and fault prediction of a rapid tunneling system according to claim 1, characterized in that: The historical operation data includes at least one of temperature, vibration frequency, load, operation efficiency and mechanical loss; the historical environmental data includes at least one of mine humidity, temperature and rock hardness.

4. The method for real-time 3D visualization and fault prediction of a rapid tunneling system according to claim 1, characterized in that: In step S4, the fault prediction model includes: 1D convolutional layer, used to extract channel features of multi-dimensional data; The LSTM module is used to mine the temporal dependency relationship based on the channel characteristics and output the fault prediction result.

5. The method for real-time 3D visualization and fault prediction of a rapid tunneling system according to claim 4, characterized in that: The LSTM module implements dynamic modeling of temporal features through forget gate, input gate, memory unit and output gate, specifically including: The forget gate is used to determine the running data that needs to be retained; The input gate is used to determine the input value of the memory unit; The memory unit is used to update the memory unit state according to the operating data and the input value; The output gate is used to generate the hidden state of the current time step.

6. The method for real-time three-dimensional visualization and fault prediction of a rapid tunneling system according to claim 1, characterized in that: Step S5 further includes: The fault prediction result is compared with a preset safety threshold range. If the fault prediction result is not within the preset safety threshold range, an audible and visual alarm is triggered, or a maintenance suggestion is generated.

7. The method for real-time 3D visualization and fault prediction of a rapid tunneling system according to claim 1, characterized in that: Step S3 further includes: The three-dimensional visualization display is performed through the digital twin model, including at least the real-time motion trajectory, real-time operating status and dynamic rendering of environmental adaptability parameters of the rapid tunneling system.

Images