Coal mine equipment state perception method based on large model and digital twinning

Abstract

The application discloses a coal mine equipment state perception method based on a large model and digital twinning, relates to the technical field of state perception, and comprises the following steps: inputting a multi-modal data set into a quantum convolutional encoder, compressing features through a quantum state superposition principle, simultaneously selecting key feature fragments by using an attention gate mechanism, and generating a multi-scale digital twin of the equipment; according to the multi-scale digital twin, a dynamic correlation model between equipment components is established by using a space-time graph Transformer network, the interaction strength between components is quantified by using a graph attention mechanism, and a neural differential equation is used to describe the equipment state evolution process to obtain an equipment state evolution model; the equipment state evolution model is combined with a generative adversarial network, and a generator is used to simulate the equipment degradation path; and the application adaptively filters key dynamic fragments through the attention gate mechanism, breaks through the classical convolution serial bottleneck and the limitation of the fixed attention rule, realizes exponential dimension reduction, and accurately captures the degradation-related space-time pattern.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of state perception, and in particular to a coal mine equipment state perception method based on large models and digital twinning. BACKGROUND

[0002] With the rapid development of industry and intelligent manufacturing, coal mine equipment state perception technology plays an important role in improving production efficiency, ensuring safe operation and reducing maintenance costs. Traditional state perception methods mainly rely on sensor data acquisition and rule-based fault diagnosis technology. These methods achieve state evaluation by real-time monitoring of equipment operating parameters, and to some extent support maintenance decisions for coal mine equipment. With the development of Internet of Things and big data technology, perception methods based on machine learning have gradually emerged, such as using support vector machines or traditional neural networks to classify and predict equipment states. In addition, digital twinning technology, as a new industrial modeling method, has shown significant potential in equipment monitoring, fault prediction and operation optimization by constructing virtual models of equipment and mapping states with real-time data. In particular, digital twinning combined with machine learning can achieve dynamic simulation and analysis of equipment operating states, providing a new approach to state perception under complex conditions.

[0003] However, there is still room for improvement in existing state perception methods. First, existing multi-scale digital twinning construction schemes usually rely on classical convolution or graph convolution networks, and the feature compression process lacks the exponential parallel advantage of quantum superposition. Moreover, the key dynamic segment screening still mainly relies on fixed attention or manual rules, making it difficult to adaptively capture time-space patterns highly related to equipment degradation, resulting in insufficient expression of local anomalies by digital twins. Second, equipment state evolution modeling mostly uses discrete-time recurrent networks or ordinary differential equation neural networks. The former ignores the continuous interaction of component graph structures, and the latter lacks explicit quantification of interaction strength, resulting in significant long-term prediction bias of state evolution models under high load conditions. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a coal mine equipment state perception method based on large models and digital twinning to solve the problem of insufficient expression of local anomalies by existing digital twins.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] The present application provides a coal mine equipment state perception method based on large models and digital twinning, which includes,

[0008] Obtaining original data and physical constraints of the equipment in the coal mine, preprocessing the original data through a physical information processing flow, and converting the physical constraints of the equipment into constraint conditions and applying them to the preprocessing process to obtain a multi-modal data set;

[0009] Inputting the multi-modal data set into a quantum convolutional encoder, compressing features through the superposition principle of quantum states, selecting key feature segments through an attention gate mechanism, and generating a multi-scale digital twin of the equipment;

[0010] According to the multi-scale digital twin, a dynamic association model between equipment components is established using a spatio-temporal graph Transformer network, the interaction strength between components is quantified through a graph attention mechanism, and a device state evolution model is obtained by using a neural differential equation to describe the evolution process of the equipment state;

[0011] Combining the device state evolution model with a generative adversarial network, simulating the degradation path of the equipment through a generator, distinguishing real and simulated data through a discriminator, and optimizing through game training to obtain a state prediction model;

[0012] Performing a prediction task through the state prediction model, generating a device state report, and simulating and verifying the verification result through the multi-scale digital twin, and feeding back the verification result to the physical information processing flow for optimization.

[0013] As a preferred scheme of the coal mine equipment state perception method based on large models and digital twins, the original data and physical constraints of the equipment in the coal mine are obtained, the original data is preprocessed through a physical information processing flow, and the physical constraints of the equipment are converted into constraint conditions and applied to the preprocessing process to obtain a multi-modal data set, specifically:

[0014] Collecting three-dimensional point cloud data, RGB image streams, and acceleration and angular velocity data, and performing time synchronization to generate time-aligned multi-modal data streams, and collecting physical characteristics of the equipment;

[0015] Converting the physical characteristics into constraint conditions, inputting the time-aligned multi-modal data streams into the physical information processing flow, performing data cleaning and noise filtering, and generating point cloud feature vectors, visual feature tensors, and motion time series data;

[0016] After standardizing the point cloud feature vectors, visual feature tensors, and motion time series data, a multi-modal data set is generated.

[0017] As a preferred scheme of the coal mine equipment state perception method based on large models and digital twins, the multi-modal data set is input into a quantum convolutional encoder, the feature compression is performed through the superposition principle of quantum states, and the key feature segments are selected through the attention gate mechanism to generate a multi-scale digital twin of the equipment, specifically:

[0018] inputting the multi-modal data set into a quantum convolutional encoder, performing quantum feature encoding on the point cloud feature vector and the visual feature tensor based on the superposition principle of quantum states, and generating a low-dimensional compressed feature representation;

[0019] inputting the low-dimensional compressed feature representation into an attention gate network, calculating the correlation scores of each feature segment and the key components of the equipment, screening the feature segments related to the running state of the equipment, and generating a multi-modal feature sequence;

[0020] performing three-dimensional geometric modeling on the point cloud feature vector to generate a mesh model of the equipment, combining the mesh model with the multi-modal feature sequence, and generating a multi-scale digital twin through a multi-scale feature fusion mechanism.

[0021] As a preferred scheme of the coal mine equipment state perception method based on large models and digital twins, wherein: according to the multi-scale digital twin, a dynamic correlation model between equipment components is established using a spatio-temporal graph Transformer network, and the interaction strength between components is quantified through a graph attention mechanism, specifically:

[0022] The geometric and behavioral features of the equipment components are extracted from the multi-scale digital twin to construct an equipment component graph, which is input into the spatio-temporal graph Transformer network. The equipment component graph includes component node features and edges. The spatio-temporal graph Transformer network calculates the graph attention weight according to the physical connection between components, and updates the node features through a multi-layer graph attention mechanism to generate a dynamic correlation model.

[0023] The dynamic correlation model is aggregated to output an interaction strength matrix between components.

[0024] As a preferred scheme of the coal mine equipment state perception method based on large models and digital twins, wherein: the device state evolution process is described using a neural differential equation to obtain a device state evolution model, specifically:

[0025] The time series features and interaction strength matrix of the equipment components in the dynamic correlation model are input into the neural differential equation;

[0026] The time series features are input into the neural differential equation, and the adaptive time step integration method is used to simulate the state evolution path of the equipment under different working conditions. The state evolution path is then feature extracted to generate a state feature vector.

[0027] The state feature vector and the interaction strength matrix are fused to generate a device state evolution model.

[0028] As a preferred scheme of the coal mine equipment state perception method based on large model and digital twinning provided in the application, wherein: the equipment state evolution model is combined with the generative adversarial network, the generator is used to simulate the equipment degradation path, the discriminator is used to distinguish the real and simulated data, the game training is used for optimization, and the state prediction model is obtained, specifically:

[0029] The equipment state feature vector is extracted from the equipment state evolution model and input into the generative adversarial network.

[0030] The generator network and the discriminator network are constructed in the generative adversarial network, the generator network simulates the degradation path of the equipment under different working conditions through a multi-layer fully connected network to generate a simulated state sequence, the discriminator network classifies the simulated state sequence and the real state data through a convolutional neural network to output a true or false discrimination result, and the real state data is extracted from historical coal mine equipment operation records.

[0031] The parameters of the generator network and the discriminator network are initialized, and the number of game training iterations is set.

[0032] The generator network and the discriminator network are alternately optimized, the parameters of the discriminator network are fixed, the generator network is optimized based on the true or false discrimination result using the generative adversarial loss function, the parameters of the generator network are fixed, and the discriminator network is optimized based on the true or false discrimination result using the discriminative loss function.

[0033] The generative adversarial loss function and the discriminative loss function are calculated, when the number of iterations is reached, the parameters of the generator network are saved, the equipment degradation path is generated, and the state prediction model is output.

[0034] As a preferred scheme of the coal mine equipment state perception method based on large model and digital twinning provided in the application, wherein: the generator network is optimized based on the true or false discrimination result using the generative adversarial loss function, which means that the generative adversarial loss function adjusts the weights of the generator network based on the true or false discrimination result output by the discriminator network through back propagation and gradient descent.

[0035] As a preferred scheme of the coal mine equipment state perception method based on large model and digital twinning provided in the application, wherein: the generator network is optimized based on the true or false discrimination result using the generative adversarial loss function, which means that the generator network is optimized based on the true or false discrimination result output by the discriminator network through back propagation and gradient descent.

[0036] As a preferred scheme of the coal mine equipment state perception method based on large model and digital twin, wherein: the simulation verification through the multi-scale digital twin is to input the equipment state report into the multi-scale digital twin, simulate the future running state of the equipment, generate a virtual running track, calculate the deviation from the state prediction result, and generate a verification report according to the deviation.

[0037] As a preferred scheme of the coal mine equipment state perception method based on large model and digital twin, wherein: the verification result is fed back to the physical information processing flow for optimization, specifically:

[0038] The deviation and high deviation area are extracted from the verification report as feedback data.

[0039] The feedback data is input into the physical information processing flow, the constraint condition weight is adjusted through error analysis, and the multi-modal data set is updated.

[0040] The beneficial effects of the present application are: the quantum convolution encoder utilizes the superposition principle to compress high-dimensional features in parallel, and simultaneously uses the attention gating mechanism to adaptively filter key dynamic segments, breaking through the classical convolution serial bottleneck and the limitations of fixed attention rules, achieving exponential dimension reduction and accurately capturing degenerative correlation spatiotemporal patterns, significantly improving the expression ability of multi-scale digital twins to local anomalies; in addition, the strength of mechanical coupling between time-space graph Transformer quantization components is input into the neural differential equation continuous evolution, overcoming the defects of discrete recursive networks ignoring graph interaction and ordinary neural differential equations lacking strength quantization, achieving a significant improvement in long-term prediction accuracy under high-load working conditions. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0042] Fig. 1 The flowchart of the coal mine equipment state perception method based on large model and digital twin.

[0043] Fig. 2 The flowchart of the multi-modal data set generation.

[0044] Fig. 3 The flowchart of digital twin construction and state evolution modeling.

[0045] Fig. 4 The flowchart of state prediction model optimization and verification feedback. DETAILED DESCRIPTION

[0046] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0047] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited by the specific embodiments disclosed below.

[0048] Secondly, "one embodiment" or "embodiment" referred to herein means that a specific feature, structure or characteristic can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an embodiment that is separate or alternative to other embodiments.

[0049] Reference Figs. 1-4 For one embodiment of the present application, the embodiment provides a coal mine equipment state perception method based on large model and digital twinning, comprising the following steps:

[0050] S1: Obtain the original data of the coal mine underground multi-sensor network and the device physical constraints, preprocess the original data through the physical information processing flow, and convert the device physical constraints into constraint conditions and apply them to the preprocessing process to obtain a multi-modal data set;

[0051] S1.1: Obtain three-dimensional point cloud data by laser radar sensors installed on a coal mining machine or a hydraulic support, capture the spatial geometric information of the equipment, and record the RGB image stream by an underground vision camera, capture the appearance and running environment of the equipment, and collect acceleration and angular velocity data by inertia measurement tools installed on moving parts (such as the cutting head of the coal mining machine); synchronize the three-dimensional point cloud data, RGB image stream and acceleration and angular velocity data through a sliding window mechanism, the specific method is to take a unified clock as a reference, align the timestamps output by each sensor, and ensure the consistency of the three-dimensional point cloud data, RGB image stream and acceleration and angular velocity data in time, thereby generating a time-aligned multi-modal data stream.

[0052] S1.2: Construct a physical constraint model based on the physical characteristics of the coal mine equipment, wherein the physical characteristics include the mechanical motion characteristics of the coal mining machine or the thermodynamic characteristics of the hydraulic function, and the physical constraint model includes kinematic constraints (such as joint motion range limits) and thermodynamic constraints (such as heat dissipation rate), which are derived from the technical specifications of the equipment; convert the physical constraint model into constraint conditions, and realize the kinematic constraints through joint angle thresholds:

[0053] ;

[0054] in, This indicates the lower limit of the joint angle, expressed in radians, and is derived from the minimum range of motion specified in the equipment documentation. This indicates the joint angle of the coal mining machine's cutting head or hydraulic support, expressed in radians. This indicates the upper limit of the joint angle, in radians, and is derived from the maximum range of motion specified in the equipment documentation.

[0055] Thermodynamic constraints are achieved through the rate of heat dissipation:

[0056] ;

[0057] in, This indicates the rate at which the equipment dissipates heat, measured in watts. This indicates the upper limit of the heat dissipation rate, measured in watts (W), and is derived from the equipment heat balance manual.

[0058] The constraints include restrictions on deviations from the expected kinematic behavior and restrictions on thermodynamic equilibrium.

[0059] The time-aligned multimodal data stream is input into the Physical Information Neural Process (PIN), a processing method capable of handling heterogeneous data and adhering to physical laws. The PSN performs data cleaning, removing outliers caused by dust interference, etc.; calculates the real-time signal-to-noise ratio (SNR) of the 3D point cloud data, RGB image stream, and acceleration and angular velocity data in the time-aligned multimodal data stream; adjusts the noise covariance matrix of the Kalman filter based on the SNR; applies Kalman filtering to smooth the point coordinate sequence of the 3D point cloud data to generate point cloud feature vectors; applies Kalman filtering to smooth the pixel value sequence of the RGB image stream to generate visual feature tensors; and applies Kalman filtering to smooth the temporal sequence of the acceleration and angular velocity data to generate motion temporal data.

[0060] The point cloud feature vectors are standardized by normalizing the coordinate values ​​to a unit range, adjusting the numerical distribution of the visual feature tensor to zero mean and unit variance, and normalizing the acceleration and angular velocity values ​​of the motion time series data to a uniform range. The standardized point cloud feature vectors, visual feature tensors, and motion time series data are combined to generate a unified format and scale multimodal dataset.

[0061] S2: Input the multimodal dataset into the quantum convolutional encoder, compress features through the principle of quantum state superposition, and select key feature segments using the attention gating mechanism to generate a multi-scale digital twin of the device;

[0062] The multi-modal data set is input into the quantum convolutional encoder, and the quantum convolutional encoder performs quantum feature coding on the point cloud feature vectors and visual feature tensors in the multi-modal data set based on the superposition principle of quantum states, simulates the superposition state of the feature distribution through a quantum circuit, maps the high-dimensional point cloud feature vectors and visual feature tensors to a low-dimensional space, and generates low-dimensional compressed feature representations.

[0063] The quantum convolutional encoder training process is as follows: collect historical multi-modal data sets, input the historical multi-modal data sets into the quantum convolutional encoder, and the quantum convolutional encoder simulates the feature distribution of the point cloud feature vectors and visual feature tensors through a quantum circuit, generates low-dimensional compressed feature representations based on the superposition principle of quantum states; the reconstruction accuracy of the low-dimensional compressed feature representations and the original point cloud feature vectors and visual feature tensors is evaluated by minimizing the feature reconstruction loss function; the quantum circuit parameters are adjusted through the parameterized quantum circuit using the variational quantum algorithm and gradient descent (learning rate is 0.001) to optimize the feature compression accuracy until the feature reconstruction loss function no longer decreases, the quantum convolutional encoder training is completed, and further, the quantum circuit includes 4 layers of RY-RZ rotation gates and CZ entanglement gates, and the rotation angle is a trainable parameter.

[0064] The low-dimensional compressed feature representations are input into the attention gate mechanism, which evaluates each feature segment in the low-dimensional compressed feature representations one by one, calculates the matching degree of the feature segment and the key component (here taking the coal mining machine cutting head and hydraulic support as an example) running parameters through cosine similarity, and generates a correlation score; according to the correlation score, the attention gate mechanism filters out feature segments with a score higher than a preset correlation threshold, which is set to 0.8 based on statistical analysis of historical coal mine equipment running data, which can effectively distinguish feature segments highly related to equipment running state and exclude low-correlation noise data; if it is lower than 0.8, the feature segment may contain irrelevant information, causing distortion of the multi-modal feature sequence, and if it is higher than 0.8, it may miss some secondary but useful features; the filtered feature segments reflect information related to the equipment running state, such as cutting head vibration mode or hydraulic support pressure change, and are integrated into a multi-modal feature sequence.

[0065] The point cloud feature vector is extracted from the multi-modal data set, the surface points in the point cloud feature vector are extracted using a Poisson surface reconstruction method, and a mesh model of the device is generated, specifically: the Poisson surface reconstruction method estimates the continuous geometric shape of the device surface through the point coordinates and normal vectors in the point cloud feature vector, extracts the surface points, constructs a triangulation mesh according to the extracted surface points, and generates a mesh model of the device; the mesh model and the multi-modal feature sequence are input into a multi-scale feature fusion mechanism, the multi-scale feature fusion mechanism performs initial alignment on the geometric details of the mesh model and the dynamic information of the multi-modal feature sequence at a low resolution level, and extracts rough device shape and operating state features; at a high resolution level, the fine geometric structure of the mesh model and the time sequence dynamics of the multi-modal feature sequence are further aligned to generate fusion features, and the fusion features at low and high resolutions are integrated to generate a virtual representation of the device at different resolution levels, forming a multi-scale digital twin.

[0066] Preferably, compared with existing coal mine equipment state perception methods, the application adopts quantum convolutional encoder and attention gate mechanism in the multi-scale digital twin construction link, significantly improves the feature compression efficiency and key information retention accuracy, and realizes parallel dimension reduction of high-dimensional point cloud and image features based on the superposition principle of quantum state, breaks through the classical convolution bottleneck, reduces the calculation load while retaining the complete geometric-behavior correlation; the attention gate mechanism accurately selects dynamic segments related to the running height of the coal cutter cutting head and the hydraulic support, effectively filters out noise introduced by underground dust and light interference, avoids information distortion or redundancy caused by traditional fixed threshold methods, and the multi-scale feature fusion mechanism further realizes seamless connection of coarse and fine granularity, ensures that the digital twin quickly responds to the overall posture at low resolution and accurately reproduces local vibration and pressure fluctuation at high resolution, provides high-fidelity, multi-level virtual entities for subsequent state evolution modeling, and significantly enhances the accuracy and real-time performance in complex underground environments.

[0067] S3: According to the multi-scale digital twin, a dynamic association model between device components is established using a spatio-temporal graph Transformer network, the interaction strength between components is quantified through a graph attention mechanism, and a device state evolution model is obtained by using a neural differential equation to describe the device state evolution process;

[0068] S3.1: Extract the geometric features and behavior features of the equipment components from the multi-scale digital twin, construct the equipment component graph according to the extracted geometric features and behavior features, and take the cutting head of the coal mining machine and the hydraulic support and other components as the nodes and the physical connection of the mechanical coupling as the edge; input the equipment component graph into the space-time graph Transformer network, and the space-time graph Transformer network evaluates the edge weight according to the physical connection of the mechanical coupling to generate the graph attention weight; the multi-layer graph attention mechanism fuses the geometric features and behavior features of the coal mining machine cutting head and the hydraulic support based on the graph attention weight, and updates the node features in the equipment component graph through layer-by-layer weighted aggregation, and each layer of aggregation adjusts the feature contribution according to the graph attention weight; integrate the updated node features to generate a dynamic correlation model.

[0069] The training process of the space-time graph Transformer network: collect historical multi-scale digital twins, including geometric and running state labels, and input them into the space-time graph Transformer network to construct the equipment component graph and obtain the historical dynamic correlation model; the matching degree of the historical dynamic correlation model and the actual label is evaluated by minimizing the interactive prediction loss function, so as to adjust the parameters of the space-time graph Transformer network until the interactive prediction loss function reaches the maximum training round, and the parameters of the space-time graph Transformer network are saved.

[0070] S3.2: Extract the time sequence feature and interaction intensity matrix from the dynamic correlation model, input the time sequence feature into the neural differential equation, and the neural differential equation is constructed based on the change rate of the equipment running parameter, and the expression is:

[0071] ;

[0072] Among them, represents the equipment state over time , that is, the dynamic evolution speed of the equipment state in the time dimension, represents the time varying time sequence feature, represents the weight coefficient of the time sequence feature, which is obtained by statistically analyzing the influence of vibration frequency and pressure change on the equipment state, represents the interaction intensity matrix, represents the weight coefficient of the interaction intensity matrix, which is obtained by analyzing the strength of the mechanical coupling between components, and describes the dynamic evolution law of the equipment under normal operation or high load working condition.

[0073] Solve the neural differential equation by the classical fourth-order Runge-Kutta method, and the fixed step size is 1 second, according to the weight coefficients and The simulation device evolves the state path under normal operation or high load working conditions; the vibration frequency change trend, pressure change rate and operation working condition duration are extracted from the state evolution path to generate a state feature vector; the state feature vector and the interaction strength matrix are fused by weighted splicing, the splicing weight of the state feature vector is 0.6, and the splicing weight of the interaction strength matrix is 0.4, which is determined according to historical data cross-validation; the information of the two is integrated to generate a device state evolution model to describe the dynamic evolution in the time dimension.

[0074] Preferably, unlike the existing coal mine equipment state perception method, the present application ingeniously fuses the space-time graph Transformer and the neural differential equation to enhance the accuracy and adaptability of the state evolution model; the graph Transformer captures the real-time mechanical interaction between the cutting head and the support through the attention mechanism, avoiding the loss of continuous conduction details by the conventional discrete model; the neural differential equation deduces the continuous trajectory of vibration and pressure by means of the fourth-order Runge-Kutta smoothing, eliminating the jump error caused by time step discretization; after the weighted fusion of the time sequence features and the interaction matrix, the collaborative degradation process of the components under high load is completely restored, laying a reliable foundation for downstream prediction, and greatly improving the robustness and predictability of the perception function in the variable scene underground.

[0075] S4: Combine the device state evolution model with the generative adversarial network, simulate the device degradation path through the generator, distinguish the real and simulated data using the discriminator, and optimize using game training to obtain a state prediction model.

[0076] S4.1: Extract the device state feature vector from the device state evolution model and input it into the generative adversarial network; initialize the generator network and the discriminator network of the generative adversarial network, the generator network includes three fully connected layers with node numbers of 512, 256 and 128 respectively, and uses ReLU activation function; the discriminator network includes three convolutional layers with output channel numbers of 64, 128 and 256 respectively, the convolution kernel size is 3x3, the step is 1, the padding is 1, and the ReLU activation function and the maximum pooling layer (kernel size 2x2, step 2) are connected behind; input the device state feature vector into the generator network, the generator network maps the degradation path of the simulated coal cutting head vibration frequency attenuation and hydraulic support pressure drop layer by layer, and obtains the simulated state sequence; input the simulated state sequence and the real state data into the discriminator network, the real state data comes from the historical coal mine equipment operation record, the discriminator network extracts the features of the simulated state sequence layer by layer, outputs the true or false discrimination result, the value is the probability between 0 and 1, and represents the authenticity of the simulated state sequence.

[0077] S4.2: Initialize the generator network parameters as a random Gaussian distribution (mean 0, standard deviation 0.01), and the discriminator network parameters as a random Gaussian distribution (mean 0, standard deviation 0.02). Based on the scale of the historical operation records of coal mine equipment, set the number of game training iterations to 10000. If it is less than 10000, it will lead to underfitting, making the discriminator network insufficient in distinguishing ability, and the true and false discrimination results fluctuate greatly. If it is higher than 10000, it will cause the generator network to overfit the training data, leading to prediction distortion in new working conditions underground.

[0078] The process of alternately optimizing is as follows: fixing the discriminator network parameters, inputting the simulated state sequence output by the generator network into the discriminator network to obtain the true and false discrimination results, adjusting the generator network weights based on the true and false discrimination results using the generative adversarial loss function through back propagation and gradient descent (learning rate is, for example, 0.0002), and using the generative adversarial loss function in the form of binary cross entropy to gradually approach the distribution of the simulated state sequence to the real state data; then fixing the generator network parameters, adjusting the discriminator network weights based on the true and false discrimination results using the discriminative loss function through back propagation and Adam optimizer (learning rate is, for example, 0.0002), and improving the discrimination accuracy of real and simulated data;

[0079] Repeat the above alternating optimization process to calculate the generative adversarial loss function and discriminative loss function in real time until the preset 10000 iterations are reached; after the training is completed, save the optimized generator network parameters to obtain the state prediction model.

[0080] S5: Perform the prediction task through the state prediction model, generate the equipment state report, and perform simulation verification through the multi-scale digital twin to feed back the verification results to the physical information neural network for optimization.

[0081] S5.1: Input the point cloud feature vector in the multi-modal data set into the generator network of the state prediction model. The generator network maps the point cloud spatial geometric information through the first full connection layer; input the visual feature tensor into the generator network, and the second full connection layer fuses the image mode and geometric information; input the motion time series data into the generator network, and the third full connection layer integrates the vibration frequency and pressure change; the generator network propagates forward layer by layer, simulates the future vibration frequency of the cutting head of the coal mining machine gradually decaying, the pressure of the hydraulic support continuously decreasing, and the working condition continuously extending, and outputs the state prediction result.

[0082] The state prediction result is post-processed, the root mean square deviation of the vibration frequency is calculated through a sliding window with a window size of 50 to obtain a device health score ranging from 0 to 100, and the higher the value, the healthier the device; an abnormal threshold is set based on statistical analysis of historical coal mine equipment operation records, and the abnormal threshold is the historical 95% quantile period of the vibration frequency or pressure sequence; if 5 consecutive sampling points in the sliding window exceed the abnormal threshold, the window corresponding to the time period is marked as an abnormal flag; the pressure and vibration trends in the state prediction result are fitted through linear regression, and the degradation slope is extracted as the degradation trend; the device health score, abnormal flag and degradation trend in the device state report are integrated.

[0083] S5.2: input the device health score, abnormal flag and degradation trend in the device state report into the multi-scale digital twin, the multi-scale digital twin loads the device coarse grid model and operating state features in the low resolution level, simulates the overall motion trajectory of the coal mining machine cutting head and hydraulic support in the future time period; the multi-scale digital twin loads the fine grid structure and time sequence dynamics in the high resolution level, simulates the second-by-second change of the cutting head vibration frequency and the hydraulic support pressure, generates a virtual running trajectory; the virtual running trajectory is aligned point by point with the vibration, pressure and working condition sequence in the state prediction result, the mean square error is calculated as the deviation, and compared with the preset deviation threshold, the deviation threshold is 0.05, which is set based on the vibration / pressure mean square error statistics of normal working conditions and slight abnormalities in the historical coal mine equipment operation records, if higher than 0.05, mark as failed, lower than 0.05, mark as passed; the deviation value, whether lower than the deviation threshold or not, is integrated into a verification report.

[0084] S5.3: extract the high deviation area of the deviation value 0.05 from the verification report, for example, the prediction error of the coal mining machine cutting head vibration frequency in the 120-150 seconds; input the deviation value and the high deviation area as feedback data into the physical information processing process; the physical information processing process identifies the three-dimensional point cloud data, RGB image stream and acceleration and angular velocity data corresponding to the high deviation area through error analysis; adjust the constraint condition weight according to the high deviation area, apply the adjusted constraint condition to the time-aligned multi-modal data stream, re-execute the median filter, Kalman filter and constraint check to generate the optimized point cloud feature vector, visual feature tensor and motion time sequence data; combine the optimized point cloud feature vector, visual feature tensor and motion time sequence data into an updated multi-modal data set to complete the current coal mine equipment state perception task.

[0085] The embodiment also provides a computer device suitable for the coal mine equipment state perception method based on a large model and digital twinning, which comprises a memory and a processor.

[0086] The computer device can be a terminal, and the computer device comprises a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0087] The embodiment also provides a storage medium having a computer program stored thereon, and the program is executed by a processor to implement the coal mine equipment state perception method based on a large model and digital twinning. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

[0088] To sum up, the application: by using superposition principle in parallel compression of high-dimensional features through quantum convolution encoder, and by using attention gating mechanism to adaptively filter key dynamic segments, the classical convolution serial bottleneck and the fixed attention rule limitation are broken through, the exponential dimension reduction is realized, and the degenerate related space-time patterns are accurately captured, so that the expression ability of the multi-scale digital twin to local anomalies is significantly improved; in addition, by quantifying the mechanical coupling strength between the space-time graph Transformer components and inputting the neural differential equation continuous evolution, the defects of ignoring graph interaction of the discrete recursive network and missing strength quantization of the ordinary neural differential equation are overcome, and the effect of greatly improving the long-term prediction accuracy under high load working conditions is realized.

[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A coal mine equipment state perception method based on large models and digital twinning, characterized in that: The application relates to a method for constructing a state prediction model of a coal mine device, comprising the following steps: obtaining original data and physical constraints of a coal mine device, preprocessing the original data through a physical information processing flow, and converting the physical constraints of the device into constraint conditions to be applied to the preprocessing process to obtain a multi-modal data set; inputting the multi-modal data set into a quantum convolutional encoder, compressing features through the principle of superposition of quantum states, and selecting key feature fragments through an attention gate mechanism to generate a multi-scale digital twin of the device, specifically: inputting the multi-modal data set into a quantum convolutional encoder, performing quantum feature encoding on point cloud feature vectors and visual feature tensors based on the principle of superposition of quantum states to generate low-dimensional compressed feature representations; inputting the low-dimensional compressed feature representations into an attention gate network, calculating the correlation scores of each feature fragment and the key components of the device, screening feature fragments related to the running state of the device, and generating a multi-modal feature sequence; performing three-dimensional geometric modeling on the point cloud feature vectors to generate a mesh model of the device, combining the mesh model with the multi-modal feature sequence, and generating a multi-scale digital twin through a multi-scale feature fusion mechanism; establishing a dynamic association model between device components using a spatio-temporal graph Transformer network according to the multi-scale digital twin, quantifying the interaction strength between components through a graph attention mechanism, and describing the evolution process of the device state using a neural differential equation to obtain a device state evolution model; the dynamic association model between device components is established using a spatio-temporal graph Transformer network according to the multi-scale digital twin, and the interaction strength between components is quantified through a graph attention mechanism, specifically: extracting geometric and behavioral features of device components from the multi-scale digital twin, constructing a device component graph, and inputting the device component graph into a spatio-temporal graph Transformer network, wherein the device component graph comprises component node features and edges, the spatio-temporal graph Transformer network calculates graph attention weights according to the physical connection between components, and updates node features through a multi-layer graph attention mechanism to generate a dynamic association model; performing feature aggregation on the dynamic association model to output an interaction strength matrix between components; combining the device state evolution model with a generative adversarial network, simulating a device degradation path through a generator, distinguishing real and simulated data through a discriminator, and optimizing through game training to obtain a state prediction model, specifically: extracting device state feature vectors from the device state evolution model and inputting the device state feature vectors into the generative adversarial network; constructing a generator network and a discriminator network in the generative adversarial network, the generator network simulates a device degradation path under different working conditions through a multi-layer fully connected network to generate a simulated state sequence, the discriminator network classifies the simulated state sequence and real state data through a convolutional neural network to output a true or false discrimination result, and the real state data is extracted from historical coal mine device operation records; initializing the parameters of the generator network and the discriminator network, and setting the number of game training iterations. alternately optimizing the generator network and the discriminator network, fixing parameters of the discriminator network, optimizing the generator network based on the authenticity discrimination result using a generative adversarial loss function, fixing parameters of the generator network, optimizing the discriminator network based on the authenticity discrimination result using a discriminative loss function; calculating the adversarial loss function and the discriminative loss function, saving parameters of the generator network when the number of iterations is reached, and obtaining a state prediction model; performing a prediction task through the state prediction model, generating a device state report, and performing simulation verification through a multi-scale digital twin, and feeding back a verification result to the physical information processing flow for optimization.

2. The large model and digital twin-based coal mine equipment state perception method according to claim 1, characterized in that: obtaining original data and device physical constraints in a coal mine, preprocessing the original data through the physical information processing flow, and converting the device physical constraints into constraint conditions and applying them to the preprocessing process to obtain a multi-modal data set, specifically: collecting three-dimensional point cloud data, RGB image streams, and acceleration and angular velocity data, and performing time synchronization to generate time-aligned multi-modal data streams, and collecting physical characteristics of the device; converting the physical characteristics into constraint conditions, inputting the time-aligned multi-modal data streams into the physical information processing flow, performing data cleaning and noise filtering to generate point cloud feature vectors, visual feature tensors, and motion time series data; standardizing the point cloud feature vectors, visual feature tensors, and motion time series data to generate the multi-modal data set.

3. The large model and digital twin-based coal mine equipment state perception method according to claim 2, characterized in that: The device state evolution model is obtained by using a neural differential equation to describe the device state evolution process, specifically: inputting the time series features and the interaction strength matrix of the device components in the dynamic correlation model into the neural differential equation as inputs; inputting the time series features into the neural differential equation, simulating the state evolution path of the device under different working conditions through an adaptive time step integration method, and extracting features from the state evolution path to generate a state feature vector; fusing the state feature vector and the interaction strength matrix to generate the device state evolution model.

4. The large model and digital twin-based coal mine equipment state perception method of claim 3, wherein: The generator network is optimized based on the authenticity discrimination result using a generative adversarial loss function, which means that the generator network weight is adjusted through backpropagation and gradient descent based on the authenticity discrimination result output by the discriminator network.

5. The large model and digital twin-based coal mine equipment state perception method according to claim 4, characterized in that: The device state report is generated by performing a prediction task through the state prediction model, which means that the multi-modal data set is input into the state prediction model to generate a state prediction result, and the device health score, abnormal flag, and degradation trend are extracted and integrated into a structured device state report.

6. The large model and digital twin based coal mine equipment state perception method according to claim 5, characterized in that: The verification report is generated by inputting the device state report into the multi-scale digital twin, simulating the future running state of the device, calculating the deviation from the state prediction result, and generating a verification report according to the deviation.

7. The large model and digital twin based coal mine equipment state perception method according to claim 6, characterized in that: The verification result is fed back to the physical information processing flow for optimization, specifically: extracting the deviation and high deviation area from the verification report as feedback data; inputting the feedback data into the physical information processing flow, adjusting the constraint condition weight through error analysis, and updating the multi-modal data set.

Images