System and method for identifying working state of mining excavator

Abstract

The embodiment of the invention provides a system and a method for identifying the working state of a mining excavator, and the method comprises the steps: collecting a vibration signal and a hydraulic signal of the mining excavator in real time through a sensor assembly; performing feature extraction on the vibration signal and the hydraulic signal by adopting a neural network model based on deep learning so as to excavate time sequence dynamic characteristics of the mining excavator in a working state, and further performing inter-feature fine-grained response interaction on the vibration frequency domain feature and the hydraulic time sequence feature to more comprehensively reflect the operation state of the mining excavator, obtaining the multi-mode joint representation of the working state of the mining excavator, and determining whether the working state of the mining excavator is abnormal or not based on the multi-mode joint representation of the working state of the mining excavator. By means of the mode, accurate recognition and anomaly detection of the working state of the mining excavator can be achieved, and reliable guarantee is provided for mine production safety and excavator operation efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of intelligent identification, and more specifically, to a system and method for identifying the working state of a mining excavator. Background Art

[0002] As a key device in mine exploitation, the real-time monitoring and abnormal identification of the working state of a mining excavator are of great significance for ensuring production safety and improving the operation efficiency of the equipment. However, the mine environment is complex and changeable, and the equipment will be affected by various factors during operation, such as load changes, hydraulic system fluctuations, etc. These factors may cause the equipment to malfunction, which may further lead to failures or safety accidents. However, there are many deficiencies in the existing technologies for identifying the working state of mining excavators, and it is difficult to meet the high-precision monitoring requirements under complex working conditions.

[0003] First of all, the traditional single-sensor monitoring method is difficult to comprehensively capture the operating state of the equipment. Especially under complex working conditions, single-modal data (such as only relying on vibration signals or hydraulic signals) cannot fully reflect the comprehensive operating state of the equipment, which easily leads to misjudgment or missed judgment. Secondly, the existing methods lack an efficient fusion mechanism for multi-modal data and fail to fully utilize the correlation and complementarity between vibration signals and hydraulic signals, resulting in insufficient robustness of state identification. Especially in the complex and changeable mine environment, the working state of the equipment is affected by various factors, and it is difficult for single features or simple fusion methods to cope with complex working condition changes.

[0004] Therefore, an optimized system for identifying the working state of a mining excavator is expected. Summary of the Invention

[0005] The present invention aims to at least solve one of the technical problems existing in the prior art, and provides a system and method for identifying the working state of a mining excavator.

[0006] In a first aspect, the embodiments of the present invention provide a system for identifying the working state of a mining excavator, including:

[0007] A data acquisition module, configured to acquire the real-time original data stream of the mining excavator collected by the sensor assembly, where the real-time original data stream includes the time queues of vibration signals and hydraulic signals;

[0008] A wavelet transform module, configured to perform wavelet transform on the time queue of the vibration signals to obtain a vibration time-frequency diagram;

[0009] A frequency domain feature extraction module, configured to extract the vibration frequency domain features of the vibration time-frequency diagram to obtain a vibration time-frequency feature coding vector;

[0010] A sequence coding module, configured to perform sequence coding on the time queue of the hydraulic signals to obtain a hydraulic time series gradient coding feature vector;

[0011] A heterogeneous time-series feature fast interaction module is used to perform feature interaction based on the implicit key time-series feature anchoring between vibration and hydraulics on the vibration time-frequency feature encoding vector and the hydraulic time-series gradient encoding feature vector to obtain a multi-modal joint encoding vector of the working state of the mining excavator;

[0012] A working state recognition module is used to determine whether there is an abnormality in the working state of the mining excavator based on the multi-modal joint encoding vector of the working state of the mining excavator.

[0013] In some possible embodiments, the frequency-domain feature extraction module is used for:

[0014] Input the vibration time-frequency diagram into a vibration frequency-domain feature extractor based on the Mobile-Former model to obtain the vibration time-frequency feature encoding vector.

[0015] In some possible embodiments, the sequence encoding module is used for:

[0016] Input the time queue of the hydraulic signal into a sequence encoder based on the LSTM-RNN model to obtain the hydraulic time-series gradient encoding feature vector.

[0017] In some possible embodiments, the heterogeneous time-series feature fast interaction module includes:

[0018] A heterogeneous feature extraction unit is used to extract the heterogeneous implicit features of the hydraulic time-series gradient encoding feature vector and the vibration time-frequency feature encoding vector to obtain a set of hydraulic time-series gradient feature principal component implicit encoding vectors and vibration time-frequency implicit feature encoding vectors;

[0019] A feature fast interaction unit is used to perform implicit key clue anchoring and feature fine-grained interaction on the set of hydraulic time-series gradient feature principal component implicit encoding vectors and the vibration time-frequency implicit feature encoding vectors to obtain the multi-modal joint encoding vector of the working state of the mining excavator.

[0020] In some possible embodiments, the heterogeneous implicit feature extraction unit includes:

[0021] A hydraulic time-series gradient feature principal component analysis sub-unit is used to perform hydraulic time-series gradient feature principal component analysis on the hydraulic time-series gradient encoding feature vector to obtain a set of hydraulic time-series gradient feature principal component encoding vectors;

[0022] A hydraulic time - series gradient implicit feature extraction subunit, which is used to perform hydraulic time - series gradient implicit feature extraction based on point convolution encoding on each hydraulic time - series gradient feature principal component encoding vector in the set of the hydraulic time - series gradient feature principal component encoding vectors to obtain the set of the hydraulic time - series gradient feature principal component implicit encoding vectors;

[0023] A vibration time - frequency implicit feature extraction subunit, which is used to perform vibration time - frequency implicit feature extraction based on point convolution encoding on the vibration time - frequency feature encoding vector to obtain the vibration time - frequency implicit feature encoding vector.

[0024] In some possible embodiments, the feature fast interaction unit includes:

[0025] An implicit key clue anchoring subunit, which is used to input the vibration time - frequency implicit feature encoding vector and the set of the hydraulic time - series gradient feature principal component implicit encoding vectors into an implicit key clue anchoring network to obtain a {vibration time - frequency implicit feature encoding vector, hydraulic time - series gradient feature principal component implicit encoding anchored vector} feature pair;

[0026] A feature fine - grained interaction subunit, which is used to input the {vibration time - frequency implicit feature encoding vector, hydraulic time - series gradient feature principal component implicit encoding anchored vector} feature pair into a feature fine - grained interaction network to obtain the multi - modal joint encoding vector of the working state of the mining excavator.

[0027] In some possible embodiments, the implicit key clue anchoring subunit is used to:

[0028] Calculate the similarity between the vibration time - frequency feature encoding vector and each hydraulic time - series gradient feature principal component implicit encoding vector in the set of the hydraulic time - series gradient feature principal component implicit encoding vectors to obtain a set of similarities, and extract the position coefficient corresponding to the maximum similarity value in the set of similarities;

[0029] Based on the position coefficient corresponding to the maximum similarity value, anchor the hydraulic time - series gradient feature principal component implicit encoding vector in the set of the hydraulic time - series gradient feature principal component implicit encoding vectors that has the highest matching similarity with the vibration time - frequency feature encoding vector to obtain the hydraulic time - series gradient feature principal component implicit encoding anchored vector;

[0030] Perform feature pair matching on the vibration time - frequency implicit feature encoding vector and the hydraulic time - series gradient feature principal component implicit encoding anchored vector to obtain the {vibration time - frequency implicit feature encoding vector, hydraulic time - series gradient feature principal component implicit encoding anchored vector} feature pair.

[0031] In some possible embodiments, the working state recognition module includes:

[0032] Input the multi-modal joint coding vector of the working state of the mining excavator into an anomaly detector based on a classifier to obtain a detection result, where the detection result is used to indicate whether there is an anomaly in the working state of the mining excavator.

[0033] In a second aspect, an embodiment of the present invention provides a method for identifying the working state of a mining excavator, including:

[0034] Obtain the real-time raw data stream of the mining excavator collected by the sensor assembly, where the real-time raw data stream includes the time queues of vibration signals and hydraulic signals;

[0035] Perform wavelet transform on the time queue of the vibration signal to obtain a vibration time-frequency diagram;

[0036] Extract the vibration frequency domain features of the vibration time-frequency diagram to obtain a vibration time-frequency feature coding vector;

[0037] Perform sequence coding on the time queue of the hydraulic signal to obtain a hydraulic time series gradient coding feature vector;

[0038] Perform fast interaction of heterogeneous time series features based on implicit key clue anchoring on the vibration time-frequency feature coding vector and the hydraulic time series gradient coding feature vector to obtain a multi-modal joint coding vector of the working state of the mining excavator;

[0039] Based on the multi-modal joint coding vector of the working state of the mining excavator, determine whether there is an anomaly in the working state of the mining excavator.

[0040] In some possible embodiments, the extracting the vibration frequency domain features of the vibration time-frequency diagram to obtain a vibration time-frequency feature coding vector includes:

[0041] Input the vibration time-frequency diagram into a vibration frequency domain feature extractor based on the Mobile-Former model to obtain the vibration time-frequency feature coding vector.

[0042] Compared with the prior art, the embodiment of the present invention provides a mining excavator working state identification system and method, which collects the vibration signal and hydraulic signal of the mining excavator in real time through the sensor component, uses a neural network model based on deep learning to extract the features of the vibration signal and the hydraulic signal to mine the time series dynamic characteristics of the mining excavator in the working state, and further performs fine-grained response interaction between the vibration frequency domain features and the hydraulic time series features to more comprehensively reflect the operating state of the mining excavator, obtains a multi-modal joint representation of the working state of the mining excavator, and determines whether the working state of the mining excavator is abnormal based on this. In this way, accurate identification and abnormality detection of the working state of the mining excavator can be achieved, providing reliable protection for mine production safety and excavator operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A block diagram of a system for identifying a working state of a mining excavator according to an embodiment of the present invention;

[0045] Figure 2 A data flow diagram of a mining excavator working state identification system according to an embodiment of the present invention;

[0046] Figure 3 A block diagram of a fast interaction module of heterogeneous time series features in a mining excavator working state recognition system according to an embodiment of the present invention;

[0047] Figure 4 The figure is a flow chart of a method for identifying a working state of a mining excavator according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0049] Unless otherwise specifically stated, the technical terms or scientific terms used in the embodiments of the present invention should have the ordinary meanings understood by those of ordinary skill in the art to which the present invention pertains. The use of "including" or "comprising" in the embodiments of the present invention neither limits the mentioned shapes, numbers, steps, actions, operations, components, elements and / or their groups, nor excludes the presence or addition of one or more other different shapes, numbers, steps, actions, operations, components, elements and / or their groups, or the addition of these. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the quantity and order of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "a plurality" means two or more, unless otherwise specifically and clearly defined.

[0050] Unless otherwise specifically stated, the relative settings, numerical expressions and numerical values of the components and steps described in these embodiments do not limit the scope of the present invention. At the same time, it should be understood that for the sake of convenience of description, the dimensions of the various parts shown in the drawings are not drawn in actual proportional relationship. For technologies, methods and devices known to those of ordinary skill in the relevant field, they may not be discussed in detail, but in appropriate cases, the shown technologies, methods and devices should be regarded as part of the authorization specification. In all the examples shown and discussed here, any specific other example may have different values. It should be noted that: similar symbols and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0051] In the description of the embodiments of the present invention, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In the embodiments of the present invention, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in the embodiments of the present invention and the features of different embodiments or examples.

[0052] In the present invention, a flowchart is used to illustrate the operations performed by the system according to an embodiment of the present invention. It should be understood that the operations before or below are not necessarily executed precisely in sequence. Instead, various steps can be processed in reverse order or simultaneously as needed. Also, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0053] Next, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments of the present invention. It should be understood that the present invention is not limited by the exemplary embodiments described herein.

[0054] In the technical solution of the present invention, a working state recognition system for a mining excavator is proposed. Figure 1 It is a block diagram of a working state recognition system for a mining excavator according to an embodiment of the present invention. Figure 2 It is a schematic diagram of data flow of a working state recognition system for a mining excavator according to an embodiment of the present invention. As Figure 1 and Figure 2 shown, a working state recognition system 300 for a mining excavator according to an embodiment of the present invention includes: a data acquisition module 310, configured to acquire a real-time raw data stream of the mining excavator collected by a sensor assembly, where the real-time raw data stream includes a time queue of vibration signals and hydraulic signals; a wavelet transform module 320, configured to perform wavelet transform on the time queue of the vibration signals to obtain a vibration time-frequency diagram; a frequency domain feature extraction module 330, configured to extract vibration frequency domain features of the vibration time-frequency diagram to obtain a vibration time-frequency feature coding vector; a sequence coding module 340, configured to perform sequence coding on the time queue of the hydraulic signals to obtain a hydraulic time series gradient coding feature vector; a heterogeneous time series feature fast interaction module 350, configured to perform feature interaction based on the anchoring of implicit key time series features between vibration and hydraulic on the vibration time-frequency feature coding vector and the hydraulic time series gradient coding feature vector to obtain a multi-modal joint coding vector of the working state of the mining excavator; and a working state recognition module 360, configured to determine whether there is an abnormality in the working state of the mining excavator based on the multi-modal joint coding vector of the working state of the mining excavator.

[0055] In particular, the data acquisition module 310 is used to acquire the real-time raw data stream of the mining excavator collected by the sensor component, and the real-time raw data stream includes the time queue of the vibration signal and the hydraulic signal. It should be understood that the mining excavator is a complex engineering machinery, and its working state is affected by many factors. The vibration signal and the hydraulic signal are two types of key data reflecting the operating state of the equipment. The vibration signal can directly reflect the dynamic characteristics of the mechanical components, while the hydraulic signal can reveal the working efficiency and control accuracy of the hydraulic system. In addition, there is a certain correlation and complementarity between the vibration signal and the hydraulic signal. For example, the abnormality of the hydraulic system may cause the vibration of the mechanical components to intensify, and the failure of the mechanical components may also affect the stability of the hydraulic system. By comprehensively analyzing these two signals, it is possible to more comprehensively judge whether the equipment is abnormal, and avoid misjudgment or missed judgment that may be caused by single signal analysis. In one example, the acquisition of the vibration signal and the hydraulic signal can be completed by a vibration sensor and a hydraulic sensor. Specifically, the vibration sensor is installed at the key parts of the excavator (such as the engine, hydraulic pump, slewing mechanism, etc.) to monitor the mechanical vibration of the equipment in real time; the hydraulic sensor is installed in the hydraulic system to monitor the pressure, flow and other parameters of the hydraulic oil. Sensors collect data at a high frequency to form a time queue to ensure the real-time and continuity of the data.

[0056] In particular, the wavelet transform module 320 is used to perform a wavelet transform on the time queue of the vibration signal to obtain a vibration time-frequency diagram. Considering the complex and changeable mining environment, the vibration signal generated by the mining excavator during operation not only contains the mechanical characteristics of the equipment itself, but also contains a large amount of external interference information. These factors work together to make the vibration signal present non-stationary and nonlinear characteristics. Although traditional time domain analysis methods (such as mean, variance, etc.) or Fourier transforms can reflect the global frequency domain characteristics of vibration signals, they cannot locate the time point when the vibration frequency changes, and cannot effectively analyze non-stationary signals. Wavelet transform can simultaneously reflect the time domain and frequency domain information of the signal, avoiding the problem of frequency domain information loss in traditional methods, thereby more comprehensively capturing the characteristics of the vibration signal. Therefore, in the technical solution of the present invention, the time queue of the vibration signal is subjected to a wavelet transform to obtain a vibration time-frequency diagram. In this process, wavelet transform can obtain the time-frequency distribution of the signal in different frequency ranges by multi-scale decomposition of the vibration signal, and present it in the form of a time-frequency diagram, where the horizontal axis represents time, the vertical axis represents frequency, and the color depth represents the energy intensity of the signal. In this way, the time point when the frequency change occurs in the signal can be accurately located, which is particularly important for identifying transient anomalies (such as impact and collision). Through wavelet transform, the system can more deeply explore the frequency domain features hidden in the vibration signal, thereby providing richer information support for subsequent feature extraction and state recognition.

[0057] In particular, the frequency domain feature extraction module 330 is used to extract the vibration frequency domain features of the vibration time-frequency graph to obtain the vibration time-frequency feature encoding vector. In the technical solution of the present invention, the vibration time-frequency graph is input into the vibration frequency domain feature extractor based on the Mobile-Former model to obtain the vibration time-frequency feature encoding vector. Considering that the vibration time-frequency graph obtained by wavelet transform contains rich information of the vibration signal in the time and frequency dimensions. However, this information is usually high-dimensional and complex. Directly using the time-frequency graph to identify the working state of the excavator is not only computationally intensive, but also difficult to capture key features. Therefore, it is necessary to perform vibration frequency domain features on the vibration time-frequency graph to mine the key information hidden in the vibration time-frequency graph. Traditional feature analysis methods are difficult to effectively process it. The Mobile-Former architecture combines the lightweight design of MobileNet with the powerful representation ability of Transformer, which not only ensures computational efficiency but also retains the richness of feature expression. Compared with traditional convolutional neural networks, Mobile-Former can not only identify local features, but also capture the correlation between global features through the self-attention mechanism, which is particularly important when analyzing non-stationary signals. Therefore, in the technical solution of the present invention, the vibration time-frequency diagram is input into the vibration frequency domain feature extractor based on the Mobile-Former model to capture the long-distance dependency and local details between different vibration frequencies, so as to obtain the vibration time-frequency feature coding vector. In this way, the actual working state of the mining excavator can be reflected more comprehensively and meticulously, providing a solid data foundation for subsequent state evaluation.

[0058] Specifically, the sequence encoding module 340 is configured to perform sequence encoding on the time queue of the hydraulic signal to obtain a hydraulic time series gradient encoding feature vector. In the technical solution of the present invention, the time queue of the hydraulic signal is input into a sequence encoder based on the LSTM-RNN model to obtain the hydraulic time series gradient encoding feature vector. It should be understood that the mining operation environment is harsh and the working conditions vary widely. The hydraulic system of a mining excavator will exhibit different dynamic characteristics under different operating states. These characteristics not only include instantaneous pressure changes, but also involve long-term operation effects and cumulative effects, which makes it difficult for traditional static analysis methods to accurately describe the working state of the hydraulic system. In addition, the changes in hydraulic signals do not exist in isolation, but are closely connected to the data at other time points, forming a continuous time series. Traditional analysis methods often fail to capture this information correlation in the time dimension, resulting in an incomplete or inaccurate understanding of the working state of the mining excavator. In the technical solution of the present invention, the time queue of the hydraulic signal is input into a sequence encoder based on the LSTM-RNN model to accurately extract key features from the complex and variable hydraulic signal data stream that can reflect the operating conditions of the mining excavator, and obtain the hydraulic time series gradient encoding feature vector. It is worth mentioning that through its internal gating mechanism, such as the forget gate, input gate, and output gate, LSTM-RNN can effectively learn and remember the long-term dependencies in the sequence while filtering out unimportant information. In this way, LSTM-RNN can more precisely capture the subtle changes in hydraulic signals and their development trends over time, providing a solid foundation for subsequent state assessment.

[0059] In particular, the heterogeneous time series feature fast interaction module 350 is used to perform feature interaction on the vibration time-frequency feature encoding vector and the hydraulic time series gradient encoding feature vector based on the anchoring of implicit key time series features between vibration and hydraulic pressure to obtain a multi-modal joint encoding vector of the working state of the mining excavator. It should be understood that the mining operation environment is complex and varied, and the working state of the mining excavator is affected by many factors. The vibration signal and the hydraulic signal provide key information on the operation of the equipment from the perspectives of mechanical vibration and hydraulic system performance, respectively. Although the features extracted from the vibration signal and the hydraulic signal by the deep learning model have certain representation capabilities, these two signals belong to heterogeneous data, distributed in different feature spaces, and have different semantics and forms of expression. For example, the vibration signal reflects the state of the mechanical parts through the frequency domain features, while the hydraulic signal reflects the performance of the hydraulic system through the time series features. If these two features are simply spliced ​​or processed independently, not only will feature redundancy and noise interference result, but also the deep correlation between the vibration feature and the hydraulic feature will be ignored. Therefore, it is necessary to deeply fuse the vibration time-frequency feature coding vector and the hydraulic time series gradient coding feature vector through an efficient feature interaction mechanism to explore the implicit correlation between the two, so as to more comprehensively reflect the operating status of the equipment. Therefore, in the technical solution of the present invention, the vibration time-frequency feature coding vector and the hydraulic time series gradient coding feature vector are quickly interacted based on the heterogeneous time series features anchored by implicit key clues to obtain the multimodal joint coding vector of the working state of the mining excavator. In particular, in this process, firstly, the most representative feature subset is efficiently screened out through dimensionality reduction methods such as feature principal component analysis and implicit feature extraction mechanisms, and non-high-value dimensions and noise parts are filtered out, thereby greatly compressing the dimension of feature representation and reducing computational complexity, ensuring that the subsequent model can focus more on those feature information that are truly useful for mining excavator state identification, and improving the model's discrimination ability. Secondly, by introducing the implicit key clue anchoring network, the system can enable different feature sets to establish a one-to-one mapping or anchor point relationship based on implicit semantic clues, thereby achieving local alignment and global consistency of multi-source features. In this way, the heterogeneity problem in feature fusion is effectively solved, ensuring that feature interaction can be carried out on the basis of semantic consistency, and improving the accuracy and robustness of feature representation. Figure 3As shown, the heterogeneous time-series feature fast interaction module 350 includes: a heterogeneous feature extraction unit 351, configured to extract heterogeneous implicit features of the hydraulic time-series gradient coding feature vector and the vibration time-frequency feature coding vector to obtain a set of hydraulic time-series gradient feature principal component implicit coding vectors and vibration time-frequency implicit feature coding vectors; a feature fast interaction unit 352, configured to perform implicit key clue anchoring and feature fine-grained interaction on the set of hydraulic time-series gradient feature principal component implicit coding vectors and the vibration time-frequency implicit feature coding vectors to obtain the multi-modal joint coding vector of the working state of the mining excavator.

[0060] Specifically, the heterogeneous feature extraction unit 351 is used to extract the heterogeneous implicit features of the hydraulic time series gradient coding feature vector and the vibration time-frequency feature coding vector to obtain a set of hydraulic time series gradient feature principal component implicit coding vectors and a vibration time-frequency implicit feature coding vector. In an embodiment of the present invention, first, the hydraulic time series gradient coding feature vector is subjected to hydraulic time series gradient feature principal component analysis to obtain a set of hydraulic time series gradient feature principal component coding vectors. It should be understood that in actual operation, the time queue of the hydraulic signal contains a large amount of information, which may contain a large number of repeated or insignificant data points without processing, that is, feature redundancy. Therefore, the hydraulic time series gradient coding feature vector is subjected to hydraulic time series gradient feature principal component analysis. By performing feature principal component analysis on the hydraulic time series gradient coding feature vector, the original high-dimensional vibration feature can be converted into a set of more compact and more representative principal component coding vectors to obtain a set of hydraulic time series gradient feature principal component coding vectors. In this way, not only can the data dimension and computational complexity be effectively reduced, but also noise and irrelevant variables can be removed, thereby improving the efficiency and effect of subsequent processing steps. Next, the hydraulic time gradient implicit feature extraction based on point convolution coding is performed on each hydraulic time gradient feature principal component coding vector in the set of the hydraulic time gradient feature principal component coding vector to obtain the set of the hydraulic time gradient feature principal component implicit coding vector. Furthermore, the vibration time-frequency feature coding vector is subjected to vibration time-frequency implicit feature extraction based on point convolution coding to obtain the vibration time-frequency implicit feature coding vector. Considering that the features in the vibration time-frequency feature coding vector are often hidden in the high-dimensional data space, and considering that the understanding of the real operating state of the mining excavator is highly dependent on the interaction between different dimensions. Therefore, in order to accurately capture and understand the working state of the mining excavator, it is necessary to adopt advanced technical means to mine the key information hidden in the complex high-dimensional data and analyze the interaction between the dimensions, so as to more accurately reflect the actual operation of the equipment and potential problems. Traditional analysis methods are often limited to focusing on local spatial information, while ignoring the ability to integrate information across dimensions. Point convolution coding integrates information directly at the feature channel level in a non-local way by using cross-dimensional weight mapping without considering the locality of the original space. In the technical scheme of the present invention, the vibration time-frequency feature coding vector is subjected to vibration time-frequency implicit feature extraction based on point convolution coding to obtain a vibration time-frequency implicit feature coding vector. In this way, the vibration feature expression capability is enhanced, thereby improving the accuracy of identifying the working state of the mining excavator, and ensuring the safety and efficiency of mining production.Similarly, for each hydraulic time - series gradient feature principal - component encoding vector in the set of the hydraulic time - series gradient feature principal - component encoding vectors, perform hydraulic time - series gradient implicit feature extraction based on point - convolution encoding to deeply mine and enhance the key information in the original hydraulic signal, so as to obtain a set of hydraulic time - series gradient feature principal - component implicit encoding vectors. In a specific example of the present invention, the following heterogeneous implicit feature extraction formula is used to extract the heterogeneous implicit features of the hydraulic time - series gradient encoding feature vector and the vibration time - frequency feature encoding vector to obtain a set of hydraulic time - series gradient feature principal - component implicit encoding vectors and vibration time - frequency implicit feature encoding vectors; where, the heterogeneous implicit feature extraction formula is:

[0061]

[0062] H1 = sigmoid[Conv 1×1 (V1)]

[0063]

[0064] Among them, V2 represents the hydraulic time - series gradient encoding feature vector, PCA(·) is the feature principal - component analysis operation, U2 represents the hydraulic time - series gradient feature principal - component orthogonal matrix, where, v 21 , v 22 and v 2m respectively represent the 1st, 2nd, and mth hydraulic time - series gradient feature principal - component encoding vectors in the set of the hydraulic time - series gradient feature principal - component encoding vectors, U2 T is the transpose matrix of U2, λ2 is the hydraulic time - series gradient feature diagonal matrix, λ 21 and λ 2m are the eigenvalues corresponding to v 21 and v 2m respectively, V1 represents the vibration time - frequency feature encoding vector, Conv 1×1 is point - convolution encoding, sigmoid(·) represents the sigmoid function, H1 represents the vibration time - frequency implicit feature extraction to obtain the vibration time - frequency implicit feature encoding vector, h 21 , h 22 , h 2i and h 2m respectively represent the 1st, 2nd, ith, and mth hydraulic time - series gradient feature principal - component implicit encoding vectors in the set of the hydraulic time - series gradient feature principal - component implicit encoding vectors, H2 represents the set of the hydraulic time - series gradient feature principal - component implicit encoding vectors.

[0065] Specifically, the feature fast interaction unit 352 is configured to perform implicit key clue anchoring and feature fine-grained interaction on the set of the hydraulic time-sequence gradient feature principal component implicit encoding vectors and the vibration time-frequency implicit feature encoding vectors to obtain the multi-modal joint encoding vectors of the working state of the mining excavator. In an embodiment of the present invention, first, the vibration time-frequency implicit feature encoding vectors and the set of the hydraulic time-sequence gradient feature principal component implicit encoding vectors are input into an implicit key clue anchoring network to obtain a feature pair of {vibration time-frequency implicit feature encoding vectors, hydraulic time-sequence gradient feature principal component implicit encoding anchored vectors}.

[0066] Specifically, the vibration time-frequency implicit feature encoding vectors and the set of the hydraulic time-sequence gradient feature principal component implicit encoding vectors are input into an implicit key clue anchoring network to obtain a feature pair of {vibration time-frequency implicit feature encoding vectors, hydraulic time-sequence gradient feature principal component implicit encoding anchored vectors}. More specifically, the specific steps of inputting the vibration time-frequency implicit feature encoding vectors and the set of the hydraulic time-sequence gradient feature principal component implicit encoding vectors into the implicit key clue anchoring network are as follows:

[0067] Calculate the similarity between each of the hydraulic time-sequence gradient feature principal component implicit encoding vectors in the vibration time-frequency feature encoding vectors and the set of the hydraulic time-sequence gradient feature principal component implicit encoding vectors to obtain a set of similarities, and extract the position coefficient corresponding to the maximum similarity value in the set of similarities; based on the position coefficient corresponding to the maximum similarity value, anchor the hydraulic time-sequence gradient feature principal component implicit encoding vector in the set of the hydraulic time-sequence gradient feature principal component implicit encoding vectors that has the highest matching similarity with the vibration time-frequency feature encoding vectors to obtain the hydraulic time-sequence gradient feature principal component implicit encoding anchored vector; perform feature pair matching on the vibration time-frequency implicit feature encoding vectors and the hydraulic time-sequence gradient feature principal component implicit encoding anchored vectors to obtain the feature pair of {vibration time-frequency implicit feature encoding vectors, hydraulic time-sequence gradient feature principal component implicit encoding anchored vectors}. That is, by calculating the correlation between the vibration time-frequency implicit feature encoding vectors and the set of the hydraulic time-sequence gradient feature principal component implicit encoding vectors, and based on this, extracting the most valuable feature pair, semantic alignment between multi-dimensional feature spaces is achieved, enabling the vibration signal and the hydraulic signal to be associated and interact in a consistent manner.

[0068] Specifically, during the process of generating feature pairs, the implicit key clue anchoring network not only promotes the establishment of deep associations between vibration signals and hydraulic signals, but also enhances the model's ability to understand complex working condition changes. For example, during excavation operations, if an abnormal wear begins to occur in a certain mechanical component, the vibration signal may show minor but significant changes, and the pressure fluctuations in the hydraulic system may also change accordingly. Through the implicit key clue anchoring network, these subtle associations can be accurately captured, thereby providing a solid foundation for subsequent feature interactions, and further providing strong support for the accurate identification and abnormal detection of the working state of mining excavators. In a specific example of the present invention, the set of the vibration time-frequency implicit feature coding vectors and the hydraulic time-series gradient feature principal component implicit coding vectors is input into the implicit key clue anchoring network according to the following key clue anchoring formula to obtain the {vibration time-frequency implicit feature coding vector, hydraulic time-series gradient feature principal component implicit coding anchor vector} feature pair; wherein, the key clue anchoring formula is:

[0069]

[0070] F bestpair ={H1; h 2k}

[0071] wherein, <·> represents the inner product, ‖·‖ is the Euclidean norm for calculating vectors, ε is the modulation coefficient, arg max j (·) represents returning the j value corresponding to the maximum value, k represents the position coefficient of the corresponding maximum similarity matching value in the set of hydraulic time-series gradient feature principal component implicit coding vectors, h 2k represents the hydraulic time-series gradient feature principal component implicit coding anchor vector, and F bestpair represents the {vibration time-frequency implicit feature coding vector, hydraulic time-series gradient feature principal component implicit coding anchor vector} feature pair.

[0072] Furthermore, input the feature pair of {vibration time-frequency implicit feature coding vector, hydraulic time-series gradient feature principal component implicit coding anchor vector} into the feature fine-grained interaction network to obtain the multi-modal joint coding vector of the working state of the mining excavator. Since the information dimensions carried by the vibration signal and the hydraulic signal are different, directly merging them may lose some important non-linear feature associations. The feature fine-grained interaction network can capture long-distance dependence relationships and multi-dimensional semantic interactions on the basis of existing semantic alignment by introducing advanced technologies such as the multi-head self-attention mechanism, thereby endowing higher flexibility for feature fusion. This mechanism enables the system to more accurately understand and analyze the complex relationship between the vibration and hydraulic signals under complex mine operation conditions, ensuring that no key information that may affect the equipment state is missed. In a specific example of the present invention, the feature pair of {vibration time-frequency implicit feature coding vector, hydraulic time-series gradient feature principal component implicit coding anchor vector} is input into the feature fine-grained interaction network according to the following feature fine-grained interaction formula to obtain the multi-modal joint coding vector of the working state of the mining excavator; wherein, the feature fine-grained interaction formula is:

[0073]

[0074] wherein, h 2k T is the transposed vector of h 2k , softmax is the activation function, S represents the scale of the hydraulic time-series gradient feature principal component implicit coding anchor vector, represents matrix multiplication, α and β are weighted hyperparameters, and V f represents the multi-modal joint coding vector of the working state of the mining excavator.

[0075] In particular, the working state recognition module 360 is configured to determine whether there is an abnormality in the working state of the mining excavator based on the multi-modal joint coding vector of the working state of the mining excavator. In the technical solution of the present invention, the multi-modal joint coding vector of the working state of the mining excavator is input into an anomaly detector based on a classifier to obtain a detection result, and the detection result is used to indicate whether there is an abnormality in the working state of the mining excavator. By inputting the multi-modal joint coding vector of the working state of the mining excavator into the anomaly detector, the state change of the device can be monitored in real time, and potential problems can be discovered at an early stage. Among them, the anomaly detector determines whether the current working state is normal according to the multi-modal joint characterization of the working state of the mining excavator in the multi-modal joint coding vector of the working state of the mining excavator. Once an anomaly is detected, the system can immediately issue an alarm to remind the operator to take necessary measures. In this way, the accurate recognition and anomaly detection of the working state of the mining excavator are realized, which not only provides safety guarantee for mine production, but also improves the intelligent level of equipment management, ensuring the efficiency and safety of mine operations.

[0076] As described above, the mining excavator working state recognition system 300 according to the embodiment of the present invention can be implemented in various wireless terminals, such as a server with a mining excavator working state recognition algorithm. In a possible implementation manner, the mining excavator working state recognition system 300 according to the embodiment of the present invention can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the mining excavator working state recognition system 300 can be a software module in the operating system of the wireless terminal, or can be an application program developed for the wireless terminal; of course, the mining excavator working state recognition system 300 can also be one of the many hardware modules of the wireless terminal.

[0077] Alternatively, in another example, the mining excavator working state recognition system 300 and the wireless terminal can also be separate devices, and the mining excavator working state recognition system 300 can be connected to the wireless terminal through a wired and / or wireless network and transmit interaction information according to a predefined data format.

[0078] Furthermore, a method for recognizing the working state of a mining excavator is also provided.

[0079] Figure 4 It is a flowchart of the method for recognizing the working state of a mining excavator according to the embodiment of the present invention. As Figure 4As shown, the method for identifying the working state of a mining excavator according to an embodiment of the present invention includes the following steps: S1, acquiring a real-time raw data stream of the mining excavator collected by a sensor component, wherein the real-time raw data stream includes a time queue of a vibration signal and a hydraulic signal; S2, performing a wavelet transform on the time queue of the vibration signal to obtain a vibration time-frequency diagram; S3, extracting the vibration frequency domain features of the vibration time-frequency diagram to obtain a vibration time-frequency feature coding vector; S4, performing sequence coding on the time queue of the hydraulic signal to obtain a hydraulic time series gradient coding feature vector; S5, performing a fast interaction of heterogeneous time series features anchored by implicit key clues on the vibration time-frequency feature coding vector and the hydraulic time series gradient coding feature vector to obtain a multimodal joint coding vector of the working state of the mining excavator; S6, determining whether the working state of the mining excavator is abnormal based on the multimodal joint coding vector of the working state of the mining excavator.

[0080] In summary, the method for identifying the working state of a mining excavator according to an embodiment of the present invention is explained, which collects the vibration signal and hydraulic signal of the mining excavator in real time through the sensor component, and uses a neural network model based on deep learning to extract features of the vibration signal and the hydraulic signal to mine the time series dynamic characteristics of the mining excavator in the working state, and further performs fine-grained response interaction between features on the vibration frequency domain features and the hydraulic time series features to more comprehensively reflect the operating state of the mining excavator, and obtain a multi-modal joint representation of the working state of the mining excavator, and based on this, determine whether the working state of the mining excavator is abnormal. In this way, accurate identification and abnormality detection of the working state of the mining excavator can be achieved, providing reliable protection for mine production safety and excavator operation efficiency.

[0081] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A mining excavator working status identification system, characterized in that: include: A data acquisition module, used to acquire a real-time raw data stream of the mining excavator collected by the sensor assembly, wherein the real-time raw data stream includes a time queue of a vibration signal and a hydraulic signal; A wavelet transform module, used for performing wavelet transform on the time queue of the vibration signal to obtain a vibration time-frequency diagram; A frequency domain feature extraction module, used to extract the vibration frequency domain features of the vibration time-frequency graph to obtain a vibration time-frequency feature encoding vector; A sequence encoding module, used for performing sequence encoding on the time queue of the hydraulic signal to obtain a hydraulic time series gradient encoding feature vector; A heterogeneous time series feature fast interaction module is used to perform feature interaction on the vibration time-frequency feature coding vector and the hydraulic time series gradient coding feature vector based on anchoring of implicit key time series features between vibration and hydraulic pressure to obtain a multi-modal joint coding vector of the working state of the mining excavator; The working state identification module is used to determine whether the working state of the mining excavator is abnormal based on the multi-modal joint coding vector of the working state of the mining excavator.

2. The mining excavator working status identification system according to claim 1, characterized in that: The frequency domain feature extraction module is used to: The vibration time-frequency diagram is input into a vibration frequency domain feature extractor based on the Mobile-Former model to obtain the vibration time-frequency feature encoding vector.

3. The mining excavator working status identification system according to claim 2, characterized in that: The sequence encoding module is used to: The time queue of the hydraulic signal is input into a sequence encoder based on the LSTM-RNN model to obtain the hydraulic time series gradient encoding feature vector.

4. The mining excavator working status identification system according to claim 3, characterized in that: The heterogeneous time series feature fast interaction module includes: A heterogeneous feature extraction unit, used to extract heterogeneous implicit features of the hydraulic time series gradient coding feature vector and the vibration time-frequency feature coding vector to obtain a set of hydraulic time series gradient feature principal component implicit coding vectors and a vibration time-frequency implicit feature coding vector; The feature rapid interaction unit is used to perform implicit key clue anchoring and feature fine-grained interaction on the set of the hydraulic time series gradient feature principal component implicit coding vectors and the vibration time-frequency implicit feature coding vectors to obtain the multi-modal joint coding vector of the working state of the mining excavator.

5. The mining excavator working status identification system according to claim 4, characterized in that: The heterogeneous implicit feature extraction unit comprises: A hydraulic time series gradient feature principal component analysis subunit, used for performing a hydraulic time series gradient feature principal component analysis on the hydraulic time series gradient encoding feature vector to obtain a set of hydraulic time series gradient feature principal component encoding vectors; A hydraulic time series gradient implicit feature extraction subunit, used for performing hydraulic time series gradient implicit feature extraction based on point convolution coding on each hydraulic time series gradient feature principal component coding vector in the set of hydraulic time series gradient feature principal component coding vectors to obtain the set of hydraulic time series gradient feature principal component implicit coding vectors; The vibration time-frequency implicit feature extraction subunit is used to perform vibration time-frequency implicit feature extraction based on point convolution coding on the vibration time-frequency feature coding vector to obtain the vibration time-frequency implicit feature coding vector.

6. The mining excavator working status identification system according to claim 5, characterized in that: The feature rapid interaction unit includes: An implicit key clue anchoring subunit, used for inputting the set of the vibration time-frequency implicit feature coding vector and the hydraulic time series gradient feature principal component implicit coding vector into an implicit key clue anchoring network to obtain a feature pair of {vibration time-frequency implicit feature coding vector, hydraulic time series gradient feature principal component implicit coding anchor vector}; The feature fine-grained interaction subunit is used to input the {vibration time-frequency implicit feature coding vector, hydraulic time series gradient feature principal component implicit coding anchor vector} feature pair into the feature fine-grained interaction network to obtain the multi-modal joint coding vector of the working state of the mining excavator.

7. The mining excavator working status identification system according to claim 6, characterized in that: The implicit key clue anchoring subunit is used to: Calculating the similarity between the vibration time-frequency feature coding vector and each hydraulic time series gradient feature principal component implicit coding vector in the set of the hydraulic time series gradient feature principal component implicit coding vector to obtain a similarity set, and extracting the position coefficient corresponding to the maximum similarity value in the similarity set; Based on the position coefficient corresponding to the maximum similarity value, anchoring the hydraulic time series gradient feature principal component implicit coding vector with the highest matching similarity to the vibration time-frequency feature coding vector in the set of the hydraulic time series gradient feature principal component implicit coding vectors to obtain the hydraulic time series gradient feature principal component implicit coding anchor vector; The vibration time-frequency implicit feature coding vector and the hydraulic time series gradient feature principal component implicit coding anchor vector are matched with each other to obtain the {vibration time-frequency implicit feature coding vector, hydraulic time series gradient feature principal component implicit coding anchor vector} feature pair.

8. The mining excavator working status identification system according to claim 7, characterized in that: The working status identification module comprises: The multimodal joint encoding vector of the working state of the mining excavator is input into a classifier-based anomaly detector to obtain a detection result, and the detection result is used to indicate whether there is an abnormality in the working state of the mining excavator.

9. A method for identifying the working state of a mining excavator, characterized in that: include: Acquire a real-time raw data stream of the mining excavator collected by the sensor assembly, wherein the real-time raw data stream includes a time queue of a vibration signal and a hydraulic signal; Performing wavelet transformation on the time sequence of the vibration signal to obtain a vibration time-frequency diagram; Extracting the vibration frequency domain features of the vibration time-frequency graph to obtain a vibration time-frequency feature encoding vector; Performing sequence encoding on the time queue of the hydraulic signal to obtain a hydraulic time series gradient encoding feature vector; Performing a fast interaction of heterogeneous time series features based on implicit key clue anchoring on the vibration time-frequency feature coding vector and the hydraulic time series gradient coding feature vector to obtain a multi-modal joint coding vector of the working state of the mining excavator; Based on the multi-modal joint coding vector of the working state of the mining excavator, it is determined whether the working state of the mining excavator is abnormal.

10. The method for identifying the working state of a mining excavator according to claim 9, characterized in that: The step of extracting the vibration frequency domain features of the vibration time-frequency graph to obtain a vibration time-frequency feature coding vector includes: The vibration time-frequency diagram is input into a vibration frequency domain feature extractor based on the Mobile-Former model to obtain the vibration time-frequency feature encoding vector.

Images