Vertical shaft cage guide anomaly detection method based on multi-mode sound and vibration fusion and deep learning
By employing multimodal acoustic-vibration fusion and deep learning methods, early, accurate, and interpretable warnings for coal mine vertical shaft conveyors are achieved, solving the problems of low detection efficiency and insufficient accuracy in existing technologies. This approach also demonstrates strong noise robustness and engineering applicability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CITIC HIC KAICHENG INTELLIGENT EQUIP CO LTD
- Filing Date
- 2026-02-11
- Publication Date
- 2026-05-05
AI Technical Summary
In existing technologies, the detection methods for coal mine vertical shaft guideways are inefficient, highly subjective, unable to provide real-time early warnings, and insensitive to complex or early minor faults. Furthermore, the fusion of single-modal data is insufficient, resulting in inadequate diagnostic accuracy and robustness.
By employing a multimodal acoustic-vibration fusion and deep learning approach, vibration and sound pressure sensors are installed on the cage to synchronously collect signals, perform multimodal feature extraction and deep fusion, utilize cross-modal attention and graph neural networks for feature fusion, and perform temporal modeling through a gated recurrent unit network. The model is then optimized by combining a focus loss function to achieve early and accurate early warning.
It achieves comprehensive, blind-spot-free monitoring of vertical shaft tank passages, improves the ability to detect early faults, has strong noise robustness and interpretability, and supports all-weather unmanned automatic monitoring and early warning.
Smart Images
Figure CN121980424A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mine safety monitoring and intelligent diagnosis technology, and in particular to a method for detecting anomalies in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning. Background Technology
[0002] Coal mine vertical shaft hoisting cages are key guiding structures for the safe and stable operation of hoisting cages. Common problems such as loosening, wear, deformation, or joint failures can directly threaten the safety of mine hoisting. Traditional detection methods mainly rely on periodic manual inspections or monitoring by single sensors such as vibration sensors, which have problems such as low efficiency, strong subjectivity, inability to provide real-time early warnings, and insensitivity to complex or early-stage minor faults.
[0003] Vibration signals can effectively reflect the low-frequency overall vibration and deformation modes of the tank structure, but their ability to capture high-frequency transient impacts such as micro-cracks and particle collisions is limited. Sound signals, on the other hand, are rich in broadband information and are particularly good at capturing transient events such as high-frequency friction and impact generated at the contact surface between the tank and the rollers, but their characterization of low-frequency resonance of the structure is weak. Existing deep learning-based methods mostly use single-mode data or simple multi-mode feature splicing, failing to deeply integrate the complementary advantages of sound and vibration signals from the physical mechanism and data level, resulting in insufficient diagnostic accuracy and robustness for early and complex faults in complex operating environments. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for detecting abnormalities in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning. This method enables synchronous and accurate acquisition and deep fusion analysis of acoustic-vibration signals, and can adaptively learn the temporal evolution law of faults to achieve early, accurate, and interpretable early warning of abnormalities in tank passages.
[0005] The technical solution adopted in this invention is: a method for detecting anomalies in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning, comprising the following steps: Step 1: Synchronous acquisition and preprocessing of multimodal data; Vibration acceleration sensors are installed at the rigid connection points of the cage's guide frame, and sound pressure sensors are installed inside or on top of the cage; a multi-channel synchronous data acquisition card is used to synchronously acquire vibration signals during the cage's operation at the same time reference. With sound signals The acquired signals are time-aligned and divided into analysis sample segments of fixed duration. Step 2: Multimodal feature extraction; including: Vibration feature extraction: Extract the wavelet packet energy entropy features of the vibration signal to form a vibration feature vector; Sound feature extraction: Extracting handcrafted time-frequency domain features of the sound signal and deep acoustic features from a convolutional neural network based on Mel spectrograms to form a sound feature vector; Step 3: Deep feature fusion based on cross-modal attention and graph neural networks; The design incorporates a fusion module with a bidirectional cross-attention mechanism and a graph convolutional network to deeply fuse vibration and sound feature vectors, yielding the final deeply fused feature vector. ; Step 4: Temporal modeling and anomaly detection; The time-series continuous fused feature vectors are input into the gated recurrent unit network for modeling, and the output is the anomaly probability representing the abnormal state of the tank passage. Step 5: Model Training and Optimization; The model was trained using historical normal and abnormal data, and optimized using the Focal Loss function. Step Six: Online Monitoring and Tiered Early Warning; The trained model is deployed to edge computing devices in the mine or cloud servers; the collected acoustic and vibration data is processed in real time to calculate the probability of anomalies. Establish multi-level early warning thresholds: when When this occurs, a basic warning is triggered, prompting the user to pay attention; when multiple windows are in succession... When this occurs, a medium-level alarm is triggered, prompting a request to arrange maintenance; when If the value continues to rise rapidly and exceeds 0.95, an emergency alarm will be triggered, and it is recommended to stop the machine immediately for inspection.
[0006] Specifically, in step one, the sound signal... Preprocessing also includes: For sound signals Pre-emphasis processing is performed to enhance high-frequency components: ;in ; Background noise collected during the stationary period of the hoist was then utilized. The audio signal during operation is subjected to spectral subtraction and noise reduction processing to suppress downhole environmental noise interference. ;in, and These are the short-time amplitude spectra of the operating signal and the noise signal, respectively. , The number of frequency segments for a sound signal. For over-subtraction factor, This is the lower limit parameter of the spectrum.
[0007] Specifically, the vibration feature extraction in step two is as follows: Vibration signals Four-level wavelet packet decomposition was performed to obtain 16 sub-band signals. ; Calculate the energy entropy of each sub-band as a feature: In the formula, Indicates the first Energy entropy of each sub-band Indicates the first The first sub-band One unit of energy Indicates the first The first sub-band The expression for the sub-band signal of each component; The energy entropy calculation results of the 16 sub-frequency band signals are used to construct the vibration characteristic vector: .
[0008] Specifically, the sound feature extraction in step two includes: Step 2.1, Time-Frequency Domain Manual Features: Calculate the time-frequency domain features of the audio signal after spectral subtraction and noise reduction: short-time zero-crossing rate (ZCR), short-time energy, spectral centroid (SC), spectral roll-off point (SR), and sub-band energy ratio (SER) based on the tank duct fault characteristics. These features together constitute the manual feature vector. ; ; Step 2.2, Depth Acoustic Features: Convert the sound signal into a Mel-spectrum matrix. The input is a lightweight multi-branch convolutional neural network (CNN); this network contains two branches that focus on low-frequency periodic textures and high-frequency random impacts, respectively, to extract high-level abstract acoustic features. The formula is as follows: In the formula, For activation function, This indicates a splicing operation. This indicates the convolution calculation of two branches; High-level abstract acoustic features Compared with handmade feature vectors The sound features are then spliced and fused, and then reduced in dimensionality using a fully connected layer to obtain a deeply fused sound feature vector. ; In the formula, This represents the computation function for the fully connected layer.
[0009] More specifically, the time-frequency domain feature calculation in step 2.1 is as follows: Assumption The time-domain discrete representation is a frame with a length of Discrete sound signals , Its frequency domain expression is , , Let the number of frequency segments of the sound signal be the number of segments; then the formula for calculating the time-frequency domain characteristics is as follows: Short-time zero crossing rate: In the formula, For symbolic functions, the expression is: ; Short-term energy: In the formula, For window functions, this invention uses a Hamming window, where n represents the nth Hamming window; Spectral centroid: ; The roll-off point represents the frequency at which the accumulated spectral energy reaches a certain proportion of the total energy, and is used to describe the tail characteristics of the spectrum; the roll-off point is the smallest index that satisfies the following conditions. : In the formula, Given the roll-off ratio, the frequency corresponding to the spectral roll-off point is: In the formula, The sampling frequency; The sub-band energy ratio divides the spectrum into multiple sub-bands and calculates the proportion of energy in each sub-band to the total energy. It is used to describe the frequency domain distribution of energy, and its formula is as follows: In the formula, This represents the number of spectral subbands of the sound signal. For the first The energy carried by a person.
[0010] Specifically, step three includes: Step 3.1, Dual-channel cross-attention enhancement: Calculate the attention weights of vibration features on sound features respectively. Attention weights of sound features to vibration features ; ; ;in, For the learnable weight matrix; generate the enhanced features: , ; Step 3.2, Graph Convolution Fusion: Construct a graph from the dual-modal features of each time sample. , where nodes Represent and Edge weights are initialized using a learnable correlation matrix; graph convolution operations are performed to aggregate cross-modal information. ;in, and The first Layer and first The graph convolution node matrix of the layer, It is an adjacency matrix with self-connections. Its degree matrix, For the first Layer trainable weights The activation function is used; after two layers of graph convolution, the output node features are obtained. ,Will conduct The concatenation yields the final deep fusion feature vector. .
[0011] Specifically, step four involves: processing the deep fusion feature sequences obtained from continuous time windows. The input is fed into the gated recurrent unit (GRU) network to learn the temporal evolution pattern of faults. The specific calculation formula is as follows: First, calculate the candidate hidden states. : In the formula, For activation function, For hidden layer weights, This indicates that the door is being reset. This indicates the final hidden state of the previous layer. This indicates element-wise multiplication. Further calculation of the final hidden state : In the formula, This represents the update gate. The final hidden state passes through a fully connected layer and a sigmoid activation function, outputting the anomaly probability of the current sequence. : In the formula, This represents the Sigmoid activation function. This represents the weight matrix of the output layer. This represents the output layer bias matrix. This indicates the final hidden state.
[0012] Specifically, step five, which uses the Focal Loss function for optimization, involves the following steps: ; in Represents the focus loss function. Indicates the true label, and These are used to control the ratio of positive to negative samples, and their values range from [value range missing]. , This is called the focusing parameter, and its value range is... ;when At that time, Focal Loss degenerates into cross-entropy loss. The larger the value, the greater the penalty for easily classified samples.
[0013] Due to the adoption of the technical solution described above, the present invention has the following advantages: 1. Multimodal physical complementarity: By comprehensively utilizing the sensitivity of vibration signals to low-frequency structural faults and the sensitivity of sound signals to high-frequency transient events, a more comprehensive and blind-spot-free monitoring of the health status of the tank passage is achieved.
[0014] 2. Deep Feature Fusion: The innovative “cross-attention + graph neural network” fusion module can dynamically learn and model the complex nonlinear correlations and complementary relationships between acoustic and vibration signals, breaking through the limitations of simple feature splicing and significantly improving the representational ability of fused features.
[0015] 3. Strong noise robustness: A spectral subtraction and noise reduction preprocessing procedure for sound signals was designed to address the harsh acoustic environment downhole, effectively improving the quality and reliability of sound modal data.
[0016] 4. Strong early warning capability: The GRU network can effectively capture the temporal dynamic characteristics of fault occurrence and development. Combined with the focus loss function to pay attention to difficult samples, the model is more sensitive to early and weak anomalies.
[0017] 5. Decision interpretability: The system can not only output the probability of anomalies, but also provide contribution analysis of each modality, which helps maintenance personnel to quickly locate the source of the fault and improves the engineering practical value of the method.
[0018] 6. High engineering applicability: The solution supports edge-cloud collaborative computing, is compatible with existing mine industrial networks, and can realize all-weather unmanned automatic monitoring and early warning. Attached Figure Description
[0019] Figure 1 This is a schematic diagram of the overall process of the present invention. Detailed Implementation
[0020] The present invention will be further explained and described below with reference to the accompanying drawings and embodiments. However, this should not be construed as limiting the scope of protection of the present invention. The purpose of disclosing the present invention is to protect all technical improvements within the scope of the present invention.
[0021] Combined with appendix Figure 1 The method for detecting anomalies in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning, as shown, includes the following steps: Step 1: Synchronous acquisition and preprocessing of multimodal data; Vibration acceleration sensors are installed at the rigid connections of the four guide frames of the cage, and sound pressure sensors with wind noise shields are installed inside or on top of the cage; a multi-channel synchronous data acquisition card is used, and the sampling rate of the acquisition card is set to: vibration channel audio channel Vibration signals during the operation of the tank conveyor are collected synchronously using the same time reference. With sound signals Collect three months of normal operation data, as well as labeled abnormal data including known loosening and wear faults; time-align the collected signals, and all data are divided into 2-second analysis sample segments.
[0022] For sound signals First, pre-emphasis processing is performed to enhance high-frequency components: ;in ; Background noise collected during the stationary period of the hoist was then utilized. The audio signal during operation is subjected to spectral subtraction and noise reduction processing to suppress downhole environmental noise interference. ;in, and These are the short-time amplitude spectra of the operating signal and the noise signal, respectively. , The number of frequency segments for a sound signal. For over-subtraction factor, This is the lower limit parameter of the spectrum.
[0023] Step 2: Multimodal feature extraction; including: Step 2.1, Vibration Feature Extraction: Extracting vibration signals... Four-level wavelet packet decomposition was performed to obtain 16 sub-band signals. ; Calculate the energy entropy of each sub-band as a feature: In the formula, Indicates the first Energy entropy of each sub-band Indicates the first The first sub-band One unit of energy Indicates the first The first sub-band The expression for the sub-band signal of each component; The energy entropy calculation results of the 16 sub-frequency band signals are used to construct the vibration characteristic vector: .
[0024] Step 2.2, Sound Feature Extraction; including: Step 2.2.1, Time-Frequency Domain Manual Features: Calculate the time-frequency domain features of the audio signal after spectral subtraction and noise reduction: short-time zero-crossing rate (ZCR), short-time energy, spectral centroid (SC), spectral roll-off point (SR), and sub-band energy ratio (SER) based on the tank duct fault characteristics. These features together constitute the manual feature vector. ; ; Assumption The time-domain discrete representation is a frame with a length of Discrete sound signals , Its frequency domain expression is , , Let the number of frequency segments of the sound signal be the number of segments; then the formula for calculating the time-frequency domain characteristics is as follows: Short-time zero crossing rate: In the formula, For symbolic functions, the expression is: Short-time zero-crossing rate measures the number of times a signal crosses zero within a unit of time, and is often used to distinguish between low ZCR for voiced sounds and high ZCR for unvoiced sounds. Short-term energy: In the formula, As a window function, this invention uses a Hamming window, where n represents the nth Hamming window; short-time energy reflects the intensity change of the signal within a short time window and is an important indicator for distinguishing speech segments from silence segments.
[0025] Spectral centroid: The centroid of the spectrum represents the center of gravity of the spectrum and reflects the brightness of the sound. The more high-frequency components there are, the higher the centroid is. The roll-off point represents the frequency at which the accumulated spectral energy reaches a certain proportion of the total energy, typically 85%, and is used to describe the tail characteristics of the spectrum. The roll-off point is the smallest index that satisfies the following conditions. : In the formula, As the roll-off ratio, which is usually taken as 0.85, the frequency corresponding to the roll-off point is: In the formula, The sampling frequency is a key feature; this feature has a good ability to distinguish audio types. The sub-band energy ratio divides the spectrum into multiple sub-bands and calculates the proportion of energy in each sub-band to the total energy. It is used to describe the frequency domain distribution of energy, and its formula is as follows: In the formula, This represents the number of spectral subbands of the sound signal. For the first The energy of each sub-band; this ratio reflects the distribution of energy in different frequency bands.
[0026] Step 2.2.2, Depth Acoustic Features: Convert the sound signal into a Mel-spectrum matrix. The input is a lightweight multi-branch convolutional neural network (CNN); this network contains two branches that focus on low-frequency periodic textures and high-frequency random impacts, respectively, to extract high-level abstract acoustic features. The formula is as follows: In the formula, For activation function, This indicates a splicing operation. This indicates the convolution calculation of two branches.
[0027] High-level abstract acoustic features Compared with handmade feature vectors The sound features are then spliced and fused, and then reduced in dimensionality using a fully connected layer to obtain a deeply fused sound feature vector. ; In the formula, This represents the computation function for the fully connected layer.
[0028] Step 3: Deep feature fusion based on cross-modal attention and graph neural networks; The design incorporates a fusion module with a bidirectional cross-attention mechanism and a graph convolutional network to deeply fuse vibration and sound feature vectors, yielding the final deeply fused feature vector. ;include: Step 3.1, Dual-channel cross-attention enhancement: Calculate the attention weights of vibration features on sound features respectively. Attention weights of sound features to vibration features ; ; ; in, For the learnable weight matrix; generate the enhanced features: , ; Step 3.2, Graph Convolution Fusion: Construct a graph from the dual-modal features of each time sample. , where nodes Represent and Edge weights are initialized using a learnable correlation matrix; graph convolution operations are performed to aggregate cross-modal information. ;in, and The first Layer and first The graph convolution node matrix of the layer, It is an adjacency matrix with self-connections. Its degree matrix, For the first Layer trainable weights The activation function is used; after two layers of graph convolution, the output node features are obtained. ,Will conduct The concatenation yields the final deep fusion feature vector. .
[0029] Step 4: Temporal modeling and anomaly detection; The temporally continuous fused feature vectors are input into a gated recurrent unit network for modeling, and the output is the probability of anomalies representing abnormal conditions in the tank passage; specifically: The fused feature sequence obtained from the continuous time window The input is fed into the gated recurrent unit (GRU) network to learn the temporal evolution pattern of the fault, as shown in the following formula: First, calculate the candidate hidden states. : In the formula, For activation function, For hidden layer weights, This indicates that the door is being reset. This indicates the final hidden state of the previous layer. This indicates element-wise multiplication. Further calculation of the final hidden state : In the formula, This represents the update gate. The final hidden state passes through a fully connected layer and a sigmoid activation function, outputting the anomaly probability of the current sequence. : In the formula, This represents the Sigmoid activation function. This represents the weight matrix of the output layer. This represents the output layer bias matrix. This indicates the final hidden state.
[0030] Step 5: Model Training and Optimization; 80% of the data was used as the training set and 20% as the test set; the optimizer was Adam, with an initial learning rate of 0.001. The model was trained using historical normal and abnormal data. To address the data imbalance problem caused by the limited number of abnormal samples in real-world scenarios, the Focal Loss function was used for optimization. ;in Represents the focus loss function. Indicates the true label, and These are used to control the ratio of positive to negative samples, and their values range from [value range missing]. , This is called the focusing parameter, and its value range is... The aim is to reduce the weight of easily classified samples, thus allowing the model to focus more on difficult samples during training; when At that time, Focal Loss degenerates into cross-entropy loss. The larger the value, the greater the penalty for easily classified samples.
[0031] Step Six: Online Monitoring and Tiered Early Warning; The trained model is deployed to edge computing devices in the mine or a cloud server; a one-month online test is conducted; the collected acoustic and vibration data is processed in real time to calculate the probability of anomalies. Establish multi-level early warning thresholds: when When this occurs, a basic warning is triggered, prompting the user to pay attention; when multiple windows are in succession... When this occurs, a medium-level alarm is triggered, prompting a request to arrange maintenance; when When the value continues to rise rapidly and exceeds 0.95, an emergency alarm is triggered, and it is recommended to immediately stop the machine for inspection. At the same time, the system can combine methods such as gradient backpropagation to calculate the contribution of vibration and sound modes to the current decision, providing maintenance personnel with interpretable prompts such as whether vibration or sound is dominant.
[0032] Through the above embodiments, the system successfully issued warnings for two potential loosening of guide bolts, with an anomaly probability consistently between 0.8 and 0.9, which were confirmed by manual verification. Under normal operating conditions, the system's false alarm rate is less than 0.5%. Each warning was attributed to 70% vibration and 30% sound, allowing maintenance personnel to focus their inspections on structural connections and quickly pinpoint the problem. These embodiments demonstrate the effectiveness, accuracy, and engineering applicability of the method of this invention.
[0033] The parts of this invention not described in detail are prior art.
[0034] The embodiments selected herein for the purpose of disclosing the inventive objectives are currently considered suitable; however, it should be understood that the invention is intended to include all variations and modifications of the embodiments that fall within the scope of this concept and invention.
Claims
1. A method for detecting anomalies in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning, characterized in that, Includes the following steps: Step 1: Synchronous acquisition and preprocessing of multimodal data; Install vibration acceleration sensors at the rigid connection of the guide frame of the cage, and install sound pressure sensors inside or on top of the cage. A multi-channel synchronous data acquisition card was used to synchronously acquire vibration signals during the operation of the tank guideway at the same time reference. With sound signals ; The acquired signals are time-aligned and divided into analysis sample segments of fixed duration; Step 2: Multimodal feature extraction; including: Vibration feature extraction: Extract the wavelet packet energy entropy features of the vibration signal to form a vibration feature vector; Sound feature extraction: Extracting handcrafted time-frequency domain features of the sound signal and deep acoustic features from a convolutional neural network based on Mel spectrograms to form a sound feature vector; Step 3: Deep feature fusion based on cross-modal attention and graph neural networks; The design incorporates a fusion module with a bidirectional cross-attention mechanism and a graph convolutional network to deeply fuse vibration and sound feature vectors, yielding the final deeply fused feature vector. ; Step 4: Temporal modeling and anomaly detection; The time-series continuous fused feature vectors are input into the gated recurrent unit network for modeling, and the output is the anomaly probability representing the abnormal state of the tank passage. Step 5: Model Training and Optimization; The model was trained using historical normal and abnormal data, and optimized using the Focal Loss function. Step Six: Online Monitoring and Tiered Early Warning; The trained model is deployed to edge computing devices in the mine or cloud servers; the collected acoustic and vibration data is processed in real time to calculate the probability of anomalies. Establish multi-level early warning thresholds: when When this occurs, a basic warning is triggered, prompting the user to pay attention; when multiple windows are in succession... When this occurs, a medium-level alarm is triggered, prompting a request to arrange maintenance; when If the value continues to rise rapidly and exceeds 0.95, an emergency alarm will be triggered, and it is recommended to immediately stop the machine for inspection.
2. The method for anomaly detection in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning according to claim 1, characterized in that: In step one, the sound signal Preprocessing also includes: For sound signals Pre-emphasis processing is performed to enhance high-frequency components: ;in ; Background noise collected during the stationary period of the hoist was then utilized. The audio signal during operation is subjected to spectral subtraction and noise reduction processing to suppress downhole environmental noise interference. ;in, and These are the short-time amplitude spectra of the operating signal and the noise signal, respectively. , The number of frequency segments for a sound signal. For over-subtraction factor, This is the lower limit parameter of the spectrum.
3. The method for anomaly detection in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning according to claim 1, characterized in that: The vibration feature extraction in step two specifically involves: Vibration signals Four-level wavelet packet decomposition was performed to obtain 16 sub-band signals. ; Calculate the energy entropy of each sub-band as a feature: In the formula, Indicates the first Energy entropy of each sub-band Indicates the first The first sub-band One unit of energy Indicates the first The first sub-band The expression for the sub-band signal of each component; The energy entropy calculation results of the 16 sub-frequency band signals are used to construct the vibration characteristic vector: .
4. The method for anomaly detection in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning according to claim 1, characterized in that: The sound feature extraction in step two includes: Step 2.1, Time-Frequency Domain Manual Features: Calculate the time-frequency domain features of the audio signal after spectral subtraction and noise reduction: short-time zero-crossing rate (ZCR), short-time energy, spectral centroid (SC), spectral roll-off point (SR), and sub-band energy ratio (SER) based on the tank duct fault characteristics. These features together constitute the manual feature vector. ; ; Step 2.2, Depth Acoustic Features: Convert the sound signal into a Mel-spectrum matrix. The input is a lightweight multi-branch convolutional neural network (CNN); this network contains two branches that focus on low-frequency periodic textures and high-frequency random impacts, respectively, to extract high-level abstract acoustic features. The formula is as follows: In the formula, For activation function, This indicates a splicing operation. This indicates the convolution calculation of two branches; High-level abstract acoustic features Compared with handmade feature vectors The sound features are then spliced and fused, and then reduced in dimensionality using a fully connected layer to obtain a deeply fused sound feature vector. ; In the formula, This represents the computation function for the fully connected layer.
5. The method for anomaly detection in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning according to claim 4, characterized in that: The time-frequency domain feature calculation in step 2.1 is specifically as follows: Assumption The time-domain discrete representation is a frame with a length of Discrete sound signals , Its frequency domain expression is , , Let the number of frequency segments of the sound signal be the number of segments; then the formula for calculating the time-frequency domain characteristics is as follows: Short-time zero crossing rate: In the formula, For symbolic functions, the expression is: ; Short-term energy: In the formula, For window functions, this invention uses a Hamming window, where n represents the nth Hamming window; Spectral centroid: ; The roll-off point represents the frequency at which the accumulated spectral energy reaches a certain proportion of the total energy, and is used to describe the tail characteristics of the spectrum; the roll-off point is the smallest index that satisfies the following conditions. : In the formula, Given the roll-off ratio, the frequency corresponding to the spectral roll-off point is: In the formula, The sampling frequency; The sub-band energy ratio divides the spectrum into multiple sub-bands and calculates the proportion of energy in each sub-band to the total energy. It is used to describe the frequency domain distribution of energy, and its formula is as follows: In the formula, This represents the number of spectral subbands of the sound signal. For the first The energy carried by a person.
6. The method for anomaly detection in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning according to claim 5, characterized in that: Step three specifically includes: Step 3.1, Dual-channel cross-attention enhancement: Calculate the attention weights of vibration features on sound features respectively. Attention weights of sound features to vibration features ; ; ;in, For the learnable weight matrix; generate the enhanced features: , ; Step 3.2, Graph Convolution Fusion: Construct a graph from the dual-modal features of each time sample. , where nodes Represent and Edge weights are initialized using a learnable correlation matrix; graph convolution operations are performed to aggregate cross-modal information. ;in, and The first Layer and first The graph convolution node matrix of the layer, It is an adjacency matrix with self-connections. Its degree matrix, For the first Layer trainable weights The activation function is used; after two layers of graph convolution, the output node features are obtained. ,Will conduct The concatenation yields the final deep fusion feature vector. .
7. The method for anomaly detection in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning according to claim 6, characterized in that: Step four specifically involves: processing the deep fusion feature sequences obtained from continuous time windows. The input is fed into the gated recurrent unit (GRU) network to learn the temporal evolution pattern of faults. The specific calculation formula is as follows: First, calculate the candidate hidden states. : In the formula, For activation function, For hidden layer weights, This indicates that the door is being reset. This indicates the final hidden state of the previous layer. This indicates element-wise multiplication. Further calculation of the final hidden state : In the formula, This represents the update gate. The final hidden state passes through a fully connected layer and a sigmoid activation function, outputting the anomaly probability of the current sequence. : In the formula, This represents the Sigmoid activation function. This represents the weight matrix of the output layer. This represents the output layer bias matrix. This indicates the final hidden state.
8. The method for anomaly detection in vertical shaft tank passages based on multimodal acoustic-vibration fusion and deep learning according to claim 7, characterized in that: Step five, which uses the Focal Loss function for optimization, specifically involves: ; in Represents the focus loss function. Indicates the true label, and These are used to control the ratio of positive to negative samples, and their values range from [value range missing]. , This is called the focusing parameter, and its value range is... ;when At this point, FocalLoss degenerates into cross-entropy loss. The larger the value, the greater the penalty for easily classified samples.