A method and system for evaluating power equipment status based on machine learning

Abstract

The present application provides a method and system for evaluating the status of power equipment based on machine learning, which relates to the field of equipment detection technology. The method acquires channel status information by deploying Wi-Fi transmitters and receivers around the power equipment, and combines the mechanical vibration or partial discharge signals collected by acoustic sensors deployed around the equipment, and respectively uses deep denoising and adaptive filtering-time-frequency transformation to extract key features. Subsequently, the denoised CSI features and the filtered acoustic features are input into a multimodal fusion model to achieve an accurate assessment of the operating status of the power equipment. This solution can not only capture the weak fault signs of the equipment under the background of strong electromagnetic interference and complex noise, but also has the advantages of non-invasive, low power consumption and real-time online monitoring, providing reliable support for the safe operation and intelligent operation and maintenance of the power system.

Description

Technical Field

[0001] The present application relates to the field of equipment detection, and specifically, to a method and system for evaluating the status of power equipment based on machine learning. Background Art

[0002] During operation, power equipment generates a variety of physical signals, such as electromagnetic radiation, mechanical vibration, and partial discharge (PD). Traditional condition monitoring methods mainly target one type of signal. For example, infrared temperature measurement is used to detect the temperature rise on the surface of the equipment, but it is easily affected by the ambient temperature and can only monitor surface abnormalities; partial discharge detection (such as ultra-high frequency sensors, ultra-high frequency antennas, etc.) can sense insulation discharge signals, but these electromagnetic methods are sensitive to external wireless interference and the sensors are expensive and complex to install. For example, although the UHF sensor is sensitive when installed inside the transformer, it is easily interfered by communication signals when installed externally.

[0003] The above methods are subject to noise interference and deployment cost limitations in complex field environments, making it difficult to obtain equipment status information in a timely and comprehensive manner. This has prompted us to look for new non-invasive, low-cost sensing methods. Wi-Fi communication devices are now almost everywhere. If their wireless signals can be used as sensing media and combined with sensitive acoustic sensors, it is expected to break through the limitations of traditional methods. However, this also brings new challenges: Wi-Fi signals are in a complex electromagnetic environment and are susceptible to multipath and interference; acoustic sensing may pick up environmental noise. Therefore, innovative technical solutions are needed to integrate these two methods to reliably extract the status characteristics of power equipment in a strong noise background. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present application provides a method and system for evaluating the status of power equipment based on machine learning.

[0005] In a first aspect, the present application provides a method for evaluating the state of an electric power device based on machine learning, comprising:

[0006] Deploy a Wi-Fi transmitter and at least one receiver around the power equipment to collect channel status information;

[0007] Deploy at least one acoustic sensor around the power equipment to collect acoustic signals generated during the operation of the power equipment;

[0008] Performing deep denoising on the channel state information to generate a CSI feature for characterizing the operating state of the power equipment;

[0009] Adaptively filtering and time-frequency transforming the acoustic signal to generate acoustic features for characterizing the electric power equipment in different frequency bands;

[0010] The CSI features and the acoustic features are input into a multimodal fusion model, the operating status of the power equipment is evaluated through an attention mechanism, and a power equipment status evaluation result is generated.

[0011] As an optional implementation manner, collecting the channel status information includes:

[0012] Deploy a plurality of receiving terminals at multiple locations around the power equipment so that the receiving terminals form a distributed MIMO array;

[0013] After the deployment of the plurality of receiving ends is completed, performing phase calibration and environmental baseline scanning on the initial channel state information received by each of the receiving ends to generate reference data of the environmental reflection path and a phase calibration result;

[0014] The initial channel state information is corrected based on the reference data and the phase calibration result.

[0015] As an optional implementation manner, the deploying at least one acoustic sensor around the electric power equipment to collect acoustic signals generated during the operation of the electric power equipment includes:

[0016] Arrange a plurality of acoustic sensors at multiple locations around the power equipment so that the acoustic sensors form an acoustic sensor array and collect acoustic signals generated during the operation of the power equipment;

[0017] In response to the completion of the deployment of the acoustic sensor array, a known calibration sound source signal is provided to the acoustic sensor, and the calibration sound source signal received by each acoustic sensor is analyzed to obtain the gain and relative delay of the acoustic sensor;

[0018] Based on the gain and the relative delay, beamforming is performed on actual acoustic signals collected by the acoustic sensor array within a target frequency band.

[0019] As an optional implementation manner, when collecting channel state information and collecting acoustic signals, the method further includes:

[0020] Based on the metal shell structure of the power equipment and the on-site noise bandwidth, a mixed calibration signal is generated; wherein the mixed calibration signal includes: a radio frequency modulation component and an ultrasonic pulse component;

[0021] Injecting the mixed calibration signal into the environment surrounding the power equipment, so that the receiving end and the acoustic sensor array receive the mixed calibration signal simultaneously;

[0022] Respectively analyzing the radio frequency modulation component and the ultrasonic pulse component received by the Wi-Fi receiving end and the acoustic sensor array to obtain relative delay and amplitude difference;

[0023] The channel state information and the acoustic signal are cross-modally calibrated according to the relative time delay and amplitude difference.

[0024] As an optional implementation manner, the generating of the CSI feature for characterizing the operating state of the power equipment includes:

[0025] Constructing a convolutional autoencoder, taking the subcarrier amplitude and phase matrix of the channel state information as input, and obtaining network parameters for separating noise components through training;

[0026] The channel state information is input into the convolutional autoencoder to obtain a denoised CSI feature.

[0027] As an optional implementation manner, the step of adaptively filtering and time-frequency transforming the acoustic signal to generate acoustic features for characterizing the electric power equipment in different frequency bands includes:

[0028] Performing short-time Fourier transform on the acoustic signal to obtain an initial time-frequency spectrum;

[0029] Using a convolutional neural network to perform feature enhancement on the initial time-frequency spectrum to generate a filtered time-frequency graph;

[0030] The filtered time-frequency graph is used as an acoustic feature.

[0031] As an optional implementation, the multimodal fusion model includes:

[0032] Construct a sensor graph network, treat each Wi-Fi receiver and acoustic sensor as a node in the graph, and establish topological connections based on the spatial location and signal correlation between nodes;

[0033] Using a graph neural network to propagate and aggregate multimodal features of the nodes, wherein the features of each node include the denoised CSI features and / or filtered acoustic features;

[0034] The state assessment result of the power equipment is generated at the output end of the graph neural network.

[0035] As an optional implementation manner, the evaluation of the operating state of the power equipment through the attention mechanism includes:

[0036] Read the feature vector recorded by each Wi-Fi receiver node and acoustic sensor node within the preset time window from the timestamp buffer of the node, and calculate the arrival time difference between the nodes based on the propagation speed of sound waves and electromagnetic waves or the device layout distance;

[0037] In response to detecting that a plurality of nodes have pulse spikes at similar times, evaluating the instantaneous association between pairs of nodes based on the arrival time differences and amplitude changes, and establishing or updating corresponding edges in the sensor graph network;

[0038] In the attention layer of the graph neural network, the arrival time difference and the amplitude change information are input into a trainable attention function to calculate the attention coefficient of each edge;

[0039] Perform multiple rounds of message propagation and aggregation on feature vectors of adjacent nodes based on the attention coefficients to obtain updated feature representations of each node;

[0040] The updated feature representation is input into an output layer to generate an operating status evaluation of the electric power equipment.

[0041] In a second aspect, the present application provides a power equipment status assessment system based on machine learning, comprising:

[0042] The first collection module is configured with a Wi-Fi transmitter and at least one receiver around the power equipment to collect channel status information;

[0043] A second acquisition module is provided with at least one acoustic sensor deployed around the electric power equipment to collect acoustic signals generated during the operation of the electric power equipment;

[0044] A first processing module, configured to perform deep denoising on the channel state information to generate a CSI feature for characterizing the operating state of the power equipment;

[0045] A second processing module, used for performing adaptive filtering and time-frequency transformation on the acoustic signal to generate acoustic features for characterizing the electric power equipment in different frequency bands;

[0046] An evaluation module is used to input the CSI feature and the acoustic feature into a multimodal fusion model, evaluate the operating status of the power equipment through an attention mechanism, and generate a power equipment status evaluation result.

[0047] Compared with the existing technology, by cleverly combining Wi-Fi CSI technology with acoustic sensing technology and adopting deep denoising, adaptive filtering and multimodal fusion models, this application can accurately capture the state changes of power equipment in strong electromagnetic interference and noise environments. Compared with traditional methods, this solution can not only use Wi-Fi signals to perform non-contact monitoring of electromagnetic disturbances around the equipment, but also use acoustic sensors to extract mechanical vibration or partial discharge signals in a noisy background. The two complement each other to make early fault detection more comprehensive and stable. The introduction of deep learning allows the system to achieve higher-precision noise suppression and feature extraction in both the adaptive filtering and time-frequency analysis stages, especially the multimodal fusion of graph neural networks, which further improves the ability to identify the health status of equipment under complex working conditions. At the same time, the solution can be deployed on edge computing terminals to reduce the need for equipment modification and downtime maintenance, effectively reduce implementation costs and improve real-time and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a method for evaluating the state of an electric power device based on machine learning provided in an embodiment of the present application;

[0049] Figure 2 A flowchart for generating a CSI feature for characterizing the operating state of the power equipment provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of a power equipment status assessment system based on machine learning provided in an embodiment of the present application.

[0051] Explanation of reference numerals: 10, first acquisition module; 20, second acquisition module; 30, first processing module; 40, second processing module; 50, evaluation module. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0053] See also Figure 1 As shown, it is a flow chart of a method for evaluating the state of electric equipment based on machine learning provided in an embodiment of the present application, the method comprising steps S101 to S105, wherein:

[0054] S101: deploying a Wi-Fi transmitter and at least one receiver around the power equipment to collect channel status information;

[0055] S102: deploying at least one acoustic sensor around the electric power equipment to collect acoustic signals generated during the operation of the electric power equipment;

[0056] S103: performing deep denoising on the channel state information to generate a CSI feature for characterizing the operating state of the power equipment;

[0057] S104: performing adaptive filtering and time-frequency transformation on the acoustic signal to generate acoustic features for characterizing the electric power equipment in different frequency bands;

[0058] S105: Input the CSI feature and the acoustic feature into a multimodal fusion model, evaluate the operating status of the power equipment through an attention mechanism, and generate a power equipment status evaluation result.

[0059] This application combines Wi-Fi CSI technology with acoustic sensing technology, and uses algorithms such as deep learning and graph neural networks for data fusion. Among them, Wi-Fi CSI, namely channel status information, can capture changes in electromagnetic signals caused by equipment, which are closely related to the mechanical vibration, electromagnetic interference, and partial discharge of the equipment. However, due to the serious electromagnetic interference in industrial environments, extracting effective features directly from CSI data may be affected by noise and environmental changes. Therefore, deep denoising methods can be used for preprocessing to extract reliable signal features.

[0060] On the other hand, acoustic signals can effectively reflect the mechanical state of the equipment, vibration characteristics, and ultrasonic signals during partial discharge. These acoustic signals are naturally resistant to electromagnetic interference, so they can complement the deficiencies of Wi-Fi CSI data, especially in noisy environments, where acoustic signals can still provide effective anomaly detection information. However, acoustic signals are also affected by background noise and environmental factors, and must be filtered and analyzed in time and frequency to accurately extract effective features related to the operating status of power equipment.

[0061] In order to solve the problem of spatiotemporal alignment between these two data sources, this application adopts a fusion model architecture for multimodal data fusion.

[0062] For example, the Transformer model uses the self-attention mechanism to model sequence data, which is suitable for fusing heterogeneous sequence features. A multimodal Transformer can be constructed, in which one part of the attention heads focuses on the CSI feature sequence and the other part focuses on the acoustic feature sequence, and the information interaction between the modalities is realized through the cross-attention mechanism. This intermediate layer feature fusion can achieve higher accuracy than simple feature concatenation. The attention mechanism can dynamically adjust the weights of each modal feature according to the correlation to achieve "complementary advantages", for example, when the Wi-Fi signal is noisy, it relies more on acoustic features, and vice versa.

[0063] On the other hand, if the spatial distribution of sensors is considered, GNN can be introduced to fuse information. Each sensor node (such as a Wi-Fi receiver or a microphone) is represented as a node in a graph network, and the node features are local features extracted by the sensor. Information is propagated through pre-built topological edges (for example, connections based on the distance between sensors or signal correlation). Graph neural networks can effectively utilize the spatial topology of sensor networks to achieve the fusion of distributed sensor information. Under this scheme, GNN can learn which sensor nodes contribute most to specific fault modes, thereby automatically assigning weights to each sensor source and enhancing the model's ability to extract effective information.

[0064] Through this multimodal, deep learning-driven fusion approach, the present application can effectively improve the accuracy and robustness of power equipment status assessment. In complex industrial environments, the combination of Wi-Fi CSI and acoustic signals not only supplements the information that may be missed by a single sensor, but also enhances the ability to detect abnormal conditions, especially in the early detection of minor faults such as equipment vibration and partial discharge. In addition, the multimodal fusion model based on graph neural networks can dynamically adjust the weights of different sensor data, thereby automatically optimizing the evaluation effect under different working conditions, and ultimately achieving efficient monitoring and fault warning of power equipment.

[0065] Regarding S101 above:

[0066] When conducting online monitoring of power equipment at industrial sites, traditional single-point sensors often have difficulty meeting the requirements of remote, non-invasive, and multi-directional coverage. In order to overcome these limitations and obtain stable data in a strong electromagnetic interference environment, this application proposes to deploy a Wi-Fi transmitter and at least one receiver around the power equipment, thereby utilizing the high sensitivity of wireless signals to changes in the surrounding environment to capture the weak interference caused by the operating status of the equipment. Through this operation, long-distance, high-resolution real-time monitoring can be achieved without destroying the original structure of the equipment, laying a data foundation for subsequent deep denoising and multimodal fusion.

[0067] Channel State Information (CSI) represents the amplitude and phase response of each subcarrier of the Wi-Fi wireless channel. It is high-dimensional information that is extremely sensitive to environmental changes. During the propagation of the multi-carrier signal emitted by the Wi-Fi router in space, it will be reflected, scattered and attenuated by power equipment and other environmental factors, forming channel state information (CSI). CSI can be subdivided into the amplitude and phase characteristics of several subcarriers. Once the device vibrates or internal discharge generates electromagnetic disturbances, the amplitude and phase of these subcarriers will change slightly over time. According to the principles of electromagnetics, when the signal path length, reflector position or medium characteristics change, different CSI fluctuations will be recorded at the receiving end.

[0068] It is worth noting that it is necessary to select a Wi-Fi transmitter with appropriate power and frequency band (such as a 2.4GHz or 5GHz router) and ensure that its transmitting antenna points to or covers the main parts of the target device. The receiving end can use a network card with a dedicated driver to extract the subcarrier amplitude and phase of each data packet in real time through the configuration software. In order to enhance resistance to strong industrial interference, a preliminary scan of the channel environment can be performed around the deployment area to select channels with lower interference or use directional antennas to reduce invalid multipath. If necessary, power threshold setting and timing synchronization can also be performed at the receiving end to ensure that data collection is completed within the same time window, which is convenient for subsequent algorithm alignment and fusion.

[0069] In specific implementation, the Wi-Fi transmitter can be installed in a safe area around the power equipment, such as the outer guardrail of the transformer or the distribution room channel, to ensure that the transmitted signal can basically cover the main body of the equipment. Subsequently, at least one receiver with CSI extraction function is deployed in different directions around the equipment, and it is connected to the local edge computing device through a wired or wireless network. The transmitter periodically sends detection data packets to each receiver, and each receiver records the corresponding CSI sequence and marks the timestamp. The CSI data of multiple receivers are finally aggregated into the edge computing device for subsequent deep denoising and multimodal models. This general process is applicable to different types of power equipment such as transformers, switchgear, rotating motors, etc., and only needs to be adjusted appropriately according to the site environment and equipment size.

[0070] For example, take the main transformer of a 110kV substation as an example: install an industrial-grade Wi-Fi router on the outer guardrail around it, and set the transmission power to 20dBm; deploy two receiving-end network cards at important locations in front, behind and on the sides of the transformer, and modify the driver on the local industrial computer to record the CSI frame of each receiving end. Due to the interference of high-voltage lines, metal shielding and strong arc welding machines in the substation, the surrounding 2.4GHz and 5GHz frequency bands can be scanned for interference first, and channels with fewer conflicts can be selected. In addition, a metal wire mesh can be installed on the outside of the receiving antenna to suppress strong noise intrusion. About 200 frames of CSI data are collected per second and GPS time synchronization is used to ensure that different receiving ends complete the recording in the same sampling window.

[0071] This method can stably obtain CSI sequence data around the transformer without affecting normal operation, which can be used for fault warning and vibration analysis.

[0072] In this way, by deploying Wi-Fi transmitters and at least one receiver around power equipment to collect channel status information, the electromagnetic perturbations caused by the operation of the equipment can be continuously captured in a non-invasive and low-cost manner, and reflected in high-dimensional CSI data. Compared with traditional sensors, this deployment method can cover a wide range of monitoring targets and does not require additional equipment modification or downtime maintenance, thus effectively saving manpower and material resources.

[0073] In addition, in order to improve the spatial resolution capability of the system, as an optional implementation, collecting the channel state information includes:

[0074] Deploy a plurality of receiving terminals at multiple locations around the power equipment so that the receiving terminals form a distributed MIMO array;

[0075] After the deployment of the plurality of receiving ends is completed, performing phase calibration and environmental baseline scanning on the initial channel state information received by each of the receiving ends to generate reference data of the environmental reflection path and a phase calibration result;

[0076] The initial channel state information is corrected based on the reference data and the phase calibration result.

[0077] In the monitoring of power equipment in industrial sites, if only a single Wi-Fi receiver is relied upon, blind spots may easily occur or it may be difficult to accurately locate the source of the fault. In order to improve the sensitivity to the changes in the spatial distribution of power equipment, the present invention further proposes to deploy multiple receivers at multiple locations around the equipment to form a distributed MIMO array.

[0078] The original intention of doing this is that when the signal from the transmitting end reaches different receiving ends, any slight disturbance caused by the equipment or environment will be reflected in different ways in the amplitude and phase changes of each CSI (channel state information), thereby obtaining more comprehensive and detailed environmental information.

[0079] The basic principle of a distributed MIMO array is to treat the multi-carrier signals recorded by multiple receivers as a set of multi-dimensional measurements, and to use the spatial gain of multiple antennas / multi-points to distinguish device locations or failure modes.

[0080] According to electromagnetic propagation theory, if the relative positions between the receiving ends meet a certain spacing (such as several wavelengths), rich reflection and scattering components can be captured on different paths. Through phase calibration, the CSI data at these different points can be synthesized in the same reference coordinate system, so that the subsequent algorithm can eliminate the random initial phase offset and perform baseline compensation for the slow-changing factors of the industrial environment (such as temperature and humidity changes). Environmental baseline scanning is mainly used to identify fixed reflection paths, such as walls, metal components, etc., to generate an "environmental reflection path baseline data" to distinguish CSI disturbances that are truly caused by changes in power equipment.

[0081] In addition, in industrial environments, high-voltage lines, metal casings, and external devices are prone to multipath fading or mutual interference. In order to mitigate these effects, it is necessary to perform initial phase calibration on the distributed receiving end. The specific method is to let the Wi-Fi transmitter send a known reference signal to each receiving end, compare the deviation between the received amplitude and phase values ​​and the ideal values, and correct the phase drift of each network card or antenna. Then perform an environmental baseline scan: let the transmitter transmit a specific signal when the device is powered off or the device is in stable working conditions, record the environmental static reflection path data, and generate "environmental reflection path benchmark data."

[0082] Once the normal monitoring phase is entered, the system can subtract the fixed reflection components represented by the baseline in real time to highlight the CSI features corresponding to the equipment changes. Finally, by inputting the corrected CSI data into the subsequent deep denoising or multimodal fusion algorithm, equipment failures can be identified more accurately.

[0083] In the specific implementation, you can first select several key points around the equipment (such as the outer guardrail around the equipment, the top beam of the channel, the side of the equipment, etc.), and install multiple receivers that support the CSI extraction function. Each receiver is in the same subnet and connected to the Wi-Fi transmitter. After completing the hardware layout, perform two-stage operations:

[0084] Phase 1 (Phase calibration and baseline generation): Let the transmitter send a reference signal when the device is unloaded or in a steady state. Each receiver records the initial CSI and calibrates the phase offset. Then, a static scan of the environment is performed to obtain a fixed reflection path.

[0085] Phase 2 (online monitoring): During normal operation, the receiver collects CSI at preset time intervals or in continuous mode, and uses calibration parameters and baseline data to correct the CSI in real time, so that the subsequent processing algorithm can accurately extract the CSI disturbance caused by changes in equipment operation.

[0086] For example, take a motor device as an example: a receiving network card is installed in each of the four directions with a radius of about 3 meters around the motor to form a rectangular distributed MIMO layout; the transmitting end is set at a height of 2.5 meters directly in front of the motor. After the deployment is completed, the motor is first put into standby mode, and the router sends about 1,000 frames of test signals to each receiving end to calibrate the antenna gain and phase offset of each receiving end; then the multipath reflection characteristics in the static environment are recorded and stored as "benchmark data of the environmental reflection path". When the motor is officially running, each receiving end collects CSI at a rate of 200 frames per second, and corrects the CSI by calling the calibration parameters and baseline data at the local edge computing end. This method can identify the impact of motor speed fluctuations and minor bearing faults on CSI. Compared with the single receiving end mode, the fault detection accuracy of the distributed MIMO layout is significantly improved.

[0087] In this way, by deploying receivers at multiple locations around the power equipment to form a distributed MIMO array, and performing phase calibration and environmental baseline scanning on the initial CSI, the uncertainty caused by multipath and industrial interference can be effectively reduced, and the CSI disturbance caused by changes in the equipment itself can be highlighted. With the help of calibration and baseline data, the amplitude and phase of the CSI information corrected are more reliable and distinguishable in subsequent deep learning or fusion model processing. At the same time, the multi-point layout also greatly improves the spatial resolution, and can perceive weak anomalies such as partial discharge and mechanical vibration at an early stage, providing more comprehensive and accurate data support for power equipment operation monitoring and fault warning.

[0088] Regarding S102 above:

[0089] During the operation of power equipment, mechanical vibration, partial discharge, and internal friction of the motor will be accompanied by the generation of acoustic signals. Compared with pure electromagnetic methods, these acoustic signals are more robust to high electromagnetic interference in certain situations and can more sensitively capture weak physical changes. Based on this motivation, the present invention proposes to deploy at least one acoustic sensor around the power equipment, and use its sensitive capture ability of acoustic features such as mechanical vibration and partial discharge to achieve multi-angle monitoring of the equipment's operating status, thereby complementing wireless signals such as Wi-Fi CSI and improving the accuracy and timeliness of fault detection.

[0090] Acoustic sensors (such as high-sensitivity microphones or MEMS microphone arrays) can convert sound waves propagating in air or solid media into electrical signals. When power equipment is in operation, if mechanical looseness, bearing wear, partial discharge or resonance occurs, pulsed or continuous sound wave energy will be generated in the audible or ultrasonic frequency band. Although industrial noise will also appear in sensor acquisition, as long as adaptive filtering and time-frequency analysis are performed in subsequent processing, the sound wave components related to the characteristics of the power equipment can be separated and identified.

[0091] It should be noted that when selecting sensors, the frequency range of the failure modes of concern to power equipment needs to be considered.

[0092] For example, partial discharge signals usually appear in the ultrasonic frequency band (above 20kHz), while motor vibrations may be concentrated in the low-frequency to several kilohertz range. In order to take into account wide-area frequency band acquisition, a MEMS microphone with a wider bandwidth can be selected, and a dust and water-proof cover can be installed on its periphery, and a metal shielding net can be used to reduce the interference of strong electromagnetic fields on the microphone circuit. In order to ensure the effectiveness of array measurement, the sensor should be kept at a certain distance from the main noise source as much as possible, and a reference microphone should be set up according to the site conditions to assist the subsequent adaptive filtering and noise suppression algorithms.

[0093] In this embodiment, at least one acoustic sensor can be installed around or on the top of the power equipment (such as a transformer or a motor). If a microphone array is used, it is necessary to assign an accurate timestamp to each microphone through a unified clock or GPS synchronization method to facilitate subsequent beamforming or delay estimation of multi-channel signals. The acoustic sampling rate can be configured according to the target fault type. For example, a sampling rate higher than 48kHz can be used for partial discharge, and a higher or lower sampling rate can be selected for mechanical vibration scenarios. The acoustic data collected by the sensor is then sent to a local or edge computing device in digital form for real-time or quasi-real-time filtering and feature extraction.

[0094] For example, take the ultrasonic detection of the main transformer of a substation as an example: two microphone units are installed on the top and side of the transformer, and the bandwidth can cover the ultrasonic range of 20kHz~80kHz. In order to resist the strong noise of the nearby cooling fan, a reference microphone is installed on the wall of the substation away from the equipment, which is used as a noise reference for adaptive filtering. The array records data at a uniform 48kHz sampling rate and collects multi-channel acoustic streams with an industrial computer. The environmental background noise is then removed by the LMS filtering algorithm, and the time-frequency characteristics of the partial discharge pulse are mined by STFT or CWT. Once partial discharge occurs in the transformer winding, this array can capture high-energy pulses in the frequency band of tens of kHz at an early stage, providing a basis for maintenance personnel to carry out timely maintenance.

[0095] In this way, by deploying at least one acoustic sensor around the power equipment, sensitive capture of mechanical vibration, partial discharge and other potential acoustic features is achieved. Compared with traditional pure electromagnetic detection methods, acoustic sensing has higher anti-electromagnetic interference capabilities in some environments, and can still extract key fault features in noisy sites; after complementing other modal data such as Wi-Fi CSI, it can further improve the system's detection rate for weak anomalies. In addition, acoustic deployment has good flexibility, and non-invasive monitoring can be achieved without changing the equipment structure or modifying the high-voltage parts, which greatly saves maintenance costs and improves safety.

[0096] As an optional implementation manner, collecting the acoustic signal generated during the operation of the electric power equipment includes:

[0097] Arrange a plurality of acoustic sensors at multiple locations around the power equipment so that the acoustic sensors form an acoustic sensor array and collect acoustic signals generated during the operation of the power equipment;

[0098] In response to the completion of the deployment of the acoustic sensor array, a known calibration sound source signal is provided to the acoustic sensor, and the calibration sound source signal received by each acoustic sensor is analyzed to obtain the gain and relative delay of the acoustic sensor;

[0099] Based on the gain and the relative delay, beamforming is performed on actual acoustic signals collected by the acoustic sensor array within a target frequency band.

[0100] In order to further enhance the ability of acoustic sensors to capture fault signals and obtain the ability to distinguish the direction of the sound source, the present invention deploys a number of acoustic sensors at multiple locations around the power equipment to form an acoustic sensor array. Compared with a single microphone, the array can capture the target signal while also locating the source of the fault through array processing algorithms (such as beamforming or delay estimation). This can better highlight key signals in noisy industrial sites, suppress environmental noise, and achieve high-resolution detection of partial discharge or mechanical anomalies.

[0101] The beamforming principle of the acoustic sensor array is based on the synthesis of multi-channel delay differences: if the fault sound of the equipment is transmitted to each sensor from different locations, the time arrival difference (TDOA) reflects the relative spatial relationship between the sound source and each sensor. By providing a known calibration sound source signal, the gain and relative delay of each microphone in the array can be accurately determined, thus laying the foundation for subsequent array processing of the measured data. During formal monitoring, if each sensor is synchronized and amplitude-phase corrected based on this calibration information, beamforming can be performed within the target frequency band to enhance the acoustic signal from a specific direction and reduce noise in irrelevant directions.

[0102] It should be noted that in order to successfully obtain the gain and relative delay of the acoustic array, it is necessary to provide a known calibration sound source signal to each sensor after the array is deployed.

[0103] For example, a pulse or sine sweep is sent at a uniform position or a measured and located position. The calibration signals received by each microphone will have a certain amplitude and phase difference due to different positions. These differences can be calculated to obtain the "channel gain" and "relative delay" of each microphone. In a noisy environment, frequency domain correlation or wavelet analysis can be used to extract these key information. Once the calibration data is obtained, the system can perform array processing (beamforming, coherent superposition, etc.) on the actual acoustic signals in the target frequency band, thereby achieving highly sensitive fault acoustic detection.

[0104] In specific implementation, several microphones can be deployed at selected points around the power equipment so that the entire array covers the main directions and heights of the sound that the equipment may generate. After deployment, a known calibration sound source signal is provided to the array (which can be emitted at a certain reference position, fixed frequency or sweep mode), and the calibration pulses received by each microphone and their arrival time and amplitude are recorded. Combined with the geometric position parameters between the microphones, the gain and relative delay are calculated and stored as "array calibration data". During normal working hours, each microphone captures the signal at a set sampling rate (such as 48kHz). The back-end algorithm first uses the calibration data for amplitude and phase compensation and performs beamforming, and then enhances the sound source in the target frequency band and outputs it to the upper-level adaptive filtering or fault diagnosis module.

[0105] For example, in a transformer monitoring scenario, in order to accurately identify the ultrasonic signal generated by partial discharge on the top of the equipment, three microphones can be evenly arranged above the transformer to form a triangular array structure. After the layout is completed, a calibration speaker is installed on the fixed bracket next to the transformer to send a sound source signal to the array in the form of a swept frequency pulse of 20kHz~40kHz. After each microphone records the signal, the system uses the time delay and amplitude difference between each other to obtain the channel gain and phase correction parameters of the array. When the monitoring is started, the microphone array will first form the received signal through the beam and focus it on the top area of ​​the transformer to capture weak partial discharge acoustic pulses; if the pulse energy is detected to exceed the normal threshold, the system will immediately trigger an alarm and transmit the data back to the operation and maintenance center.

[0106] In this way, by using an acoustic sensor array and calibrating the sound source signal to obtain gain and relative delay, the acoustic signal of the fault source can be directionally enhanced or located in the target frequency band, thereby effectively suppressing environmental noise and improving the system's detection accuracy and real-time performance for weak acoustic events such as partial discharge or mechanical abnormalities.

[0107] As an optional implementation manner, when collecting channel state information and collecting acoustic signals, the method further includes:

[0108] Based on the metal shell structure of the power equipment and the on-site noise bandwidth, a mixed calibration signal is generated; wherein the mixed calibration signal includes: a radio frequency modulation component and an ultrasonic pulse component;

[0109] Injecting the mixed calibration signal into the environment surrounding the power equipment, so that the Wi-Fi receiving end and the acoustic sensor array receive the mixed calibration signal simultaneously;

[0110] Respectively analyzing the radio frequency modulation component and the ultrasonic pulse component received by the Wi-Fi receiving end and the acoustic sensor array to obtain relative delay and amplitude difference;

[0111] The channel state information and the acoustic signal are cross-modally calibrated according to the relative time delay and amplitude difference.

[0112] In industrial environments, relying solely on independent Wi-Fi channel status information (CSI) or acoustic signals may result in detection deviations due to environmental interference, multipath effects, or complex noise distribution. In order to achieve precise cross-modal alignment and calibration in strong interference sites, the present invention further proposes a "hybrid calibration signal", which contains both RF modulation components and ultrasonic pulse components. By injecting this signal into the environment around the power equipment, the Wi-Fi receiver and the acoustic sensor array can "sense" and record a unified reference at the same time, thereby correcting the relative delay and amplitude difference between the two modal data and improving the accuracy of subsequent multimodal fusion.

[0113] In the hybrid calibration signal, the RF modulation component can be captured by the Wi-Fi receiver and used to perform benchmark calibration on the phase and amplitude of each subcarrier; while the ultrasonic pulse component is received by the acoustic sensor and used to correct the gain or time difference of arrival (TDOA) between acoustic arrays.

[0114] Once the RF and acoustic components are transmitted simultaneously in the same time window, both the Wi-Fi end and the acoustic end can compare the actual measured amplitude, phase and delay based on this, and compare them with the ideal or known modulation, and then obtain a unified benchmark across modes. If the power equipment itself has a metal shell or other structure, the impact of the metal enclosed or semi-enclosed environment on the propagation of RF and acoustic waves should be considered when designing the hybrid calibration signal, and the pulse power or frequency range should be adjusted appropriately to ensure that both modes can be detected and analyzed at the same time.

[0115] According to the metal shell structure of the power equipment and the ambient noise bandwidth, the mixed calibration signal can be designed as follows:

[0116] RF modulation component: Send short pulses or specific pilot codes in BPSK / QPSK and other methods in the 2.4GHz or 5GHz frequency band;

[0117] Ultrasonic pulse component: A pulse sequence is emitted between 20kHz and 50kHz. The pulse width and interval can be determined by the equipment size and environmental noise level.

[0118] It should be noted that before transmitting the signal, it is necessary to ensure that the Wi-Fi end and the acoustic array have a synchronized or approximately synchronized time reference so that the receiving end can determine the arrival delay difference of the two modal signals. During subsequent analysis, the Wi-Fi end extracts the amplitude and phase changes of the pilot subcarrier, and the acoustic end analyzes the peak time and energy of the ultrasonic pulse. By comparing the delay and amplitude differences between the two, the previously collected real device data can be cross-modally calibrated.

[0119] In a specific implementation, a mixed calibration signal can be generated on an edge computing device or a dedicated signal generator, and then the RF modulation component is simultaneously sent out through a Wi-Fi transmitter, while the ultrasonic pulse component is simultaneously radiated through a neighboring transmitter or ultrasonic transducer, so that the two signals are injected into the environment at a similar time.

[0120] Among them, after detecting the calibration signals of their respective frequency bands, the Wi-Fi receiver and the acoustic array record the waveform and timestamp respectively, and upload the results to the local processing unit. The processing unit calculates the "cross-modal calibration coefficient" based on the differences in amplitude, phase, and arrival time between the actual collected RF modulation and ultrasonic pulses, and finally calibrates and synchronizes the subsequently collected CSI data and acoustic data, thereby reducing the cross-modal misalignment caused by environmental multipath or interference.

[0121] For example, take a closed metal shell transformer as an example. Considering that its shell has a certain shielding effect on RF waves, but a ventilation grille is provided on the top to facilitate the penetration of ultrasonic signals. At this time, a multi-band signal generator can be arranged outside the transformer: a short BPSK pulse train is generated at 2.4GHz as the RF modulation component, and a pulse train between 20kHz and 40kHz is emitted from the ultrasonic transducer.

[0122] After the Wi-Fi receiver captures the BPSK pilot, it analyzes its phase alignment, and the acoustic microphone array records the peak moment of the ultrasonic pulse. Due to the different attenuation and reflection modes caused by the metal casing, the delay and amplitude ratio measured at both ends also changes accordingly. The system calculates the calibration coefficients such as relative delay (Δt) and amplitude offset (ΔA) to correct the CSI and acoustic waveforms actually collected later. This hybrid calibration method can significantly improve the accuracy of cross-modal information alignment and reduce the errors caused by stray reflections on the device.

[0123] In this way, by generating a mixed calibration signal containing RF modulation and ultrasonic pulse components and allowing the Wi-Fi receiver and the acoustic sensor array to receive it simultaneously, the time delay and amplitude difference between the two modes can be accurately obtained, cross-modal calibration can be achieved, and the detection accuracy and data alignment efficiency in complex industrial environments can be significantly improved.

[0124] Regarding S103 above:

[0125] In the strong electromagnetic interference and multipath reflection environment of industrial sites, the directly collected Wi-Fi channel status information (CSI) is often mixed with a large amount of random noise or slowly changing environmental factors, making it difficult to accurately reflect the true operating status of power equipment. To this end, the present invention proposes to perform deep denoising on the CSI sequence through a deep learning algorithm, thereby suppressing environmental noise and multipath interference while retaining key features, so as to obtain clean CSI features that can better characterize equipment failures or state changes.

[0126] The core idea of ​​deep denoising is to utilize neural networks (such as convolutional autoencoders, GANs, or RNN variants) to learn a clean prior distribution of CSI data.

[0127] During the training phase, a large amount of CSI data under normal working conditions can be collected as positive samples, and some simulated or real noise can be injected into them to construct a training set. The network gradually masters the ability to filter random noise by minimizing the reconstruction error or adversarial loss.

[0128] At the same time, the network can also capture the characteristics of weak disturbances of power equipment in the frequency domain or time series, making the denoised CSI more stable and more sensitive. According to theoretical and experimental comparisons, deep networks have stronger nonlinear fitting capabilities than traditional filters and can better identify subtle fault signs in multipath reflections or strong electromagnetic interference.

[0129] In the specific implementation, it is necessary to first perform necessary preprocessing on the CSI sequence, such as removing mean drift, eliminating hardware phase offset, interpolating and aligning timestamps, etc. Then the preprocessed sequence is input into the deep denoising network for frame-by-frame or small batch inference, and the network output is the "denoised CSI feature".

[0130] In addition, to ensure sufficient adaptability to short-term burst interference, multi-scale convolution kernels or recurrent units can be designed in the network, and time smoothing constraints can be introduced in the loss function to ensure that the CSI after denoising still retains the transient components required for fault changes.

[0131] It should be noted that if a generative adversarial network is used, a discriminator module can be introduced to determine whether the output CSI presents a real distribution, so as to further improve the denoising effect.

[0132] In specific implementation, a deep denoising model can be deployed in an edge computing device (or cloud server) and the acquired raw CSI data can be processed frame by frame or segment by segment. The specific process includes:

[0133] Preprocessing: Receive the original CSI frame, align the amplitudes of different subcarriers and antennas, and interpolate or slice the time axis.

[0134] Network input: Encapsulate the preprocessed CSI data into specific tensors or time series segments as the input of the neural network.

[0135] Inference: The network performs forward propagation and outputs denoised CSI features. During offline training or online fine-tuning, known labeled or unlabeled normal data can be used for model training or updating.

[0136] Output: The output “clean CSI features” are stored or sent to subsequent modules (multimodal fusion, anomaly detection, etc.) for further analysis.

[0137] For example, in a substation test scenario, about 200 frames of CSI data are collected per second, and a set of convolutional autoencoders are loaded into the industrial computer to perform real-time denoising.

[0138] The autoencoder can adopt a four-layer convolution and deconvolution structure, and use a large number of CSI sequences collected under normal working conditions, plus some Gaussian noise and pulse interference synthetic samples for supervised learning during training.

[0139] During the online inference phase, each batch (10 frames) of CSI data can be inferred through the network in only about 10 milliseconds to output the denoised feature tensor. Compared with the results of using traditional mean filtering or sliding windows, the CSI curve after deep denoising can still remain relatively stable during high interference periods and sensitively capture the amplitude and phase micro-movements caused by partial discharge.

[0140] In this way, the CSI features obtained through deep denoising greatly improve the robustness to industrial noise and random interference compared to the original CSI, while maintaining sensitivity to minor fault signs of power equipment (such as partial discharge or vibration), providing high-quality input for subsequent multimodal fusion and fault assessment, thereby significantly improving the detection accuracy and stability of the system under complex working conditions.

[0141] See also Figure 2 , Figure 2 A flowchart for generating a CSI feature for characterizing the operating state of the power equipment provided in an embodiment of the present application includes steps S201 to S202, wherein:

[0142] S201: construct a convolutional autoencoder, taking the subcarrier amplitude and phase matrix of the channel state information as input, and obtaining network parameters for separating noise components through training;

[0143] S202: Input the channel state information into the convolutional autoencoder to obtain denoised CSI features.

[0144] The channel state information (CSI) obtained at industrial sites often has high-intensity noise interference and multipath distortion, which makes it difficult for the original amplitude or phase sequence to directly reflect the subtle fault signs of power equipment. Traditional filters and simple time domain / frequency domain processing methods often find it difficult to simultaneously retain small vibration characteristics and suppress environmental noise. Based on this motivation, this application uses convolutional autoencoders (CAEs) to learn and separate noise components, so that the network can extract CSI information that can better represent the status of the equipment at the "deep feature" level.

[0145] When building a convolutional autoencoder, the network structure needs to be customized according to the dimension and sampling frequency of CSI. If CSI is input in the two-dimensional form of "subcarrier × time", a two-dimensional convolution kernel can be used to capture the time trend and the correlation between subcarriers at the same time; if CSI becomes a three-dimensional tensor (subcarrier × time × antenna) in a multi-antenna scenario, it can be further expanded to three-dimensional convolution or a block strategy can be adopted. In the training phase, a large amount of CSI data under normal industrial working conditions can be used as the main sample, and some simulated interference or real noise records can be mixed into it to form a supervised or semi-supervised training process. The final network parameters can distinguish the difference between environmental interference and the intrinsic signal of the equipment.

[0146] For the above S201, a convolution encoder can be designed to subject the original CSI matrix to several layers of convolution and pooling in sequence; a deconvolution decoder can be designed to be symmetrical or partially symmetrical with the encoder structure to restore the input dimension; training is performed on the data set to minimize the reconstruction error and regularization term to obtain the final network weights.

[0147] For the above S202, the channel state information can be input into the convolutional autoencoder to obtain new CSI data online or offline; the trained model is input for forward reasoning, and the denoised CSI features are output; the denoised features are passed to the subsequent state evaluation or multimodal fusion stage.

[0148] For example, in a 110kV substation environment, about 200 frames of CSI data (including 64 subcarriers and 2 antenna dimensions) are collected per second and integrated into The two-dimensional matrix is ​​then input into the convolutional autoencoder. The encoding end uses two layers of two-dimensional convolution (kernel size 3×3), and the decoding end uses two layers of deconvolution for step-by-step recovery. The training data includes normal working condition CSI and interference such as artificially injected pulses and random Gaussian noise.

[0149] After several rounds of iterative training, the model can complete the denoising output of each frame of data in a short time during online inference, smoothing the CSI curve with obvious jitter, thereby enhancing the subsequent perception of tiny discharges or vibration disturbances.

[0150] In this way, by using a convolutional autoencoder to separate the noise component in the channel state information (CSI), the key characteristics of the operating status of power equipment can be retained in a complex industrial environment, the sensitivity and accuracy of fault detection can be significantly improved, and high-quality input data can be provided for subsequent multimodal fusion.

[0151] Regarding S104 above:

[0152] During the operation of power equipment, mechanical vibration, bearing wear, and partial discharge may generate a variety of acoustic signals covering low frequencies to ultrasonic frequency bands. However, these original signals are usually masked by industrial noise, environmental sound sources, and equipment operation sounds, and it is difficult to directly find weak fault signs from the time domain waveform. In order to overcome this problem, the present invention will first perform adaptive filtering on the collected acoustic signals to suppress relatively stable or known mode noise sources in real time, and then separate the key features in different frequency bands through time-frequency transformation, so that the system can more easily identify abnormal waveforms or pulses exhibited by the equipment in different operating frequency bands.

[0153] Among them, adaptive filtering mainly obtains environmental noise components based on the sensor array or reference channel, automatically adjusts the filter coefficients, and subtracts components similar to environmental noise from the target signal while retaining short-term abnormal pulses or vibration information.

[0154] When performing time-frequency transformation subsequently, it can be decided whether to use short-time Fourier transform (STFT), continuous wavelet transform (CWT) or other multi-resolution analysis methods according to the target fault type. By mapping the one-dimensional time series to a two-dimensional time-frequency spectrum, the transient energy or frequency drift characteristics in the special frequency band area are highlighted, thereby generating acoustic characteristics used to characterize the power equipment in different frequency bands.

[0155] In the specific implementation, after the sensor is laid out, a suitable preamplifier circuit is set for each microphone and the initial sampling rate is determined. Then, an adaptive filtering process is designed at the data acquisition end: if the LMS (least mean square) algorithm is used, the filter coefficients are updated each time during the acquisition according to the reference noise channel or the error signal of the previous frame to suppress steady-state or predictable noise components.

[0156] After filtering, the output pure signal is divided into several short-time windows, time-frequency transformation is performed window by window, and the corresponding time-frequency spectrum or wavelet coefficient map is generated for each window.

[0157] In addition, to facilitate subsequent fusion or deep learning, these time-frequency graphs can be packaged and timestamped, and then sent to edge computing devices for pattern recognition or anomaly detection.

[0158] For example, when performing acoustic monitoring on a large transformer, the system sets up two microphones: one close to the transformer casing and the other away from the equipment and close to the cooling fan to obtain the main noise reference. The sampling rate is set to 48kHz. During operation, the local industrial computer performs LMS adaptive filtering on the reference channel and the target channel, and updates the filter coefficients every second to adapt to the changes in the fan roar and environmental noise.

[0159] The filtered target signal is divided into short-term segments of 500ms, and STFT is performed on each segment to generate a time spectrum. Subsequently, a sharp pulse component is observed in the segment of tens of kilohertz, which is basically consistent with the typical pattern of partial discharge in the existing data, so the system issues an early warning.

[0160] Through the combination of adaptive filtering and time-frequency transformation, the present invention can extract key acoustic features in different frequency bands of power equipment in a noisy environment, significantly improve the ability to capture signs of partial discharge, vibration or mechanical failure, and provide a high-quality data foundation for multimodal fusion analysis.

[0161] As an optional implementation manner, the generating of acoustic features for characterizing the electric power equipment in different frequency bands includes:

[0162] Performing short-time Fourier transform on the acoustic signal to obtain an initial time-frequency spectrum;

[0163] Using a convolutional neural network to perform feature enhancement on the initial time-frequency spectrum to generate a filtered time-frequency graph;

[0164] The filtered time-frequency graph is used as an acoustic feature.

[0165] In industrial noise environments, mechanical vibrations, partial discharges or other abnormal signals generated by power equipment often cover acoustic components in different frequency bands. If we only stay in the time domain analysis, it is easy to be overwhelmed by high-intensity noise or spurious waveforms. Therefore, the present invention further proposes to extract acoustic features in the time-frequency domain, and perform feature enhancement on the initial time-frequency spectrum through a convolutional neural network (CNN). This idea is based on short-time Fourier transform (STFT) or other time-frequency analysis methods to map one-dimensional acoustic data to a two-dimensional time-frequency graph, and then use the image processing capabilities of CNN to suppress noise in the time-frequency domain and highlight key fault frequency bands, thereby realizing adaptive mining of acoustic signals in multiple frequency bands.

[0166] In the specific analysis process, the acoustic signal is first segmented according to a certain window (such as hundreds of milliseconds), and STFT is performed on each segment to transform the time domain signal into a two-dimensional amplitude spectrum or complex spectrum. In this way, the energy distribution of the sound wave in different frequency bands and different time slices can be clearly presented on the time-frequency plane.

[0167] Since there is often strong background noise or continuous machine roar at industrial sites, a convolutional neural network is used to enhance the features of the initial time-frequency spectrum, focusing on the sharp pulses and harmonic components that may appear in the target frequency band. Through multi-layer convolution kernel learning, CNN can perform targeted noise filtering or contrast enhancement for features such as partial discharge pulse bands, harmonic trends, and mechanical vibration textures in the time-frequency graph, thereby obtaining a "filtered time-frequency graph."

[0168] In the specific implementation, the acoustic signal is first adaptively filtered to remove constant noise, and then the cleaned waveform is divided into several short-time windows (e.g., each window length is 0.5 seconds, and the frame shift is 0.25 seconds). STFT is performed on each short-time window to generate a time-frequency spectrum. Usually, the amplitude spectrum can be retained and logarithm compression can be taken to make it more suitable for CNN processing.

[0169] Subsequently, the pre-trained CNN model is loaded, which consists of several layers of convolution, pooling and activation functions. The input is the time-frequency image block and the output is the enhanced feature map.

[0170] Finally, these enhanced time-frequency diagrams are further marked with timestamps and summarized into time-frequency sequences for use by multimodal fusion or fault identification modules.

[0171] For example, when performing time-frequency analysis on the ultrasonic signal of a high-voltage switchgear, acoustic data can be collected at a sampling rate of 48kHz, STFT is first performed with a window length of 200ms and 50% overlap, and the obtained amplitude spectrum is converted into a grayscale image. After the grayscale image is feature-enhanced by a CNN model pre-trained on normal and faulty acoustic data, the local discharge pulses in the time-frequency domain appear as stripes or spots of higher brightness, while most of the irrelevant noise is suppressed. If high-energy pulses appear in the same frequency band in multiple consecutive frames of time-frequency images, the system will issue an alarm and upload the data to the operation and maintenance platform for on-site maintenance personnel to further determine the location and nature of the fault.

[0172] Regarding the above S105:

[0173] After the above processing, two types of feature data are obtained: one is the denoised CSI features provided by each Wi-Fi receiver (for example, expressed in the form of feature vectors), and the other is the filtered acoustic feature map or vector representation provided by each acoustic sensor. These features cover the wireless channel changes and acoustic signal changes caused by the operating status of the equipment. Next, these two types of features are jointly analyzed through a multimodal fusion model to evaluate the health status or operating status of the power equipment.

[0174] In the fusion stage, a variety of deep learning architectures can be adopted, such as Transformer-based attention fusion and graph neural network (GNN)-based topological fusion.

[0175] The Transformer model uses the self-attention mechanism to model sequence data, which is very suitable for fusing heterogeneous sequence features. A multimodal Transformer can be constructed, in which one part of the attention head focuses on the CSI feature sequence and the other part focuses on the acoustic feature sequence, and the cross-attention mechanism is used to achieve information interaction between the modalities. This intermediate layer feature fusion can achieve higher accuracy than simple feature concatenation. The attention mechanism can dynamically adjust the weights of each modal feature according to the correlation to achieve "complementary advantages", for example, when the Wi-Fi signal is noisy, it relies more on acoustic features, and vice versa.

[0176] On the other hand, if the spatial distribution of sensors is considered, GNN can be introduced to fuse information. Each sensor node (such as a Wi-Fi receiver or a microphone) is represented as a node in a graph network. The node features are local features extracted by the sensor, and information is propagated through pre-built topological edges (for example, connections based on the distance between sensors or signal correlation). Graph neural networks can effectively utilize the spatial topology of sensor networks to achieve the fusion of distributed sensor information. For example, in the fault diagnosis of high-speed rail bogies, GNN combined with multi-sensor data significantly improved the diagnostic accuracy. Under this scheme, GNN can learn which sensor nodes contribute most to specific fault modes, thereby automatically assigning weights to each sensor source and enhancing the model's ability to extract effective information.

[0177] Whether it is Transformer or GNN, the output of the fusion model is an evaluation result of the current equipment status (such as "normal", "partial discharge abnormality" or specific fault type).

[0178] As an optional implementation, the multimodal fusion model includes:

[0179] Construct a sensor graph network, treat each Wi-Fi receiver and acoustic sensor as a node in the graph, and establish topological connections based on the spatial location and signal correlation between nodes;

[0180] Using a graph neural network to propagate and aggregate multimodal features of the nodes, wherein the features of each node include the denoised CSI features and / or filtered acoustic features;

[0181] The state assessment result of the power equipment is generated at the output end of the graph neural network.

[0182] As an optional implementation manner, the evaluation of the operating state of the power equipment through the attention mechanism includes:

[0183] Read the feature vector recorded by each Wi-Fi receiver node and acoustic sensor node within the preset time window from the timestamp buffer of the node, and calculate the arrival time difference between the nodes based on the propagation speed of sound waves and electromagnetic waves or the device layout distance;

[0184] In response to detecting that a plurality of nodes have pulse spikes at similar times, evaluating the instantaneous association between pairs of nodes based on the arrival time differences and amplitude changes, and establishing or updating corresponding edges in the sensor graph network;

[0185] In the attention layer of the graph neural network, the arrival time difference and the amplitude change information are input into a trainable attention function to calculate the attention coefficient of each edge;

[0186] Perform multiple rounds of message propagation and aggregation on feature vectors of adjacent nodes based on the attention coefficients to obtain updated feature representations of each node;

[0187] The updated feature representation is input into an output layer to generate an operating status evaluation of the electric power equipment.

[0188] Among them, the multimodal fusion model is constructed in the form of a "sensor graph network", which regards each Wi-Fi receiver and acoustic sensor as a node in the graph, and establishes topological connections based on the spatial position and signal correlation between the nodes, and finally generates the status assessment results of the power equipment at the output end of the graph neural network.

[0189] In the specific implementation, first, a node ID is assigned to each Wi-Fi receiver and each acoustic sensor deployed on site, and the position coordinates of each node in the physical space (such as the radius from the device, installation height, etc.) are recorded.

[0190] If two nodes are close to each other, or their collected signals (such as CSI change patterns and acoustic features) are highly correlated over a period of time, an edge is added to the graph, and the edge weight is initially set to the average value or according to the distance decay rule.

[0191] In this way, a "sensor graph" can be formed, where the number of graph nodes is the number of sensors, and the graph edges represent the possible information associations or spatial neighbor relationships between sensors.

[0192] Furthermore, the "denoised CSI features" and "filtered acoustic features" are extracted for each node respectively. For example, if a node only has Wi-Fi, only the CSI features are retained, and if a node only has acoustic sensors, only the acoustic features are retained. If a node reuses two types of sensor information, they can also be spliced ​​into a joint vector.

[0193] In edge computing devices or cloud servers, these node features are fed into the input layer of the graph neural network in sequence to perform message propagation and aggregation.

[0194] Furthermore, the graph neural network utilizes the edge connectivity between nodes to share the feature vector of each node with neighboring nodes in stages according to a pre-set number of iterations or network layers.

[0195] In each round of iteration, the node updates its hidden representation based on its own features and neighbor features (weighted by edge weights or attention coefficients), and finally forms a high-level feature vector of multimodal fusion after several iterations.

[0196] The node representation processed in this way not only covers the Wi-Fi or acoustic characteristics of the node itself, but also integrates the environmental information of neighboring nodes, which can more comprehensively reflect the overall status of the power equipment.

[0197] At the output of the last layer of the graph neural network, classification or regression operations are performed for each node or the entire graph to identify whether the current device is in a normal state, a minor fault, or a serious fault.

[0198] It should be noted that if only the whole machine status needs to be output, all node features can be globally pooled (such as averaging or maximizing) at the end of the GNN and then enter an MLP (multi-layer perceptron) to obtain the final status classification result.

[0199] In this implementation, the graph neural network can use GAT (Graph Attention Network), GCN (Graph Convolutional Network) or other variants, depending on the timeliness and accuracy requirements of the industrial site. The output power equipment status includes multiple types such as "normal", "partial discharge", and "abnormal mechanical vibration", so that operation and maintenance personnel can detect potential faults as early as possible.

[0200] After completing the construction of the sensor graph network and the fusion of basic multimodal features, the present invention further uses information such as arrival time difference (TDOA) and amplitude change in the attention layer of the graph neural network to dynamically weight the pulse spikes that appear at similar times in multiple nodes, and finally obtains the operating status evaluation of the power equipment.

[0201] In the specific implementation, each Wi-Fi receiver and acoustic sensor node maintains a timestamp buffer locally to record the feature vectors (CSI or acoustic) in the recent period. When the system enters the evaluation window, the feature vectors in the window are read from the buffer of each node and pre-aligned.

[0202] If there is GPS synchronization or a unified clock, the CSI and acoustic signal data collected by each node at the same time or approximate time can be accurately confirmed.

[0203] Furthermore, after detecting that pulse spikes or abnormal amplitude jumps occur in several nodes at similar times (such as the difference does not exceed a certain threshold), the arrival time difference between these node pairs is estimated based on the propagation speed of sound waves and electromagnetic waves, and the locations of devices and nodes.

[0204] If the arrival time difference matches the theoretical value (or baseline data) and the amplitude changes of each node are in the same direction, it means that they are likely to capture the same fault event. For this node pair, an edge can be established or strengthened in the sensor graph network to increase its edge weight.

[0205] In the attention layer of the graph neural network, the above-mentioned arrival time difference, amplitude change and other information are input into the trainable attention function (such as using affine transformation or MLP for inner product mapping) to obtain the attention coefficient of each edge. Compared with fixed edge weights, this attention coefficient can be dynamically adjusted with the intensity and time synchronization of the event, so that the node pairs with high correlation with the fault have a higher weight ratio in the message propagation link.

[0206] Furthermore, based on the calculated attention coefficient, the graph neural network performs several rounds of message passing and aggregation on the feature vectors of adjacent nodes. Each time, the information from the neighboring nodes is superimposed on the current node and activated or normalized while considering the edge weights.

[0207] If a node continues to receive high attention input from multiple neighbors, the implicit representation of the node will be gradually strengthened during the iteration, fully reflecting its importance in this failure event.

[0208] The updated feature representation of each node is input into the final output layer of the network (which can be a layer of MLP or global pooling + classifier) ​​to obtain an overall judgment on the operating status of the power equipment.

[0209] For example, for transformers, multiple categories of results such as "normal / partial discharge / abnormal vibration" can be output; for motors, states such as "normal / bearing fault / overload" can be determined.

[0210] Once the model determines that there is an anomaly or potential fault in the system, the edge computing device will send an alarm message to the operation and maintenance center along with attention information between node pairs to indicate which area the source of the fault may be concentrated in or which sensor node first captures the anomaly.

[0211] On this basis, this application can accurately identify and synchronously capture nodes of abnormal signals in graph neural networks through attention mechanisms and arrival time difference analysis in industrial sites with strong interference and multi-node distribution, thereby achieving early fault warning and reducing false alarm rates.

[0212] Based on the same inventive concept, the embodiment of the present application also provides a machine learning-based power equipment status assessment system corresponding to a machine learning-based power equipment status assessment method. Since the principle of solving the problem by the system in the embodiment of the present application is similar to the above-mentioned machine learning-based power equipment status assessment method in the embodiment of the present application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be repeated.

[0213] Reference Figure 3 FIG. 1 is a schematic diagram of a power equipment status assessment system based on machine learning provided in an embodiment of the present application, wherein the system includes:

[0214] The first collection module 10 is configured with a Wi-Fi transmitter and at least one receiver around the power equipment to collect channel status information;

[0215] The second acquisition module 20 has at least one acoustic sensor deployed around the power equipment to collect acoustic signals generated during the operation of the power equipment;

[0216] A first processing module 30, configured to perform deep denoising on the channel state information to generate a CSI feature for characterizing the operating state of the power equipment;

[0217] The second processing module 40 is used to perform adaptive filtering and time-frequency transformation on the acoustic signal to generate acoustic features for characterizing the electric power equipment in different frequency bands;

[0218] The evaluation module 50 is used to input the CSI feature and the acoustic feature into a multimodal fusion model, evaluate the operating status of the power equipment through an attention mechanism, and generate a power equipment status evaluation result.

[0219] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

Claims

1. A method for evaluating the state of power equipment based on machine learning, characterized in that: include: Deploy a Wi-Fi transmitter and at least one receiver around the power equipment to collect channel status information; Deploy at least one acoustic sensor around the power equipment to collect acoustic signals generated during the operation of the power equipment; Performing deep denoising on the channel state information to generate a CSI feature for characterizing the operating state of the power equipment; Adaptively filtering and time-frequency transforming the acoustic signal to generate acoustic features for characterizing the electric power equipment in different frequency bands; The CSI features and the acoustic features are input into a multimodal fusion model, the operating status of the power equipment is evaluated through an attention mechanism, and a power equipment status evaluation result is generated.

2. The method according to claim 1, characterized in that: Collecting the channel status information includes: Deploy a plurality of receiving terminals at multiple locations around the power equipment so that the receiving terminals form a distributed MIMO array; After the deployment of the plurality of receiving ends is completed, performing phase calibration and environmental baseline scanning on the initial channel state information received by each of the receiving ends to generate reference data of the environmental reflection path and a phase calibration result; The initial channel state information is corrected based on the reference data and the phase calibration result.

3. The method according to claim 2, characterized in that The step of deploying at least one acoustic sensor around the electric power equipment to collect acoustic signals generated during the operation of the electric power equipment includes: Arrange a plurality of acoustic sensors at multiple locations around the power equipment so that the acoustic sensors form an acoustic sensor array and collect acoustic signals generated during the operation of the power equipment; In response to the completion of the deployment of the acoustic sensor array, a known calibration sound source signal is provided to the acoustic sensor, and the calibration sound source signal received by each acoustic sensor is analyzed to obtain the gain and relative delay of the acoustic sensor; Based on the gain and the relative delay, beamforming is performed on actual acoustic signals collected by the acoustic sensor array within a target frequency band.

4. The method according to claim 3, characterized in that: When collecting channel status information and acoustic signals, it also includes: Based on the metal shell structure of the power equipment and the on-site noise bandwidth, a mixed calibration signal is generated; wherein the mixed calibration signal includes: a radio frequency modulation component and an ultrasonic pulse component; Injecting the mixed calibration signal into the environment surrounding the power equipment, so that the receiving end and the acoustic sensor array receive the mixed calibration signal simultaneously; Respectively analyzing the radio frequency modulation component and the ultrasonic pulse component received by the Wi-Fi receiving end and the acoustic sensor array to obtain relative delay and amplitude difference; The channel state information and the acoustic signal are cross-modally calibrated according to the relative time delay and amplitude difference.

5. The method according to claim 4, characterized in that The generating of the CSI feature for characterizing the operating state of the electric power equipment includes: Constructing a convolutional autoencoder, taking the subcarrier amplitude and phase matrix of the channel state information as input, and obtaining network parameters for separating noise components through training; The channel state information is input into the convolutional autoencoder to obtain a denoised CSI feature.

6. The method according to claim 5, characterized in that The generating of acoustic characteristics for characterizing the electric power equipment in different frequency bands comprises: Performing short-time Fourier transform on the acoustic signal to obtain an initial time-frequency spectrum; Using a convolutional neural network to perform feature enhancement on the initial time-frequency spectrum to generate a filtered time-frequency graph; The filtered time-frequency graph is used as an acoustic feature.

7. The method according to claim 6, characterized in that The multimodal fusion model includes: Construct a sensor graph network, treat each Wi-Fi receiver and acoustic sensor as a node in the graph, and establish topological connections based on the spatial location and signal correlation between nodes; Using a graph neural network to propagate and aggregate multimodal features of the nodes, wherein the features of each node include the denoised CSI features and / or filtered acoustic features; The state assessment result of the power equipment is generated at the output end of the graph neural network.

8. The method according to claim 7, characterized in that The evaluation of the operating state of the power equipment by the attention mechanism includes: Read the feature vector recorded by each Wi-Fi receiver node and acoustic sensor node within the preset time window from the timestamp buffer of the node, and calculate the arrival time difference between the nodes based on the propagation speed of sound waves and electromagnetic waves or the device layout distance; In response to detecting that a plurality of nodes have pulse spikes at similar times, evaluating the instantaneous association between pairs of nodes based on the arrival time differences and amplitude changes, and establishing or updating corresponding edges in the sensor graph network; In the attention layer of the graph neural network, the arrival time difference and the amplitude change information are input into a trainable attention function to calculate the attention coefficient of each edge; Perform multiple rounds of message propagation and aggregation on feature vectors of adjacent nodes based on the attention coefficients to obtain updated feature representations of each node; The updated feature representation is input into an output layer to generate an operating status evaluation of the electric power equipment.

9. A power equipment status assessment system based on machine learning, characterized in that: include: The first collection module is configured with a Wi-Fi transmitter and at least one receiver around the power equipment to collect channel status information; A second acquisition module is provided with at least one acoustic sensor deployed around the electric power equipment to collect acoustic signals generated during the operation of the electric power equipment; A first processing module, configured to perform deep denoising on the channel state information to generate a CSI feature for characterizing the operating state of the power equipment; A second processing module, used for performing adaptive filtering and time-frequency transformation on the acoustic signal to generate acoustic features for characterizing the electric power equipment in different frequency bands; An evaluation module is used to input the CSI feature and the acoustic feature into a multimodal fusion model, evaluate the operating status of the power equipment through an attention mechanism, and generate a power equipment status evaluation result.

Images