Fault large model construction method, system, device and medium for electric power inspection

Abstract

The present application relates to the technical field of intelligent monitoring and fault diagnosis of power equipment, and discloses a fault large model construction method, system, equipment and medium for power inspection, which comprises the following steps: a first network is used to decouple the multi-scale features of the main modal perception data of the equipment to be inspected, so as to obtain intrinsic feature components and disturbance feature components; a second network is used to recombine the environment conditions of the two components based on the auxiliary modal perception data of the equipment to be inspected, so as to obtain multi-modal scene features; a third network and a fourth network are used to sequentially perform cross-modal semantic space alignment and hierarchical attention fusion based on a physical prior mask on the multi-modal scene features, so as to obtain fusion features; a multi-task perception compatible decoder is used to decode the fusion features, so as to obtain fault detection results; and these network structures are sequentially connected to form a fault large model for fault diagnosis of the equipment to be inspected; and the robustness and accuracy of the fault identification of the power equipment in a complex environment are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring and fault diagnosis technology for power equipment, and in particular to a method, system, equipment and medium for constructing large-scale fault models for power inspection. Background Technology

[0002] With the development of smart grids, online monitoring technology based on multi-sensor fusion has gradually replaced traditional single-threshold alarms. Currently, mainstream monitoring solutions typically utilize devices such as visible light cameras, infrared thermal imagers, and acoustic signature collectors to acquire multimodal data, and then use deep learning models (such as CNN and Transformer) for feature extraction and fault diagnosis. However, this method suffers from several drawbacks: 1) lacks adaptive decoupling capabilities to complex environmental interference, making it difficult to distinguish between "equipment-specific features" and "environmental disturbance features." In harsh environments, the model often incorrectly encodes environmental noise such as raindrop obstruction and background electromagnetic clutter as fault features, resulting in extremely high false alarm rates in noisy or low-light scenarios. 2) Forcibly splices visual, auditory, and electrical signals based solely on statistical correlation, ignoring the inherent physical laws governing power equipment faults. This makes existing fusion mechanisms (such as standard self-attention mechanisms) prone to incorrectly associating random burst noises (such as bird calls and vehicle passing sounds) with electromagnetic interference signals, leading to diagnostic results that contradict physical principles. 3) It is difficult to effectively perform deep semantic alignment between one-dimensional high-frequency pulse time-series signals and two-dimensional visual / time-frequency maps, resulting in insufficient early warning capabilities for latent faults.

[0003] Therefore, it is evident that constructing a large-scale fault diagnosis model that can effectively address the problems of existing multimodal monitoring methods for power equipment being greatly affected by strong interference such as rain, fog, and electromagnetic noise in complex substation environments, and lacking adherence to the laws of physical propagation, leading to spurious correlations and insufficient robustness in fault identification, has become a pressing technical issue for those skilled in the art. Summary of the Invention

[0004] This invention provides a method, system, equipment, and medium for constructing a large-scale fault model for power line inspection. By constructing a large-scale fault diagnosis model that integrates data-driven and physical mechanism-driven approaches, it solves the problems of existing multimodal monitoring methods for equipment under inspection being greatly affected by strong interference such as rain, fog, and electromagnetic noise in complex substation environments, and lacking adherence to the laws of physical propagation, which leads to the model being prone to false correlations and insufficient robustness in fault identification.

[0005] To address the aforementioned technical problems, the first aspect of this invention provides a method for constructing a large-scale fault model for power line inspection, comprising: A first network is constructed; the first network is used to decouple the main modal sensing data of the equipment to be inspected from multiple scales to obtain the inherent feature components representing the equipment body information and the disturbance feature components representing the environmental noise; the main modal sensing data is other modal data other than environmental noise data; A second network is constructed; the second network is used to reorganize the inherent feature components and the disturbance feature components based on the auxiliary modal perception data of the equipment to be inspected, and obtain multimodal scene features adapted to the current environment; the auxiliary modal perception data includes the environmental noise data; A third network is constructed; the third network is used to perform cross-modal semantic space alignment of the multimodal scene features to obtain a global alignment result; A fourth network is constructed; the fourth network is used to perform hierarchical attention fusion based on physical prior masks on the global alignment results to obtain fused features; A multi-task perception compatible decoder is constructed; the multi-task perception compatible decoder is used to decode the fused features to obtain the fault detection result of the equipment to be inspected; The first network, the second network, the third network, the fourth network, and the multi-task perception compatible decoder are cascaded to form a large fault model.

[0006] A second aspect of the present invention provides a fault large model construction system for power line inspection, comprising: The first network construction module is used to construct a first network; the first network is used to perform multi-scale feature decoupling on the main modal sensing data of the equipment to be inspected, to obtain the inherent feature components representing the equipment body information and the disturbance feature components representing the environmental noise; the main modal sensing data is modal data other than environmental noise data; The second network construction module is used to construct a second network; the second network is used to reorganize the inherent feature components and the disturbance feature components based on the auxiliary modal perception data of the equipment to be inspected, and obtain multimodal scene features adapted to the current environment; the auxiliary modal perception data includes the environmental noise data; The third network construction module is used to construct a third network; the third network is used to perform cross-modal semantic space alignment on the multimodal scene features to obtain a global alignment result; The fourth network construction module is used to construct a fourth network; the fourth network is used to perform hierarchical attention fusion based on physical prior masks on the global alignment result to obtain fused features; A decoder construction module is used to construct a multi-task perception compatible decoder; the multi-task perception compatible decoder is used to decode the fused features to obtain the fault detection results of the equipment to be inspected; The large model generation module is used to cascade the first network, the second network, the third network, the fourth network, and the multi-task perception compatible decoder to form a large fault model.

[0007] A third aspect of the present invention provides an electronic device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the fault large model construction method for power inspection as described above.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium comprising a stored computer program, wherein when the device containing the computer-readable storage medium executes the computer program, it implements the fault large model construction method for power inspection as described above.

[0009] Compared with the prior art, the beneficial effects of the embodiments of the present invention are as follows: The first network is constructed to forcibly separate the "device body features" and "environmental disturbance features" in the feature space of the acquired signals. This enables the fault model containing this network to suppress environmental noise and effectively avoid misleading fault identification due to severe weather (such as rain and fog) or strong electromagnetic interference. The second network is constructed to enable the fault model containing this network to flexibly adjust its judgment strategy according to the field environment, achieving true scene adaptation. The third network is constructed to perform cross-modal semantic space alignment of multimodal scene features, eliminating the semantic gap between different sensors and ensuring that the fused features remain semantically consistent. In the feature fusion stage, a "thing-device-body feature" is constructed. The "prior mask" mechanism forcibly ignores noise signals that violate physical laws within the time window, enabling the fault model containing this network to not only possess the strong fitting ability of deep learning but also the logical judgment ability of a physics expert. This fundamentally suppresses the problems of pure data-driven models easily learning false correlations and having poor anti-interference capabilities. The constructed multi-task perception compatible decoder can simultaneously output detection results from multiple dimensions and is compatible with the needs of different inspection tasks. Through the dual drive of data-driven and physical mechanisms, the resistance to strong interference in complex environments is effectively enhanced, and the robustness and accuracy of fault identification of power equipment in scenarios such as low light and strong noise are significantly improved. Attached Figure Description

[0010] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of a method for constructing a large fault model for power inspection, provided in a certain embodiment of the present invention; Figure 2 This is a structural diagram of a fault large model construction system for power inspection provided in a certain embodiment of the present invention; Figure 3 This is a structural diagram of an electronic device provided in a certain embodiment of the present invention; Figure label: Among them, 10 is the first network construction module; 20 is the second network construction module; 30 is the third network construction module; 40 is the fourth network construction module; 50 is the decoder construction module; 60 is the large model generation module; 5000 is the electronic device; 5001 is the processor; 5002 is the bus; 5003 is the memory; and 5004 is the transceiver. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings and examples. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0013] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. Those skilled in the art will be able to understand the specific meaning of the above terms in this application according to the specific circumstances.

[0014] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is merely for describing specific embodiments and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0015] In one embodiment, such as Figure 1As shown, the first aspect of the present invention provides a method for constructing a large fault model for power line inspection, comprising: S1. Construct a first network; the first network is used to decouple the main modal sensing data of the equipment to be inspected from multiple scales to obtain the inherent feature components representing the equipment body information and the disturbance feature components representing the environmental noise; the main modal sensing data is other modal data besides the environmental noise data; S2. Construct a second network; the second network is used to reorganize the inherent feature components and the disturbance feature components based on the auxiliary modal perception data of the equipment to be inspected, and obtain multimodal scene features adapted to the current environment; the auxiliary modal perception data includes the environmental noise data; S3. Construct a third network; the third network is used to perform cross-modal semantic space alignment on the multimodal scene features to obtain a global alignment result; S4. Construct a fourth network; the fourth network is used to perform hierarchical attention fusion based on physical prior masks on the global alignment results to obtain fused features; S5. Construct a multi-task perception compatible decoder; the multi-task perception compatible decoder is used to decode the fused features to obtain the fault detection result of the equipment to be inspected; S6. The first network, the second network, the third network, the fourth network and the multi-task perception compatible decoder are cascaded to form a large fault model.

[0016] Specifically, this invention constructs a large-scale fault model for diagnosing faults in the equipment under inspection by collecting multimodal sensing data of the equipment under various environmental conditions (sunny days, rainy days, foggy days, nighttime, days with strong electromagnetic interference, etc.) and the corresponding inspection task text (including inspection points, inspection paths, and task priorities, etc.) and fault inspection results (equipment operation status assessment results, fault types, fault locations, etc.). The equipment under inspection includes insulators of transmission lines, transformers in substations, etc. To overcome the problem of single-mode sensitivity to environmental interference in existing technologies, this invention further subdivides the sensing data into primary mode sensing data and auxiliary mode sensing data. The auxiliary mode sensing data includes environmental noise data for environmental perception and control (such as ambient wind sound, rain sound, and substation background noise collected by microphones), meteorological sensor data (temperature, humidity, wind speed, and light intensity), and electromagnetic interference sensor data (power frequency magnetic field, high-frequency noise intensity), etc., to help the model understand the current environmental state. The main modal sensing data includes data other than environmental noise data, such as visual modal data: visual images and surface temperature distribution images of the equipment to be inspected are acquired using visible light cameras and infrared thermal imagers, respectively, to characterize the external structural state and thermal characteristics of the equipment; acoustic and vibration modal data: acoustic signals and mechanical vibration responses of the equipment to be inspected during operation are acquired using microphones and vibration sensors, to characterize the operating mechanical characteristics of the equipment; pulse current signal data: HFCT, as a fixed partial discharge pulse acquisition device, is deployed at the grounding wire, cable shielding layer, or critical grounding circuit of key power equipment. When the mechanical dog or other inspection equipment arrives at the corresponding inspection point, the real-time pulse data of HFCT is read through the edge gateway in the station to characterize the internal insulation state and pulse current response characteristics of the equipment to be inspected. These modal data do not include environmental noise data (i.e., they do not include independent channels that directly record environmental noise).

[0017] The collected raw data underwent preprocessing to improve accuracy. For example, for visual modal data, size normalization was followed by histogram equalization; for acoustic and vibration modal data, short-time Fourier transform was used to generate time-frequency graphs (window length 1024, overlap 512); for pulse current signal data, bandpass filtering (300kHz~30MHz) was used to remove power frequency and high-frequency interference, and the data was normalized to [-1,1]. All data were time-synchronized, normalized, and labeled with fault tags (fault type, location, level), and divided into training and validation sets in a 7:3 ratio to form a large fault model sample set for fault diagnosis of the equipment under inspection.

[0018] The first network can also be considered as a feature decoupling network. It adopts a structure of multi-scale feature extraction sub-network + two-branch decoupling sub-network. It takes the main modality perception data as input, extracts features through the multi-scale feature extraction sub-network, and obtains multi-scale feature maps, which are then input into the two-branch decoupling sub-network. It consists of two parallel decoupling branches (such as fully connected layers or small convolutional layers): one outputs intrinsic feature components (tensor size is the same as the input feature map, representing the ontological information of the equipment to be inspected); the other outputs perturbation feature components (tensor of the same size, representing changes caused by the environment).

[0019] The second network can also be considered as an environmental condition reorganization network. It adopts a cascaded structure of conditional encoding subnetwork + activation function. It encodes the auxiliary modal data through the conditional encoding subnetwork to obtain the environmental condition vector. Then, using the environmental condition features as input, it generates reorganization weights through the Sigmoid activation function to weight the intrinsic feature components and the perturbation feature components, thereby obtaining multimodal scene features adapted to the current field environment.

[0020] The third network can also be considered a cross-modal semantic space alignment network. It adopts a cascaded structure of global pooling layer + linear projection layer + modal encoder. After compressing multimodal scene features into embedding vectors, it achieves global alignment through two types of constraints: spatial correspondence constraint: at the same spatial location, the semantic embedding vectors of different modalities are as close as possible, and a bidirectional contrastive learning strategy is used to ensure that the same visual component corresponds in the semantic space; semantic consistency constraint: the semantic representation of the same device state (such as normal, overheat) is clustered consistently in different modalities, eliminating the semantic shift caused by modal differences. Finally, it outputs a global alignment result in which all modalities are in the same semantic space and there is a one-to-one correspondence between space and semantics.

[0021] The fourth network can also be considered as a physical prior mask hierarchical attention fusion network. This network uses the physical knowledge of the equipment to be inspected as a prior mask to guide the attention mechanism to perform hierarchical fusion. First, cross-modal fusion between acoustic vibration and pulse is performed based on the prior mask, and visual-guided cross-attention is performed based on the fusion result. After double-layer attention weighted fusion, the output is a fusion feature that incorporates physical priors and focuses on key areas.

[0022] The multi-task perception-compatible decoder adopts a cascaded structure of gated attention module + shared Transformer decoding backbone subnetwork + task-aware routing layer subnetwork + parallel multi-decoder head, which is compatible with different fault detection tasks. It fuses, decodes, and classifies the task embedding vectors and fusion features corresponding to the perception data and then inputs them into the decoder head for processing to obtain the fault detection results of the equipment to be inspected. The decoder head includes at least a fault classification head, a fault location head, and a severity assessment head; each task head has independent top-level parameters, while the bottom-level decoding features are completely shared, achieving multi-task perception compatibility without the need to design a separate decoding link for a single task.

[0023] Finally, the constructed networks and modules are connected sequentially according to the data flow: the output of the first network is connected to the input of the second network, the output of the second network is connected to the input of the third network, the output of the third network is connected to the input of the fourth network, and the output of the fourth network is connected to the input of the multi-task-aware compatible decoder, forming a complete end-to-end cascaded fault model. Each layer group consists of multiple functionally defined sublayers connected in series or parallel, and data flow between layers is achieved through feature maps / feature vectors. The entire process can be implemented using a general deep learning framework.

[0024] During model training, the above-annotated sample set (covering transformer samples of different seasons, weather, and fault types, labeled with corresponding fault type, location, and severity) was used, with the optimizer being Adam and the learning rate being 1e. -4 The batch size is 32, and the training run is 100 epochs, with end-to-end training convergence aimed at achieving the preset prediction accuracy. Alternatively, the training process can also employ a joint loss function (a weighted sum of classification cross-entropy loss and cross-modal alignment loss; where classification cross-entropy loss is used for device state discrimination, and cross-modal alignment loss is used to constrain the consistency of different modalities in the semantic space) to achieve synergistic optimization of classification accuracy and modality consistency.

[0025] In actual reasoning, real-time sensing data of the equipment under inspection is acquired through a multimodal sensor array integrated on a substation inspection robot (such as a quadruped robot) or a fixed monitoring terminal. All sensors are synchronized in time using a high-precision clock synchronization protocol (such as IEEE 1588 PTP) to ensure that the collected pulse, acoustic vibration, and visual data have uniform and accurate timestamp labels. The acquired real-time sensing data is input into a trained fault model (deployed on the inspection robot or UAV edge computing unit in the substation), which directly outputs complete detection results of the fault type, location, and severity of the equipment under inspection.

[0026] In one embodiment, the first network is specifically used for: The main modality sensing data is subjected to feature extraction through a multi-scale feature extraction sub-network to obtain a multi-scale feature map. The multi-scale feature map is input into a two-branch decoupled subnetwork containing an intrinsic branch and a perturbation branch. The intrinsic branch is used to perform global average pooling on the multi-scale feature map to obtain the intrinsic feature components, while the perturbation branch is used to perform global standard deviation pooling on the multi-scale feature map to obtain the perturbation feature components.

[0027] Specifically, this invention employs a multi-scale feature extraction sub-network (divided into three independent modal streams based on modality, without interference) composed of three parallel convolutional coding branches to extract features from the main modality perception data, resulting in multi-scale feature maps. Specifically, a ResNet-50 network with the last fully connected layer removed and the 3rd, 4th, and 5th residual blocks extracted is used as the visual branch, taking visual modality data as input for feature extraction and outputting multi-scale visual feature maps (sizes of 28×28×512, 14×14×1024, and 7×7×2048, respectively). A temporal convolutional network (TCN) is used as the acoustic branch, containing four residual blocks with dilation coefficients of 1, 2, 4, and 8, each with a kernel size of 3 and 256 output channels. It takes acoustic modality data as input and outputs multi-scale acoustic feature maps. A 1D-CNN one-dimensional convolutional network is used as the pulse branch: Conv1D(1,64,7)→BatchNorm→ReLU→MaxPool(3)→Conv1D(64,128,5)→Conv1D(128,256,3)→GlobalAvgPool. It takes pulse current signal data as input and outputs high-frequency pulse time-series feature maps. These extracted features together constitute a multi-scale feature map.

[0028] For each modality's feature map, a two-branch decoupling subnetwork containing intrinsic and perturbation branches is used for feature decoupling. In the intrinsic branch, global average pooling is performed on the input feature map, which means calculating the arithmetic mean of each channel of the input content along the time dimension as an intrinsic statistical descriptor characterizing the overall activation intensity of that channel. Since the vibration modes of the mechanical structure of the equipment under inspection (such as contacts and insulators) are relatively stable, the signals generated have a consistent average energy in different periods, so this descriptor effectively characterizes the intrinsic features of the equipment. In parallel, in the perturbation branch, global standard deviation pooling is performed on the input feature map, which means calculating the standard deviation of each channel of the input content along the time dimension as a perturbation statistical descriptor characterizing the degree of fluctuation of that channel on the time axis. The above intrinsic / perturbation statistical descriptors are used to generate channel-by-channel gating weights. Then, the original multi-scale feature map is modulated channel-by-channel using these gating weights to obtain the intrinsic feature components and perturbation feature components of that modality. The inherent feature components and the perturbation feature components are feature maps that are identical to the original multi-scale feature map. Their temporal and spatial resolutions are consistent, and the sampling positions of acoustic vibrations and pulses along the time dimension are preserved.

[0029] The disturbance branch first extracts candidate disturbance features representing the degree of fluctuation through global standard deviation pooling; however, large fluctuations are not directly equivalent to environmental disturbances and need to be further distinguished by combining auxiliary modal sensing data and fault labels or fault physical priors. Features consistent with environmental noise but inconsistent with the fault mechanism are identified as disturbance feature components, while those consistent with the fault label or fault physical prior are retained. In addition, intrinsic feature components are not limited to stable or low-fluctuation features, but also refer to features related to equipment structure, operating status, material aging, or stable fault mechanism, covering changes caused by normal stable response and actual faults; disturbance feature components refer to external interference features that are related to environmental data and do not meet the consistency constraints of fault labels or fault physical priors. Even if an actual fault has high fluctuations, as long as it conforms to the fault label or fault physical prior such as electro-acoustic delay, it is classified as an intrinsic feature component rather than a disturbance.

[0030] The intrinsic feature components of the visual flow are used to characterize the structural morphology and stable texture information of the equipment, while the perturbation feature components of the visual flow are used to characterize visual perturbations caused by environmental factors such as illumination, occlusion, smoke, and electromagnetic reflection. The intrinsic feature components of the acoustic vibration flow are used to characterize the mechanical vibration and acoustic response of the equipment under stable operating conditions, while the perturbation feature components of the acoustic vibration flow are used to characterize signal disturbances caused by non-equipment-specific factors such as background noise and environmental vibration. The intrinsic feature components of the pulse flow focus on key insulation defect fingerprints such as the rising edge and pulse width of the partial discharge waveform, while the perturbation feature components of the pulse flow are used to characterize the white noise substrate and carrier communication interference in the field. In addition, each mode's dual-branch decoupling subnetwork uses orthogonal constraint loss during the training phase to reduce the correlation between intrinsic feature components and perturbation feature components in the feature space. During training, the correlation between intrinsic feature components and perturbation feature components in the feature space is reduced by minimizing the orthogonal constraint loss, thereby improving their discriminability.

[0031] In addition, during feature extraction, VGG, DenseNet, EfficientNet, or a Transformer-based visual encoder (ViT) can be used for visual data; for acoustic vibration / pulse data, it can be replaced with a long short-term memory network (LSTM), gated recurrent unit (GRU), or WaveNet and other temporal processing networks; the decoupling strategy can also adopt a decoupling architecture based on variational autoencoder (VAE) or generative adversarial network (GAN) to separate intrinsic features from perturbation features.

[0032] Therefore, this invention distinguishes between environmental disturbances and real faults during the decoupling stage, which reduces the probability of environmental noise being misjudged as a fault and avoids real faults being filtered out as disturbances, providing a stable input with fault discrimination significance for the subsequent fusion of multimodal features.

[0033] In one embodiment, the second network is specifically used for: Multidimensional feature mapping is performed on the auxiliary modal perception data to obtain a scene condition vector representing the real-time state of the current environment; Using the scenario condition vector as the control signal, the recombination weights are calculated using the Sigmoid activation function; The inherent feature components and the perturbation feature components are weighted based on the recombination weights to obtain the multimodal scene features; wherein, the multimodal scene features include visual modal features, acoustic and vibration modal features, and impulse modal features; Specifically, the second network constructed in this invention dynamically weights the inherent feature components and perturbation feature components according to the real-time environment, suppresses strong interference in the current environment, and enables the generated scene adaptive multimodal scene features to adapt to the scene environment, and uses them as the input of the subsequent multimodal feature fusion encoder, thereby solving the problem of model diagnosis drift in different environments.

[0034] Temperature (normalized to [0,1]), humidity (normalized), and background electromagnetic noise intensity (normalized after logarithmic compression) in the auxiliary modal sensing data are input into a conditional coding sub-network (three fully connected layers with 32, 64, and 128 neurons, respectively, and ReLU activation function) for processing to map them into low-dimensional conditional vectors. The output is a scene conditional vector used to characterize the environmental change characteristics (or real-time environmental state) of the current detection scene.

[0035] Since the scene condition vector is a one-dimensional feature vector (dimension 128), and the intrinsic feature components and perturbation feature components of each mode are multi-dimensional feature maps with channel number C, height H, and width W, this embodiment first projects the scene condition vector to the same number of channels C through a fully connected layer, and then uses tensor broadcasting to copy and expand it in the spatial dimension H×W to align it with the feature map, thereby modulating each channel and each spatial position element by element.

[0036] For each mode, the aligned scene condition vector is used as the control signal, and the recombination weights corresponding to that mode are calculated using the Sigmoid activation function. This process is shown in the following formula: In the formula, For restructuring weights; Use the Sigmoid activation function; , These are learnable parameters; This is the scene condition vector.

[0037] Based on the calculated recombination weights, the intrinsic feature components and their perturbation feature components under each modality are weighted to obtain the adaptive scene features corresponding to that modality—visual modality features, acoustic-vibration modality features, and impulse modality features. These features are combined to form multimodal scene features; the weighting process is shown in the following formula: In the formula, For adaptive scene features; These are inherent characteristic components; These are the perturbation characteristic components.

[0038] Through the above-mentioned recombination method, it is possible to achieve high interference scenarios (such as...) Suppressing perturbation components yields multimodal adaptive features adapted to the current scene. Furthermore, the reconstruction employs channel-wise (element-wise) modulation, preserving the temporal and spatial resolutions of the feature map, thus retaining the sampling positions of acoustic vibrations and pulses along the time dimension. In addition to using Sigmoid gated units to generate dynamic weights, the environmental reconstruction process can also employ mechanisms such as Feature Linear Modulation (FiLM), Adaptive Instance Normalization (AdaIN), or Spatial Transformation Network (STN) to perform affine transformations or spatial corrections on the principal modal features using environmental condition vectors.

[0039] The conditional coding subnetwork in this invention can extract low-dimensional conditional vectors describing the current environmental state from environmental noise in real time, and dynamically generate recombination weights accordingly. These weights are not fixed values, but are continuously adjusted with environmental changes. The entire process does not require manual setting of thresholds or switching of modes, achieving complete self-adaptation. Compared with hard thresholds or discrete switches, the continuous weights generated by Sigmoid enable a smooth transition between intrinsic features and perturbation features, avoiding instability in inspection perception and diagnosis caused by feature mutations. Compared with traditional methods that use fixed preprocessing, this method can dynamically adjust feature expression according to environmental changes, effectively avoiding false alarms or missed alarms and inspection perception deviations caused by environmental mutations.

[0040] In one embodiment, the third network is specifically used for: The multimodal scene features are compressed to obtain a global feature embedding vector; The similarity between the global feature embedding vectors is calculated, and based on the similarity calculation results, a cross-modal alignment loss is constructed using a bidirectional contrastive learning strategy that incorporates a temperature coefficient. The modal encoder is trained using the cross-modal alignment loss, and the trained modal encoder is used to perform cross-modal semantic space alignment on the multimodal scene features to obtain the global alignment result; wherein, the global alignment result includes acoustic alignment features, pulse alignment features and visual alignment features; Specifically, due to the differences in physical properties of visual, acoustic, and pulse modes, their feature distributions are inconsistent. This invention constructs a multimodal feature fusion encoder, first unifies the multimodal semantic space, and then uses the physical law of partial discharge electro-acoustic propagation delay to constrain attention fusion. By using physical laws to establish rules for the model, it solves the problem of erroneous association noise in pure data-driven models, outputs highly reliable fusion features, and eliminates false associations.

[0041] For multimodal scene features, that is, scene adaptive features for each modality, they can first be mapped by the corresponding modality encoder to obtain modality alignment features with complete temporal and spatial resolution and preservation of each sampling position along the temporal dimension. Then, through an alignment projection head composed of a global pooling layer and a linear projection layer, they are compressed into a fixed-length low-dimensional global feature embedding vector. This embedding vector only retains the overall semantic information of the modality, eliminates local detail redundancy, unifies the feature dimensions and expression forms of different modalities, and solves the problem that images, time series, and pulse signals cannot be directly compared.

[0042] It should be noted that the global feature embedding vectors mentioned above, obtained through global pooling, are only used to calculate the cross-modal alignment loss and do not participate in subsequent fusion. The global alignment features output by the modal encoder (i.e., visual alignment features, acoustic-vibration alignment features, and pulse alignment features) retain complete temporal and spatial resolution, serving as input for subsequent hierarchical attention fusion based on physical prior masks. The cross-modal semantic space alignment here is achieved at the global semantic level—by applying the contrastive loss described later to the global feature embedding vectors of each modality, the modal encoder is trained so that its output falls within a consistent cross-modal semantic space, without requiring precise alignment of different modalities at every time position. Since both acoustic-vibration alignment features and pulse alignment features retain their sampling positions along the time dimension, their temporal correspondence is preserved. In subsequent fusion, the cross-modal comparison of the two at each time position is completed by the learnable projection matrix of the attention, while the physical prior mask constrains the allowed interaction time positions based on the pulse triggering time and acoustic-vibration sampling time. Therefore, alignment at the global semantic level is sufficient to support fusion.

[0043] Cosine similarity is used as a quantitative indicator of semantic consistency to calculate the similarity between global embedding vectors of different modalities. This similarity is used to measure the semantic matching degree between modalities. The closer the similarity value is to 1, the stronger the semantic association between the two modal features; the closer it is to 0, the greater the semantic difference.

[0044] A large number of multimodal data segments were collected (each segment containing scene features such as vision, acoustic vibration, and pulse from the same device at the same time). The three modal features within each segment were positive sample pairs. At the same time, modal features randomly extracted from different segments (different devices) constituted negative sample pairs. Subsequently, a bidirectional contrastive learning strategy incorporating a temperature coefficient was used to construct a cross-modal alignment loss, which includes a visual retrieval acoustic vibration loss and an acoustic vibration retrieval visual loss.

[0045] Visual Retrieval of Vibration Embedsion Loss: For a batch containing N segments, each segment has a visual embedding and an acoustic embedding. When calculating the loss, the visual image is used as the query; positive samples are the vibration embeddings of the same segment, and negative samples are the vibration embeddings of all other segments within the batch. This loss is expressed by the following formula: In the formula, For visual retrieval of acoustic vibration loss; molecular part The denominator represents the similarity of positive sample pairs from the same device; the denominator is the sum of the similarity of all samples (including negative samples) in the current batch. Embedding vectors for visual features from the same device (positive sample pairs) With acoustic vibration feature embedding vector Cosine similarity between them; The number of samples; The temperature coefficient is preferably 0.07 or 0.1. Embed the current visual feature vector with the first element in the batch. Each acoustic vibration feature embedding vector The similarity between them.

[0046] Acoustic vibration retrieval visual loss: Symmetrically, acoustic vibration is used as the query, and visual signals are used as positive and negative samples. This loss is expressed by the following formula: In the formula, Visual loss due to acoustic vibration retrieval; Cosine similarity between the acoustic and vibrational feature embedding vectors and the visual feature embedding vectors of the same device; The current acoustic vibration feature embedding vector and the first in the batch A visual feature embedding vector The similarity between them.

[0047] By combining the calculated visual retrieval acoustic loss and acoustic retrieval visual loss, the cross-modal alignment loss can be obtained: In the formula, This represents the cross-modal alignment loss.

[0048] The modal encoder (such as Meta-Transformer or ImageBind) for each modality is trained with the goal of minimizing the cross-modal alignment loss. After training, the parameters of the modal encoder are fixed. For a new test sample, the scene features of the three modalities are input into the corresponding encoders respectively, mapping to obtain global alignment features located in the same alignment semantic space, namely visual alignment features, acoustic-vibration alignment features, and pulse alignment features. The above-mentioned modal alignment features are already located in the same alignment semantic space—embedded modalities with the same device state (such as "normal" or "winding deformation") are close to each other in this space, while embedded features of different states are far apart. The visual alignment features, acoustic-vibration alignment features, and pulse alignment features together constitute the global alignment result, which serves as the input for subsequent hierarchical attention fusion based on physical prior masks. Optionally, the embedding vectors of the three modal alignment features after global pooling can be concatenated (or added, averaged) along the channel dimension to obtain a global semantic representation that characterizes the overall semantic consistency across modalities. In addition, it should be noted that the cross-modal alignment loss can also be obtained by calculating the bidirectional loss of each of the three modal features pairwise and then adding them together.

[0049] This invention employs a bidirectional comparison strategy: it requires that the embedding of modality A can correctly identify its positive sample from the embedding set of modality B (and vice versa). This bidirectional constraint allows the encoder of each modality to simultaneously assume the roles of "matcher" and "matched," thereby more fully mining the shared information between modalities, reducing redundant modal noise, improving the robustness of alignment, and eliminating the need for paired precise annotation, thus utilizing a large amount of unlabeled field data.

[0050] In one embodiment, the fourth network is specifically used for: The physical prior mask is constructed based on the physical law of electro-acoustic propagation delay of partial discharge phenomenon; The physical prior mask is superimposed on the attention weight matrix between the acoustic-vibration alignment feature and the pulse alignment feature to calculate the constrained cross-modal attention and obtain the acoustic-vibration fusion feature. A query vector is constructed based on the acoustic-vibration fusion features, and a key vector and a value vector are constructed using the visual alignment features to calculate visually guided cross-attention and output the fusion features.

[0051] Specifically, to address the problem that traditional attention mechanisms are prone to incorrect association with environmental noise, this invention first separates the global alignment results along the channel dimension, extracting the corresponding visual alignment features, acoustic alignment features, and pulse alignment features. Then, it constructs a priori masks by introducing the physical law of electro-acoustic propagation delay of partial discharge to constrain the feature interaction process. Furthermore, it employs a two-stage hierarchical attention mechanism (first acoustic-pulse fusion, then visual-guided fusion) to improve the physical interpretability of the fusion, suppress non-physically related false associations, and enhance anti-interference capabilities.

[0052] This invention utilizes the time difference of arrival (TDOA) between pulse signals (electrical signals, propagating at the speed of light) and acoustic vibration signals (mechanical waves, propagating at the speed of sound) to construct a physical prior mask. In one embodiment, the construction of the physical prior mask based on the physical law of electro-acoustic propagation time delay due to partial discharge phenomena includes: The first sampling time and the second sampling time are determined based on the main modality sensing data, and the time difference between the first sampling time and the second sampling time is calculated. The physical prior mask is constructed based on the maximum propagation distance of the internal sound waves of the equipment to be inspected and the speed of sound in the medium, combined with the time difference.

[0053] This invention determines the first sampling time, which is the pulse signal triggering time, based on the master mode sensing data. The second sampling time, which is the sampling time of the acoustic vibration signal. Meanwhile, let the maximum propagation distance of the internal sound waves of the device under test be... The speed of sound in the medium is The tolerance is If both originate from the same discharge event, then the time difference... Physical constraints must be met Based on this principle, a physical prior mask matrix is ​​constructed. : = The purpose of the aforementioned mask matrix is ​​to forcibly reset the attention weight to 0 when the time window of the sound signal does not conform to the physical propagation law (e.g., sudden environmental noise), thereby cutting off false associations. In addition, the construction of the physical prior mask is not limited to "hard threshold" truncation, but can also be designed as a "soft mask" based on Gaussian distribution, that is, as the degree of deviation of the time difference from the physical propagation law increases, the attention weight decays exponentially.

[0054] When partial discharge occurs, the electrical pulse signal (high-frequency current or ultra-high-frequency electromagnetic wave) propagates at the speed of light, while the ultrasonic signal propagates at the speed of sound. The time difference (delay) between the two signals reaching the sensor is strictly physically related to the distance from the sensor to the discharge source. This invention constructs this delay law as a priori mask and directly superimposes it onto the attention weight matrix between the acoustic vibration and the pulse. This forces the attention mechanism to prioritize feature locations that conform to the physical propagation delay, which not only reduces the dependence of the data-driven model on a large number of labeled samples but also gives the fusion result a clear physical meaning, making it easier for maintenance personnel to understand and verify.

[0055] First, perform the first-level restricted attention calculation: use a linear projection matrix to map the impulse alignment features to the query. Mapping acoustic vibration alignment features to bonds Sum Physical prior mask The values ​​are added additively to the attention score to calculate the constrained cross-modal attention, a process expressed as follows: In the formula, For cross-modal attention; The acoustic-vibration fusion feature means that for each partial discharge pulse feature, the acoustic-vibration response information with a physically reasonable time delay relationship with it is aggregated from the acoustic-vibration mode, thereby enhancing the partial discharge feature that has both electro-acoustic correspondence and suppressing irrelevant noise. For dimensions.

[0056] Furthermore, cross-modal attention mechanisms can also employ variations such as multi-head attention, sparse attention, or linear attention, as long as their computation process is explicitly constrained by physical propagation laws (such as TDOA), they are all within the scope of protection of this invention.

[0057] The second level of visual-guided full-scale interaction is then performed: The acoustic-vibration fusion features are globally averaged (or adaptively pooled) over the time dimension to obtain a global feature vector. This vector is then mapped to a query vector through a linear transformation. The visual alignment feature map is then transformed into a key vector and a value vector through two linear transformations. The acoustic-vibration fusion features are used as query cues to retrieve visual features, achieving "sound-based electrical location determination," and subsequently, the cross-attention of the visual guidance is calculated. In the formula, For cross-attention weights, used to represent the attention weights of partial-spot correlation fusion features to visual spatial locations; This refers to the visual cross-fusion feature, which is also the local fusion feature vector; This is the query vector corresponding to the acoustic-vibration fusion feature; , These are the key vectors and value vectors corresponding to the visual alignment features.

[0058] To achieve adaptive fusion of acoustic-vibration fusion features and local fusion feature vectors, this invention introduces a gated fusion unit to calculate the fusion weights: In the formula, For weight fusion.

[0059] Finally, the locally fused feature vectors are broadcast along the time dimension and then used... and The acoustic-vibration fusion features and the local fusion feature vectors are weighted separately to obtain the final fused features. Through this fusion method, the present invention can automatically adjust the contribution of visual or acoustic-vibration information based on the current fault characteristics, improving the accuracy and robustness of multimodal fusion.

[0060] This invention constructs a mask based on the physical law of partial discharge electro-acoustic propagation delay, superimposes it onto acoustic-vibration-pulse attention to obtain acoustic-vibration fusion features, and then uses these acoustic-vibration fusion features as a query to guide visual attention and finally output the fusion features. The hierarchical fusion design adopted realizes the progressive integration of multimodal information and uses the global spatial information of the visual modality to assist decision-making. This enables the final fusion features to simultaneously contain the physical fingerprint of partial discharge and the macroscopic state information of the equipment, thereby improving the comprehensive discrimination capability of complex faults. The fusion features serve as the input of the subsequent multi-task perception compatible decoder to generate fault detection results and inspection perception results.

[0061] In one embodiment, the construction of the multi-task-aware compatible decoder is specifically used for: The task embedding vector corresponding to the main modality perception data and the fusion feature are fused together to obtain the task adaptation feature; The task adaptation features are decoded to obtain multi-task shared context features; The multi-task shared context features are routed to obtain task-specific features, which are then sent to the decoding head for decoding, and the fault detection results are output.

[0062] Specifically, this invention uses a multi-task perception compatible decoder built through modular design to decode fused features and output multi-dimensional fault detection results.

[0063] First, the task embedding vector corresponding to the main modality perception data is mapped to a gated vector with the same number of fusion feature channels through a fully connected layer. This vector represents the importance of each feature channel to the current task. The fusion features are then gated and weighted along the channel dimension using the gated vector to obtain weighted fusion features. Subsequently, the task embedding vector is passed through another fully connected layer to obtain a bias vector, which is then added to the gated features to form task-adaptive features. These features are biased towards the channel modes required by the current task.

[0064] Subsequently, a shared Transformer decoding backbone sub-network is used to perform deep decoding of task-adaptive features, extracting multi-task shared contextual features. This backbone shares parameters across all tasks, avoiding the need to maintain a separate decoder for each task. It adopts a standard Transformer decoder structure, but only uses the decoder layer (no encoder, as the fused features can be considered encoder output), including: positional encoding: since the fused feature sequence length is 128 (which can be considered as a time step or spatial location), a learnable positional encoding is added; the first layer: a self-attention module (8 heads, 64 dimensions), followed by a feedforward network (FFN, intermediate dimension 256, ReLU activation) and layer normalization; the second layer: a self-attention + FFN with the same structure, stacked in L=4 layers, the input being the concatenation of task-adaptive features and positional encoding, and the output being multi-task shared contextual features. This backbone does not contain any task-specific parameters, so regardless of the task being performed, the multi-task shared contextual features computed forward are based on the same set of weights. During training, gradients from different tasks accumulate and update these shared weights, enabling the backbone to learn a universal representation beneficial to all tasks.

[0065] Then, the shared context features are converted into task-specific features through the task-aware routing layer sub-network and routed to the corresponding decoder head (output head). The routing layer dynamically selects which feature channels / positions to pass to which decoder head based on the task embedding vector. This sub-network takes the shared context features and the task embedding vector as input. First, the task embedding vector is passed through a small network (a two-layer MLP with an output dimension of 128) to generate a routing weight vector to represent the importance of each sequence position to the task. Then, the routing weights are used to weight the shared context features along the sequence dimension. Finally, the weighted features are globally averaged along the sequence dimension to obtain the task-specific feature vector (for localization tasks, global pooling is not performed; instead, the sequence dimension is retained to output a heatmap. Therefore, the routing layer can be configured in two modes: pooling mode for classification / regression and retention mode for localization).

[0066] Finally, the task-specific features are fed into the corresponding decoding heads, such as the fault classification head (composed of a global average pooling layer + 2 fully connected layers + a Softmax output layer, outputting the overall device status and specific fault type), the fault location head (composed of 2 convolutional layers + a Sigmoid segmentation output layer, outputting pixel-level fault region segmentation results, and simultaneously outputting fault rectangle coordinates via a bounding box generation unit), and the severity assessment head (composed of a global average pooling layer + 2 fully connected layers + a level output layer, outputting the mild / moderate / severe fault level), which outputs the fault detection results for the equipment to be inspected, including fault classification results, fault location results, and fault severity assessment results. Additionally, the decoding head may also include an inspection task optimization head (composed of a global average pooling layer + 2 fully connected layers + a level output layer, which determines the handling recommendation level based on the fault severity assessment results and fault type, i.e., severe corresponds to immediate re-inspection, moderate corresponds to planned re-inspection, and mild corresponds to no processing for now), used to generate dynamic optimization results for the inspection task, including the handling recommendation level.

[0067] This invention introduces task embedding vectors as gating signals, enabling the decoder to dynamically adapt to current inspection needs without retraining or switching models. By extracting shared context features across multiple tasks, it avoids the computational overhead of maintaining a separate complete decoder for each task. Based on the input task embedding vector, different parts of the shared context features are dynamically routed to the corresponding decoding heads, avoiding parameter inflation caused by fully connected mapping. Simultaneously, each task focuses only on its most relevant feature subspace, improving decoding efficiency and interpretability. When new inspection tasks (such as life prediction or vibration waveform analysis) need to be added, only a new output branch needs to be added to the routing layer, and a task embedding vector needs to be defined for the new task. No other structural changes are required, and the performance of existing tasks remains unaffected. This "hot-swappable" characteristic allows the large model to continuously expand as power inspection needs evolve, reducing long-term maintenance costs.

[0068] It should be noted that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order requirement for the execution of these steps, and they can be executed in other orders.

[0069] In another embodiment, such as Figure 2 As shown, a second aspect of the present invention provides a fault large model construction system for power line inspection, comprising: The first network construction module 10 is used to construct a first network; the first network is used to perform multi-scale feature decoupling on the main modal sensing data of the equipment to be inspected, to obtain the inherent feature components representing the equipment body information and the disturbance feature components representing the environmental noise; the main modal sensing data is modal data other than environmental noise data; The second network construction module 20 is used to construct a second network; the second network is used to reorganize the inherent feature components and the disturbance feature components based on the auxiliary modal perception data of the equipment to be inspected, and obtain multimodal scene features adapted to the current environment; the auxiliary modal perception data includes the environmental noise data; The third network construction module 30 is used to construct a third network; the third network is used to perform cross-modal semantic space alignment on the multimodal scene features to obtain a global alignment result; The fourth network construction module 40 is used to construct a fourth network; the fourth network is used to perform hierarchical attention fusion based on physical prior masks on the global alignment result to obtain fused features; The decoder construction module 50 is used to construct a multi-task perception compatible decoder; the multi-task perception compatible decoder is used to decode the fused features to obtain the fault detection result of the equipment to be inspected; The large model generation module 60 is used to cascade the first network, the second network, the third network, the fourth network and the multi-task perception compatible decoder to form a large fault model.

[0070] It should be noted that each module in the aforementioned fault model construction system for power line inspection can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module. For specific limitations regarding the fault model construction system for power line inspection, please refer to the limitations of the fault model construction method for power line inspection described above; both have the same function and role, and will not be repeated here.

[0071] A third aspect of the present invention provides an electronic device comprising: Processor, memory, and bus; The bus is used to connect the processor and the memory; The memory is used to store operation instructions; The processor is configured to execute operations corresponding to a fault large model construction method for power inspection as shown in the first aspect of this application by invoking the operation instructions.

[0072] In one alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 5000 includes a processor 5001 and a memory 5003. The processor 5001 and the memory 5003 are connected, for example, via a bus 5002. Optionally, the electronic device 5000 may also include a transceiver 5004. It should be noted that in practical applications, the transceiver 5004 is not limited to one type, and the structure of this electronic device 5000 does not constitute a limitation on the embodiments of this application.

[0073] Processor 5001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 5001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0074] Bus 5002 may include a path for transmitting information between the aforementioned components. Bus 5002 may be a PCI bus or an EISA bus, etc. Bus 5002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0075] The memory 5003 may be a ROM or other type of static storage device capable of storing static information and instructions, RAM or other type of dynamic storage device capable of storing information and instructions, or it may be an EEPROM, CD-ROM or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0076] The memory 5003 is used to store application code that executes the scheme of this application, and its execution is controlled by the processor 5001. The processor 5001 is used to execute the application code stored in the memory 5003 to implement the content shown in any of the foregoing method embodiments.

[0077] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers.

[0078] The fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a fault large model construction method for power inspection as shown in the first aspect of this application.

[0079] Another embodiment of this application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.

[0080] Furthermore, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0081] In summary, this invention relates to the field of intelligent monitoring and fault diagnosis technology for power equipment, and discloses a method, system, device, and medium for constructing a large-scale fault model for power inspection. The method involves: a first network decoupling multi-scale features from the main modal perception data of the equipment to be inspected, obtaining intrinsic feature components and perturbation feature components; a second network recombining environmental conditions from the two components based on the auxiliary modal perception data of the equipment to be inspected, obtaining multi-modal scene features; a third and fourth network sequentially performing cross-modal semantic space alignment and hierarchical attention fusion based on physical prior masks on the multi-modal scene features, obtaining fused features; a multi-task perception compatible decoder decoding the fused features to obtain fault detection results; and cascading these network structures sequentially to form a large-scale fault model for fault diagnosis of the equipment to be inspected, effectively improving the robustness and accuracy of power equipment fault identification in complex environments.

[0082] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0083] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the technical principles of this invention, and these improvements and substitutions should also be considered within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the scope of the claims.

Claims

1. A method for constructing a large-scale fault model for power line inspection, characterized in that, include: Construct the first network; The first network is used to perform multi-scale feature decoupling on the main modal sensing data of the equipment to be inspected, so as to obtain the inherent feature components representing the equipment body information and the disturbance feature components representing the environmental noise. The main modal sensing data is modal data other than environmental noise data; A second network is constructed; the second network is used to reorganize the inherent feature components and the disturbance feature components based on the auxiliary modal perception data of the equipment to be inspected, and obtain multimodal scene features adapted to the current environment. The auxiliary modal sensing data includes the environmental noise data; A third network is constructed; the third network is used to perform cross-modal semantic space alignment of the multimodal scene features to obtain a global alignment result; A fourth network is constructed; the fourth network is used to perform hierarchical attention fusion based on physical prior masks on the global alignment results to obtain fused features; Construct a multi-task-aware compatible decoder; The multi-task perception compatible decoder is used to decode the fused features to obtain the fault detection result of the equipment to be inspected; The first network, the second network, the third network, the fourth network, and the multi-task perception compatible decoder are cascaded to form a large fault model.

2. The method for constructing a large-scale fault model for power line inspection according to claim 1, characterized in that, The first network is specifically used for: The main modality sensing data is subjected to feature extraction through a multi-scale feature extraction sub-network to obtain a multi-scale feature map. The multi-scale feature map is input into a two-branch decoupled subnetwork containing an intrinsic branch and a perturbation branch. The intrinsic branch is used to perform global average pooling on the multi-scale feature map to obtain the intrinsic feature components, while the perturbation branch is used to perform global standard deviation pooling on the multi-scale feature map to obtain the perturbation feature components.

3. The method for constructing a large-scale fault model for power line inspection according to claim 1, characterized in that, The second network is specifically used for: Multidimensional feature mapping is performed on the auxiliary modal perception data to obtain a scene condition vector representing the real-time state of the current environment; Using the scenario condition vector as the control signal, the recombination weights are calculated using the Sigmoid activation function; The inherent feature components and the perturbation feature components are weighted based on the recombination weights to obtain the multimodal scene features.

4. The method for constructing a large-scale fault model for power line inspection according to claim 1, characterized in that, The third network is specifically used for: The multimodal scene features are compressed to obtain a global feature embedding vector; The similarity between the global feature embedding vectors is calculated, and based on the similarity calculation results, a cross-modal alignment loss is constructed using a bidirectional contrastive learning strategy that incorporates a temperature coefficient. The modal encoder is trained using the cross-modal alignment loss, and the trained modal encoder is used to perform cross-modal semantic space alignment of the multimodal scene features to obtain the global alignment result.

5. The method for constructing a large-scale fault model for power line inspection according to claim 1, characterized in that, The global alignment result includes acoustic alignment features, pulse alignment features, and visual alignment features; among which... The fourth network is specifically used for: The physical prior mask is constructed based on the physical law of electro-acoustic propagation delay of partial discharge phenomenon; The physical prior mask is superimposed on the attention weight matrix between the acoustic-vibration alignment feature and the pulse alignment feature to calculate the constrained cross-modal attention and obtain the acoustic-vibration fusion feature. A query vector is constructed based on the acoustic-vibration fusion features, and a key vector and a value vector are constructed using the visual alignment features to calculate visually guided cross-attention and output the fusion features.

6. The method for constructing a large-scale fault model for power line inspection according to claim 5, characterized in that, The physical prior mask is constructed based on the physical laws governing electro-acoustic propagation delay caused by partial discharge phenomena, including: The first sampling time and the second sampling time are determined based on the main modality sensing data, and the time difference between the first sampling time and the second sampling time is calculated. The physical prior mask is constructed based on the maximum propagation distance of the internal sound waves of the equipment to be inspected and the speed of sound in the medium, combined with the time difference.

7. The method for constructing a large-scale fault model for power line inspection according to claim 1, characterized in that, The construction of the multi-task-aware compatible decoder is specifically used for: The task embedding vector corresponding to the main modality perception data and the fusion feature are fused together to obtain the task adaptation feature; The task adaptation features are decoded to obtain multi-task shared context features; The multi-task shared context features are routed to obtain task-specific features, which are then sent to the decoding head for decoding, and the fault detection results are output.

8. A fault large-scale model construction system for power line inspection, characterized in that, include: The first network construction module is used to build the first network; The first network is used to perform multi-scale feature decoupling on the main modal sensing data of the equipment to be inspected, so as to obtain the inherent feature components representing the equipment body information and the disturbance feature components representing the environmental noise. The main modal sensing data is modal data other than environmental noise data; The second network construction module is used to construct a second network; the second network is used to reorganize the inherent feature components and the disturbance feature components based on the auxiliary modal perception data of the equipment to be inspected, and obtain multimodal scene features adapted to the current environment; the auxiliary modal perception data includes the environmental noise data; The third network construction module is used to construct a third network; the third network is used to perform cross-modal semantic space alignment on the multimodal scene features to obtain a global alignment result; The fourth network construction module is used to construct a fourth network; the fourth network is used to perform hierarchical attention fusion based on physical prior masks on the global alignment result to obtain fused features; The decoder building block is used to build multi-task-aware compatible decoders; The multi-task perception compatible decoder is used to decode the fused features to obtain the fault detection result of the equipment to be inspected; The large model generation module is used to cascade the first network, the second network, the third network, the fourth network, and the multi-task perception compatible decoder to form a large fault model.

9. An electronic device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the fault large model construction method for power inspection as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein when the device containing the computer-readable storage medium executes the computer program, it implements the fault large model construction method for power inspection as described in any one of claims 1 to 7.

Images