Power inspection method and system for multi-modal sparse trans-attention anomaly detection

Abstract

The invention discloses a multi-modal sparse trans-attention anomaly detection power inspection method and system, and the method comprises the steps: obtaining multi-modal data in an inspection region in an inspection process, carrying out the time-space alignment of the multi-modal data, obtaining the multi-modal sample data of a unified time sequence, carrying out the feature extraction of each modal sample data, and carrying out the feature extraction of each modal sample data; multi-modal sample features are obtained; carrying out feature saliency confidence coefficient calculation on the multi-modal sample features, and carrying out sparse token processing on the multi-modal sample features based on the feature saliency confidence coefficient to obtain core features; according to the confidence coefficient of each mode, dynamically selecting the multi-mode core features for feature fusion to obtain multi-mode fusion features, and inputting the multi-mode fusion features into a pre-constructed anomaly scoring module for anomaly judgment; and a confidence distribution diagram of the corresponding modal data position of the inspection area is generated based on the output result of the anomaly scoring module and is used for assisting in judging the anomaly reason and position of the corresponding inspection area.

Description

Technical Field

[0001] This invention relates to the field of power line inspection technology, and in particular to a power line inspection method and system for detecting multimodal sparse cross-attention anomalies. Background Technology

[0002] In the field of power grid inspection, common solutions employ drones, inspection robots, or handheld terminals to simultaneously collect multi-source sensor data, including visible light video, infrared thermal imaging, laser point clouds, and acoustic data. Feature extraction is performed on each modality separately, followed by fusion analysis to detect faults such as high temperatures, partial discharge, and equipment loosening. Existing multimodal fusion methods in power grid inspection often employ feature stitching, weighted fusion, or end-to-end learning based on dense attention deep networks. However, these existing technologies suffer from the following technical drawbacks: 1. High computational complexity of multimodal fusion; dense attention or fully connected fusion struggles to meet latency and computational constraints in edge or near-real-time scenarios. 2. Inability to dynamically perceive and adaptively adjust the reliability of each modality's data, leading to increased false alarms or missed alarms due to changes in illumination, noise interference, occlusion, or missing modalities. 3. Poor interpretability of multimodal fusion results; difficulty in clearly identifying the source of abnormal modalities and specific equipment areas, hindering rapid decision-making by on-site maintenance personnel. 4. Lack of effective online adaptive and drift protection mechanisms, making it difficult for models to maintain stable performance under long-term operation and environmental changes. Summary of the Invention

[0003] One objective of this invention is to provide a power line inspection method and system for multimodal sparse cross-attention anomaly detection. The method and system perform spatiotemporal alignment of different modal data during the power line inspection process, extract features from each modal data in the spatiotemporally aligned multimodal data using a lightweight feature extraction network, and perform sparse tokenization on the extracted features based on the evaluation results of feature saliency and confidence. This allows the invention to effectively reduce the computational complexity of the model while ensuring its accuracy. Simultaneously, the spatiotemporally aligned features enable the model computation to meet low-latency scenarios, reducing spatiotemporal misalignment of inspection results caused by latency.

[0004] Another objective of this invention is to provide a power line inspection method and system for multimodal sparse cross-attention anomaly detection. The method and system construct a dynamically adjusted and confidence-driven sparse feature interaction method. Furthermore, this invention employs a joint anomaly scoring mechanism of self-supervised reconstruction error and cross-modal comparative learning distance for the fused multimodal features, enabling the fused multimodal features to achieve adaptive adjustment and perception, thereby effectively identifying abnormal interference in the multimodal fused features and improving the accuracy of multimodal feature fusion representation.

[0005] Another objective of this invention is to provide a power inspection method and system for multimodal sparse cross-attention anomaly detection. The method and system, based on the closed-loop monitoring method of anomaly scores output by different modal confidence distributions, can effectively assist in locating the source of anomalies and the abnormal equipment parts. This significantly improves the interpretability of the multimodal fusion features in this invention, and can effectively enhance the auxiliary judgment of the source and cause of anomalies by maintenance personnel when facing abnormal situations.

[0006] Another objective of this invention is to provide a power line inspection method and system for multimodal sparse cross-attention anomaly detection. The method and system construct a self-supervised model for environmental changes and modal drift. Using the self-supervised model, sparse attention weights or anomaly scoring thresholds can be adaptively adjusted when the environment changes and modal drift occurs, thereby ensuring the stability of the multimodal model under long-term and complex environmental changes.

[0007] To achieve at least one of the above-mentioned objectives, the present invention further provides a power line inspection method for multimodal sparse cross-attention anomaly detection, the method comprising:

[0008] Multimodal data within the inspection area is acquired during the inspection process, and the multimodal data is spatiotemporally aligned to obtain multimodal sample data with a unified time series. Feature extraction is performed on each type of sample data to obtain multimodal sample features.

[0009] The confidence score of the feature saliency of the multimodal sample features is calculated, and the multimodal sample features are sparsely tokenized based on the confidence score of the feature saliency to obtain the core features;

[0010] Based on the confidence level of each modality, multimodal core features are dynamically selected for feature fusion to obtain multimodal fusion features. The multimodal fusion features are then input into a pre-built anomaly scoring module for anomaly judgment.

[0011] Based on the output of the anomaly scoring module, a confidence distribution map is generated, which includes the location of the modal data corresponding to the inspection area, to help determine the cause and location of the anomaly in the corresponding inspection area.

[0012] According to a preferred embodiment of the present invention, multimodal sensing data including visible light video, infrared thermal image, laser point cloud and acoustic signal are acquired within the inspection area, and the timestamp of each sensor corresponding to the multimodal sensing data is acquired, and the installation pose data of each modal sensor is acquired. Based on the timestamp and installation pose data, each modal sensing data is spatiotemporally registered using a time synchronization algorithm and a spatial calibration method to obtain multimodal sample data with a unified time series.

[0013] According to another preferred embodiment of the present invention, the confidence calculation method for feature saliency includes: firstly, using a lightweight convolutional network to perform spatial-temporal feature extraction on the multimodal sample data to obtain multimodal sample features; then, calculating the energy intensity, gradient rate of change, and attention response of each modality based on the multimodal sample data; calculating the confidence of the energy intensity, gradient rate of change, and attention response of each modality respectively; and further weighting and summing to obtain the confidence value within each modality.

[0014] According to another preferred embodiment of the present invention, the confidence calculation method for feature saliency includes: after obtaining the confidence value within each modality, configuring a confidence threshold to adaptively filter the sparsity key tokens of the multimodal sample features to obtain the core features of the multimodal samples, wherein the core features are high-confidence key sample features among the multimodal sample features, dynamically selecting the high-confidence key sample features to interact with the core features of different modalities, wherein an attention mechanism is used to select high-confidence key sample features for cross-modal feature matching and fusion to obtain multimodal fused features.

[0015] According to another preferred embodiment of the present invention, the method for constructing the anomaly scoring module includes: pre-configuring a self-supervised reconstruction network and a cross-modal learning contrast space; uniformly encoding multimodal features using a multimodal encoder; fusing and encoding the multimodal features using a fusion encoder to obtain intermediate variables; further reconstructing each modal feature of the intermediate variables using the corresponding modal decoder; calculating the reconstruction loss of each modal feature; further calculating the total reconstruction loss of the multimodal features; and using the total reconstruction loss as the reconstruction error of the anomaly scoring module, wherein the reconstruction loss is calculated based on a specific distance metric of different modalities.

[0016] According to another preferred embodiment of the present invention, the method for constructing the anomaly scoring module includes: projecting the multimodal features into the cross-modal learning comparison space; performing loss comparison between different modalities for the same inspection sample in the cross-modal learning comparison space; wherein the distance between different modalities is calculated based on cosine similarity and the comparison loss is calculated; and a joint anomaly score is obtained by weighted summation based on the comparison loss between different modalities and the reconstruction error of the self-supervised reconstruction network as the core scoring logic of the anomaly scoring module.

[0017] According to another preferred embodiment of the present invention, after calculating the joint anomaly score, the joint anomaly score is mapped to the spatial coordinates of each modality, and a confidence distribution map or anomaly heatmap of the corresponding location is constructed based on the spatial coordinates of each modality. The confidence distribution map or anomaly heatmap is used to assist in determining the current anomaly source and regional location.

[0018] According to another preferred embodiment of the present invention, the method pre-constructs a self-supervised neural network for dynamically updating sparse attention weights or anomaly scoring thresholds. If the current environment undergoes a sudden change or modal drift, the self-supervised neural network automatically updates recent normal samples as input samples and automatically updates the sparse attention weights or anomaly scoring thresholds corresponding to the normal samples, so that the multimodal detection data remains stable.

[0019] To achieve at least one of the above-mentioned objectives, the present invention further provides a power inspection system for multimodal sparse cross-attention anomaly detection, wherein the system executes the above-mentioned power inspection method for multimodal sparse cross-attention anomaly detection.

[0020] The present invention further provides a computer-readable storage medium storing a computer program, which is executed by a processor to implement the above-described multimodal sparse cross-attention anomaly detection method for power line inspection. Attached Figure Description

[0021] Figure 1 The diagram shows a flowchart of a power line inspection method for detecting multimodal sparse cross-attention anomalies. Detailed Implementation

[0022] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0023] It is understood that the term "a" should be understood as "at least one" or "one or more," that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0024] Please combine Figure 1 This invention discloses a power line inspection method and system for multimodal sparse cross-attention anomaly detection, wherein the method mainly includes the following steps:

[0025] S01. Acquire multimodal data within the inspection area during the inspection process, and perform spatiotemporal alignment on the multimodal data to obtain multimodal sample data with a unified time series. Extract features from each type of sample data to obtain multimodal sample features.

[0026] S02. Calculate the confidence level of the feature saliency of the multimodal sample features, and perform sparse tokenization on the multimodal sample features based on the confidence level of the feature saliency to obtain the core features;

[0027] S03. Dynamically select multimodal core features based on the confidence level of each modality and perform feature fusion to obtain multimodal fusion features. Input the multimodal fusion features into a pre-built anomaly scoring module for anomaly judgment.

[0028] S04. Based on the output of the anomaly scoring module, generate a confidence distribution map including the location of the modal data corresponding to the inspection area, which is used to assist in determining the cause and location of the anomaly in the corresponding inspection area.

[0029] Specifically, the multimodal data in this invention can be acquired from visible light video, infrared thermal imaging, laser point clouds, and acoustic signals, with each modality being sensed and acquired by different sensors. Furthermore, this invention performs spatiotemporal registration on the multimodal data. The spatiotemporal registration method includes: acquiring the timestamp of each sensor corresponding to the multimodal sensing data, acquiring the installation pose data of each modal sensor, and performing spatiotemporal registration on each modal sensing data using a time synchronization algorithm and a spatial calibration method based on the timestamps and installation pose data to obtain a unified time-series multimodal sample data. It should be noted that the time synchronization algorithm can employ clock synchronization algorithms, including but not limited to unified hardware trigger signals or bidirectional time synchronization protocols, to synchronize the clocks of different sensors. The spatial calibration method can employ, including but not limited to, SLAM-based calibration methods.

[0030] Furthermore, the confidence calculation method for the feature saliency includes: firstly, using a lightweight convolutional network to extract spatial-temporal features from the multimodal sample data; in another preferred embodiment of the present invention, a Transformer coding network can be used to extract the spatial-temporal features of the multimodal sample data to obtain multimodal sample features; and then, based on the multimodal sample data, the energy intensity, gradient rate of change, and attention response of each modality are calculated, and the confidence of the energy intensity, gradient rate of change, and attention response of each modality is calculated respectively, and then a weighted sum is obtained to obtain the confidence value of the feature saliency within each modality.

[0031] It should be noted that the energy intensity of each modal data described in this invention is represented in different forms in the corresponding modality. For example, in visible light video, energy intensity can be represented as visible light intensity; in infrared thermal imaging, it can be represented as target thermal sensing intensity; in laser point cloud, it can be represented as the intensity of reflected point cloud laser; and in acoustic signal, it can be represented as acoustic signal intensity.

[0032] The formula for the confidence level of energy intensity is as follows:

[0033] ;

[0034] Where c represents the confidence level, e represents the energy intensity, k represents the k-th mode, N represents the total number of feature tokens, z represents the feature tokens, i represents the sequence of feature tokens, and j in the summation symbol represents the index variable for traversing all N feature tokens. This represents the Euclidean norm.

[0035] The corresponding formula for calculating the confidence level of the gradient rate of change is:

[0036] ;

[0037] Where c represents the confidence level, g represents the gradient, k represents the k-th mode, D represents the dimension of the feature token, i represents the sequence of feature tokens, and d in the summation operator represents the index variable for traversing each component of the feature token. In this context, d represents the d-th scalar component. For the partial differential operator in the time t direction, Let be a partial differential operator in the x-direction of space. Let be a partial differential operator in the y-direction of space.

[0038] The corresponding formula for calculating the confidence score of the attention response is:

[0039] ;

[0040] Where c represents confidence, a superscript represents attention, k represents the k-th modality, i represents the sequence of feature tokens, L represents the total number of layers in the Transformer encoder, l in the summation symbol represents the index variable for traversing all Transformer layers, and N represents the total number of feature tokens. This represents the attention weight for the j-th token when computing the new representation of the i-th token, where In the summation notation, j represents the summation of the attention of all feature tokens from 1 to N, except when j=i. This represents the self-attention weight matrix of the k-th mode in the l-th layer of the Transformer.

[0041] In this invention, the confidence value of the feature saliency within each mode can be obtained by weighted summation of the aforementioned energy intensity confidence, gradient rate of change confidence, and attention response confidence. Since the weighted summation formula is a conventional formula, it will not be described in detail here.

[0042] Furthermore, this invention performs sparsification processing on multimodal samples using the confidence values ​​of the feature saliency. The specific method includes: after obtaining the confidence values ​​of the feature saliency within each modality, configuring a confidence threshold to adaptively filter the sparsification key tokens of the multimodal sample features to obtain the core features of the multimodal samples. The core features are the high-confidence key sample features among the multimodal sample features. The high-confidence key sample features are dynamically selected to interact with the core features of different modalities. An attention mechanism is used to select high-confidence key sample features for cross-modal feature matching and fusion to obtain multimodal fused features.

[0043] The normalized multimodal feature token confidence score obtained by normalizing the confidence score of the multimodal feature token is denoted as: ,in The subscript m represents the modality index, the subscript i represents the feature token index, and c represents the confidence level. The normalization process can employ existing techniques, which will not be elaborated upon in this invention. Based on the normalized feature token confidence levels, the feature tokens with the highest confidence ranking and a proportion of p are selected as the core features of the multimodal samples. It should be noted that the multimodal feature fusion method in this invention can refer to existing techniques, which will not be elaborated upon in this invention.

[0044] In another preferred embodiment of the present invention, the method for constructing the anomaly scoring module includes: pre-configuring a self-supervised reconstruction network and a cross-modal learning contrast space; uniformly encoding multimodal features using a multimodal encoder; fusing and encoding the multimodal features using a fusion encoder to obtain intermediate variables; further reconstructing each modal feature of the intermediate variables using the corresponding modal decoder; calculating the reconstruction loss of each modal feature; further calculating the total reconstruction loss of the multimodal features; and using the total reconstruction loss as the reconstruction error of the anomaly scoring module, wherein the reconstruction loss is calculated based on a specific distance metric of different modalities. It should be noted that the multimodal encoder includes, but is not limited to, different modal encoders such as CNN-based encoders, Transformer-based point cloud encoders, and Transformer audio encoders; and further, a fusion encoder is used to represent the features of different modalities in a unified form and perform feature fusion. The fusion encoder is preferably a cross-modal attention fusion encoder. The corresponding decoder can be a decoder of its respective modality. The aforementioned encoders, fusion encoders, and decoders are all prior art, and the present invention will not elaborate on them further. The reconstruction loss of each modality is defined as... In this formula, the subscript 're' indicates reconstruction, 'L' represents the loss function, and the superscript 'm' indicates a specified mode. Indicates the feature of the current mode m. represents the features of the reconstructed mode m. d represents the distance between identical modes m, where the distance is calculated differently for different specified modes m. The formula for calculating the summation loss of all modes is: M represents the total number of all modes.

[0045] Furthermore, the construction method of the anomaly scoring module includes: projecting the multimodal features into the cross-modal learning comparison space; comparing the losses between different modalities for the same inspection sample in the cross-modal learning comparison space; wherein the distance between different modalities is calculated based on cosine similarity, and the comparison loss is calculated; and a joint anomaly score is obtained by weighted summation based on the comparison loss between different modalities and the reconstruction error of the self-supervised reconstruction network, which serves as the core scoring logic of the anomaly scoring module. It should be noted that the formula for projecting each modal feature into the cross-modal learning comparison space in this invention can be expressed as: , where g m This represents the projection head for the corresponding modality m. For a batch of B samples, a cross-modal contrastive loss is constructed. The cross-modal contrastive loss is preferably calculated using the InfoNCE loss. Specifically, this invention preferably uses the corresponding modality m as the anchor point, calculates the InfoNCE loss between it and all other modalities, and performs a weighted sum to obtain a joint anomaly score. It should be noted that the InfoNCE loss in this invention is a self-supervised contrastive learning loss function, which learns meaningful representations by comparing positive sample pairs with negative sample pairs.

[0046] It is worth mentioning that after calculating the joint anomaly score, the joint anomaly score is mapped to the spatial coordinates of each mode according to its own mode type, and a confidence distribution map or anomaly heat map of the corresponding location is constructed based on the spatial coordinates of each mode. The confidence distribution map or anomaly heat map is used to help determine the current anomaly source and regional location.

[0047] This invention also constructs an online adaptive learning model. The specific method includes: pre-constructing a self-supervised neural network for dynamically updating sparse attention weights or anomaly scoring thresholds. In one preferred embodiment of this invention, the self-supervised neural network can be an online learning network used to dynamically learn the determination of environmental change features and model drift features. If the current environment undergoes a sudden change or modal drift occurs, the self-supervised neural network automatically updates recent normal samples as input samples and automatically updates the sparse attention weights or anomaly scoring thresholds corresponding to the normal samples, so that the multimodal detection data remains stable.

[0048] The processes described in the flowcharts above, as disclosed in the embodiments of this invention, can be implemented as computer software programs. Embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the methods of this application are not limited to the aforementioned functions. It should be noted that the computer-readable medium described above can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wire segments, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical fibers, RF, etc., or any suitable combination thereof.

[0049] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0050] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the stated principles, the implementation of the present invention may have any variations or modifications.

Claims

1. A power line inspection method for multimodal sparse cross-attention anomaly detection, characterized in that, The method includes: Multimodal data within the inspection area is acquired during the inspection process, and the multimodal data is spatiotemporally aligned to obtain multimodal sample data with a unified time series. Feature extraction is performed on each type of sample data to obtain multimodal sample features. The confidence score of the feature saliency of the multimodal sample features is calculated, and the multimodal sample features are sparsely tokenized based on the confidence score of the feature saliency to obtain the core features; Based on the confidence level of each modality, multimodal core features are dynamically selected for feature fusion to obtain multimodal fusion features. The multimodal fusion features are then input into a pre-built anomaly scoring module for anomaly judgment. Based on the output of the anomaly scoring module, a confidence distribution map is generated, which includes the location of the modal data corresponding to the inspection area, to help determine the cause and location of the anomaly in the corresponding inspection area.

2. The power line inspection method for multimodal sparse cross-attention anomaly detection according to claim 1, characterized in that, The system acquires multimodal sensing data, including visible light video, infrared thermal image, laser point cloud, and acoustic signal, within the inspection area. It also acquires the timestamp of each sensor corresponding to the multimodal sensing data and the installation pose data of each modal sensor. Based on the timestamp and installation pose data, the system performs spatiotemporal registration of each modal sensing data using a time synchronization algorithm and a spatial calibration method to obtain multimodal sample data with a unified time series.

3. The power line inspection method for multimodal sparse cross-attention anomaly detection according to claim 1, characterized in that, The confidence calculation method for feature saliency includes: firstly, using a lightweight convolutional network to perform spatial-temporal feature extraction on the multimodal sample data to obtain multimodal sample features; then, calculating the energy intensity, gradient rate of change, and attention response of each modality based on the multimodal sample data; calculating the confidence of the energy intensity, gradient rate of change, and attention response of each modality respectively; and further weighting and summing to obtain the confidence value within each modality.

4. The power line inspection method for multimodal sparse cross-attention anomaly detection according to claim 3, characterized in that, The confidence calculation method for feature saliency includes: after obtaining the confidence value within each modality, configuring a confidence threshold to adaptively filter the sparsity key tokens of the multimodal sample features to obtain the core features of the multimodal samples. The core features are the key sample features with high confidence among the multimodal sample features. The high-confidence key sample features are dynamically selected to interact with the core features of different modalities. The attention mechanism is used to select the high-confidence key sample features for cross-modal feature matching and fusion to obtain multimodal fusion features.

5. The power line inspection method for multimodal sparse cross-attention anomaly detection according to claim 1, characterized in that, The method for constructing the anomaly scoring module includes: pre-configuring a self-supervised reconstruction network and a cross-modal learning contrast space; using a multimodal encoder to uniformly encode multimodal features; using a fusion encoder to fuse and encode multimodal features to obtain intermediate variables; further reconstructing each modal feature of the intermediate variables using the corresponding modal decoder; calculating the reconstruction loss of each modal feature; further calculating the total reconstruction loss of the multimodal features; and using the total reconstruction loss as the reconstruction error of the anomaly scoring module, wherein the reconstruction loss is calculated based on a specific distance metric of different modalities.

6. The power line inspection method for multimodal sparse cross-attention anomaly detection according to claim 1, characterized in that, The method for constructing the anomaly scoring module includes: projecting the multimodal features into the cross-modal learning comparison space; comparing the losses between different modalities for the same inspection sample in the cross-modal learning comparison space; calculating the distance between different modalities based on cosine similarity and calculating the comparison loss; and weighting and summing the comparison loss between different modalities and the reconstruction error of the self-supervised reconstruction network to obtain a joint anomaly score as the core scoring logic of the anomaly scoring module.

7. The power line inspection method for multimodal sparse cross-attention anomaly detection according to claim 1, characterized in that, After calculating the joint anomaly score, the joint anomaly score is mapped to the spatial coordinates of each modality, and a confidence distribution map or anomaly heatmap is constructed based on the spatial coordinates of each modality. The confidence distribution map or anomaly heatmap is used to help determine the current anomaly source and regional location.

8. The power line inspection method for multimodal sparse cross-attention anomaly detection according to claim 1, characterized in that, The method pre-constructs a self-supervised neural network for dynamically updating sparse attention weights or anomaly scoring thresholds. If the current environment undergoes a sudden change or modal drift, the self-supervised neural network automatically updates recent normal samples as input samples and automatically updates the sparse attention weights or anomaly scoring thresholds corresponding to the normal samples, so that the multimodal detection data remains stable.

9. A power line inspection system for multimodal sparse cross-attention anomaly detection, characterized in that, The system executes a power line inspection method for multimodal sparse cross-attention anomaly detection as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program that is executed by a processor to implement a power line inspection method for multimodal sparse cross-attention anomaly detection as described in any one of claims 1-8.

Images