Insulator defect detection method and system based on low-rank adaptation fusion

Abstract

The invention provides an insulator defect detection method and system based on low-rank adaptation fusion, and relates to the technical field of insulator defect detection, and the method comprises the steps: collecting an insulator image or video stream under a power transmission and transformation line as an input image; preprocessing the input image to obtain a standardized image; the method comprises the following steps of: extracting a multi-modal feature by adopting a convolutional neural network and Transform mixed backbone network; differentiated processing is carried out on the multi-modal features, and efficient parameter updating and fusion are carried out through a low-rank adaptation mechanism; defect category detection is carried out based on the fused multi-modal features; screening is carried out through a confidence coefficient threshold value and a non-maximum suppression algorithm, and a final detection result is output; according to the method, while the detection precision and robustness are ensured, the model calculation and storage overhead is effectively reduced, and differential feature extraction of multiple defect types, cross-modal information deep fusion and real-time efficient detection in a complex electric power scene are realized.

Description

Technical Field

[0001] This invention relates to the field of insulator defect detection technology, and more specifically, to an insulator defect detection method and system based on low-rank adaptive fusion. Background Technology

[0002] Insulators, as key components in power transmission and transformation lines, play a vital role in supporting conductors and providing electrical insulation. Insulators operating outdoors for extended periods are susceptible to various defects caused by natural factors such as rain, snow, wind, sand, pollution, and lightning strikes, as well as human or environmental stresses such as mechanical and thermal stresses. These defects can lead to damage, cracks, contamination, and flashover marks. If these defects are not detected and addressed promptly, they can result in decreased insulation performance, flashover discharge, and even large-scale power outages, posing a serious threat to the safe operation of the power grid.

[0003] In the existing technology, the detection of insulator defects mainly includes the following categories: (1) Manual inspection: The inspection is carried out by maintenance personnel or by using inspection robots to observe with the naked eye or to obtain images through photography, video recording and other means for manual analysis. This method has problems such as low detection efficiency, high labor intensity, susceptibility to subjective factors, and safety hazards in high-altitude or high-voltage environments; (2) Traditional image processing methods: The computer vision algorithms such as edge detection, grayscale analysis, and texture analysis are used to identify defects. These methods are sensitive to changes in image background, changes in lighting and dirt occlusion, have poor robustness, and are difficult to take into account the detection of multiple defect types; (3) Deep learning detection methods: In recent years, the detection methods that introduce deep learning architectures such as convolutional neural networks (CNN) and Transformer have significantly improved the detection accuracy, such as Faster R-CNN, YOLO series, DETR, etc. Some studies have introduced multimodal technology to combine visual-language models (such as Grounding) While DINO (Digital Insulator Detection Object) is applied to open-vocabulary detection tasks, these methods still suffer from the following shortcomings in insulator defect detection: Insufficient performance in small target detection: Insulator defects often occupy only a few pixels in high-resolution inspection images, leading to high difficulty in feature learning and a high false negative rate; Complex and easily confused backgrounds: Actual transmission line images contain interfering backgrounds such as sky, trees, and towers, which vary significantly under different lighting, weather, and angles, easily causing false detections; Diverse defect types: Different defects differ significantly in morphology, color, and texture, making it difficult for a single feature extraction channel to simultaneously recognize all types of defects; Limited generalization ability: Existing single-model structures show significant performance degradation in new scenes and new defect types, and are highly dependent on the distribution of training samples; Large model parameters and decreased inference efficiency: Some multimodal methods increase model complexity after introducing advanced feature fusion, which is not conducive to online detection or edge deployment. Therefore, existing technologies still have significant shortcomings in high-precision detection of insulator defects, specialized feature learning for different defect types, robustness in complex backgrounds, and real-time performance. Summary of the Invention

[0004] The purpose of this invention is to provide an insulator defect detection method and system based on low-rank adaptive fusion. By introducing a multi-expert low-rank adaptive fusion mechanism, the model can effectively reduce the computation and storage overhead of the model while ensuring detection accuracy and robustness. This enables the extraction of differentiated features of various defect types, deep fusion of cross-modal information, and real-time and efficient detection in complex power scenarios, so as to meet the practical application needs of intelligent inspection of power transmission and transformation equipment.

[0005] The technical solution of this invention is as follows: In a first aspect, this application provides an insulator defect detection method based on low-rank adaptive fusion, which includes the following steps: S1. Collect images or video streams of insulators under power transmission and transformation lines using drones, inspection robots, or fixed monitoring equipment as input images; S2. Perform distortion correction, scale normalization, contrast enhancement, and data augmentation on the input image to obtain a standardized image; S3. A backbone network combining convolutional neural networks and Transformers is used to extract multimodal features from standardized images. S4. Differentiate the processing of multimodal features and use a low-rank adaptation mechanism to efficiently update and fuse the parameters of the processed multimodal features; S5. Defect category detection based on updated and fused multimodal features; S6. Based on the detection results, the results are filtered using a confidence threshold and a non-maximum suppression algorithm, and the final detection results are output.

[0006] Furthermore, the computational process for extracting multimodal features of standardized images using a backbone network that combines convolutional neural networks and Transformers includes: For the visual branch, a backbone network combining convolutional neural networks and Transformers is used to extract features: The text branch is processed by an encoder to transform the defect description statement into semantic features: In the formula, For visual branching features, This represents a backbone network that combines convolutional neural networks and Transformers. To standardize images, For text semantic features, Indicates encoder, This represents a defect description statement. These are the parameters for the visual branch and the text branch, respectively.

[0007] Furthermore, the aforementioned differentiated processing of multimodal features includes setting up an expert network during the fusion stage and utilizing each expert to optimize for different defect feature distributions. The calculation formula includes: In the formula, Provided by experts Represents a network of k experts. For visual branching features, These are textual semantic features.

[0008] Furthermore, the computational process described above for efficiently updating and fusing the processed multimodal features using a low-rank adaptation mechanism includes: Low-rank decomposition is used for each expert parameter update: The final fused features are calculated using a gating mechanism: In the formula, This refers to the low-rank update term of the parameters of the expert weight matrix. , All are expert matrix parameters. for The transpose of the matrix, where rank is the permutation function, and m and n are the number of rows and columns of the expert weight matrix. To represent the fusion features, k represents the k-th expert network, and K represents the total number of expert networks. These are the weighting coefficients. Provided by experts For gated network functions, , These are the parameters of the k-th and j-th gating networks, respectively.

[0009] Furthermore, the calculation process for defect category detection based on the updated and fused multimodal features includes: The fused features are fed into the detection head: Define the target detection loss function: In the formula, For defect categories, For bounding box, As a feature of fusion, This is the detection head. To detect head parameters, Let the target detection loss function be... These are the weighting coefficients. For classifying losses, The bounding box regression loss is used, where y represents the true defect category. b This represents the actual bounding box information.

[0010] Furthermore, the calculation formula for filtering and outputting the final detection result based on the detection results using a confidence threshold and a non-maximum suppression algorithm includes: In the formula, For the final test results, This is a non-maximum suppression algorithm. For the predicted defect category, For the predicted bounding box information, The confidence score is... This is the confidence threshold.

[0011] Secondly, this application provides an insulator defect detection system based on low-rank adaptive fusion, comprising: The image acquisition module is used to acquire images or video streams of insulators under power transmission and transformation lines via drones, inspection robots, or fixed monitoring equipment as input images. The data preprocessing module is used to perform distortion correction, scale normalization, contrast enhancement, and data augmentation on the input image to obtain a standardized image. The multimodal feature extraction module is used to extract multimodal features of normalized images using a backbone network that combines convolutional neural networks and Transformers. The multi-expert low-rank adaptive fusion module is used to perform differentiated processing on multimodal features and to efficiently update and fuse the processed multimodal features through a low-rank adaptive mechanism. The defect category detection module is used to detect defect categories based on updated and fused multimodal features; The results output module is used to filter the detection results using a confidence threshold and a non-maximum suppression algorithm, and then output the final detection results.

[0012] Furthermore, the computational process for extracting multimodal features of standardized images using a backbone network that combines convolutional neural networks and Transformers includes: For the visual branch, a backbone network combining convolutional neural networks and Transformers is used to extract features: The text branch is processed by an encoder to transform the defect description statement into semantic features: In the formula, For visual branching features, This represents a backbone network that combines convolutional neural networks and Transformers. To standardize images, For text semantic features, Indicates encoder, This represents a defect description statement. These are the parameters for the visual branch and the text branch, respectively.

[0013] Furthermore, the aforementioned differentiated processing of multimodal features includes setting up an expert network during the fusion stage and utilizing each expert to optimize for different defect feature distributions. The calculation formula includes: In the formula, Provided by experts Represents a network of k experts. For visual branching features, These are textual semantic features.

[0014] Furthermore, the computational process described above for efficiently updating and fusing the processed multimodal features using a low-rank adaptation mechanism includes: Low-rank decomposition is used for each expert parameter update: The final fused features are calculated using a gating mechanism: In the formula, This refers to the low-rank update term of the parameters of the expert weight matrix. , All are expert matrix parameters. for The transpose of the matrix, where rank is the permutation function, and m and n are the number of rows and columns of the expert weight matrix. To represent the fusion features, k represents the k-th expert network, and K represents the total number of expert networks. These are the weighting coefficients. Provided by experts For gated network functions, , These are the parameters of the k-th and j-th gating networks, respectively.

[0015] Furthermore, the calculation process for defect category detection based on the updated and fused multimodal features includes: The fused features are fed into the detection head: Define the target detection loss function: In the formula, For defect categories, For bounding box, As a feature of fusion, This is the detection head. To detect head parameters, Let the target detection loss function be... These are the weighting coefficients. For classifying losses, The bounding box regression loss is used, where y represents the true defect category. b This represents the actual bounding box information.

[0016] Furthermore, the calculation formula for filtering and outputting the final detection result based on the detection results using a confidence threshold and a non-maximum suppression algorithm includes: In the formula, For the final test results, This is a non-maximum suppression algorithm. For the predicted defect category, For the predicted bounding box information, The confidence score is... This is the confidence threshold.

[0017] Thirdly, this application provides an electronic device, comprising: Memory, used to store one or more programs; processor; When one or more of the above programs are executed by the above processor, an insulator defect detection method and system based on low-rank adaptive fusion as described in any of the first aspects above is implemented.

[0018] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an insulator defect detection method based on low-rank adaptive fusion as described in any of the first aspects above.

[0019] Compared with the prior art, the present invention has at least the following advantages or beneficial effects: (1) The present invention provides an insulator defect detection method and system based on low-rank adaptive fusion, which effectively improves the detection accuracy of small target defects and complex backgrounds by adopting a multi-expert mechanism to perform differentiated feature learning for different defect types; (2) This invention introduces low-rank adaptation (LoRA) technology, which greatly reduces the number of model parameters and computational overhead, making it easier for edge devices to be deployed and for real-time detection; (3) This invention enhances the robustness of the model under different scenarios, lighting and weather conditions by multimodal feature fusion, thereby reducing false negatives and false positives; (4) The gating mechanism of the present invention can adaptively schedule the contribution ratio of different experts according to the input characteristics, and flexibly respond to various defect types and working condition changes; (5) This invention can be seamlessly integrated with front-end data acquisition equipment such as drones and inspection robots to meet the real-time and security requirements of power inspection tasks. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the steps of an insulator defect detection method based on low-rank adaptive fusion according to the present invention. Figure 2 This is a schematic structural block diagram of an electronic device according to an embodiment of the invention.

[0022] Icons: 101, memory; 102, processor; 103, communication interface. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0025] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0026] It should be noted that, in this document, the term "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0027] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other.

[0028] Example 1 Please see Figure 1 , Figure 1 The diagram shows the steps of an insulator defect detection method and system based on low-rank adaptive fusion provided in an embodiment of this application.

[0029] In a first aspect, this application provides an insulator defect detection method based on low-rank adaptive fusion, which includes the following steps: S1. Collect images or video streams of insulators under power transmission and transformation lines using drones, inspection robots, or fixed monitoring equipment as input images; S2. Perform distortion correction, scale normalization, contrast enhancement, and data augmentation on the input image to obtain a standardized image; S3. A backbone network combining convolutional neural networks and Transformers is used to extract multimodal features from standardized images. S4. Differentiate the processing of multimodal features and use a low-rank adaptation mechanism to efficiently update and fuse the parameters of the processed multimodal features; S5. Defect category detection based on updated and fused multimodal features; S6. Based on the detection results, the results are filtered using a confidence threshold and a non-maximum suppression algorithm, and the final detection results are output.

[0030] As a preferred implementation, the computational process for extracting multimodal features of a normalized image using a backbone network that combines convolutional neural networks and Transformers includes: For the visual branch, a backbone network combining convolutional neural networks and Transformers is used to extract features: The text branch is processed by an encoder to transform the defect description statement into semantic features: In the formula, For visual branching features, This represents a backbone network that combines convolutional neural networks and Transformers. To standardize images, For text semantic features, Indicates encoder, This represents a defect description statement. These are the parameters for the visual branch and the text branch, respectively.

[0031] As a preferred implementation, the differentiated processing of multimodal features includes setting up an expert network during the fusion stage and utilizing each expert to optimize for different defect feature distributions. The calculation formula includes: In the formula, Provided by experts Represents a network of k experts. For visual branching features, These are textual semantic features.

[0032] As a preferred implementation, the computational process for efficient parameter updating and fusion of processed multimodal features through a low-rank adaptation mechanism includes: Low-rank decomposition is used for each expert parameter update: The final fused features are calculated using a gating mechanism: In the formula, This refers to the low-rank update term of the parameters of the expert weight matrix. , All are expert matrix parameters. for The transpose of the matrix, where rank is the permutation function, and m and n are the number of rows and columns of the expert weight matrix. To represent the fusion features, k represents the k-th expert network, and K represents the total number of expert networks. These are the weighting coefficients. Provided by experts For gated network functions, , These are the parameters of the k-th and j-th gating networks, respectively.

[0033] As a preferred implementation, the calculation process for defect category detection based on the updated and fused multimodal features includes: The fused features are fed into the detection head: Define the target detection loss function: In the formula, For defect categories, For bounding box, As a feature of fusion, For detection head. To detect head parameters, Let the target detection loss function be... These are the weighting coefficients. For classifying losses, The bounding box regression loss is used, where y represents the true defect category. b This represents the actual bounding box information.

[0034] As a preferred implementation, the calculation formula for filtering and outputting the final detection result based on the detection results using a confidence threshold and a non-maximum suppression algorithm includes: In the formula, For the final test results, This is a non-maximum suppression algorithm. For the predicted defect category, For the predicted bounding box information, The confidence score is... This is the confidence threshold.

[0035] Example 2 Embodiment 2 of this application provides an insulator defect detection system based on low-rank adaptive fusion, comprising: The image acquisition module is used to acquire images or video streams of insulators under power transmission and transformation lines via drones, inspection robots, or fixed monitoring equipment as input images. The data preprocessing module is used to perform distortion correction, scale normalization, contrast enhancement, and data augmentation on the input image to obtain a standardized image. The multimodal feature extraction module is used to extract multimodal features of normalized images using a backbone network that combines convolutional neural networks and Transformers. The multi-expert low-rank adaptive fusion module is used to perform differentiated processing on multimodal features and to efficiently update and fuse the processed multimodal features through a low-rank adaptive mechanism. The defect category detection module is used to detect defect categories based on updated and fused multimodal features; The results output module is used to filter the detection results using a confidence threshold and a non-maximum suppression algorithm, and then output the final detection results.

[0036] As a preferred implementation, the computational process for extracting multimodal features of a normalized image using a backbone network that combines convolutional neural networks and Transformers includes: For the visual branch, a backbone network combining convolutional neural networks and Transformers is used to extract features: The text branch is processed by an encoder to transform the defect description statement into semantic features: In the formula, For visual branching features, This represents a backbone network that combines convolutional neural networks and Transformers. To standardize images, For text semantic features, Indicates encoder, This represents a defect description statement. These are the parameters for the visual branch and the text branch, respectively.

[0037] As a preferred implementation, the differentiated processing of multimodal features includes setting up an expert network during the fusion stage and utilizing each expert to optimize for different defect feature distributions. The calculation formula includes: In the formula, Provided by experts Represents a network of k experts. For visual branching features, These are textual semantic features.

[0038] As a preferred implementation, the computational process for efficient parameter updating and fusion of processed multimodal features through a low-rank adaptation mechanism includes: Low-rank decomposition is used for each expert parameter update: The final fused features are calculated using a gating mechanism: In the formula, This refers to the low-rank update term of the parameters of the expert weight matrix. , All are expert matrix parameters. for The transpose of the matrix, where rank is the permutation function, and m and n are the number of rows and columns of the expert weight matrix. To represent the fusion features, k represents the k-th expert network, and K represents the total number of expert networks. These are the weighting coefficients. Provided by experts For gated network functions, , These are the parameters of the k-th and j-th gating networks, respectively.

[0039] As a preferred implementation, the calculation process for defect category detection based on the updated and fused multimodal features includes: The fused features are fed into the detection head: Define the target detection loss function: In the formula, For defect categories, For bounding box, As a feature of fusion, For detection head. To detect head parameters, Let the target detection loss function be... These are the weighting coefficients. For classifying losses, The bounding box regression loss is used, where y represents the true defect category. b This represents the actual bounding box information.

[0040] As a preferred implementation, the calculation formula for filtering and outputting the final detection result based on the detection results using a confidence threshold and a non-maximum suppression algorithm includes: In the formula, For the final test results, This is a non-maximum suppression algorithm. For the predicted defect category, For the predicted bounding box information, The confidence score is... This is the confidence threshold.

[0041] Example 3 Please see Figure 2 , Figure 2 This is a schematic structural block diagram of an electronic device provided in an embodiment of this application.

[0042] An electronic device includes a memory 101, a processor 102, and a communication interface 103. The memory 101, processor 102, and communication interface 103 are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules. The processor 102 executes the software programs and modules stored in the memory 101 to perform various functional applications and data processing. The communication interface 103 can be used for signaling or data communication with other node devices.

[0043] The memory 101 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0044] The processor 102 can be an integrated circuit chip with signal processing capabilities. The processor 102 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0045] It is understood that the structure shown in the figure is for illustrative purposes only. An insulator defect detection method and system based on low-rank adaptive fusion may include more or fewer components than those shown in the figure, or have a different configuration. The components shown in the figure can be implemented using hardware, software, or a combination thereof.

[0046] In the embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The embodiments described above are merely illustrative. For example, the flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0047] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0048] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0049] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0050] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this application. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for detecting insulator defects based on low-rank adaptive fusion, characterized in that, Includes the following steps: S1. Collect images or video streams of insulators under power transmission and transformation lines using drones, inspection robots, or fixed monitoring equipment as input images; S2. Perform distortion correction, scale normalization, contrast enhancement, and data augmentation on the input image to obtain a standardized image; S3. A backbone network combining convolutional neural networks and Transformers is used to extract multimodal features from standardized images. S4. Differentiate the processing of multimodal features and use a low-rank adaptation mechanism to efficiently update and fuse the parameters of the processed multimodal features; S5. Defect category detection based on updated and fused multimodal features; S6. Based on the detection results, the results are filtered using a confidence threshold and a non-maximum suppression algorithm, and the final detection results are output.

2. The insulator defect detection method based on low-rank adaptive fusion as described in claim 1, characterized in that, The computational process for extracting multimodal features of normalized images using a backbone network that combines convolutional neural networks and Transformers includes: For the visual branch, a backbone network combining convolutional neural networks and Transformers is used to extract features: The text branch is processed by an encoder to transform the defect description statement into semantic features: In the formula, For visual branching features, This represents a backbone network that combines convolutional neural networks and Transformers. To standardize images, For text semantic features, Indicates encoder, This represents a defect description statement. These are the parameters for the visual branch and the text branch, respectively.

3. The insulator defect detection method based on low-rank adaptive fusion as described in claim 2, characterized in that, The differentiated processing of multimodal features includes setting up an expert network during the fusion stage and using each expert to optimize for different defect feature distributions. The calculation formula includes: In the formula, Provided by experts Represents a network of k experts. For visual branching features, These are textual semantic features.

4. The insulator defect detection method based on low-rank adaptive fusion as described in claim 3, characterized in that, The computational process of efficiently updating and fusing the processed multimodal features through a low-rank adaptation mechanism includes: Low-rank decomposition is used for each expert parameter update: The final fused features are calculated using a gating mechanism: In the formula, This refers to the low-rank update term of the parameters of the expert weight matrix. , All are expert matrix parameters. for The transpose of the matrix, where rank is the permutation function, and m and n are the number of rows and columns of the expert weight matrix. To represent the fusion features, k represents the k-th expert network, and K represents the total number of expert networks. These are the weighting coefficients. Provided by experts For gated network functions, , These are the parameters of the k-th and j-th gating networks, respectively.

5. The insulator defect detection method based on low-rank adaptive fusion as described in claim 4, characterized in that, The calculation process for defect category detection based on updated and fused multimodal features includes: The fused features are fed into the detection head: Define the target detection loss function: In the formula, For defect categories, For bounding box, As a feature of fusion, For detection head. To detect head parameters, Let the target detection loss function be... These are the weighting coefficients. For classifying losses, The bounding box regression loss is used, where y represents the true defect category. b This represents the actual bounding box information.

6. The insulator defect detection method based on low-rank adaptive fusion as described in claim 1, characterized in that, The calculation formula for filtering and outputting the final detection result based on the detection results using a confidence threshold and a non-maximum suppression algorithm includes: In the formula, For the final test results, This is a non-maximum suppression algorithm. For the predicted defect category, For the predicted bounding box information, The confidence score is... This is the confidence threshold.

7. An insulator defect detection system based on low-rank adaptive fusion, characterized in that, include: The image acquisition module is used to acquire images or video streams of insulators under power transmission and transformation lines via drones, inspection robots, or fixed monitoring equipment as input images. The data preprocessing module is used to perform distortion correction, scale normalization, contrast enhancement, and data augmentation on the input image to obtain a standardized image. The multimodal feature extraction module is used to extract multimodal features of normalized images using a backbone network that combines convolutional neural networks and Transformers. The multi-expert low-rank adaptive fusion module is used to perform differentiated processing on multimodal features and to efficiently update and fuse the processed multimodal features through a low-rank adaptive mechanism. The defect category detection module is used to detect defect categories based on updated and fused multimodal features; The results output module is used to filter the detection results using a confidence threshold and a non-maximum suppression algorithm, and then output the final detection results.

8. An electronic device, characterized in that, include: Memory, used to store one or more programs; processor; When the processor executes the one or more programs, it implements an insulator defect detection method based on low-rank adaptive fusion as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements an insulator defect detection method based on low-rank adaptive fusion as described in any one of claims 1-6.

Images