Visual defect detection method and system based on context example learning

Abstract

The invention provides a visual defect detection method and system based on context example learning, and relates to the field of power equipment detection.The method comprises the steps that on the basis of a power equipment image, text description of the power equipment image is generated through a large model; inputting the image and the text description of the power equipment into a trained anomaly detection network, judging the running state of the power equipment, and obtaining a detection result of whether the power equipment is abnormal or not; wherein the anomaly detection network takes a query image and a text description thereof as well as a plurality of sample images and text descriptions thereof of the same equipment as input during each training, and performs judgment by comparing image-text features of the query image and image-text features of the sample images; learning whether a query image is abnormal or normal under the condition of a given context example; according to the method, the multi-mode learning capability of a large model is combined, and the visual defects of the power equipment are efficiently detected through context example input of images and texts.

Description

Technical Field

[0001] The present disclosure relates to power equipment detection technology, which belongs to the technical field of intelligent maintenance and automatic detection of power equipment, and specifically to a visual defect detection method and system based on context example learning. Background Art

[0002] Visual defect detection of power equipment is crucial to ensuring the safety and reliability of the power system. In modern power networks, equipment failure may not only cause power outages, but may even cause more serious safety hazards. Therefore, it is particularly important to conduct visual defect detection of power equipment on a regular basis.

[0003] Although traditional manual inspection methods are widely used, they are inefficient and easily affected by human factors, which increases the risk of missed detection or misjudgment. Inspectors need to have rich experience and expertise to identify potential defects in a timely manner. However, due to the wide variety of power equipment and the fact that they often operate in high-risk environments, the reliability of manual inspections is greatly limited. In addition, existing image processing systems are usually designed based on fixed algorithms, lack flexibility and adaptability, and are difficult to cope with the diverse and complex characteristics of power equipment. Among them, although deep learning-based image processing has solved some problems, it is limited by the training effect of the data set, especially when data is insufficiently labeled, and the detection performance is not high. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a visual defect detection method and system based on contextual example learning, which combines the multimodal learning ability of a large model and realizes efficient detection of visual defects of power equipment through contextual example input of images and texts.

[0005] According to some embodiments, the present disclosure adopts the following technical solutions:

[0006] A method for visual defect detection based on contextual example learning, comprising:

[0007] Acquire an image of the electric power equipment to be inspected;

[0008] Based on the power equipment image, a text description of the power equipment image is generated using a large model;

[0009] Input the power equipment image and text description into the trained anomaly detection network to judge the operating status of the power equipment and obtain the detection result of whether the power equipment is abnormal;

[0010] Among them, the anomaly detection network, during each training, takes a query image and its text description, and several sample images of the same device and their text descriptions as input, makes judgments by comparing the image and text features of the query image with the image and text features of the sample images, and learns to make abnormal and normal judgments on the query image under the condition of given context examples.

[0011] Furthermore, the method of using a large model to generate a text description of an image of power equipment is to use a pre-trained text-visual multimodal large model to automatically generate a text description for the image of power equipment, wherein the text description includes equipment appearance information and defect inference information.

[0012] Further, the anomaly detection network includes a query image feature extraction branch, an example image feature extraction branch, a memory bank for example image feature storage, and feature comparison learning;

[0013] Wherein, the feature extraction branch of the query image is used to extract query image and text features;

[0014] The feature extraction branch of the example image is used to extract the features of the example image and text;

[0015] The memory library is used to store normal example graphic features and abnormal example graphic features of different types of power equipment;

[0016] The feature contrast learning is based on query image features and example image features, and adopts contrastive learning loss for learning, which shortens the distance between the features of the query image and the example image features of similar power equipment in the memory library, and pushes away the distance between the features of the query image and the example image features of other power equipment, thereby making the features more discriminative.

[0017] Furthermore, the feature extraction branch of the query image and the feature extraction branch of the example image adopt the same structure, including an image feature extractor and a text feature extractor, which are respectively used to extract visual features and text features of the input power equipment image and its text description, and finally splice the visual features and text features to obtain image and text features.

[0018] Furthermore, the memory library is specifically constructed as follows:

[0019] The power equipment images in the dataset are taken as example images. Based on the pre-labeled equipment types and operating status, the normal example image and text features and abnormal example image and text features of different types of power equipment are extracted through the feature extraction branch of the example images to form a memory library.

[0020] Furthermore, the discrimination by comparing the graphic features of the query image with the graphic features of the normal and abnormal example images is to calculate the similarity between the graphic features of the query image and the graphic features of the example images in the memory library through a similarity function, and judge the operating status based on the similarity.

[0021] According to some embodiments, the present disclosure adopts the following technical solutions:

[0022] A visual defect detection system based on contextual example learning, comprising:

[0023] The image acquisition module is configured to: acquire an image of the electric power equipment to be detected;

[0024] The text generation module is configured to: generate a text description of the power equipment image using the large model based on the power equipment image;

[0025] The anomaly detection module is configured to: input the power equipment image and text description into the trained anomaly detection network, judge the operating status of the power equipment, and obtain the detection result of whether the power equipment is abnormal;

[0026] Among them, the anomaly detection network, during each training, takes a query image and its text description, and several sample images of the same device and their text descriptions as input, makes judgments by comparing the image and text features of the query image with the image and text features of the sample images, and learns to make abnormal and normal judgments on the query image under the condition of given context examples.

[0027] According to some embodiments, the present disclosure adopts the following technical solutions:

[0028] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the method for visual defect detection based on context example learning is implemented.

[0029] According to some embodiments, the present disclosure adopts the following technical solutions:

[0030] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method for visual defect detection based on context example learning is implemented.

[0031] According to some embodiments, the present disclosure adopts the following technical solutions:

[0032] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the method for visual defect detection based on context example learning.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] Based on natural language processing technology and contextual learning technology, the present invention proposes a visual defect detection scheme, which combines the multimodal learning ability of a large model and realizes efficient detection of visual defects of power equipment through contextual example input of images and texts; the method does not require a large amount of labeled data, and uses a few-sample learning and expert knowledge to guide the model, which can identify various visual defects such as corrosion, cracks, and burns in power equipment, providing an efficient and reliable solution for intelligent inspection and predictive maintenance of power equipment. Compared with traditional detection methods, the present invention has the following advantages:

[0035] (1) No need to annotate a large amount of data, which reduces the cost of data acquisition.

[0036] (2) Contextual example learning improves the robustness and accuracy of defect detection.

[0037] (3) It can support defect detection of various types of equipment and has wide applicability.

[0038] (4) It provides an efficient and reliable solution for intelligent inspection and predictive maintenance of power equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0040] Figure 1 A flowchart of an embodiment of the present disclosure.

[0041] Figure 2 An example diagram of a text description of an electric power device generated by using a large model according to an embodiment of the present disclosure.

[0042] Figure 3 4 is a structural diagram of an abnormal regression network according to an embodiment of the present disclosure.

[0043] Figure 4 This is a diagram of the anomaly detection network structure of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] In recent years, the rapid development of deep learning and natural language processing technology has brought new hope to power equipment defect detection. By combining visual information with text descriptions, these emerging technologies not only improve the accuracy of automated detection, but also provide the model with richer contextual information; the emergence of in-context learning has further broadened the application possibilities in this field, enabling the model to rely on a small number of examples for instant reasoning in the absence of a large amount of labeled data. This innovative learning method makes the model more adaptable and flexible, and is no longer limited to traditional training methods. In addition, combining the contextual learning capabilities of large language models with the specific characteristics of power equipment can significantly improve the efficiency and accuracy of defect detection, provide real-time feedback and decision support, thereby providing new intelligent solutions for the maintenance and management of power equipment and reducing labor costs and safety risks; therefore, the present invention proposes a visual defect detection scheme based on natural language processing technology and contextual learning technology, which combines the multimodal learning capabilities of large models and realizes efficient detection of visual defects of power equipment through contextual example input of images and texts; this method does not require a large amount of labeled data, and uses few-sample learning and expert knowledge to guide the model, which can identify various visual defects such as corrosion, cracks, and burns in power equipment, providing an efficient and reliable solution for intelligent inspection and predictive maintenance of power equipment.

[0048] Example 1

[0049] In one embodiment of the present disclosure, a method for visual defect detection based on context example learning is provided, comprising:

[0050] Acquire an image of the electric power equipment to be inspected;

[0051] Based on the power equipment image, a text description of the power equipment image is generated using a large model;

[0052] Input the power equipment image and text description into the trained anomaly detection network to judge the operating status of the power equipment and obtain the detection result of whether the power equipment is abnormal;

[0053] Among them, the anomaly detection network, during each training, takes a query image and its text description, and several sample images of the same device and their text descriptions as input, makes judgments by comparing the image and text features of the query image with the image and text features of the sample images, and learns to make abnormal and normal judgments on the query image under the condition of given context examples.

[0054] Furthermore, the method of using a large model to generate a text description of an image of power equipment is to use a pre-trained text-visual multimodal large model to automatically generate a text description for the image of power equipment, wherein the text description includes equipment appearance information and defect inference information.

[0055] Further, the anomaly detection network includes a query image feature extraction branch, an example image feature extraction branch, a memory bank for example image feature storage, and feature comparison learning;

[0056] Wherein, the feature extraction branch of the query image is used to extract query image and text features;

[0057] The feature extraction branch of the example image is used to extract the features of the example image and text;

[0058] The memory library is used to store normal example graphic features and abnormal example graphic features of different types of power equipment;

[0059] The feature contrast learning is based on query image features and example image features, and adopts contrastive learning loss for learning, which shortens the distance between the features of the query image and the example image features of similar power equipment in the memory library, and pushes away the distance between the features of the query image and the example image features of other power equipment, thereby making the features more discriminative.

[0060] Furthermore, the feature extraction branch of the query image and the feature extraction branch of the example image adopt the same structure, including an image feature extractor and a text feature extractor, which are respectively used to extract visual features and text features of the input power equipment image and its text description, and finally splice the visual features and text features to obtain image and text features.

[0061] Furthermore, the memory library is specifically constructed as follows:

[0062] The power equipment images in the dataset are taken as example images. Based on the pre-labeled equipment types and operating status, the normal example image and text features and abnormal example image and text features of different types of power equipment are extracted through the feature extraction branch of the example images to form a memory library.

[0063] Furthermore, the discrimination by comparing the graphic features of the query image with the graphic features of the normal and abnormal example images is to calculate the similarity between the graphic features of the query image and the graphic features of the example images in the memory library through a similarity function, and judge the operating status based on the similarity.

[0064] As an embodiment, the present invention discloses a method for visual defect detection based on contextual example learning, which combines the multimodal learning ability of a large model and realizes efficient detection of visual defects of power equipment through contextual example input of images and texts. The specific implementation process is described below from the perspective of model and network training. Figure 1 As shown, specifically:

[0065] Step 1: Prepare the dataset: Build a high-quality and diverse dataset.

[0066] Visible light images of normal and abnormal operating conditions of various power equipment are collected, and the equipment type and normal and abnormal conditions are classified and labeled according to the equipment type, operating conditions, and defect severity.

[0067] Specifically, the following points should be noted when collating and annotating the visible light image dataset of power equipment:

[0068] First, the types of power equipment in the dataset are diverse, covering various power facilities such as transmission lines, transformers, switchgear, insulators, etc. This diversity enables the model to learn the performance characteristics of different devices in different states during the training process, ensuring that it can be generalized to a variety of equipment scenarios in practical applications, thereby improving the flexibility and wide applicability of detection.

[0069] Secondly, for the same type of power equipment, the data set contains a large number of normal samples, because in actual operation most of the equipment is in normal operation. However, although the number of abnormal samples is relatively small, we try to include as many as possible, and try to cover different fault types and manifestations as much as possible, such as corrosion, cracks, burning and other fault types, to help the model accurately identify potential faults and avoid the model biased towards the normal state.

[0070] Finally, based on expert knowledge, each device is labeled in detail, not only indicating the specific type of the device, but also classifying and labeling its normal or abnormal status.

[0071] Step 2: Generate text description: Use the large model to perform semantic interpretation of power equipment images.

[0072] A pre-trained text-visual multimodal large model is used to automatically generate text descriptions for each power equipment image in the dataset. These descriptions not only include equipment appearance information, but also possible defect inference information.

[0073] Specifically, a multimodal large model such as Tongyi Qianwen can be used to automatically generate a text description of the input power equipment, such as Figure 2 As shown, while inputting an image, a prompt word is input, such as: "Please describe the picture above." or "Please describe the power equipment in the picture, and describe the status of the power equipment. Is there any abnormality?" Based on the prompt word and the input image, the multimodal capability of the large model enables it to extract complex features from the image and convert these features into natural language descriptions.

[0074] The generated text description covers the appearance characteristics of the power equipment, such as equipment type, structural composition, color, shape and other visual information, and also makes further descriptions and inferences on the detailed performance of the equipment.

[0075] This automatically generated text description greatly reduces the burden of manual annotation, ensuring that each image sample is accompanied by accurate and detailed semantic information, providing a richer reference for subsequent equipment fault diagnosis and analysis.

[0076] Step 3: Adapt the encoder: Adapt the encoder based on CLIP.

[0077] A device normal and abnormal discriminator is constructed by combining a feature encoder and an adapter based on the pre-trained text-visual multimodal large model CLIP (Contrastive Language-Image Pretraining). The adapter is tuned using the text description and normal and abnormal labels of the power equipment images generated above. The encoders here include a visual encoder (CLIP-V) and a text encoder (CLIP-T). The training method used for the adaptation of the two is the same. Here, the adaptation of the text encoder (CLIP-T) is taken as an example for detailed explanation.

[0078] The text encoder of the pre-trained text-vision large model CLIP (denoted as CLIP-T) is adapted to the task of power equipment defect detection, so that the extracted features are more discriminative for the task of anomaly detection of power equipment.

[0079] Specifically, an anomaly regression network based on the combination of CLIP-T text feature encoder and text adapter is constructed to predict the anomaly score of the equipment description text, so as to judge whether the equipment status is normal or abnormal. The text adapter adopts a multi-layer perceptron structure, which aims to adapt the general text encoder to the power equipment defect detection task.

[0080] The core components of the abnormal regression network are as follows: Figure 3 As shown, it includes three modules:

[0081] 1) CLIP-T text encoder: As part of a pre-trained multimodal large model, CLIP-T is able to extract high-dimensional feature representations from text. By inputting the text description generated by the image of the power equipment, CLIP-T can encode the text description into a D-dimensional text feature vector F t ∈R D , which captures the semantic information in the text and its relevance to the visual content.

[0082] 2) Text adapter based on multi-layer perceptron MLP: text feature vector F generated by CLIP-T t It is then passed to a multi-layer perceptron for feature adaptation and mapping. The role of MLP is to further learn and extract deep features that are helpful for classification through multi-layer nonlinear transformations, thereby improving the discrimination of features in different categories.

[0083] 3) Anomaly Regressor: The features processed by MLP are finally passed to an anomaly score regressor to predict the status of the power equipment. This process can be formally expressed as:

[0084] s = Regressor(MLP(F t ))

[0085] Among them, s is an anomaly score of [0,1], which indicates the prediction of the equipment operation status, 0 for normal and 1 for abnormal.

[0086] During model training, the parameters of the CLIP-T text encoder are frozen, and only the parameters of the subsequent text adapter are optimized. This method can reduce the complexity of training and prevent the model from losing the multimodal semantic associations learned in the pre-training stage during fine-tuning. At the same time, the fine-tuning process uses the fork entropy loss function to optimize the model parameters. The loss function is defined as:

[0087]

[0088] Where N is the number of training samples, y i is the true label of the ith sample (normal is 0, abnormal is 1), s i It is the probability output of the model's prediction for this sample.

[0089] Through the above training method, the trained CLIP-T text encoder is obtained as a part of the feature extraction branch of the anomaly detection network. In the later training process of the anomaly detection network, the encoder does not need to be trained again.

[0090] Step 4: Training of anomaly detection network based on contextual example learning.

[0091] An anomaly detection network based on contextual example learning is constructed. The network learns how to judge whether a query image is abnormal or normal when given a contextual example. In each training, a query image and its text description, together with several normal or abnormal sample images of the same device and their corresponding text descriptions are given as input, and the judgment is made by comparing the graphic and text features of the query image with the graphic and text features of the normal and abnormal sample images.

[0092] Specifically, we construct an anomaly detection network based on contextual example learning. The network input is a query image randomly selected from the dataset and its corresponding text description, as well as several randomly selected example images of similar devices and their text descriptions. Figure 4 As shown in the figure, the anomaly detection network consists of several modules: a feature extraction branch for query images, a feature extraction branch for example images, a memory bank for storing example image features, and feature contrast learning.

[0093] The query image feature extraction branch and the example image feature extraction branch use the same structure, but with different inputs, such as Figure 4 As shown, each feature extraction branch includes an image feature extractor and a text feature extractor. The image feature extractor f v () The visual encoder (CLIP-V) of the pre-trained text-vision large model CLIP is followed by a visual adapter implemented by MLP, and the text feature extractor f t () consists of the above text encoder CLIP-T followed by a tuned text adapter. When the image I and text T are input, the two feature extractors are represented as:

[0094] f v (I) = MLP (CLIP-V (I))

[0095] f t (T) = MLP (CLIP - T (T))

[0096] When the query image I q and text T q When the visual and text feature extractors are used to obtain the visual features f v (I q )∈R D and text features f t (I q )∈R D . Meanwhile, the example image and (i∈{1,…,S}, S is the number of example images) are extracted by visual and text feature extractors respectively to obtain features. and The visual features and text features of each image are concatenated to obtain a 2D image-text feature.

[0097] Here, unlike directly comparing the query image and text features with the example image and text features, this embodiment constructs a memory library M∈R of example images 2C×2D , used to save the normal and abnormal features of different power equipment. The normal features here are the graphic features of different types of equipment under normal operating conditions, and the abnormal features are the graphic features of different types of equipment under normal operating conditions. Among them, C is the total number of power equipment categories, and 2C means that each category contains one normal feature and one abnormal feature. When an example image is input, according to the type of power equipment to which the example image belongs and the normal or abnormal operating state, the graphic features of the example image are used to update the corresponding graphic features in the memory library using the average sliding update strategy, which is formally expressed as:

[0098]

[0099] in, [;] indicates feature concatenation, For example image ; μ∈[0,1] is the forgetting coefficient, M[j] is the jth item in the memory library, and the corresponding here is the image and text features of the same device type and the same operating status, that is, take the image and text features in the memory library that match the example image Graphic features of devices with the same device type and operating status.

[0100] In the whole network, the parameters of CLIP-V and CLIP-T are frozen, and only the randomly initialized visual adapter and the tuned text adapter are learned, and the contrastive learning loss InfoNCE is used for learning, which is defined as follows:

[0101]

[0102] Where N is the total number of query images, sim(,) calculates the similarity function of two image-text feature vectors, For the query image The graphic features of For images The corresponding feature item M(j) in the memory library is the jth item in the memory library, 4CD is the total size of the memory library, exp() is the exponential function, and τ is the temperature constant. This loss brings the features of the query image closer to the prototype features of the same type of power equipment in the memory library that belong to the normal or abnormal category, and pushes the distance to other features away, making the features more discriminative.

[0103] Step 5: Testing phase: Use the trained anomaly detection network for inference.

[0104] Input a test image and use the multimodal large model to get its text description. Then input the image and text into the feature extraction branch of the query image of the contextual example learning anomaly detection network to obtain the image and text features of the query image. Based on the image and text features, perform anomaly discrimination and get the result of whether the device in the image is abnormal.

[0105] Specifically, when a test image is input, the network extracts the image and text features, and calculates the similarity with the image and text features in the memory library through the sim(,) function, selects the image and text features with the highest similarity, and judges whether the test image has any abnormality based on the normal and abnormal operating status of the image and text features.

[0106] Example 2

[0107] In one embodiment of the present disclosure, a visual defect detection system based on contextual example learning is provided, comprising:

[0108] The image acquisition module is configured to: acquire an image of the electric power equipment to be detected;

[0109] The text generation module is configured to: generate a text description of the power equipment image using the large model based on the power equipment image;

[0110] The anomaly detection module is configured to: input the power equipment image and text description into the trained anomaly detection network, judge the operating status of the power equipment, and obtain the detection result of whether the power equipment is abnormal;

[0111] Among them, the anomaly detection network, during each training, takes a query image and its text description, and several sample images of the same device and their text descriptions as input, makes judgments by comparing the image and text features of the query image with the image and text features of the sample images, and learns to make abnormal and normal judgments on the query image under the condition of given context examples.

[0112] Example 3

[0113] In one embodiment of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method for visual defect detection based on context example learning is implemented.

[0114] Example 4

[0115] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method of visual defect detection based on context example learning is implemented.

[0116] Example 5

[0117] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the method for visual defect detection based on context example learning.

[0118] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0120] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A method for visual defect detection based on contextual example learning, characterized in that: include: Acquire an image of the electric power equipment to be inspected; Based on the power equipment image, a text description of the power equipment image is generated using a large model; Input the power equipment image and text description into the trained anomaly detection network to judge the operating status of the power equipment and obtain the detection result of whether the power equipment is abnormal; Among them, the anomaly detection network, during each training, takes a query image and its text description, and several sample images of the same device and their text descriptions as input, makes judgments by comparing the image and text features of the query image with the image and text features of the sample images, and learns to make abnormal and normal judgments on the query image under the condition of given context examples.

2. A method for visual defect detection based on contextual example learning as claimed in claim 1, characterized in that: The method of using a large model to generate a text description of an image of power equipment is to use a pre-trained text-visual multimodal large model to automatically generate a text description for the image of power equipment, wherein the text description includes equipment appearance information and defect inference information.

3. A method for visual defect detection based on contextual example learning as claimed in claim 1, characterized in that: The anomaly detection network includes a query image feature extraction branch, an example image feature extraction branch, a memory bank for example image feature storage, and feature comparison learning; Wherein, the feature extraction branch of the query image is used to extract query image and text features; The feature extraction branch of the example image is used to extract the features of the example image and text; The memory library is used to store normal example graphic features and abnormal example graphic features of different types of power equipment; The feature contrast learning is based on query image features and example image features, and adopts contrastive learning loss for learning, which shortens the distance between the features of the query image and the example image features of similar power equipment in the memory library, and pushes away the distance between the features of the query image and the example image features of other power equipment, thereby making the features more discriminative.

4. A method for visual defect detection based on contextual example learning as claimed in claim 3, characterized in that: The feature extraction branch of the query image and the feature extraction branch of the example image adopt the same structure, including an image feature extractor and a text feature extractor, which are respectively used to extract visual features and text features of the input power equipment image and its text description, and finally splice the visual features and text features to obtain image and text features.

5. The method for visual defect detection based on contextual example learning according to claim 3, characterized in that: The memory library is specifically constructed as follows: The power equipment images in the dataset are taken as example images. Based on the pre-labeled equipment types and operating status, the normal example image and text features and abnormal example image and text features of different types of power equipment are extracted through the feature extraction branch of the example images to form a memory library.

6. A method for visual defect detection based on contextual example learning as claimed in claim 1, characterized in that: The discrimination is performed by comparing the graphic features of the query image with the graphic features of the normal and abnormal example images, and the similarity between the graphic features of the query image and the graphic features of the example images in the memory library is calculated by a similarity function, and the operating state is judged based on the similarity.

7. A visual defect detection system based on contextual example learning, characterized in that: include: The image acquisition module is configured to: acquire an image of the electric power equipment to be detected; The text generation module is configured to: generate a text description of the power equipment image using the large model based on the power equipment image; The anomaly detection module is configured to: input the power equipment image and text description into the trained anomaly detection network, judge the operating status of the power equipment, and obtain the detection result of whether the power equipment is abnormal; Among them, the anomaly detection network, during each training, takes a query image and its text description, and several sample images of the same device and their text descriptions as input, makes judgments by comparing the image and text features of the query image with the image and text features of the sample images, and learns to make abnormal and normal judgments on the query image under the condition of given context examples.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for visual defect detection based on context example learning according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, a visual defect detection method based on context example learning as described in any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement a visual defect detection method based on context example learning as described in any one of claims 1-6.

Images