Substation defect detection method and device in small sample scene, equipment and medium

Abstract

The invention discloses a transformer substation defect detection method, device and equipment in a small sample scene and a medium, and relates to the technical field of transformer substation defect detection.The method comprises the steps that a regression network conducts spatial expansion and grid sampling on a candidate prediction frame, prototype feature embedding is conducted in combination with a category prototype vector and an attention mechanism, and a candidate prediction frame is obtained; constructing a mask stacking region propagation mechanism, and outputting a target frame prediction result; the target classification network performs screening based on the candidate prediction frame and the category prototype vector to obtain a candidate defect category set, and generates candidate defect category confidence coefficients by constructing intra-category response embedding, inter-category sorting embedding and background semantic embedding and adopting a multi-source embedding fusion mode; and introducing an image-text alignment network, and performing auxiliary scoring on the semantic consistency of the candidate prediction frames in a preset interval range to obtain calibrated candidate defect category confidence. By adopting the method, the defect detection precision under the condition of limited samples can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of substation defect detection, and in particular to a substation defect detection method, device, equipment and medium for a small sample scenario. Background Art

[0002] With the rapid adoption of intelligent inspection technology in substations, deep learning-based computer vision algorithms have been widely used in power equipment defect detection. Currently, mainstream object detection methods such as YOLO, Faster R-CNN, and DETR have been extensively studied for identifying and locating various defect targets in substation scenarios. These methods generally rely on large-scale, high-quality annotated data.

[0003] However, in actual substation inspections, due to the extremely low frequency of some defect events, such as loose screws, discoloration and rust on respirators, and peeling coatings on equipment, only a very small number of labeled defect image samples (single digit to a dozen or so) can be collected, forming a typical small-sample detection scenario. Under such conditions, mainstream object detection methods have the following typical limitations: (1) The boundaries between different defect categories are difficult to accurately delineate, resulting in unclear confidence distribution when the model outputs category probabilities. This makes it very easy for inter-class confusion to occur, leading to false detection or missed detection. (2) Since small sample targets often have weak features in images, it is difficult for the model to obtain sufficient discriminative support for bounding box regression, which leads to a significant decrease in positioning accuracy; (3) When the number of samples is limited, it is difficult for the deep feature extraction network to fully learn representative category discriminant features. The distribution of small sample categories in the feature space is easily disturbed by the overfitting of the backbone network, which ultimately leads to insufficient feature expression capabilities.

[0004] Although relevant technologies have promoted the development of small-sample defect detection to a certain extent, when faced with the challenges of scarcity of image samples and similar features between categories in power inspections, there are still technical difficulties such as low detection accuracy, weak category differentiation ability and poor model generalization. Summary of the Invention

[0005] The present invention aims to solve, at least to some extent, one of the technical problems in the related art. To this end, the present invention aims to provide a substation defect detection method, apparatus, device, and medium for a small sample scenario to improve the defect detection accuracy in the case of limited samples.

[0006] To achieve the above objectives, a first embodiment of the present invention provides a substation defect detection method for a small sample scenario, comprising: Input the substation inspection image to be detected into the backbone network and the Region Proposal Network (RPN) respectively to extract the full image features and candidate prediction boxes of the substation inspection image; The full-image features and candidate prediction boxes of the substation inspection image are input into a trained regression network that integrates prototype attention and mask propagation for processing. The regression network is used to perform spatial expansion and grid sampling on the candidate prediction boxes, embed prototype features by combining category prototype vectors with the attention mechanism, and construct a mask stacking regional propagation mechanism based on binary masks to output the target box prediction results. The full-image features and candidate prediction boxes of the substation inspection image are input into a trained target classification network for processing; the target classification network is used to screen a set of candidate defect categories based on the similarity between the candidate prediction boxes and the category prototype vectors, and generate the candidate defect category confidence of the candidate prediction boxes by constructing intra-class response embedding, inter-class ranking embedding and background semantic embedding, using a multi-source embedding fusion method; An image-text alignment network is introduced to assist in scoring the semantic consistency of candidate prediction boxes whose confidence levels of candidate defect categories are within a preset range, thereby obtaining the calibrated confidence levels of candidate defect categories. The category prototype vector is constructed during the model training process and is dynamically updated based on the momentum update mechanism.

[0007] In addition, the substation defect detection method for a small sample scenario in the above embodiment of the present invention may also have the following additional technical features: According to one embodiment of the present invention, combining the category prototype vector with the attention mechanism to perform prototype feature embedding includes: Extracting candidate region features within each of the expanded candidate prediction boxes by using the full image features and the expanded candidate prediction boxes through a region alignment operation; Using the query vector and key vector in the attention mechanism, linearly transform the candidate region features and category prototype vectors to obtain a low-dimensional embedding representation; Calculating the attention response relationship between each position of the expanded candidate prediction box and the low-dimensional embedding representation to obtain a foreground category response map and a background category response map; The foreground region embedding features generated based on the foreground category response map and the background embedding features generated based on the background category response map are concatenated to obtain regional embedding features as input to the regional propagation mechanism.

[0008] According to one embodiment of the present invention, a mask stacking region propagation mechanism is constructed based on a binary mask to output a target box prediction result, including: Constructing an input mask based on the positional relationship between the sampling points in the grid sampling and the candidate prediction box; the input mask is used to indicate whether the sampling points in the grid sampling fall into the candidate prediction box; The input mask is used as the initial input of the region propagation mechanism and is fused with the region embedding feature to generate a mask response map layer by layer through a multi-layer propagation network; The final mask response map output by the multi-layer propagation network is decoded into the normalized coordinates of the target box through coordinate integration operation.

[0009] According to one embodiment of the present invention, the image-text alignment network is introduced to perform auxiliary scoring on the semantic consistency of candidate prediction boxes whose candidate defect category confidences are within a preset interval to obtain calibrated candidate defect category confidences, including: Extract the regional visual feature vectors of the candidate prediction boxes whose confidence levels of the candidate defect categories are within a preset range, and perform similarity calculations with the category text feature matrix generated based on the natural language description of each defect category to obtain a semantic similarity score. The semantic similarity score is weightedly fused with the candidate defect category confidence of the corresponding candidate prediction box to obtain a calibrated candidate defect category confidence.

[0010] According to one embodiment of the present invention, the process of constructing a category prototype vector in model training includes: Determining a sample type of the sample set based on the number of sample images in the sample set; the sample type is a basic category or a small sample category; Extracting the average semantic features of the defect area of ​​the support set sample image in the sample set of each defect category, and selecting a corresponding aggregation method to perform feature aggregation on the average semantic features according to the sample type to which the support set sample image belongs; If the sample type is a small sample category, the average semantic features are aggregated using a mean aggregation method; If the sample type is the basic category, the average semantic features are aggregated using a K-Means clustering algorithm based on cosine distance.

[0011] According to one embodiment of the present invention, the method further includes: After each round of training iteration is completed, the momentum update method is used to update the category prototype vector that best matches the query set sample image in the defect category of the query set sample image input to the model.

[0012] According to one embodiment of the present invention, during the model training process, the loss value of the target classification network is calculated by fusing Focal Loss with a classification loss function based on collaborative optimization of positive and negative samples.

[0013] To achieve the above objectives, a second embodiment of the present invention provides a substation defect detection device for a small sample scenario, comprising: A data extraction module is used to input the substation inspection image to be detected into the backbone network and the RPN network respectively, and extract the full image features and candidate prediction boxes of the substation inspection image; A regression network processing module inputs the full-image features and candidate prediction boxes of the substation inspection image into a trained regression network that integrates prototype attention and mask propagation for processing. The regression network is used to perform spatial expansion and grid sampling on the candidate prediction boxes, embed prototype features using the category prototype vector and attention mechanism, and construct a mask stacking regional propagation mechanism based on binary masks to output target box prediction results. The target classification network processing module inputs the full-image features and candidate prediction boxes of the substation inspection image into the trained target classification network for processing; the target classification network is used to screen and obtain a set of candidate defect categories based on the similarity between the candidate prediction boxes and the category prototype vectors; constructs an intra-class response embedding, an inter-class ranking embedding, and a background semantic embedding for each candidate defect category, and generates a candidate defect category confidence score for the candidate prediction box through a multi-source embedding fusion method; The image-text alignment network processing module introduces an image-text alignment network to assist in scoring the semantic consistency of candidate prediction boxes whose candidate defect category confidence lies within a preset interval, and obtains the calibrated candidate defect category confidence; The category prototype vector is constructed during the model training process and is dynamically updated based on the momentum update mechanism.

[0014] To achieve the above-mentioned purpose, an embodiment of the third aspect of the present invention proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the substation defect detection method for the above-mentioned small sample scenario are implemented.

[0015] To achieve the above objectives, a fourth embodiment of the present invention proposes a computer-readable storage medium having a computer program stored thereon. When the computer program is executed, the steps of the substation defect detection method for the above small sample scenario are implemented.

[0016] The substation defect detection method, apparatus, equipment and medium for small sample scenarios of the embodiments of the present invention, by fusing the category prototype attention and mask propagation mechanism in the regression network, make the candidate region features closer to the discriminant features of the target category and improve the accuracy of candidate box positioning. In the classification stage, multi-source embedding such as intra-class response, inter-class sorting and background semantics is combined to effectively enhance the category separability and suppress background interference. The image-text alignment network is introduced to the low-confidence candidate box for semantic consistency calibration to reduce missed detection and false detection of small sample categories. Moreover, the category prototype vectors used by the regression network and the target classification network are optimized by momentum updating in the model training stage, which can improve the generalization ability and stability of the model in small sample scenarios, thereby significantly improving the accuracy and reliability of substation defect detection when labeled samples are scarce. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is an overall framework diagram of a small sample defect detection algorithm in one embodiment; Figure 2 Schematic diagram of a flow chart of a substation defect detection method in a small sample scenario according to an embodiment; Figure 3 A schematic diagram of data processing of a regression network in one embodiment; Figure 4 A schematic diagram of data processing of an image-text alignment network in one embodiment; Figure 5 FIG1 is an example diagram of updating a category prototype vector in one embodiment; Figure 6 Schematic diagram of the structure of a substation defect detection device for a small sample scenario in an embodiment. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0019] The following describes in detail the implementation details of the technical solutions of the embodiments of the present application.

[0020] Figure 1 The overall framework diagram of the small sample defect detection algorithm is shown. Based on this diagram and different embodiments, the small sample defect detection is described in detail below.

[0021] In one embodiment, Figure 2 As shown, a flow chart of a substation defect detection method for a small sample scenario is provided. The method may include the following steps: In step S101, the substation inspection image to be detected is input into the backbone network and the RPN network respectively, and the full image features and candidate prediction boxes of the substation inspection image are extracted.

[0022] refer to Figure 1 During the model inference phase, the input image is a substation inspection image to be tested. This image can be captured by an inspection robot, drone, or fixed monitoring equipment. The substation inspection image is fed into the DINOv2 ViT (DINOv2, a self-supervised visual feature extraction model based on the Vision Transformer architecture) visual backbone network for forward propagation. Patch token features are extracted in the middle layer and, after necessary mapping or rearrangement, multi-channel full-image features are obtained:

[0023] in, represents the input image, which is the inspection image of the substation to be detected; Represents the feature extraction process of the backbone network; Represents the full-image features of the output; represents the feature dimension, represents the height of the feature map, Indicates the width of the feature map. Full image features It retains the image's location information in the spatial dimension and carries multi-level semantic descriptions in the channel dimension, which can be directly used in subsequent candidate region alignment and prototype embedding operations.

[0024] At the same time, the substation inspection image is input into the RPN network to obtain the candidate prediction box of the substation inspection image , all candidate prediction boxes constitute a candidate prediction box set .

[0025] In step S102, the full-image features and candidate prediction boxes of the substation inspection image are input into the trained regression network integrating prototype attention and mask propagation for processing.

[0026] The extracted full-image features and candidate prediction boxes As the input of the regression network, it is used to perform subsequent processing such as candidate box expansion and grid sampling to complete accurate positioning and discrimination. Among them, the regression network is a detection model that combines the category prototype vector with the attention mechanism for feature enhancement and realizes the layer-by-layer enhancement of foreground area features based on the mask propagation mechanism. Figure 3 As shown, Figure 3 The data processing diagram of the regression network is shown below. Figure 3The data processing flow of the regression network is explained in detail.

[0027] Regression network for candidate prediction boxes Perform spatial expansion and grid sampling operations to retain more context information and enhance the feature expression ability of the candidate region. Specifically, the candidate prediction box set

[0028] Each candidate prediction box From the upper left corner With the lower right corner coordinates Indicates that, and Represent the horizontal and vertical coordinates of the image coordinate plane respectively. In the spatial expansion process, the candidate prediction box is first converted into the representation of the center point and width and height, and the minimum expansion amount is introduced :

[0029] in, is the expansion factor, ranging from 0.05 to 0.2, and Respectively represent The width and height of the candidate box. The expanded width and height are and , based on which the corresponding extended candidate prediction box set is generated .

[0030] For each expanded candidate prediction box Divide it horizontally and vertically The uniform grid area of Represents the number of divisions in the horizontal and vertical directions), and performs grid sampling to construct the corresponding coordinate sampling point set , and construct a binary mask based on it to indicate the sampling position within the candidate area.

[0031] The regression network will expand the candidate prediction box Combined with the category prototype vector, prototype features are embedded through an attention mechanism to strengthen the semantic association between candidate regions and corresponding categories in the feature space. After prototype feature embedding, a mask-stacked region propagation mechanism is constructed based on the aforementioned binary mask. This mechanism uses masks to guide feature flow in the feature space, allowing foreground features to be gradually enhanced during multi-layer propagation while suppressing background interference. Based on the features processed by the region propagation mechanism, the regression network outputs target bounding box predictions, primarily including defect location information.

[0032] It should be noted that the category prototype vector refers to a high-dimensional vector that can represent the overall semantic features of the defect category in the feature space. It can be understood as the feature "anchor" or "center" of the category. It serves as a category feature reference in the regression network and the subsequent target classification network inference process, and is used to guide downstream detectors to focus on the discriminative direction, thereby reducing feature distribution bias and enhancing generalization capabilities under small sample conditions. Among them, the category prototype vector is constructed during the training phase. During the training phase, the category prototype vector is dynamically updated through the momentum update mechanism, so that the category vector can be continuously optimized. In the inference phase, the category prototype vector remains fixed to ensure the stability of the positioning process.

[0033] In one embodiment, reference Figure 3 As shown in Figure 2, the process of embedding prototype features by combining the category prototype vector and the attention mechanism in the regression network includes the following steps: In step S201 , the regional image features within each extended candidate prediction frame are extracted by region alignment using the full image features and the expanded extended candidate prediction frame.

[0034] Utilize full-image features and expand the candidate prediction box set , through the regional feature alignment operation (RoIAlign) in the full image feature The image features within the expanded candidate prediction box are extracted by sampling. Among them, RoIAlign can accurately sample the candidate region on the feature map and ensure the spatial consistency of the features.

[0035] The extracted candidate region features are recorded as ,in Indicates the number of candidate boxes, is the feature dimension, Represents the spatial resolution within each candidate box after pooling (e.g., 14 × 14). These regional image features will serve as input for subsequent prototype feature embedding and mask propagation mechanisms.

[0036] In step S202, the query vector and key vector in the attention mechanism are used to perform a linear transformation on the candidate region features and the category prototype vector to obtain a low-dimensional embedding representation.

[0037] In order to introduce asymmetric matching capabilities, the candidate region features and the category prototype vectors are linearly mapped. In the linear mapping process, the category prototype vector is used As a reference for category features, Representation category For each category, take the mean of all its prototype sample vectors and normalize them to get a single normalized prototype sample vector for that category. By sequentially stacking the normalized prototype sample vectors, a category prototype set is formed. ,in, is the number of categories, is the feature dimension.

[0038] Based on this, we can get the following through linear mapping:

[0039]

[0040] in, Represents the set of query vectors obtained by the feature mapping of the candidate region, which is used to represent the feature representation of different spatial positions within each candidate region in the embedding space; Represents the key vector set obtained by mapping the category prototype set, which is used to represent the feature index of each category prototype in the embedding space; are respectively learnable linear mapping matrices, is the low-dimensional attention embedding dimension, Indicates that Flatten into a one-dimensional sequence of positions.

[0041] It can be understood that the category prototype sample vector can be further divided into the foreground category prototype sample vector and the background category prototype sample vector. If a linear mapping is performed on the candidate region feature and the foreground category prototype sample vector, the result is a low-dimensional embedding representation of the foreground; if a linear mapping is performed on the candidate region feature and the background category prototype sample vector, the result is a low-dimensional embedding representation of the background.

[0042] Step S203: Calculate the attention response relationship between each position of the expanded candidate prediction box and the low-dimensional embedding representation to obtain a foreground category response map and a background category response map.

[0043] Here we take the foreground category as an example. For the foreground category, we calculate the attention similarity between each position in the candidate region and all foreground category prototypes:

[0044] in, is the number of the candidate prediction box, Candidate prediction box grid position number, Category number, represents the vector inner product, is the scaling factor, is the foreground class prototype sample vector and the linear mapping matrix Got it.

[0045] The attention similarity sets of all positions and foreground category prototypes are organized into tensor form, denoted as , where each element Corresponding tensor The value at the corresponding position in .

[0046] right Perform Softmax normalization on the category dimension to obtain the foreground category response map ,in, , where represents the attention similarity matrix before normalization, Indicates normalization along the category dimension.

[0047] The background category response map is calculated in the same way as the foreground, except that its category prototype comes from the background region feature set.

[0048] In step S204, the foreground region embedding feature generated based on the foreground category response map is concatenated with the background embedding feature generated based on the background category response map to obtain a regional embedding feature as the input of the regional propagation mechanism.

[0049] Response maps for foreground categories Response plot with background categories Input to the foreground embedding transformation module and background embedding transformation module ,get:

[0050]

[0051] in, Embedding features for the foreground, is the background embedding feature, is the number of candidate prediction boxes, The number of grids divided for each candidate box, To embed feature dimensions, the transformation module and It can include operations such as linear mapping and position encoding.

[0052] Finally, the foreground embedding and the background embedding are concatenated in the feature channel dimension to obtain the regional embedding feature:

[0053] Among them, the regional embedding feature As the initial input of the regional propagation mechanism, it is used for subsequent feature information interaction and update.

[0054] In one embodiment, reference Figure 3 As shown in the figure, the process of outputting the target box prediction result by the region propagation mechanism based on binary mask to construct mask stack in the regression network includes the following steps: Step S301: construct an input mask based on the positional relationship between the sampling points in the grid sampling and the candidate prediction boxes.

[0055] Based on each expanded candidate prediction box Partitioned uniform coordinate sampling point set , combined with the candidate prediction box , constructing the input mask for the region propagation mechanism.

[0056] In practical applications, the coordinates of each sampling point in the coordinate sampling point set can be expressed as:

[0057]

[0058] in, Indicates the The upper left corner coordinate of the expanded candidate prediction box, and are the expanded width and height of the expanded candidate prediction box, respectively. , is the number of grid divisions.

[0059] Input Mask It can be defined as a binary matrix indicating whether the sampling point is located inside the corresponding candidate prediction box:

[0060] In this way, the generated input mask corresponds to the set of coordinate sampling points, which is used to clearly indicate whether the sampling point belongs to the valid area of ​​the candidate prediction box, thereby effectively distinguishing the foreground and background areas and providing initial spatial guidance for the region propagation mechanism.

[0061] In step S302, the input mask is used as the initial input of the region propagation mechanism and fused with the region embedding feature to generate a mask response map layer by layer through a multi-layer propagation network.

[0062] A multi-stage Region Propagation Network (RPNET) is designed in the regression network. The input mask is used as the initial input of the region propagation mechanism and is used together with the region embedding feature. The fusion method involves encoding the binary information of the input mask into a feature representation and then concatenating it with the regional embedding feature in the channel dimension.

[0063] The fused feature input is fed into RPNET, where the network structure of RPNET consists of several layers of lightweight residual blocks (ResBlock) and convolutional layers, combined with the activation function Sigmoid and the temperature factor , realizing the iterative generation of layer-by-layer mask response maps.

[0064] The propagation process is formally expressed as: Initialize the layer 0 mask , and mask the 0th layer and regional embedding features Splicing, we get the propagation input of the first layer. And so on, for the layer( ) can be expressed as:

[0065] in, is the input mask corresponding to the candidate prediction box, is the number of candidate prediction boxes, The number of grids divided for each candidate box, Indicates the The propagation input of the layer.

[0066] Each layer of the propagation network Receive the fused input and generate the next layer of mask response map:

[0067] in, Indicates the Lightweight propagation module of the layer, Indicates the The output of the layer lightweight propagation module, is the Sigmoid activation function. Through iterative propagation, foreground features are gradually diffused and enhanced, while background interference is gradually suppressed.

[0068] Step S303: decode the final mask response map output by the multi-layer propagation network into the normalized coordinates of the target box through a coordinate integration operation.

[0069] The final mask response map output by the multi-layer propagation network is decoded to obtain the normalized coordinates of the target box. The decoding method uses a coordinate integral operation to perform a weighted summation of the mask response map based on the spatial distribution of response intensities in the mask response map to calculate the normalized coordinates of the target box center and size.

[0070] In step S103, the full image features and candidate prediction boxes of the substation inspection image are input into the trained target classification network for processing.

[0071] In the target classification network, the candidate prediction box set is calculated Each candidate prediction box The similarity between the candidate prediction box and the category prototype vector The features of are compared with the category prototype vector. Specifically, combined with the candidate prediction box set , perform RoIAlign operation on the full image features for each candidate prediction box to extract candidate region features of fixed size ,in is the number of candidate prediction boxes, is the spatial resolution after the RoIAlign operation. The spatial dimensions of the candidate boxes are averaged and pooled to obtain the global features of the region. .

[0072] It should be noted that the global features here use the patch token features natively output by the visual backbone network DINOv2 ViT. ,in is the feature dimension, is the spatial resolution of the feature map. Is a two-dimensional matrix, a token sequence. The above global features is obtained by rearranging the patch token features, and its value is equivalent to ,Right now and Different expressions of the same feature.

[0073] Based on this, the regional global features of each candidate prediction box are and the category prototype vector (here the category prototype vector is ) to perform dot product operation to obtain the preliminary category matching score, and take the highest score from them Category, select a set of candidate categories , as the first The candidate defect categories of the candidate prediction boxes.

[0074] At the same time, the full image features Perform dot product matching on the channel dimension with the category prototype vector to obtain the category response map on the full image scale , used to express the response intensity of each spatial position to each category prototype, where represents the number of categories, Indicates the number of background prototypes.

[0075] After obtaining the candidate category set, in order to capture the difference in category distribution inside and outside the region, a three-way semantic embedding is constructed for each candidate category based on the response information of the entire image scale, namely, the intra-class distribution embedding , inter-class ranking embedding and background semantic embedding .

[0076] For intra-class distribution embedding , take the response channel of the candidate defect category, perform position distance encoding on the spatial area corresponding to the candidate prediction box (for example, add relative coordinates or distance center encoding to each spatial position), project it into a fixed dimension representation through a linear projection or convolution layer, and obtain an embedding that depicts the category distribution inside the region .

[0077] For inter-class ranking embedding , for each candidate defect category , extract its corresponding response graph channel , and combined with other category channels to form inter-category ranking features. Specifically, for other category channels except the current category, Sort the channels according to the category response intensity to obtain the sorting results ,in, Indicates the sorting operation of channel response strength. Indicates that for candidate categories The sorting result is mapped to a uniform dimension through linear interpolation upsampling. , and then generate semantic ranking embedding through the feedforward network , which is used to reflect the semantic priority and inhibition relationship of the candidate prediction box relative to other categories.

[0078] For background semantic embedding , through the regional global features of the candidate prediction box Dot product matching is performed with the background category prototype vector to obtain the background semantic embedding, which is essentially a background distribution map used to characterize the background distribution information of the candidate area.

[0079] Concatenate the three embeddings in the channel dimension to construct fusion features , as follows:

[0080] in, , , Represent the number of channels of the three-way embedding, and Represents height and width respectively.

[0081] The fusion features Input to the binary classification network , get the candidate defect categories Confidence score for the candidate prediction box:

[0082] in, Indicates that the candidate prediction box belongs to the category Confidence score.

[0083] For the background category, we also construct background ranking embedding features and embedded with background semantics After splicing, input the background two-class discrimination network to obtain the background discrimination score :

[0084] in, is the background discrimination score, It is the background binary classification network.

[0085] Final summary The score set of the foreground candidate categories of the candidate boxes and background score Splicing to form the confidence of candidate defect categories:

[0086] In step S104, a graphic-text alignment network is introduced to perform auxiliary scoring on the semantic consistency of the candidate prediction boxes whose candidate defect category confidences are within a preset interval, so as to obtain calibrated candidate defect category confidences.

[0087] In some substation defect detection tasks, especially those involving small sample categories, the detection model often has low confidence in these categories during the inference phase, and may even misclassify them as belonging to other, similar-looking categories. To improve the ability to determine the semantic consistency of candidate targets, this embodiment introduces an image-text alignment network. Using a cross-modal feature matching mechanism, the visual features of the candidate prediction box and the semantic features of the text description of the corresponding defect category are mapped into the same embedding space and a semantic similarity score is calculated between the two. This provides effective semantic supplementation during the discrimination of low-confidence samples, achieving cross-modal calibration of confidence.

[0088] Here, candidate prediction boxes whose confidence levels of candidate defect categories are within a preset interval are screened. The preset interval is used to screen candidate prediction boxes that require further semantic calibration. It is usually defined as candidate prediction boxes whose confidence levels are within an interval (e.g., 0.2 to 0.7), indicating that the candidate prediction boxes within the interval have neither extremely high nor extremely low confidence levels, and have greater uncertainty, making it suitable for introducing semantic similarity for auxiliary judgment.

[0089] The visual features of these candidate prediction boxes and the textual features of the defect categories are then fed into an image-text alignment network, where a similarity calculation is performed to generate an additional semantic consistency score. This score complements the detection model's original classification confidence. By integrating the two, a calibrated confidence score for the candidate defect category is generated, improving discrimination accuracy in low-confidence and small-sample classification scenarios.

[0090] It should be noted that in order to improve the calibration efficiency and stability, specific categories such as background and invalid categories can be excluded to prevent them from participating in the semantic calibration process.

[0091] In one embodiment, Figure 4 A data processing diagram of the image-text alignment network is shown, wherein the data processing flow of the image-text alignment network may include the following steps: Step S401: extracting the regional visual feature vectors of the candidate prediction boxes whose candidate defect category confidence lies within a preset interval, and performing similarity calculation with the category text feature matrix generated based on the natural language description of each defect category to obtain a semantic similarity score; For candidate prediction boxes whose confidence of candidate defect categories is within the preset interval, regional feature extraction is performed. Specifically, for candidate prediction boxes within the preset interval, a feature extractor matching the visual branch of the image-text model is used to crop and encode the candidate regions. The candidate regions are uniformly pooled into fixed-size image blocks through the ROIAlign operation and input into the visual encoder encode_image to obtain the regional visual feature vector in the inference stage. .

[0092] At the same time, the natural language description of each defect category is encoded based on a pre-trained image-text alignment model (such as CLIP) to obtain normalized category text features:

[0093] in, Represents the text feature vectors of different defect categories, is the total number of categories. Category descriptions can be constructed using prompt words. For example, for a defect category identified as "hxq_gjtps", "The silicone barrel of the respirator is damaged" can be used as the input text, and the text encoder encode_text in the image-text alignment model can be used to extract the corresponding feature vector.

[0094] Then, the regional visual feature vector After normalizing the category text features, similarity calculation is performed to obtain the semantic similarity score:

[0095] in, Representation category The text feature vector of Represents the regional visual feature vector The transpose of Indicates the The text feature vector of each category, is the temperature scaling factor, Indicates the candidate prediction boxes and the The semantic similarity score between categories.

[0096] Step S402 : performing weighted fusion on the semantic similarity score and the candidate defect category confidence of the corresponding candidate prediction box to obtain a calibrated candidate defect category confidence.

[0097] For candidate boxes within the threshold range, the subsequent defect category confidence (i.e., original confidence) output by the fusion detection model is calculated. and semantic similarity score , and obtain the calibrated candidate defect category confidence:

[0098] in, is the fusion coefficient, usually set between 0.5 and 0.9; is the semantic similarity score corresponding to the predicted defect category.

[0099] It should be noted that Figure 4 The interval judgment in is used to screen the predicted candidate boxes within the threshold interval. If the candidate defect category confidence of the candidate box is within the threshold interval, the original confidence corresponding to the predicted candidate box is weightedly fused with the semantic similarity score to obtain the calibrated candidate defect category confidence; if the candidate defect category confidence of the candidate box is not within the threshold interval, the original confidence remains unchanged.

[0100] Through the above-mentioned cross-modal semantic calibration method, the category determination accuracy of low-confidence candidate boxes can be effectively improved in small-sample defect detection, and the robustness and generalization ability of the model in rare defect category identification tasks can be enhanced.

[0101] It is understandable that the above embodiment is based on the trained regression network and target classification network, combined with the image-text alignment network, to obtain the detection results of small sample defects. In practical applications, the training process of the regression network and target classification network can refer to the relevant content of the above embodiment, Figure 1 ,In the model training stage, the input image can be the support set and query set ,based on the sample set division, where the support set is used for ,prototype feature construction and the query set is used as the input for ,model training.

[0102] During model training, the regression network uses a mechanism that combines prototype attention with mask propagation to combine the dynamic category prototype vector with candidate region features to generate an attention response map. The mask propagation mechanism achieves fine positioning of the candidate box and outputs the target box prediction results for the model training phase. The target classification network generates the semantic priority of the candidate category based on the inter-class ranking strategy, and simultaneously extracts three features: intra-class distribution embedding, inter-class ranking embedding, and background semantic embedding. These three embeddings are then integrated to calculate the defect category confidence of the candidate box and output the candidate defect category confidence for the model training phase. The loss values ​​of the regression network and target classification network during model training are calculated and optimized based on the loss values, ultimately obtaining the regression network and target classification network used in the model inference phase.

[0103] It should be noted that there are the following differences between the model training phase and the model inference phase: (1) During the model training phase, it is necessary to construct a category prototype vector for each defect category and dynamically update the category prototype vector; (2) There is a difference between the expanded candidate prediction box in the model training stage and the expanded candidate prediction box in the model inference stage. In the model training stage, the expanded box is constructed by the minimum expansion amount. , that is, the expansion box The dimensions are: , and calculate the expansion box With real box The outer bounding box of , as the final set of expanded candidate prediction boxes In the model inference stage, since there is no real box , the expansion frame constructed based on the minimum expansion amount can be Directly as the final set of expanded candidate prediction boxes .

[0104] (3) Loss calculation of regression network and target classification network.

[0105] The above differences (1) and (3) are described in detail below through different embodiments to clearly illustrate the training process of the regression network and the target classification network.

[0106] In one embodiment, the process of constructing the category prototype vector includes the following steps: Step S501 : determining the sample type of the sample set based on the number of sample images in the sample set.

[0107] During model training, a sample set must be constructed for model training. The substation's inspection system acquires image data showing the operating status of various equipment. These sample images can be obtained through drone inspections, fixed camera footage, or handheld inspection equipment. The sample images cover several typical defect types, including but not limited to the following seven: damaged meter casings, damaged respirator silicone barrels, abnormal oil level and seal status, discolored respirator silicone, blurred meter dials, suspended debris, and damaged insulators.

[0108] To facilitate subsequent defect detection model training, each sample image is manually or semi-automatically annotated. This annotation includes the precise location of the defect within the image (e.g., a bounding box or mask) and the defect's category label. For example, if an insulator on power transmission equipment is damaged, an annotator will select the defect area in the image and classify the annotated defect into the corresponding defect category.

[0109] Through the above method, a sample dataset covering multiple defect types can be constructed, and it is ensured that the sample images of each category have clear defect labeling information, thereby providing a reliable data basis for subsequent small sample training.

[0110] In practical applications, in order to enhance the feature expression ability under small sample conditions, the constructed sample sets of each defect category are divided into support sets and query sets according to the category. The construction of the category prototype vector here depends on the support set.

[0111] In order to simulate the problem of insufficient small sample categories in real substation inspections and support the training and evaluation of small sample detection algorithms, all sample sets are divided into types based on the number of sample instances of each defect category in the sample set.

[0112] Count the number of sample instances for each defect category in the sample set. Based on a preset threshold, categories with at least 50 samples are classified as base classes, while categories with fewer than 50 samples are classified as novel classes. Base classes include common categories such as discoloration of the respirator silicone, blurred meter dials, suspended debris, and damaged insulators. Novels include categories with fewer samples, such as damaged meter casings, damaged respirator silicone barrels, and abnormal oil level and seal conditions.

[0113] Step S502 : extracting the average semantic features of the defect areas of the support set sample images in the sample set of each defect category, and performing feature aggregation on the average semantic features by selecting a corresponding aggregation method according to the sample type to which the support set sample images belong.

[0114] In the support set, the average semantic features of the annotated defect areas in each support set sample image are extracted. Specifically, the support set sample images are forward propagated through the frozen parameter visual backbone network DINOv2 (self-supervised visual feature extraction model) to extract the patch token features of the intermediate layer. This feature represents the semantic representation of each region after the image is divided into several regions, as follows:

[0115] In the above formula, Represents the input image, where the input image is the support set sample image; Represents the feature extraction process of the backbone network; Represents the full-image features of the output; represents the feature dimension, represents the height of the feature map, Indicates the width of the feature map.

[0116] Generate the corresponding binary mask based on the marked defect area (border or mask information) In this mask matrix, the patch positions belonging to the defect area are assigned a value of 1, and the remaining positions are assigned a value of 0. For example, when the defect area is located in the 3×3 patch range in the middle of the image, the corresponding 9 positions in the mask matrix are 1, and the remaining positions are all 0.

[0117] The generated binary mask and features Multiply element by element in the spatial position and perform weighted average calculation on the non-zero position to calculate the average semantic feature of the defect area of ​​the support set sample image .

[0118] For different sample types, different feature aggregation strategies are adopted to aggregate the average semantic features.

[0119] For small sample categories, directly All The average semantic features of the region Perform arithmetic mean to obtain a single category prototype vector. Its mathematical expression is:

[0120] in, For category The prototype vector of For category The number of regions.

[0121] For the basic categories, the K-Means clustering algorithm based on cosine distance is used, and the number of clusters is preset to be , for categories The average semantic features of Cluster centers. Each cluster center By cluster The average of all eigenvectors in is:

[0122] in, Indicates the total number of preset clusters, The index of the cluster ranges from 1 to , indicating the clusters, Indicates the The number of eigenvectors of clusters, Indicates the cluster center vectors, Representation category The average semantic features of .

[0123] Final Category The category prototype vector set is expressed as:

[0124] in, Representation category A collection of prototype vectors.

[0125] The set of prototype vectors for all categories is , used as input for subsequent models.

[0126] In one embodiment, after each round of training iteration, in order to improve the representation quality and adaptability of the category prototype vector, the momentum update mechanism is used to optimize the category prototype vector, such as Figure 5 As shown, Figure 5An example diagram of the update of the category prototype vector is shown in Figure 5 Different colors in correspond to the category prototype vectors of different defect categories. Specifically, for the query set sample image ,in, represents the visual feature vector extracted from the candidate region of the query set sample image in the same way during the training phase, Indicates the first The category labels of the query set sample images are first The complete prototype vector collection , calculate the sample The cosine similarity of the prototype vector with the highest similarity is selected. As the best matching prototype, the corresponding expression is:

[0127] in, is the index value obtained by the above expression, which represents the number of the prototype vector of the category that is most similar to the sample feature. After selecting the most matching prototype vector, update it according to the momentum update formula:

[0128] in, is the momentum coefficient, which controls the balance between historical prototype information and current sample characteristics. The value can maintain the stability of the prototype vector, while smaller The value helps the prototype vector quickly adapt to the new sample feature distribution.

[0129] Through the above momentum update method, the category prototype vector can continuously integrate new sample information during the iteration process, gradually optimize the representation effect, and thus improve the detection performance in small sample scenarios.

[0130] In one embodiment, the loss of the target classification network is used to measure the accuracy of candidate boxes in category prediction and background discrimination. This loss is calculated by fusing the focal loss with a classification loss function based on the collaborative optimization of positive and negative samples to address sample imbalance and insufficient learning of difficult samples in multi-category defect detection tasks.

[0131] In order to strengthen the training weight of difficult-to-distinguish samples, the positive sample loss term of Focal Loss is introduced to reduce the focus on easy-to-distinguish samples and enhance the training weight of difficult-to-distinguish samples. The query set sample images, whose positive sample loss term is defined as:

[0132] in, Indicates the The positive sample loss of query set sample images, For the The target category probability of the query set sample images, is an adjustment factor used to reduce the focus on easily separable samples.

[0133] In the negative sample part, in order to improve the model's ability to discriminate non-target categories, the negative sample information enhancement items are designed as follows:

[0134] in, Indicates the The negative sample loss of query set sample images, Indicates the The query set sample image pairs The predicted probability of the class, is the negative sample confidence enhancement factor, which is used to highlight the impact of high-confidence misjudgment categories on the total loss.

[0135] In order to avoid the background category from excessively dominating the loss calculation process in multi-category detection tasks, a background category weight factor is introduced , define the sample loss weight coefficient as:

[0136] in, Indicates the The weight coefficient of the query set sample images, is the background category weight factor, Indicates the The true category labels of the query set sample images are adjusted by the coefficients to adjust the weights of the positive and negative sample loss terms respectively:

[0137] Finally, the weighted positive and negative loss terms are fused to obtain the sample-level classification loss:

[0138] in, and is the weighted coefficient of the positive and negative sample loss terms. The classification losses of all samples are summed and averaged to obtain the overall loss value of the target classification network in this training batch, and the network parameters are optimized and updated accordingly.

[0139] Through the above design, the classification loss function can significantly improve the model's ability to identify minority defects and difficult-to-classify samples in small sample scenarios where the number of samples is highly unbalanced and the proportion of difficult samples is high, thereby improving the overall accuracy and robustness of substation defect detection.

[0140] In practical applications, the loss value of the regression network is used to measure the deviation between the target box prediction result and the true box in terms of position, size, etc. For the loss calculation of the regression network, a hierarchical loss supervision mechanism is adopted in the training phase. That is, two types of indicators, mask loss and coordinate regression loss, are calculated at each layer of the region propagation network. Mask loss includes binary cross entropy loss and Dice similarity loss. The binary cross entropy loss is used to measure the classification error between the predicted mask and the true mask at the pixel level. The Dice similarity loss is used to reflect the degree of regional overlap between the predicted mask and the true mask. It is calculated as follows:

[0141] in, Represents the regression network The binary cross entropy loss of the layer, Represents the regression network Dice similarity loss of the layer, For the layer prediction mask, is the corresponding output mask. Output mask The generation rule is composed of the coordinate sampling point set Each sampling point With real box (i.e., the marked defect area) is formulated to define whether the sampling point is located within the corresponding real box, specifically:

[0142] Coordinate regression loss includes L1 distance loss With generalized IoU loss The L1 distance loss is used to measure the numerical difference between the predicted coordinates and the true coordinates, while the generalized IoU loss is used to comprehensively consider the overlap and relative position relationship between the candidate box and the true box. It is calculated as follows:

[0143]

[0144] in, Represents the regression network L1 distance loss of the layer, Represents the regression network The generalized IoU loss of the layer, and Respectively represent The normalized coordinates of the layer prediction and the true normalized coordinates, Represents the inverse mapping function from normalized coordinates to actual candidate box coordinates.

[0145] The final total loss of the regression network is the weighted sum of the multi-layer losses:

[0146] in, represents the total number of layers in the regression network, is the loss weight coefficient.

[0147] In each round of training, the weight parameters of the regression network and the target classification network are independently optimized using gradient backpropagation and parameter update strategies, with the loss values ​​of the regression network and the target classification network respectively serving as optimization targets. After sufficient training, the regression network can accurately predict the spatial location and size of defect targets, while the target classification network can accurately distinguish different defect categories and effectively suppress background interference.

[0148] After training, the trained regression network and target classification network showed significant advantages in small sample detection scenarios. Even under the condition of extremely limited number of defect category samples, they can still maintain high detection accuracy and robustness. The optimized regression network and target classification network are then applied to the substation inspection images to be inspected to achieve the position positioning and category recognition of defect targets in the image, thereby completing the defect detection task.

[0149] To verify the effectiveness of the proposed substation defect detection method in small-sample scenarios, performance evaluation was conducted using a test set of substation defect images from the State Grid 2022 competition. This test set contains three typical small-sample defect types: damaged meter casings, damaged respirator silicone barrels, and abnormal oil seals. The evaluation metric used was mAP@50 (mean Average Precision at IoU=0.5), a widely used metric in the field of object detection. mAP@50 represents the detection performance metric calculated by taking the average precision of each category when the intersection-over-union ratio (IoU) threshold between the predicted and ground-truth boxes is 0.5.

[0150] Under two small sample settings, 5-shot and 10-shot, where 5-shot means only 5 labeled training images are provided for each defect category; 10-shot means only 10 labeled training images are provided for each defect category, the method of the present invention is compared with the existing mainstream small sample target detection models FSCE (Few-Shot Object Detection via Contrastive Proposal Encoding), VFA (Few-Shot Object Detection via Variational Feature Aggregation) and DEViT (Detect Everything with Few Examples).

[0151] Table 1 Performance comparison of different detection methods

[0152] Experimental results show that the proposed method improves the mAP@50 index by 16.18%, 8.33% and 4.92% respectively compared with FSCE, VFA and DEViT in the 10-shot task. It also shows obvious performance advantages in the 5-shot task, with the average detection accuracy steadily improving, fully demonstrating the high precision and good generalization ability of the proposed method in the small sample defect detection task of substations.

[0153] In addition, to further verify the effectiveness of the regression network (module 1) and cross-modal semantic calibration mechanism (module 2) in the present invention that include prototype momentum update and visual fusion, a systematic ablation experiment was conducted. The experimental results are shown in Table 2.

[0154] Table 2 Ablation experiment results

[0155] Experimental results show that Module 1 effectively enhances target perception during the region candidate generation phase by introducing similarity guidance between category prototypes and region features, making the generated candidate regions more semantically aligned with the actual target regions. This improves detection accuracy by approximately 2.2% and 3.47% in the 5-shot and 10-shot scenarios, respectively. Module 2 further enhances detection performance by approximately 1.16% and 1.1% in the 5-shot and 10-shot scenarios, respectively, by introducing a semantic alignment mechanism between text category priors and visual features. In terms of overall inference efficiency, the inference time for Module 1 is 0.132 seconds, while the addition of Module 2 reduces it to 0.167 seconds, both meeting the State Grid standard of a single-image inference time of no more than 0.3 seconds.

[0156] In summary, the above experimental results fully verify the superior performance and practical application feasibility of the method of the present invention in the field of small-sample defect detection in substations, which not only ensures a significant improvement in detection accuracy but also meets the efficiency requirements of real-time reasoning.

[0157] In the above-mentioned embodiment, a regression network integrating prototype attention and mask propagation is introduced into the substation defect detection process. This effectively combines the category prototype vector with the attention mechanism during the spatial expansion and grid sampling of the candidate prediction box. This allows the feature representation of the candidate region to more closely align with the discriminant features of the target category, thereby achieving high positioning accuracy even with small sample sizes. Furthermore, the region propagation mechanism with mask stacking enhances the feature correlation between the interior and boundary regions of the candidate box, further improving the robustness and accuracy of target regression.

[0158] In the classification stage, by constructing intra-class response embedding, inter-class ranking embedding and background semantic embedding based on the category prototype vector, and using multi-source embedding fusion to generate the confidence of candidate defect categories, it can not only enhance the separability between different categories under limited sample conditions, but also suppress the risk of misclassification caused by background noise.

[0159] In addition, when processing candidate prediction boxes in the low confidence interval, a picture-text alignment network is introduced, and the original confidence is calibrated using cross-modal semantic consistency scores. This can effectively make up for the semantic judgment ability when visual information is insufficient and reduce missed detections and false detections of small sample categories.

[0160] By dynamically optimizing the class prototype vectors using a momentum update mechanism during training, the prototype representation can continuously incorporate new sample feature information and maintain stability, thereby improving the prototype's generalization and adaptability in small sample scenarios. In summary, this invention can significantly improve the positioning accuracy and classification reliability of substation defect detection in situations where labeled samples are scarce.

[0161] In one embodiment, a substation defect detection device for a small sample scenario is provided, referring to Figure 6 As shown, the substation defect detection device 500 for the small sample scenario may include: a data extraction module 501, a regression network processing module 502, a target classification network processing module 503, a graph-text alignment network processing module 504 and a model training module 505. The data extraction module 501 is used to input the substation inspection image to be detected into the backbone network and the RPN network respectively, and extract the full image features and candidate prediction boxes of the substation inspection image; The regression network processing module 502 is used to input the full-image features and candidate prediction boxes of the substation inspection image into the trained regression network that integrates prototype attention and mask propagation for processing. The regression network is used to perform spatial expansion and grid sampling on the candidate prediction boxes, combine the category prototype vector and the attention mechanism to embed the prototype features, and construct a mask stacking regional propagation mechanism based on the binary mask to output the target box prediction result. The target classification network processing module 503 is used to input the full image features and candidate prediction boxes of the substation inspection image into the trained target classification network for processing; the target classification network is used to screen a set of candidate defect categories based on the similarity between the candidate prediction boxes and the category prototype vectors, and to generate the candidate defect category confidence of the candidate prediction boxes by constructing intra-class response embedding, inter-class ranking embedding, and background semantic embedding using a multi-source embedding fusion method; The image-text alignment network processing module 504 is used to introduce the image-text alignment network to assist in scoring the semantic consistency of the candidate prediction boxes whose candidate defect category confidences are within a preset interval, and obtain the calibrated candidate defect category confidences; Among them, the category prototype vector is constructed during the model training process and dynamically updated based on the momentum update mechanism.

[0162] In one embodiment, the regression network processing module 502 is specifically configured to extract candidate region features within each expanded candidate prediction box by performing a region alignment operation using the full image features and the expanded candidate prediction box. The query vector and key vector in the attention mechanism are used to linearly transform the candidate region features and category prototype vectors to obtain a low-dimensional embedding representation; Calculate the attention response relationship between each position of the expanded candidate prediction box and the low-dimensional embedding representation to obtain the foreground category response map and the background category response map; The foreground region embedding features generated based on the foreground category response map are concatenated with the background embedding features generated based on the background category response map to obtain the regional embedding features, which serve as the input of the regional propagation mechanism.

[0163] In one embodiment, the regression network processing module 502 is specifically configured to construct an input mask based on the positional relationship between the sampling points in the grid sampling and the candidate prediction box; the input mask is used to indicate whether the sampling points in the grid sampling fall within the candidate prediction box; The input mask is used as the initial input of the region propagation mechanism and fused with the region embedding features to generate the mask response map layer by layer through a multi-layer propagation network; The final mask response map output by the multi-layer propagation network is decoded into the normalized coordinates of the target box through coordinate integration operation.

[0164] In one embodiment, the image-text alignment network processing module 504 is specifically configured to extract regional visual feature vectors of candidate prediction boxes whose confidence levels for candidate defect categories are within a preset interval, and perform similarity calculations with the category text feature matrix generated based on the natural language description of each defect category to obtain a semantic similarity score. The semantic similarity score is weightedly fused with the candidate defect category confidence of the corresponding candidate prediction box to obtain the calibrated candidate defect category confidence.

[0165] In one embodiment, the substation defect detection apparatus 500 for a small sample scenario further includes a model training module 505 for determining a sample type of the sample set based on the number of sample images in the sample set; the sample type is a basic class or a small sample class; Extract the average semantic features of the defect area of ​​the support set sample image in the sample set of each defect category, and select the corresponding aggregation method to perform feature aggregation on the average semantic features according to the sample type to which the support set sample image belongs; Among them, if the sample type belongs to the small sample category, the average semantic features are aggregated using the mean aggregation method; If the sample type is the basic category, the K-Means clustering algorithm based on cosine distance is used to aggregate the average semantic features.

[0166] In one embodiment, the model training module 505 is specifically configured to update the category prototype vector that best matches the query set sample image in the defect category input to the model using momentum update after each round of training iteration.

[0167] In one embodiment, the model training module 505 is specifically used to calculate the loss value of the target classification network during the model training process by fusing Focal Loss with a classification loss function based on collaborative optimization of positive and negative samples.

[0168] The specific limitations of the substation defect detection device 500 for small sample scenarios can be found in the limitations of the substation defect detection method for small sample scenarios described above and will not be further elaborated here. Each module within the substation defect detection device 500 for small sample scenarios described above can be implemented in whole or in part via software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor within a computer device in hardware form, or stored in memory within the computer device in software form, allowing the processor to call and execute the corresponding operations of each module.

[0169] In one embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements a substation defect detection method for a small sample scenario when executing the computer program.

[0170] In one embodiment, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, a method for detecting substation defects in a small sample scenario is implemented.

[0171] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0172] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0173] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0174] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0175] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A substation defect detection method for a small sample scenario, characterized in that: include: Input the substation inspection image to be detected into the backbone network and RPN network respectively, and extract the full image features and candidate prediction boxes of the substation inspection image; The full-image features and candidate prediction boxes of the substation inspection image are input into a trained regression network that integrates prototype attention and mask propagation for processing. The regression network is used to perform spatial expansion and grid sampling on the candidate prediction boxes, embed prototype features by combining category prototype vectors with the attention mechanism, and construct a mask stacking regional propagation mechanism based on binary masks to output the target box prediction results. Inputting the full image features and candidate prediction boxes of the substation inspection image into the trained target classification network for processing; The target classification network is used to screen a set of candidate defect categories based on the similarity between the candidate prediction box and the category prototype vector, and generate the candidate defect category confidence of the candidate prediction box by constructing intra-class response embedding, inter-class ranking embedding and background semantic embedding in a multi-source embedding fusion manner; An image-text alignment network is introduced to assist in scoring the semantic consistency of candidate prediction boxes whose confidence levels of candidate defect categories are within a preset range, thereby obtaining the calibrated confidence levels of candidate defect categories. The category prototype vector is constructed during the model training process and is dynamically updated based on the momentum update mechanism.

2. The substation defect detection method for a small sample scenario according to claim 1 is characterized in that: Combining category prototype vectors with the attention mechanism for prototype feature embedding, including: Extracting candidate region features within each of the expanded candidate prediction boxes by using the full image features and the expanded candidate prediction boxes through a region alignment operation; Using the query vector and key vector in the attention mechanism, linearly transform the candidate region features and category prototype vectors to obtain a low-dimensional embedding representation; Calculating the attention response relationship between each position of the expanded candidate prediction box and the low-dimensional embedding representation to obtain a foreground category response map and a background category response map; The foreground region embedding features generated based on the foreground category response map and the background embedding features generated based on the background category response map are concatenated to obtain regional embedding features as input to the regional propagation mechanism.

3. The substation defect detection method for a small sample scenario according to claim 2 is characterized in that: A mask stacking regional propagation mechanism is constructed based on binary masks to output target box prediction results, including: Constructing an input mask based on the positional relationship between the sampling points in the grid sampling and the candidate prediction box; the input mask is used to indicate whether the sampling points in the grid sampling fall into the candidate prediction box; The input mask is used as the initial input of the region propagation mechanism and is fused with the region embedding feature to generate a mask response map layer by layer through a multi-layer propagation network; The final mask response map output by the multi-layer propagation network is decoded into the normalized coordinates of the target box through coordinate integration operation.

4. The substation defect detection method for a small sample scenario according to claim 1 is characterized in that: The image-text alignment network is introduced to assist in scoring the semantic consistency of candidate prediction boxes whose candidate defect category confidences are within a preset interval, and to obtain the calibrated candidate defect category confidences, including: Extract the regional visual feature vectors of the candidate prediction boxes whose confidence levels of the candidate defect categories are within a preset range, and perform similarity calculations with the category text feature matrix generated based on the natural language description of each defect category to obtain a semantic similarity score. The semantic similarity score is weightedly fused with the candidate defect category confidence of the corresponding candidate prediction box to obtain a calibrated candidate defect category confidence.

5. The substation defect detection method for a small sample scenario according to claim 1 is characterized in that: The process of constructing category prototype vectors during model training includes: Determining a sample type of the sample set based on the number of sample images in the sample set; the sample type is a basic category or a small sample category; Extracting the average semantic features of the defect area of ​​the support set sample image in the sample set of each defect category, and selecting a corresponding aggregation method to perform feature aggregation on the average semantic features according to the sample type to which the support set sample image belongs; If the sample type is a small sample category, the average semantic features are aggregated using a mean aggregation method; If the sample type is the basic category, the average semantic features are aggregated using a K-Means clustering algorithm based on cosine distance.

6. The substation defect detection method for a small sample scenario according to claim 5 is characterized in that: The method further comprises: After each round of training iteration is completed, the momentum update method is used to update the category prototype vector that best matches the query set sample image in the defect category of the query set sample image input to the model.

7. The substation defect detection method for a small sample scenario according to claim 1 is characterized in that: During the model training process, the loss value of the target classification network is calculated by fusing Focal Loss with the classification loss function based on the collaborative optimization of positive and negative samples.

8. A substation defect detection device for a small sample scenario, characterized in that: include: A data extraction module is used to input the substation inspection image to be detected into the backbone network and the RPN network respectively, and extract the full image features and candidate prediction boxes of the substation inspection image; A regression network processing module inputs the full-image features and candidate prediction boxes of the substation inspection image into a trained regression network that integrates prototype attention and mask propagation for processing. The regression network is used to perform spatial expansion and grid sampling on the candidate prediction boxes, embed prototype features using the category prototype vector and attention mechanism, and construct a mask stacking regional propagation mechanism based on binary masks to output target box prediction results. The target classification network processing module inputs the full image features and candidate prediction boxes of the substation inspection image into the trained target classification network for processing; The target classification network is used to screen a set of candidate defect categories based on the similarity between the candidate prediction box and the category prototype vector; construct an intra-class response embedding, an inter-class ranking embedding, and a background semantic embedding for each candidate defect category, and generate a candidate defect category confidence score for the candidate prediction box through a multi-source embedding fusion method; The image-text alignment network processing module introduces an image-text alignment network to assist in scoring the semantic consistency of candidate prediction boxes whose candidate defect category confidence lies within a preset interval, and obtains the calibrated candidate defect category confidence; The category prototype vector is constructed during the model training process and is dynamically updated based on the momentum update mechanism.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the substation defect detection method for a small sample scenario according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the substation defect detection method for a small sample scenario according to any one of claims 1 to 7 is implemented.

Images