Substation defect identification method and system based on size model cooperation

Abstract

The application discloses a substation defect identification method and system based on size model cooperation, which comprises the following steps: S1, collecting image data and preprocessing; S2, obtaining scene understanding score, if the score is higher than the preset scene threshold, then S3 and S4 are executed synchronously; otherwise, only S3 is executed; S3, inputting the image data into the small model to obtain the detection result, if the detection result is higher than the first preset threshold, then the detection result is directly taken as the current defect identification result; if the detection result is lower than the second preset threshold, then the detection result is discarded; if the detection result is between the first preset threshold and the second preset threshold, then the regional detection result is obtained through image cropping and local large model reasoning; S4, inputting the image data into the substation defect detection large model to obtain the global detection result; S5, obtaining the final defect identification result according to the small model and / or regional and / or global detection result. The application can improve the precision of substation defect identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power equipment detection, and in particular to a substation defect identification method and system based on size model cooperation. BACKGROUND

[0002] As the core hub of the power system, the safe and stable operation of the substation directly determines the reliability of power supply. With the advancement of the intelligentization process of substations, defect identification technology based on artificial intelligence has gradually replaced traditional manual inspection and become the mainstream way of substation equipment state monitoring.

[0003] Currently, two types of AI models are mainly used in the field of substation defect identification: one is the traditional small model of each professional (such as the target detection small model based on convolutional neural network), which has small computational load and fast inference speed, but poor adaptability to complex scenes (such as low-visibility environments such as night, rainy day, foggy day, or device component occlusion, multi-defect concurrent scene), and limited recognition accuracy; the other is a multi-modal visual language large model, which has strong scene understanding and complex feature extraction capabilities, and can achieve high-precision defect identification, but has the problems of large resource consumption and long inference time, making it difficult to meet the real-time requirements of substation inspection.

[0004] To balance the recognition accuracy and efficiency, existing technologies propose a cooperative scheme of serial or parallel connection of small and large models. The serial connection scheme needs to first infer through the small model and then secondarily check the inference results by the large model, which has a long overall processing time, and the information loss in the small model inference process leads to poor checking effect of the large model; the parallel connection scheme needs to input the data into the small and large models for inference respectively, and then fuse the results, which avoids information loss, but all samples need to be inferred by the large model, resulting in great resource consumption and being unable to adapt to the hardware resource limitations of substation edge inspection devices (such as inspection robots and edge computing gateways).

[0005] In addition, the existing cooperative scheme also has the following defects: first, it does not design a scene adaptation mechanism for the scene characteristics of substation defect identification (such as diverse device types, complex defect types, and variable environmental interference), making it difficult to distinguish between simple and difficult scenes, resulting in lack of flexibility in model calling; second, it lacks effective result screening and optimization mechanism, and low-confidence results output by the small model are directly discarded or all processed by the large model, causing resource waste or accuracy loss; third, the fusion of small and large model results is only a simple superposition, without considering the confidence weight difference of the two in different defect types and different scenes, resulting in low fusion accuracy; fourth, it lacks a closed-loop data feedback mechanism, which cannot continuously optimize the performance of the small model based on actual identification results, making it difficult to improve the model generalization ability.

[0006] Therefore, there is an urgent need for a defect identification scheme that can flexibly cooperate with models of different sizes, balance identification accuracy and efficiency, and adapt to complex scenarios in substations, to solve the problems of the prior art. SUMMARY

[0007] The technical problem solved by the present application: In view of the above problems of the prior art, a substation defect identification method and system based on size model cooperation are provided to solve the problems of low efficiency, high resource consumption, poor fusion accuracy, lack of scene adaptation and data feedback mechanism in the existing size model cooperation scheme in substation defect identification.

[0008] To solve the above technical problems, the technical scheme adopted by the present application is:

[0009] A substation defect identification method based on size model cooperation, comprising the steps of:

[0010] S1, collecting image data and corresponding environment data of power equipment, and preprocessing the image data;

[0011] S2, obtaining a scene understanding score according to the preprocessed image data and corresponding environment data, if the scene understanding score is higher than a preset scene threshold, determining that the current scene is a difficult scene, and simultaneously executing steps S3 and S4; otherwise, only executing step S3 and not executing step S4;

[0012] S3, inputting the preprocessed image data into a substation defect detection small model for detection to obtain a small model detection result, if the confidence in the detection result is higher than a first preset threshold, directly taking the detection result as the current defect identification result; if the confidence is lower than a second preset threshold, discarding the detection result and ending the process; if the confidence is between the first preset threshold and the second preset threshold, obtaining a regional detection result through image cropping and local large model reasoning;

[0013] S4, inputting the preprocessed image data into a substation defect detection large model to obtain a global detection result;

[0014] S5, obtaining a final defect identification result of the substation according to the small model detection result and / or the regional detection result and / or the global detection result.

[0015] Further, in step S1, the image data is preprocessed, specifically including:

[0016] A Gaussian filter algorithm is used to calculate the Gaussian kernel and the image data through convolution operation to obtain a denoised image;

[0017] The size of the denoised image is unified to a preset size, and the pixel values of each pixel point of the normalized image are calculated by a bilinear interpolation method;

[0018] The normalized image color space is converted from an RGB space to an HSV space.

[0019] Further, in step S2, a scene understanding score is obtained according to the preprocessed image data and corresponding environment data, and specifically includes:

[0020] Texture, brightness and contrast features of the image are extracted through a feature extraction network, a last full connection layer of the network is removed, and an output of a penultimate layer of the network is reserved as an image feature vector;

[0021] Weather information in the environment data is converted into a one-hot vector, collection time is converted into a time period code, and device area information is converted into a device type code, and the environment feature vector is obtained by merging;

[0022] The image feature vector and the environment feature vector are spliced to form a fusion feature vector, and the fusion feature vector is input into a scene classification model constructed based on a multilayer perceptron, and is output through a Sigmoid function of an output layer;

[0023] The expression of the scene understanding score is as follows:

[0024] ;

[0025] In the above formula, S is a scene understanding score, are weight matrices of each layer of the multilayer perceptron, are bias terms of each layer of the multilayer perceptron, X is a fusion feature vector, are a first layer hidden layer output and a second layer hidden layer output of the multilayer perceptron.

[0026] Further, in step S3, a region detection result is obtained through image cropping and local large model inference, and specifically includes:

[0027] According to the defect candidate box coordinate information in the small model detection result, a defect candidate region image is cropped in the preprocessed image, and edge expansion is performed on the cropped region image to obtain a region-level image;

[0028] The obtained region-level image is input into a substation defect detection large model for region-level target detection to obtain a region detection result.

[0029] Further, step S5 includes:

[0030] If the detection result only includes one of the small model detection result, the region detection result or the global detection result, the result is directly taken as the final defect recognition result of the substation;

[0031] If the detection result includes two or all of the small model detection result, the region detection result and the global detection result, the intersection-over-union algorithm is used to match the defect frame positions in the region detection result and the global detection result first, if the matching is successful, it is determined that they are the same defect, the final confidence is obtained through fusion calculation, and the final defect recognition result is output; if the matching fails, the region detection result and the global detection result are output as different defect recognition results respectively.

[0032] Further, the final confidence is obtained through fusion calculation, and the final defect recognition result is output, specifically including:

[0033] The weight values of the small model detection result, the region detection result and the global detection result are determined based on the scene understanding score S and the region detection weight coefficient a;

[0034] The final confidence is calculated based on the obtained weight values;

[0035] If the final confidence is greater than a preset value, the final defect recognition result is output; if the final confidence is less than the preset value, the fusion defect recognition result is output, marked as a suspected defect, and manual review is prompted.

[0036] Further, the calculation expression of the final confidence is as follows:

[0037] ;

[0038] In the above formula, C_final is the final confidence, W1, W2 and W3 are the weight values of the small model detection result, the region detection result and the global detection result respectively, C1, C2 and C3 are the confidence scores of the small model detection result, the region detection result and the global detection result respectively, S is the scene understanding score, and a is the region detection weight coefficient.

[0039] Further, if the final confidence is less than a preset value, the fusion defect recognition result is output, marked as a suspected defect, and manual review is prompted, specifically including:

[0040] The output fusion defect recognition result is aligned with the small model detection result, the region detection result and the global detection result in defect frame position and defect type;

[0041] The absolute value difference value of the confidence scores in the small model detection result and the large model detection result is calculated, if the absolute value difference value is greater than a preset difference value, it is determined that the image data is uncertain data to be reviewed, submitted to the manual review link, and the professional operation and maintenance personnel verify the defect authenticity, defect type / level label accuracy according to the power equipment defect determination standard, and correct the label deviation;

[0042] The uncertain data confirmed by manual review is stored in a difficult sample library to provide data for subsequent small model optimization and improve the recognition ability of the small model for complex scene defects.

[0043] Further, the small model detection result, the regional detection result and the global detection result each include one or more of a defect frame position, a defect type, a defect level, a defect part and a confidence score.

[0044] A substation defect identification system based on size model cooperation includes a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to perform the steps of the substation defect identification method based on size model cooperation.

[0045] Compared with the prior art, the advantages of the present application are that:

[0046] The present application realizes effective balance between recognition accuracy and computing efficiency by introducing scene understanding scoring for adaptive scheduling of the detection process: in a conventional scene, the small model is preferred for fast reasoning, which can significantly reduce the power consumption and response time delay, meeting the needs of substation online inspection and real-time alarm; in a difficult scene, the global reasoning of the large model and the fine analysis of the large model in the local area are introduced simultaneously, which can fully utilize the strong representation ability of the large model in complex background, weak features and occlusion conditions, and improve the accuracy and robustness of defect identification. At the same time, through the multi-threshold confidence determination mechanism, the high confidence result is directly output, the low confidence result is automatically removed, and the intermediate confidence result is subjected to secondary fine reasoning, effectively reducing false positives and false negatives. The present application can reduce the overall computing resource occupation while ensuring high recognition reliability, and is suitable for complex and variable substation application environments. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The flowchart of the substation defect identification method based on size model cooperation of the embodiments of the present application. DETAILED DESCRIPTION

[0048] In order to better understand the above technical solutions, the above technical solutions will be described in detail in conjunction with the drawings of the specification and specific embodiments.

[0049] As shown in Figure 1 The substation defect identification method based on size model cooperation of the embodiments of the present application includes the following steps:

[0050] S1, collecting image data and corresponding environment data of power equipment, and pre-processing the image data;

[0051] S2, obtaining a scene understanding score according to the preprocessed image data and corresponding environment data, if the scene understanding score is higher than a preset scene threshold, determining that the current scene is a difficult scene, and synchronously executing steps S3 and S4; otherwise, only executing step S3 and not executing step S4;

[0052] S3, inputting the preprocessed image data into a substation defect detection small model for detection to obtain a small model detection result, if a confidence in the detection result is higher than a first preset threshold, directly taking the detection result as a current defect recognition result, if the confidence is lower than a second preset threshold, discarding the detection result and ending the process, and if the confidence is between the first preset threshold and the second preset threshold, obtaining a regional detection result through image cropping and local large model reasoning;

[0053] S4, inputting the preprocessed image data into a substation defect detection large model to obtain a global detection result;

[0054] S5, obtaining a final defect recognition result of the substation according to the small model detection result and / or the regional detection result and / or the global detection result.

[0055] In the embodiment, in step S1, the image data is preprocessed, specifically including:

[0056] A Gaussian kernel is calculated with the image data through a convolution operation by using a Gaussian filtering algorithm to obtain a denoised image;

[0057] The size of the denoised image is unified to a preset size, and the pixel values of each pixel point of the normalized image are calculated by using a bilinear interpolation method;

[0058] The color space of the normalized image is converted from an RGB space to an HSV space.

[0059] In a specific application embodiment, the specific implementation mode of step S1 (data acquisition and preprocessing) is as follows:

[0060] Visual image data (hereinafter referred to as "input image") of power equipment is collected by a substation deployed inspection device (such as a drone, a fixed camera, a robot), and at the same time, multi-source auxiliary data corresponding to the image (including weather information, time information, device type information, device location information at the time of collection, obtained through a GPS module, a temperature and humidity sensor and the like of the inspection device) is collected; the input image is preprocessed, including:

[0061] Image denoising: a Gaussian filtering algorithm is used; let the image input be I(x, y), and the Gaussian filtering formula is:

[0062] G(x, y) = (1 / (2πσ²))e^(-(x²+y²) / (2σ²))

[0063] wherein, sigma is the standard deviation of the Gaussian function, which determines the smoothing degree of the filter, and the Gaussian kernel is calculated by convolution operation with the input image I(x, y) to obtain the denoised image.

[0064] Image normalization: unify the image size to a preset size (such as 640*640 pixels); the formula is as follows:

[0065] Suppose the pixel coordinates of the original image are (i, j), and the corresponding normalized image coordinates are (i', j'), then i'=i* (640 / height of the original image), j'=j* (640 / width of the original image), and the pixel value of each pixel point of the normalized image is calculated by the bilinear interpolation method.

[0066] Color space conversion: convert from RGB space to HSV space to enhance the environmental light adaptability.

[0067] In this embodiment, in step S2, the scene understanding score is obtained according to the preprocessed image data and the corresponding environmental data, specifically including:

[0068] The texture, brightness and contrast features of the image are extracted by the feature extraction network, the last fully connected layer of the network is removed, and the output of the second last layer of the network is reserved as the image feature vector;

[0069] The weather information in the environmental data is converted into a one-hot vector, the collection time is converted into a time period code, and the device area information is converted into a device type code, and the environmental feature vector is obtained by merging;

[0070] The image feature vector and the environmental feature vector are spliced to form a fusion feature vector, and the fusion feature vector is input into the scene classification model constructed based on the multilayer perceptron, and the output of the Sigmoid function of the output layer is output;

[0071] The expression of the scene understanding score is as follows:

[0072] ;

[0073] In the above formula, S is the scene understanding score, are the weight matrices of each layer of the multilayer perceptron, are the bias terms of each layer of the multilayer perceptron, X is the fusion feature vector, are the first layer hidden layer output and the second layer hidden layer output of the multilayer perceptron, respectively.

[0074] In a specific application embodiment, the specific implementation mode of step S2 (scene understanding) is as follows:

[0075] The pre-processed image and corresponding environmental data (such as collection time, weather information, device location information, obtained through the GPS module of the inspection device and the temperature and humidity sensor) are input into a scene understanding module to generate a scene understanding score;

[0076] The scene understanding module is a scene classification model constructed based on a multi-layer perceptron (MLP), and the process of generating the scene understanding score includes:

[0077] S21. Extract image features: use ResNet18 network or other feature extraction network to extract texture, brightness and contrast features of the image, remove the last fully connected layer of the network, and keep the output of the second last layer as the image feature vector with a dimension of N (such as 512 dimensions);

[0078] S22. Environmental data encoding: convert weather information (such as sunny, rainy, foggy, and night) into a one-hot vector (such as sunny encoded as [1, 0, 0, 0], rainy encoded as [0, 1, 0, 0]), convert the collection time into a time period encoding (such as daytime 8:00-18:00 encoded as 1, night 18:00-8:00 encoded as 0), and convert the device area information into a device type encoding (such as outdoor area encoded as 1, indoor area encoded as 2), and combine the above encodings to obtain an environmental feature vector with a dimension of M (such as 6 dimensions);

[0079] S23. Feature fusion and score output: concatenate the image feature vector and the environmental feature vector to form a fusion feature vector with a dimension of N+M, input into a multi-layer perceptron MLP model, and output a scene understanding score (value range 0-1) through the Sigmoid function of the output layer; let the fusion feature vector be X=[x1, x2,...,xN+M], the first hidden layer output of the multi-layer perceptron be: n+m

[0080] H1=ReLU(W1X+b1);

[0081] The output of the second hidden layer is:

[0082] H2=ReLU(W2H1+b2);

[0083] The scene understanding score is:

[0084] S=Sigmoid(W3H2+b3);

[0085] Where W1, W2, W3 are weight matrices of each layer, b1, b2, b3 are bias terms of each layer, and the Sigmoid function is defined as:

[0086] Sigmoid(z)=1 / (1+e -z ); ​

[0087] Finally, the scene environment score S is obtained. The higher the scene understanding score is, the more complex the current scene is, for example, the night scene score is higher, the rainy day scene score is second, and the sunny day scene score is the lowest.

[0088] If the scene understanding score output by the scene understanding module is greater than or equal to a preset scene threshold T_scene (T_scene=0.6), it is determined that the current scene is a difficult scene, and the preprocessed complete image is input into the large model simultaneously, and the large model global inference (S4) and subsequent steps are executed simultaneously.

[0089] In this embodiment, in step S3, the region detection result is obtained by image cropping and local large model inference, specifically including:

[0090] According to the defect candidate box coordinate information in the small model detection result, a defect candidate region image is cropped in the preprocessed image, and edge expansion is performed on the cropped region image to obtain a region-level image;

[0091] The obtained region-level image is input into a substation defect detection large model for region-level target detection to obtain a region detection result.

[0092] In a specific application embodiment, the specific implementation mode of step S3 (small model inference, early return mechanism screening, and local large model inference) is as follows:

[0093] S31. Small model inference

[0094] The preprocessed image is input into a preset small model for target detection to obtain a small model detection result;

[0095] The small model is a substation defect detection model based on YOLOv8 lightweight improvement, which can be replaced by other same level lightweight models, and has target detection capability for typical defects of a substation; the model is optimized by pruning and quantization operation to adapt to edge device operation resources.

[0096] The small model detection result includes defect candidate boxes (coordinate information), defect candidate boxes (coordinate information), which are in the form of (x1, y1, x2, y2), where x1, y1, x2, and y2 represent the x and y values of the upper left corner of the candidate box and the x and y coordinate values of the lower right corner of the candidate box, respectively. The small model detection result also includes defect types and small model confidence scores (value range 0-1).

[0097] S32. Early return mechanism screening

[0098] Based on the small model confidence score and the preset threshold, including high threshold T1 and low threshold T2, wherein T1=0.8, T2=0.4, which can be verified and optimized by the historical defect sample set of the substation to determine the screening of the small model detection result:

[0099] S321. If the small model confidence score is greater than or equal to T1: it is determined that the small model detection result is reliable, and the result is directly output as the current defect identification result, the local reasoning process of the large model is skipped, and fast response is realized;

[0100] S322. If the small model confidence score is less than or equal to T2: it is determined that the possibility of detecting defects is extremely low, and the result is directly discarded to avoid subsequent invalid calculation;

[0101] S323. If T2 < small model confidence score < T1: it is determined that the small model detection result has uncertainty, the corresponding defect candidate box information is recorded, and step S33 is entered;

[0102] S33. Image cropping

[0103] Based on the defect candidate box information recorded in step S323, an image cropping module is started. The image cropping module uses a coordinate mapping algorithm to crop a defect candidate region image from the preprocessed image according to the coordinate information (x1, y1, x2, y2) of the defect candidate box, and performs edge expansion (the number of expanded pixels is 20 to avoid losing defect edge information) on the cropped region image to obtain a region-level image. The coordinates of the expanded region are as follows:

[0104] x1'=max(0,x1-20)

[0105] y1'=max(0,y1-20)

[0106] x2'=min(width,x2+20)

[0107] y2'=min(height,y2+20)

[0108] Wherein width and height represent the width and height of the image respectively. At the same time, the size is adjusted to a fixed size, such as 320x320 pixels.

[0109] S34. Local large model reasoning

[0110] In order to enable the large model to get rid of the interference of redundant background information, focus on the core area of the defect for fine analysis, so as to improve the recognition accuracy and judgment depth of the defect characteristics, the region-level image obtained in step S33 is input into the large model for region-level target detection, and the large model region detection result is obtained, including region defect frame position, region defect type, region defect level, region defect part, and large model region confidence score (value range 0-1). Through the focused analysis method, the large model can guide the calculation resources and recognition attention to the defect parts that really need to be judged, effectively reducing the interference of irrelevant information on the detection result, and further improving the analysis ability of defect details and the reliability of the detection result.

[0111] In a specific application embodiment, step S4 (global-level large model reasoning) includes:

[0112] If the scene understanding score in step S2 is greater than or equal to T_scene, the preprocessed complete image is input into the large model for global-level target detection, and the large model global detection result is obtained; the large model global detection result includes global defect frame, global defect type, global defect level, global defect part, and large model global confidence score (value range 0-1).

[0113] When the scene complexity is high (such as severe weather and dramatic light changes), relying only on local analysis may miss cross-regional associated defects or hidden problems covered by complex backgrounds. By introducing global-level reasoning, the large model can utilize its global perception ability for the complete scene to capture spatial correlation features of defects and environment (such as multi-device cooperative failure and chain defects caused by environmental factors), making up for the limited field of view of local analysis, so as to ensure the integrity and correlation of detection in complex scenes and improve the comprehensiveness of defect detection under extreme conditions.

[0114] In this embodiment, step S5 includes:

[0115] If the detection result includes only one of the small model detection result, the region detection result, or the global detection result, the result is directly taken as the final defect recognition result of the transformer substation;

[0116] If the detection result includes two or all of the small model detection result, the region detection result, and the global detection result, the intersection over union algorithm is first used to match the defect frame positions in the region detection result and the global detection result. If the matching is successful, it is determined as the same defect, the final confidence is obtained through fusion calculation, and the final defect recognition result is output. If the matching fails, the region detection result and the global detection result are output as different defect recognition results.

[0117] In this embodiment, the final confidence is obtained through fusion calculation, and the final defect recognition result is output, specifically including:

[0118] determining weight values of the small model detection result, the region detection result and the global detection result based on the scene understanding score S and the region detection weight coefficient a;

[0119] calculating a final confidence based on the obtained weight values;

[0120] If the final confidence is greater than a preset value, output the final defect recognition result; if the final confidence is less than the preset value, output the fusion defect recognition result, mark as suspected defect, and prompt manual review.

[0121] In the embodiment, the calculation expression of the final confidence is as follows:

[0122]

[0123] In the above formula, C_final is the final confidence, W1, W2 and W3 are weight values of the small model detection result, the region detection result and the global detection result respectively, C1, C2 and C3 are confidence scores of the small model detection result, the region detection result and the global detection result respectively, S is the scene understanding score, and a is the region detection weight coefficient.

[0124] In a specific application embodiment, step S5 (multi-result confidence fusion) includes:

[0125] A weighted fusion model is constructed to fuse the following three types of results to obtain the final defect recognition result:

[0126] Category one: the small model detection result output by step S321 (denoted as R1, with a confidence C1);

[0127] Category two: the large model region detection result output by step S34 (denoted as R2, with a confidence C2);

[0128] Category three: the large model global detection result output by step S4 (denoted as R3, with a confidence C3);

[0129] The fusion process specifically includes:

[0130] S51. Result matching: an intersection over union (IoU) algorithm is used to match the defect frames in R2 and R3. If IoU≥0.5, it is determined as the same defect and enters the fusion calculation; if IoU<0.5, it is determined as different defects and is kept separately; the formula is as follows:

[0131]

[0132] Wherein R2∩R3 represents the intersection part of the defect frame of R2 and the defect frame of R3, and R2∪R3 represents the union part of the defect frame of R2 and the defect frame of R3.​

[0133] S52. Weight determination: determine the weight value of each result based on the defect type and scene understanding score, the formula is as follows:

[0134] For category one result: weight W1=C1x(1-S), wherein S is the scene understanding score (S≤T_scene);

[0135] For category two result: weight W2=C2xSxα, wherein α is the area detection weight coefficient (α=0.7, determined by historical sample verification);

[0136] For category three result: weight W3=C3xSx(1-α);

[0137] S53. Fusion calculation: determine the matched defect type, defect level, and defect position by weighted voting, and calculate the confidence by weighted summation, the formula is as follows:

[0138] C_final=W1+W2+W3;

[0139] C_final is the final confidence, if C_final≥0.6, output the final defect recognition result (including the final defect frame, the final defect type, the final defect level, the final defect position, and C_final); if C_final<0.6, mark as suspected defect and prompt manual review.

[0140] In this embodiment, if the final confidence is less than the preset value, output the fusion defect recognition result, mark as suspected defect and prompt manual review, which specifically includes:

[0141] Align the output fusion defect recognition result with the small model detection result, the area detection result, and the global detection result in terms of defect frame position and defect type;

[0142] Calculate the absolute value difference of the confidence scores in the small model detection result and the large model detection result, if the absolute value difference is greater than the preset difference value, determine that the image data is uncertain data to be reviewed, submit to the manual review link, and verify the defect authenticity and defect type / level annotation accuracy by professional operation and maintenance personnel combined with the power equipment defect determination standard, and correct the annotation deviation;

[0143] Store the uncertain data confirmed by manual review in the difficult sample library to provide data for subsequent small model optimization and improve the recognition ability of the small model for complex scene defects.

[0144] In a specific application embodiment, to realize model performance iteration optimization and continuous improvement of detection accuracy, the uncertain data in the detection process is directionally selected and manually reviewed, and a difficult sample library is constructed, and the specific process is as follows:

[0145] S61. Data screening and difference analysis: Align the final defect identification results output by step S5 with the small model detection results (R1), the large model detection results (R2, R3) in terms of coordinates and categories, and focus on the confidence difference under the same defect frame - calculate the absolute value difference value AC = |Cbig - Csmall| of the small model confidence (Csmall) and the large model confidence (Cbig); if AC≥0.3, it is determined that the sample is "uncertain data" to be reviewed (such samples usually reflect the recognition deviation of the small model in complex scenes, and have high optimization value). Among them, Csmall = W1, Cbig = W2 + W3 (if there is only one large model detection result, it is directly used as a reference; if there are two, they need to be weighted according to the regional detection coefficient).

[0146] S62. Artificial review verification: The uncertain data screened out (including preprocessed images, multi-model detection results, and AC calculation values) are submitted to the artificial review link, and professional operation and maintenance personnel verify the defect authenticity, defect type / grade label accuracy according to the power equipment defect judgment standard, and correct the label deviation.

[0147] S63. Construction of difficult sample library: The effective samples confirmed by artificial review (including preprocessed images, accurate defect label information, and multi-model confidence difference data) are stored in the difficult sample library to provide high-quality data support for subsequent small model optimization (such as fine-tuning and incremental training), and help to improve the recognition ability of the model to complex scene defects.

[0148] In the embodiment, the small model detection result, the regional detection result and the global detection result each include one or more of a defect frame position, a defect type, a defect level, a defect part, and a confidence score.

[0149] The application further provides a transformer substation defect identification system based on the cooperation of large and small models, which comprises a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to perform the steps of the transformer substation defect identification method based on the cooperation of large and small models.

[0150] The system of the application corresponds to the above method and also has the advantages of the above method.

[0151] The present application realizes all or part of the processes in the above-mentioned embodiment methods, and can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of the above-mentioned method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium includes any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. The memory is used to store the computer program and / or modules, and the processor realizes various functions by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device, etc.

[0152] The above-mentioned is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiment. Any technical solution falling within the idea of the present application belongs to the protection scope of the present application. It should be noted that some improvements and refinements made by ordinary skilled in the art without departing from the principle of the present application should also be considered as the protection scope of the present application.

Claims

1. A substation defect identification method based on size model cooperation, characterized in that, The method comprises the steps of: S1, collecting image data and corresponding environment data of the power equipment, and preprocessing the image data; S2, obtaining a scene understanding score according to the preprocessed image data and the corresponding environment data, and if the scene understanding score is higher than a preset scene threshold, determining that the current scene is a difficult scene, and simultaneously executing steps S3 and S4; Otherwise, only step S3 is executed, and step S4 is not executed; S3, inputting the preprocessed image data into a substation defect detection small model for detection to obtain a small model detection result, if a confidence in the detection result is higher than a first preset threshold, directly taking the detection result as a current defect recognition result; If the confidence is lower than a second preset threshold, discarding the detection result and ending the process; If the confidence is between the first preset threshold and the second preset threshold, obtaining a regional detection result through image cropping and local large model reasoning; S4, inputting the preprocessed image data into a substation defect detection large model to obtain a global detection result; S5, obtaining a final defect recognition result of the substation according to the small model detection result and / or the regional detection result and / or the global detection result; In step S2, the scene understanding score is obtained according to the preprocessed image data and the corresponding environment data, and specifically comprises: extracting texture, brightness and contrast features of the image through a feature extraction network, removing a last full connection layer of the network, and retaining an output of a penultimate layer of the network as an image feature vector; converting weather information in the environment data into a one-hot vector, converting a collection time into a time period encoding, and converting device region information into a device type encoding, and merging to obtain an environment feature vector; splicing the image feature vector and the environment feature vector to form a fusion feature vector, and inputting the fusion feature vector into a scene classification model constructed based on a multilayer perceptron, and outputting through a Sigmoid function of an output layer; The expression of the scene understanding score is as follows: ； In the above formula, S is a scene understanding score, respectively are weight matrices of each layer of the multilayer perceptron, respectively are bias terms of each layer of the multilayer perceptron, X is a fusion feature vector, respectively are first layer hidden layer output and second layer hidden layer output of the multilayer perceptron; Step S5 comprises: If the detection result only includes one of the small model detection result, the regional detection result or the global detection result, directly taking the result as the final defect recognition result of the substation; If the detection result includes two or all of the small model detection result, the regional detection result and the global detection result, first matching the defect frame positions in the regional detection result and the global detection result by using an intersection over union algorithm, if the matching is successful, determining that they are the same defect, obtaining a final confidence through fusion calculation and outputting a final defect recognition result, if the matching fails, taking the regional detection result and the global detection result as different defect recognition results and outputting them respectively.

2. The size model based collaboration based substation defect identification method of claim 1, wherein, In step S1, the image data is preprocessed, specifically comprising: using a Gaussian filtering algorithm to calculate a Gaussian kernel and the image data through convolution operation to obtain a denoised image; uniformly setting the size of the denoised image to a preset size, and calculating pixel values of each pixel point of the normalized image through a bilinear interpolation method; converting a color space of the normalized image from an RGB space to an HSV space.

3. The size model based collaboration based substation defect identification method of claim 1, wherein, In step S3, the regional detection result is obtained through image cropping and local large model reasoning, specifically comprising: According to the defect candidate box coordinate information in the small model detection result, a defect candidate region image is cropped from the preprocessed image, and edge expansion is performed on the cropped region image to obtain a region-level image; The obtained region-level image is input into the substation defect detection large model for region-level target detection to obtain a region detection result.

4. The size model based collaboration based substation defect identification method of claim 1, wherein, The final confidence is obtained by fusion calculation and the final defect recognition result is output, specifically including: The weight values of the small model detection result, the region detection result and the global detection result are determined based on the scene understanding score S and the region detection weight coefficient a; The final confidence is calculated based on the obtained weight values; If the final confidence is greater than a preset value, the final defect recognition result is output; if the final confidence is less than the preset value, the fusion defect recognition result is output, marked as a suspected defect, and a prompt for manual review is given.

5. The size model based collaboration based substation defect identification method of claim 4, wherein, The calculation expression of the final confidence is as follows: ； In the above formula, C_final is the final confidence, W1, W2 and W3 are the weight values of the small model detection result, the region detection result and the global detection result respectively, C1, C2 and C3 are the confidence scores of the small model detection result, the region detection result and the global detection result respectively, S is the scene understanding score, and a is the region detection weight coefficient.

6. The size model based collaboration based substation defect identification method of claim 4, wherein, If the final confidence is less than the preset value, the fusion defect recognition result is output, marked as a suspected defect, and a prompt for manual review is given, specifically including: The output fusion defect recognition result is aligned with the small model detection result, the region detection result and the global detection result in terms of defect box position and defect type; The absolute value difference value of the confidence scores in the small model detection result and the large model detection result is calculated, if the absolute value difference value is greater than a preset difference value, it is determined that the image data is uncertain data to be reviewed, which is submitted to the manual review link, and the professional operation and maintenance personnel verify the defect authenticity, defect type / grade annotation accuracy according to the power equipment defect judgment standard, and correct the annotation deviation; The uncertain data confirmed by manual review is stored in the difficult sample library to provide data for subsequent small model optimization and improve the recognition ability of the small model for complex scene defects.

7. The size model based collaborative substation defect identification method according to any one of claims 1-6, characterized in that, The small model detection result, the region detection result and the global detection result each include one or more of defect box position, defect type, defect level, defect part and confidence score.

8. A substation defect identification system based on size model collaboration, comprising a microprocessor and a memory connected to each other, characterized by, The microprocessor is programmed or configured to perform the steps of the substation defect recognition method based on the size model cooperation according to any one of claims 1-7.

Images