Power equipment operation defect determination method, device, equipment, medium and product

Abstract

The invention relates to a power equipment operation defect determination method and device, equipment, a medium and a product. The method comprises the following steps: acquiring a first image corresponding to power equipment in an operation process; performing multi-scale feature extraction on the first image for different times to obtain different first feature data; performing feature processing on the first feature data to obtain corresponding second feature data; and determining the operation defect condition of the power equipment according to the second feature data. The detection effect can be improved.

Description

Technical Field

[0001] This application relates to the field of power technology, and in particular to a method, apparatus, equipment, medium, and product for determining operational defects in power equipment. Background Technology

[0002] Before the application of deep learning technology to defect detection in power equipment operation, defect detection mainly relied on the following existing technologies:

[0003] (1) Manual inspection and experience judgment: This method relies heavily on visual inspection by inspectors, which is subjective, inefficient, and has a high rate of missed inspections. The quality of the inspection is greatly affected by personnel experience, fatigue, weather, lighting, and equipment accessibility (such as high-altitude and high-voltage environments), making it difficult to achieve high-frequency and full-coverage inspections, especially for large power grid systems. Moreover, the recording and analysis of inspection results depend on manual labor, resulting in poor standardization and traceability.

[0004] (2) Online monitoring based on fixed sensors: mainly used for monitoring the status of key equipment (e.g., transformers) (e.g., oil chromatography, partial discharge, temperature or vibration, etc.), which is costly, limited in deployment and has a narrow coverage.

[0005] (3) Traditional image processing and computer vision techniques: using visible light, infrared or ultraviolet images / videos to attempt automated detection. For example, separating the target from the background based on pixel gray values ​​is sensitive to complex backgrounds such as changes in lighting and has poor robustness; extracting manually designed features (e.g., corners, edges, textures) and comparing standard templates with the images to be inspected, the performance is highly dependent on the features designed by experts. These features are difficult to comprehensively and robustly represent the complex and ever-changing defect morphology of power equipment, and have weak generalization ability. The performance of the model drops sharply in new scenes outside the training set, and it is difficult to process complex backgrounds.

[0006] With the development of deep learning technology, it has been increasingly applied to the detection of operational defects in power equipment, significantly improving the performance and efficiency of defect detection and overcoming the shortcomings of existing technologies. However, the detection effectiveness of deep learning-based power equipment defect identification still needs improvement. Summary of the Invention

[0007] Therefore, it is necessary to provide a method, apparatus, equipment, medium, and product for determining operational defects in power equipment that can improve the detection effect, in response to the above-mentioned technical problems.

[0008] In a first aspect, this application provides a method for determining operational defects in power equipment, including:

[0009] Acquire the first image of the power equipment during the operation;

[0010] Perform multi-scale feature extraction on the first image at different times to obtain different first feature data;

[0011] Perform feature processing on each of the first feature data to obtain the corresponding second feature data;

[0012] Based on the data of each secondary feature, the operational defects of the power equipment are determined.

[0013] In one embodiment, performing multi-scale feature extraction on the first image at different times to obtain different first feature data includes: for each multi-scale feature extraction, extracting features from the input data at multiple scales to obtain third feature data corresponding to the corresponding scale; performing edge feature enhancement processing on the third feature data corresponding to each scale to obtain fourth feature data corresponding to the scale; and determining the first feature data based on the fourth feature data corresponding to each scale. Wherein, if this multi-scale feature extraction is the first multi-scale feature extraction, the input data is the first image; if this multi-scale feature extraction is not the first multi-scale feature extraction, the input data is the first feature data obtained after the previous multi-scale feature extraction.

[0014] In one embodiment, edge feature enhancement processing is performed on the third feature data corresponding to each scale to obtain the fourth feature data corresponding to the scale, including: extracting edge feature data with a frequency higher than a preset frequency from the third feature data corresponding to each scale; performing feature enhancement processing on the edge feature data to obtain edge feature enhanced data; and performing feature fusion between the third feature data and the edge feature enhanced data to obtain the fourth feature data corresponding to the scale.

[0015] In one embodiment, determining the first feature data based on the fourth feature data corresponding to each scale includes: extracting features from the input data to obtain initial feature data; and concatenating and fusing the fourth feature data corresponding to each scale and the initial feature data to obtain the first feature data.

[0016] In one embodiment, feature processing is performed on each first feature data to obtain corresponding second feature data, including: based on an efficient additive attention mechanism, feature extraction is performed on the first feature data obtained by the maximum number of multi-scale feature extractions to obtain fifth feature data; feature processing is performed on the first feature data and the fifth feature data obtained after the non-maximum number of multi-scale feature extractions to obtain second feature data.

[0017] In one embodiment, feature processing is performed on the first feature data and the fifth feature data obtained after multi-scale feature extraction with a non-maximum number of iterations to obtain the second feature data. This includes: performing feature interaction processing on the first feature data obtained after multi-scale feature extraction with a non-maximum number of iterations to obtain the corresponding sixth feature data; performing feature interaction processing on the fifth feature data to obtain the corresponding seventh feature data; and fusing the seventh feature data with each of the sixth feature data based on dilated convolution to obtain the corresponding eighth feature data. The second feature data includes the seventh feature data and each of the eighth feature data.

[0018] Secondly, this application also provides a device for determining operational defects in power equipment, comprising:

[0019] The image acquisition module is used to acquire the first image corresponding to the power equipment during the operation process;

[0020] The first extraction module is used to perform multi-scale feature extraction on the first image at different times to obtain different first feature data;

[0021] The second extraction module is used to perform feature processing on each first feature data to obtain the corresponding second feature data.

[0022] The defect determination module is used to determine the operational defects of power equipment based on each second feature data.

[0023] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method provided in the first aspect.

[0024] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect.

[0025] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in the first aspect.

[0026] The aforementioned method, apparatus, equipment, medium, and product for determining operational defects in power equipment perform multi-scale feature extraction on a first image corresponding to the power equipment during operation at different times, obtaining different first feature data. It is evident that these different first feature data represent different multi-scale feature data, achieving the extraction of multi-scale information to varying degrees. Next, feature processing is performed on each first feature data to obtain second feature data, and the operational defect status is determined based on this second feature data. Because the above process achieves the extraction of multi-scale information to varying degrees, the accuracy of the determined operational defect status of the power equipment can be improved, thereby enhancing the defect detection effect. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a method for determining operational defects of power equipment in one embodiment;

[0029] Figure 2 This is a schematic diagram of the MEF-DETR model in one embodiment;

[0030] Figure 3 This is a flowchart illustrating the steps for obtaining the first feature data in one embodiment;

[0031] Figure 4A This is a schematic diagram of the processing flow at each scale in one embodiment;

[0032] Figure 4B This is a schematic diagram of the edge feature enhancement process in one embodiment;

[0033] Figure 5 This is a flowchart illustrating the step of obtaining the fourth feature data in one embodiment;

[0034] Figure 6 This is a flowchart illustrating the step of obtaining the second feature data in one embodiment;

[0035] Figure 7 This is a schematic diagram of the processing flow of self-attention AIFI in one embodiment;

[0036] Figure 8 This is a schematic diagram of the AIFI processing flow that introduces EAA in one embodiment;

[0037] Figure 9AThis is a flowchart illustrating the step of obtaining the second feature data in one embodiment;

[0038] Figure 9B This is a schematic diagram of the structure of a dilated convolutional PDC in one embodiment;

[0039] Figure 9C This is a schematic diagram of a composite structure including a PDC in one embodiment;

[0040] Figure 10 This is a structural block diagram of a power equipment operation defect determination device in one embodiment;

[0041] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] In one exemplary embodiment, a method for determining operational defects in power equipment is provided, see [link to relevant documentation]. Figure 1 The method includes:

[0044] S110, acquire the first image corresponding to the power equipment during the operation.

[0045] The first image is of the power equipment during operation.

[0046] S120, perform multi-scale feature extraction on the first image at different times to obtain different first feature data.

[0047] The extraction process involves different numbers of iterations, such as 2, 3, and 4. For example, performing multi-scale feature extraction twice on the first image yields the corresponding first feature data. Performing multi-scale feature extraction three times on the first image yields the corresponding first feature data. Performing multi-scale feature extraction four times on the first image yields the corresponding first feature data.

[0048] In practical scenarios, the methods provided in the embodiments of this application can be implemented by a Multi-scale Efficient Fusion DETR (MEF-DETR) network for detecting operational defects in power equipment. See also... Figure 2MEF-DETR comprises a backbone network, an efficient hybrid encoder, and a decoder. Four multi-scale feature enhancement (MSFE) modules are designed within the backbone network. Each MSFE module performs multi-scale feature extraction on the input data, using the output features of the second, third, and fourth MSFE modules as one first feature data, thus obtaining three first feature data.

[0049] In the original structure, the C2f module (dual convolutional branch feature fusion module) effectively promotes feature interaction and transformation through its multi-branch design, and can output rich contextual information. However, when processing the first image, the fixed receptive field of the C2f module has limitations in capturing minute defects under complex background interference and adapting to defect features with significant scale changes. Furthermore, compared to efficient designs such as depthwise separable convolution widely used in lightweight networks, the C2f module has a large number of parameters and lower computational efficiency. Therefore, this embodiment uses the MSFE module instead of the original C2f module.

[0050] S130, perform feature processing on each first feature data to obtain the corresponding second feature data.

[0051] That is, by performing feature processing on the three first feature data, three second feature data can be obtained.

[0052] In this process, each first feature data can be input into an Efficient Hybrid Encoder, which performs feature processing on each first feature data and then outputs each second feature data.

[0053] S140, Based on the second characteristic data, determine the operational defects of the power equipment.

[0054] The operational defect situation may include whether there is an operational defect in the power equipment, and if so, the type of operational defect.

[0055] Among them, see Figure 2 Each second feature data can be input into the decoder, and through the decoding process of the decoder, the operational defects of the power equipment can be obtained.

[0056] In this embodiment, multi-scale feature extraction is performed on the first image corresponding to the power equipment during the operation at different times, resulting in different first feature data. It can be seen that different first feature data are different multi-scale feature data, achieving the extraction of multi-scale information at different levels. Next, feature processing is performed on each first feature data to obtain second feature data, and the operation defect situation is determined based on the second feature data. Since the above process achieves the extraction of multi-scale information at different levels, the accuracy of the determined operation defect situation of the power equipment can be improved, thereby improving the defect detection effect.

[0057] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the step of obtaining the first feature data is refined.

[0058] See Figure 3 The steps for obtaining the first feature data include:

[0059] S310: For each multi-scale feature extraction, the input data is subjected to feature extraction at multiple scales to obtain the third feature data corresponding to the respective scale.

[0060] In the case where this multi-scale feature extraction is the first multi-scale feature extraction, the input data is the first image; in the case where this multi-scale feature extraction is not the first multi-scale feature extraction, the input data is the first feature data obtained after the previous multi-scale feature extraction.

[0061] For example, the multiple scales include four different scales: 3×3, 6×6, 9×9, and 12×12. See also Figure 4A For each scale, the input data undergoes average pooling at the corresponding scale, followed by convolution with a 1×1 kernel, and then with a 3×3 kernel. This process ensures the extraction of important information while avoiding redundant feature representations. Subsequently, the convolution results are upsampled to obtain third feature data with the same resolution as the input data, facilitating subsequent feature data fusion. Therefore, third feature data corresponding to scale 3×3, scale 6×6, scale 9×9, and scale 12×12 can be obtained. The operation flow for the third feature data at each scale can be represented by the following formula:

[0062]

[0063] In the formula, Up() represents the upsampling formula, AdaptiveAvg() represents average pooling, Conv1×1 represents convolution with a kernel size of 1×1, Conv3×3 represents convolution with a kernel size of 3×3, and F n The third feature data is represented by n×n, which represents the scale of the feature data after average pooling.

[0064] S320 performs edge feature enhancement processing on the third feature data corresponding to each scale to obtain the fourth feature data corresponding to the scale.

[0065] See Figure 4A For each scale, the upsampled third feature data is subjected to edge feature enhancement processing to obtain the corresponding fourth feature data.

[0066] In one alternative implementation, see Figure 5 The steps to obtain the fourth feature data include:

[0067] S510 extracts edge feature data with a frequency higher than the preset frequency from the third feature data corresponding to each scale.

[0068] Among them, see Figure 4B Low-frequency information can be extracted by average pooling, and the difference between the third feature data and the low-frequency information can be calculated to obtain edge feature data with a frequency higher than the preset frequency.

[0069] S520 performs feature enhancement processing on the edge feature data to obtain edge feature enhanced data.

[0070] One approach is to use convolutional operations in convolutional layers to enhance the expressive power of edge features.

[0071] S530 performs feature fusion between the third feature data and the edge feature enhancement data to obtain the fourth feature data corresponding to the scale.

[0072] That is, the third feature data and the edge feature enhancement data are added together to obtain the fourth feature data.

[0073] The expression for obtaining the fourth feature data mentioned above can be:

[0074]

[0075] Where X is the third feature data input, i.e., the aforementioned F n AvgPool() is the average pooling operation, Conv is the convolution operation, and EdgeEnhance(X) is the fourth feature data corresponding to the third feature data X.

[0076] In the above implementation, the step of obtaining the fourth feature data extracts low-frequency information through average pooling, and the high-frequency part is obtained by subtracting the third feature data from the low-frequency information. Then, edge feature enhancement data is obtained through convolution operation, thereby enhancing the expressive power of edge features. Finally, the edge feature enhancement data is fused with the third feature data to obtain the fourth feature data that retains global information and enhances edge details.

[0077] S330, determine the first feature data based on the fourth feature data corresponding to each scale.

[0078] In order to facilitate the determination of the first feature data, the fourth feature data corresponding to each scale can be aligned to the same scale through the interpolation operation, and then the first feature data can be determined based on the fourth feature data corresponding to each scale.

[0079] In one optional implementation, determining the first feature data in S330 based on the fourth feature data corresponding to each scale may include:

[0080] S1, extract features from the input data to obtain initial feature data.

[0081] In this process, features can be extracted from the input data through convolutional layers to obtain initial feature data.

[0082] S2, the fourth feature data corresponding to each scale and the initial feature data are spliced ​​and fused to obtain the first feature data.

[0083] For example, the process of performing feature enhancement processing on the edge feature data corresponding to each scale to obtain the fourth feature data corresponding to the corresponding scale, and determining the first feature data based on the fourth feature data corresponding to each scale, can be expressed as the following formula:

[0084]

[0085] In the formula, F3 represents the third feature data corresponding to scale 3×3, and x1 is the fourth feature data corresponding to F3; F6 represents the third feature data corresponding to scale 6×6, and x2 is the fourth feature data corresponding to F6; F9 represents the third feature data corresponding to scale 9×9, and x3 is the fourth feature data corresponding to F9; F 12 This represents the third feature data corresponding to a scale of 12×12, where x4 is F. 12 The corresponding fourth feature data; x is the input data, x5 is the initial feature data corresponding to x, y is the first feature data, Concat() is the concatenation function, and Conv3×3 is the convolution operation with a kernel size of 3×3.

[0086] In the above implementation, the initial feature data corresponding to the input data is also included in the splicing process, so that the first feature data retains multi-scale information and enhanced edge information, while also having global information, avoiding information omission and improving the accuracy of subsequent defect detection.

[0087] In this embodiment, feature extraction is performed on the input data at multiple scales. This allows for the extraction of multi-scale information from different receptive fields, thereby improving the detection capability for targets at different scales. Furthermore, edge enhancement processing can improve the expressive power of edge features, thereby enhancing the accuracy of subsequent defect detection.

[0088] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the step of obtaining the second feature data is refined.

[0089] See Figure 6 The steps for obtaining the second feature data include:

[0090] S610, based on an efficient additive attention mechanism, extracts features from the first feature data obtained by the maximum number of multi-scale feature extractions to obtain the fifth feature data.

[0091] Among them, the first feature data obtained from the multi-scale feature extraction with the maximum number of iterations, for example... Figure 2 The first feature data is obtained by processing the output data of the last MSFE in the convolutional layer.

[0092] S620 performs feature processing on the first and fifth feature data obtained after multi-scale feature extraction (not at the maximum number of extractions) to obtain the second feature data.

[0093] Among them, the first feature data obtained after multi-scale feature extraction with a non-maximum number of iterations, for example... Figure 2 The first feature data of the second MSFE output and the first feature data of the third MSFE output.

[0094] In real-world scenarios, after obtaining multiple first feature data through the backbone network, these first feature data are input into the Efficient Hybrid Encoder. The Efficient Hybrid Encoder performs interactive processing of the first feature data within the same scale and performs cross-scale fusion, thereby achieving efficient multi-scale feature processing, which reduces computational costs while improving real-time detection performance.

[0095] The efficient hybrid encoder extracts features through an attention-based Intra-scale Feature Interaction (AIFI) module before performing intra-scale interaction processing and cross-scale fusion. The attention mechanism can be self-attention, and the core function of self-attention AIFI is to integrate deep and shallow features output from the feature extraction network. This module enhances the correlation between deep features using self-attention, thereby extracting richer global information. Simultaneously, its design avoids redundant interactions between shallow feature layers, effectively balancing model accuracy and runtime efficiency. However, the self-attention mechanism requires calculating the key matrix K, query matrix Q, and value matrix V. Its complex matrix multiplication operations lead to high computational overhead, reducing the overall efficiency of the model. Furthermore, the analysis of self-attention AIFI mainly focuses on deep feature extraction, failing to adequately consider key details in shallow features, resulting in performance bottlenecks in small object detection.

[0096] See Figure 7 The self-attention AIFI processing procedure is as follows: The first input feature data is processed by a linear transformation of a linear layer (Linear), mapping it to a query matrix Q, a key matrix K, and a value matrix V, each with d rows and n columns. Then, the product of the transposes of Q and K is calculated, and an activation function is used to process this product into a feature matrix. This feature matrix is ​​then multiplied by V to obtain the self-attention output. In the self-attention mechanism, the computational cost increases significantly with the increase in input image size and feature dimension. The computational process of the self-attention mechanism is as follows:

[0097]

[0098] Where T is the transpose, Softmax() is the activation function, and X' is the self-attention output.

[0099] As can be seen, the computational complexity of the self-attention mechanism is n. 2 ×d, as the input first feature data increases, Q, K T The computational load and memory usage increase quadratically, resulting in low inference efficiency and high resource consumption, which in turn affects the real-time response capability of the network system.

[0100] Therefore, when constructing the feature matrix, the Efficient Additive Attention (EAA) mechanism is used instead of the self-attention mechanism mentioned above. Due to the introduction of EAA, the processing steps of the value matrix V are omitted. Only the query matrix Q and the key matrix K are integrated, and the additive rule is used to replace the traditional Hadamard product operation.

[0101] See Figure 8 The AIFI introduced by EAA, namely EAA-AIFI, proceeds as follows: The first feature data obtained from the maximum number of multi-scale feature extractions is used as input X. A query matrix Q and a key matrix K are derived through linear transformation. The query matrix Q is then associated with the learnable parameter vector Wα to generate a global attention vector α, as shown in the following formula:

[0102]

[0103] In the formula, d represents the number of elements in each column of Q. α1, α2, ..., α n Let be the component of α.

[0104] Next, the global attention vector α is pooled to obtain a single global query vector q, calculated as follows:

[0105]

[0106] In the formula, α i Q is the i-th element in the global attention vector α. i Let be the i-th column in Q, and n be the column number of Q.

[0107] Next, q and the key matrix K are cross-encoded to extract global context features. The two feature matrices are then fused using skip connections and regularization techniques, ultimately outputting the fused features. The calculation formula is as follows:

[0108]

[0109] In the formula, Q' is the query matrix after normalization, and T() is the linear transformation function.

[0110] The above formula constructs an interaction mechanism between the query matrix Q and the key matrix K through a linear transformation layer, and integrates all feature matrices in the broadcast element-wise multiplication, which significantly reduces the computational complexity. After applying linear transformation processing, the computational efficiency of EAA is improved and its computational complexity is significantly reduced.

[0111] As can be seen, EAA achieves feature fusion through linear addition, abandoning the quadratic parameter calculation method of matrix multiplication in traditional self-attention mechanisms, significantly reducing computational complexity and shortening inference latency. EAA optimizes the feature interaction process, removing the explicit association constraint between key-value matrices K and V, and instead constructing global context features through the interaction between query matrix Q and key matrix K. The resulting flexible and efficient architecture improves model computational performance while enhancing the ability to learn deep interaction patterns between features.

[0112] In this embodiment, an efficient additive attention mechanism is introduced, replacing the traditional self-attention matrix multiplication with linear addition, reducing the computational complexity from quadratic to linear, thereby improving the efficiency of defect detection.

[0113] Based on the technical solutions provided in the above embodiments, an optional embodiment is provided, in which the step of obtaining the second feature data in S620 is refined.

[0114] See Figure 9A The steps for obtaining the second feature data in S620 include:

[0115] S910, after multi-scale feature extraction with a non-maximum number of iterations, the first feature data is subjected to feature interaction processing to obtain the corresponding sixth feature data.

[0116] Specifically, for each first feature data obtained after multi-scale feature extraction (not at the maximum number of extractions), a channel attention mechanism (CA) can be used to perform feature interaction processing to obtain the sixth feature data corresponding to that first feature data.

[0117] For example, for Figure 2 The first feature data output by the second MSFE is processed by feature interaction to obtain the sixth feature data corresponding to the second MSFE; Figure 2 The first feature data output by the third MSFE is processed by feature interaction to obtain the sixth feature data corresponding to the third MSFE.

[0118] The feature interaction processing of the channel attention mechanism can be understood as a process of dynamically adjusting the importance weight of each channel by modeling the correlation between different channel features, thereby enhancing key channel features and suppressing redundant channel features.

[0119] S920, perform feature interaction processing on the fifth feature data to obtain the corresponding seventh feature data.

[0120] Among them, the channel attention mechanism, namely CA, can be used to perform feature interaction processing on the fifth feature data to obtain the seventh feature data.

[0121] S930, based on dilated convolution, fuses the seventh feature data with each of the sixth feature data to obtain the corresponding eighth feature data; wherein, the second feature data includes the seventh feature data and each of the eighth feature data.

[0122] That is, the seventh feature data is fused with the sixth feature data corresponding to the second MSFE to obtain the corresponding eighth feature data, the seventh feature data is fused with the sixth feature data corresponding to the third MSFE to obtain the corresponding eighth feature data, and the two eighth feature data and the seventh feature data are used as three second feature data.

[0123] CA and Parallel Dilated Convolution (PDC) can be set in the High-level Screening Feature Pyramid Network (HSFPN) to implement S910~S930. Since HSFPN introduces PDC, it can be called PDC-HSFPN.

[0124] In real-world scenarios, when minor operational defects exist and feature data is scarce, key features are primarily contained in lower and middle-level feature maps. The feature fusion module of RT-DETR (Real-time Target Detection Transformation Model) processes feature maps through basic concatenation or addition operations, which easily leads to information loss or degradation when fusing feature data from different levels of the pyramid. Furthermore, it struggles to effectively mine and integrate potential inter-level correlations, thus failing to obtain a comprehensive and representative feature representation. Therefore, this embodiment provides PDC-HSFPN.

[0125] The structure of dilated convolution is shown in [reference]. Figure 9B The dilated convolutional (PDC) model employs a three-branch structure, with each branch having a 3×3 kernel and dilation rates of 1, 2, and 3, respectively. By adjusting the dilation rate, the PDC can adaptively extract features at different scales, thus significantly improving the model's adaptability to photovoltaic defects at different scales. The formulas for the delimited convolution operation and normalization operation of any branch are as follows:

[0126]

[0127] In the formula, x is the input data of PDC, Conv() is the convolution function, which can also be expressed as DConv(); BN() is the normalization function, γ represents the scaling factor of the normalized result, W() represents the convolution kernel weights, σ ​​represents the variance of the normalization parameters, ε is a minimum constant to prevent the denominator from being zero, b is a constant, μ represents the variance of the convolution kernel weights, and β represents the bias term.

[0128] The output data from the three branches are concatenated, and then subjected to a 1×1 convolution to obtain the final result. This process can be represented as:

[0129]

[0130] In the formula, x1 represents the output data of the first branch, x2 represents the output data of the second branch, x3 represents the output data of the third branch, and y represents the final result obtained by concatenating the output data of the three branches and performing a 1×1 convolution.

[0131] To further enhance the ability of dilated convolution to capture multi-scale features, a composite structure is proposed, which can be called the PDC-Fusion structure. (See [link to relevant documentation]). Figure 9C See also Figure 9C The PDC-Fusion structure employs a dual-branch architecture. The first branch adjusts the feature channel dimensions using a 1×1 convolution, while the second path consists of a 1×1 convolution and a PDC. The PDC extracts multi-scale features from the image to generate rich multi-scale feature representations. Finally, the outputs of the two branches are concatenated to generate the final output data.

[0132] As can be seen, in order to effectively integrate the semantic information of deep feature maps with the positional information of shallow feature maps, this embodiment embeds the PDC-Fusion module into the high-level filtering feature pyramid network to construct a new PDC-HSFPN.

[0133] The principle of PDC-HSFPN is explained as follows:

[0134] Unlike traditional fusion strategies that simply add deep and shallow features, PDC-HSFPN uses deep features as weighting factors to filter key semantic information from shallow features before fusing them with high-level features, effectively suppressing data redundancy. This mechanism significantly improves the model's feature extraction and parsing capabilities, and its specific implementation is shown in the following formula:

[0135]

[0136]

[0137] in, For deep features, For shallow features, TC() represents transposed convolution; for deep features... Expanding using transposed convolution yields... BL() represents the bilinear interpolation function, which uses bilinear interpolation pairs. Obtain by upsampling or downsampling CA stands for Channel Attention, which transforms deep features into corresponding attention weights to filter out shallow features. Finally, these are fused to obtain... .

[0138] As can be seen, PDC-HSFPN significantly improves detection accuracy by optimizing the key region attention mechanism. This method first introduces channel attention (CA) to filter features at different levels, preserving key semantic information while effectively eliminating redundant information. Secondly, it uses deconvolution (ConvTranspose2d, CT2d) to upsample the deep feature maps, aligning their size with the shallow feature maps. Finally, the PDC-Fusion module fuses the processed deep and shallow features, enhancing the model's representational ability and achieving effective integration of semantic and detailed information.

[0139] in, Figure 2 RepC3 is a neural network module based on reparameterization convolution, mainly used for computer vision tasks such as object detection. Its core design goal is to balance model performance and computational efficiency. By dynamically adjusting the number of channels, residual connections and efficient computational structures, it reduces the number of parameters and improves speed during the inference stage.

[0140] It is evident that by introducing dilated convolution and embedding it into a high-level filtering feature pyramid network, the PDC-HSFPN architecture can be constructed, which can effectively integrate contextual information at different scales.

[0141] In this embodiment, different levels of features, namely deep features and shallow features, can be filtered through feature interaction processing. By using dilated convolution to fuse the seventh feature data with each of the sixth feature data, the deep features and shallow features can be effectively fused, realizing the effective integration of semantic information and detailed information, which is beneficial to improving the accuracy of defect detection.

[0142] In summary, the above embodiments propose an MSFE module in the backbone network to replace the traditional C2f convolution. Through multi-scale average pooling and edge information enhancement, it effectively improves the perception capability of multi-scale feature information and edge information. Then, EAA-AIFI is introduced, using additive operations to replace the dot product operation in the traditional self-attention mechanism, reducing computational complexity. Finally, based on PDC-HSFPN, the multi-scale feature integration capability is enhanced during feature fusion. Therefore, MEF-DETR can improve detection accuracy while optimizing computational efficiency, making it particularly suitable for edge devices with limited computing resources.

[0143] To evaluate the performance of MEF-DETR, experiments were conducted using a public dataset of photovoltaic cell defects. This dataset contains 12 types of photovoltaic cell defects. Five typical defects—linear cracks, broken grids, black cores, coarse lines, and star-shaped cracks—were selected as the research objects, comprising 3816 images. To balance the distribution of defect types, black cores were considered easily identifiable samples, and their sample size was reduced. The dataset was then divided into training, validation, and test sets in a 7:2:1 ratio. To enhance the model's ability to detect small-target defects such as micro-cracks, broken grids, and coarse lines, image augmentation techniques such as random mirroring, flipping transformation, and Gaussian blur were used to augment these defects. The specific distribution of each defect type in the dataset is shown in Table 1.

[0144] Table 1. Dataset Sample Distribution Table

[0145]

[0146] This experiment was conducted under a pre-defined operating system and equipped with an RTX 4090 graphics card with 24GB of VRAM. A pre-defined open-source framework was used during model building, training, and inference. The dataset was trained for 150 epochs with an initial learning rate of 0.0001 and a momentum value of 0.937. Stochastic gradient descent was used as the optimization algorithm, and a cosine annealing strategy was introduced to dynamically adjust the learning rate. Precision, recall, mean precision, number of parameters, and computational cost were selected as evaluation metrics to systematically evaluate the performance of MEF-DETR.

[0147] To verify the effectiveness of each module in MEF-DETR in improving performance, RT-DETR (Real-Time Detection Transformer, a real-time end-to-end target detection model based on the Transformer architecture) was used as the baseline model. Ablation experiments were conducted on a photovoltaic defect dataset by individually introducing the MSFE module, EAA-AIFI, and PDC-HSFPN modules. The ablation experiment results are shown in Table 2. Each module improved the detection performance of RT-DETR to varying degrees. Specifically, MEF-DETR with the MSFE module referenced the backbone network structure of the YOLOv8 model in its backbone network, replacing the c2f module in the YOLOv8 backbone network with the MSFE module. The MSFE module integrates multi-scale feature extraction and edge information enhancement, enhancing the feature extraction capability of the backbone network. The average precision (mAP@0.5) and average precision (mAP@0.5:0.95) were improved by 2% and 1.4%, respectively, while the number of parameters in the model decreased by 33% compared to the baseline model. Introducing EAA-AIFI alone improved the average precision (mAP@0.5) by 2.2% and 0.6% compared to the baseline model, respectively, while reducing computational cost by 4G. This demonstrates that the efficient additive attention mechanism is superior to the self-attention mechanism in optimizing the information interaction process and reducing the computational complexity of attention. Introducing the PDC-HSFPN structure alone, without a significant increase in the number of parameters, improved the average precision (mAP@0.5) by 1.6% and 1.3% compared to the baseline model, respectively, while reducing computational cost by 4G. This is attributed to the proposed parallel dilated convolution, which improves the model's multi-scale feature extraction capability without increasing the number of model parameters.

[0148] By simultaneously introducing the MSFE, EAA-AIFI, and PDC-HSFPN modules, the number of model parameters and computational cost were significantly reduced. The number of parameters decreased to 14.4M, a 33% reduction compared to the baseline model, and the computational cost decreased to 52G. The MEF-DETR model proposed in this application achieved an average accuracy of 89% and an average accuracy of 58.8% (mAP@0.5:0.95), respectively, demonstrating the best average accuracy. Simultaneously, the frame rate reached 122 frames per second, an 11% improvement compared to the baseline model, meeting the requirements for real-time detection. In summary, MEF-DETR significantly improved detection accuracy while significantly reducing the number of parameters and computational cost, meeting the requirements for lightweight and real-time detection. This fully demonstrates the effectiveness of MEF-DETR in photovoltaic defect detection tasks.

[0149] Table 2 Ablation Experiment Results

[0150]

[0151] To verify the performance of the MEF-DETR model, we selected the two-stage detection model Faster-RCNN (an efficient two-stage object detection algorithm) and the one-stage detection models YOLOv8m, YOLOv9m, YOLOv10m, YOLOv11m, and YOLOv12m for comparative experiments. The results of the comparative experiments are shown in Table 3.

[0152] Table 3 Comparison results of different models

[0153]

[0154] As shown in Table 3 above, the MEF-DETR model's average accuracy (mAP@0.5) and average accuracy (mAP@0.5:0.95) are 2.7% and 1.4% higher than the baseline model, respectively, while the number of parameters is reduced by 33%. This demonstrates that the model improves detection accuracy while maintaining its lightweight design. Furthermore, compared to the YOLOv11m model, which boasts the highest accuracy among YOLO models, the proposed model achieves 1.7% and 1.1% higher average accuracy (mAP@0.5) and average accuracy (mAP@0.5:0.95), respectively, exhibiting superior defect detection performance. In summary, the MEF-DETR model improves detection accuracy while maintaining a lightweight design and reducing computational load.

[0155] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0156] Based on the same inventive concept, this application also provides a power equipment operation defect determination device for implementing the above-mentioned power equipment operation defect determination method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more power equipment operation defect determination device embodiments provided below can be found in the limitations of the power equipment operation defect determination method described above, and will not be repeated here.

[0157] In one exemplary embodiment, a device for determining operational defects in power equipment is provided, see [link to relevant documentation]. Figure 10 The device includes an image acquisition module 1010, a first extraction module 1020, a second extraction module 1030, and a defect determination module 1040, wherein:

[0158] The image acquisition module 1010 is used to acquire the first image corresponding to the power equipment during the operation process;

[0159] The first extraction module 1020 is used to perform multi-scale feature extraction on the first image at different times to obtain different first feature data;

[0160] The second extraction module 1030 is used to perform feature processing on each of the first feature data to obtain the corresponding second feature data.

[0161] The defect determination module 1040 is used to determine the operational defect status of the power equipment based on each of the second feature data.

[0162] In one embodiment, the first extraction module includes: a first extraction unit, configured to extract features from the input data at multiple scales for each multi-scale feature extraction, to obtain third feature data corresponding to the corresponding scale; a first enhancement unit, configured to perform edge feature enhancement processing on the third feature data corresponding to each scale, to obtain fourth feature data corresponding to the scale; and a first determination unit, configured to determine the first feature data based on the fourth feature data corresponding to each scale; wherein, when this multi-scale feature extraction is the first multi-scale feature extraction, the input data is the first image; when this multi-scale feature extraction is not the first multi-scale feature extraction, the input data is the first feature data obtained after the previous multi-scale feature extraction.

[0163] In one embodiment, the first enhancement unit includes: a first extraction subunit, configured to extract edge feature data with a frequency higher than a preset frequency from the third feature data corresponding to each scale; a first enhancement subunit, configured to perform feature enhancement processing on the edge feature data to obtain edge feature enhanced data; and a first fusion subunit, configured to perform feature fusion between the third feature data and the edge feature enhanced data to obtain fourth feature data corresponding to the scale.

[0164] In one embodiment, the first fusion subunit is specifically used to: extract features from the input data to obtain initial feature data; and concatenate and fuse the fourth feature data corresponding to each scale with the initial feature data to obtain the first feature data.

[0165] In one embodiment, the second extraction module includes: a second extraction unit, configured to extract features from the first feature data obtained by the maximum number of multi-scale feature extractions based on an efficient additive attention mechanism, to obtain the fifth feature data; and a first processing unit, configured to perform feature processing on the first feature data obtained after the non-maximum number of multi-scale feature extractions and the fifth feature data, to obtain the second feature data.

[0166] In one embodiment, the first processing unit is specifically configured to: perform feature interaction processing on the first feature data obtained after multi-scale feature extraction at a non-maximum number of times to obtain corresponding sixth feature data; perform feature interaction processing on the fifth feature data to obtain corresponding seventh feature data; and perform fusion processing on the seventh feature data and each of the sixth feature data based on dilated convolution to obtain corresponding eighth feature data; wherein the second feature data includes the seventh feature data and each of the eighth feature data.

[0167] Each module in the aforementioned power equipment operational defect determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0168] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores relevant data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for determining operational defects in power equipment.

[0169] Those skilled in the art will understand that Figure 11The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0170] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the power equipment operation defect determination method provided in the above embodiments.

[0171] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method for determining operational defects of power equipment provided in the above embodiments.

[0172] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the power equipment operation defect determination method provided in the above embodiments.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0174] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0175] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0176] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining operational defects in power equipment, characterized in that, include: Acquire the first image of the power equipment during the operation; Perform multi-scale feature extraction on the first image at different times to obtain different first feature data; Perform feature processing on each of the first feature data to obtain the corresponding second feature data; Based on each of the second feature data, the operational defects of the power equipment are determined.

2. The method according to claim 1, characterized in that, The step of performing multi-scale feature extraction on the first image at different times to obtain different first feature data includes: For each multi-scale feature extraction, the input data is subjected to feature extraction at multiple scales to obtain the third feature data corresponding to the respective scale; The third feature data corresponding to each scale is subjected to edge feature enhancement processing to obtain the fourth feature data corresponding to the scale. The first feature data is determined based on the fourth feature data corresponding to each of the aforementioned scales; In the case where this multi-scale feature extraction is the first multi-scale feature extraction, the input data is the first image; in the case where this multi-scale feature extraction is not the first multi-scale feature extraction, the input data is the first feature data obtained after the previous multi-scale feature extraction.

3. The method according to claim 2, characterized in that, The step of performing edge feature enhancement processing on the third feature data corresponding to each scale to obtain the fourth feature data corresponding to the scale includes: Extract edge feature data with a frequency higher than the preset frequency from the third feature data corresponding to each scale; The edge feature data is subjected to feature enhancement processing to obtain edge feature enhanced data; The third feature data is fused with the edge feature enhancement data to obtain the fourth feature data corresponding to the scale.

4. The method according to claim 2, characterized in that, The step of determining the first feature data based on the fourth feature data corresponding to each of the aforementioned scales includes: Feature extraction is performed on the input data to obtain initial feature data; The fourth feature data corresponding to each scale and the initial feature data are concatenated and fused to obtain the first feature data.

5. The method according to any one of claims 1 to 4, characterized in that, The step of performing feature processing on each of the first feature data to obtain the corresponding second feature data includes: Based on an efficient additive attention mechanism, feature extraction is performed on the first feature data obtained from the maximum number of multi-scale feature extractions to obtain the fifth feature data; The first feature data and the fifth feature data obtained after multi-scale feature extraction (not at the maximum number of extractions) are subjected to feature processing to obtain the second feature data.

6. The method according to claim 5, characterized in that, The step of performing feature processing on the first feature data and the fifth feature data obtained after multi-scale feature extraction (not at the maximum number of extractions) to obtain the second feature data includes: The first feature data obtained after multi-scale feature extraction (not at the maximum number of extractions) is subjected to feature interaction processing to obtain the corresponding sixth feature data. The fifth feature data is subjected to feature interaction processing to obtain the corresponding seventh feature data; Based on dilated convolution, the seventh feature data is fused with each of the sixth feature data to obtain the corresponding eighth feature data; The second feature data includes the seventh feature data and each of the eighth feature data.

7. A device for determining operational defects in power equipment, characterized in that, include: The image acquisition module is used to acquire the first image corresponding to the power equipment during the operation process; The first extraction module is used to perform multi-scale feature extraction on the first image at different times to obtain different first feature data; The second extraction module is used to perform feature processing on each of the first feature data to obtain the corresponding second feature data. The defect determination module is used to determine the operational defect status of the power equipment based on each of the second feature data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images