Power defect detection method, device, equipment, medium and program product
By integrating feature extraction and target classification models for multi-scale feature processing, the problem of inaccurate feature extraction in power defect detection using visual language models is solved, achieving higher detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG POWER GRID CO LTD DONGGUAN POWER SUPPLY BUREAU
- Filing Date
- 2026-02-09
- Publication Date
- 2026-05-08
AI Technical Summary
In existing technologies, when using visual language models for power defect detection, feature extraction is inaccurate, leading to inaccurate defect detection.
By employing a fusion feature extraction model and a target classification model, and through multi-scale feature extraction, feature fusion, discriminative feature extraction, and classification, the probability of occurrence of each type of power defect is obtained.
It improves the accuracy and generalization of power defect detection, can handle images with poor lighting and shooting angles, and enhances the matching degree and robustness between features and power defects.
Smart Images
Figure CN121997146A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of power grid technology, and in particular to a method, apparatus, equipment, medium, and program product for detecting power defects. Background Technology
[0002] In power grids, there are often various electrical defects, such as bird nests on power lines, fires in power equipment, and oil stains on power equipment, which can affect the stable operation of the power grid. Therefore, it is necessary to conduct electrical defect detection.
[0003] In existing technologies, the detection of power defects typically involves inputting the image to be detected into a visual language model to obtain a defect description text, which is then used by staff to determine whether a power defect exists.
[0004] However, because the visual language model includes many features unrelated to defects in the features obtained from image processing, the resulting defect description text is inaccurate, which in turn leads to inaccurate defect detection. Summary of the Invention
[0005] The power defect detection method, apparatus, equipment, medium, and program products provided in this application are intended to solve the problem of inaccurate defect detection caused by the use of visual language models in the prior art.
[0006] In a first aspect, embodiments of this application provide a power defect detection method, including:
[0007] The acquired power image to be detected and the preset defect detection prompt text are input into the fusion feature extraction model to obtain the fusion feature matrix. The fusion feature extraction model is a pre-trained visual language model that obtains the feature matrix based on the image and text.
[0008] By using a target classification model, multi-scale feature extraction processing is performed on the fused feature matrix to obtain multiple first feature matrices;
[0009] The second feature matrix is obtained by fusing all the first feature matrices using the target classification model.
[0010] The second feature matrix is subjected to discriminative feature extraction processing using the target classification model to obtain the third feature matrix;
[0011] The target classification model is used to classify the third feature matrix to obtain the probability of occurrence of each power defect. The target classification model is a pre-trained deep neural network model used to obtain the probability of occurrence of each power defect based on the fused feature matrix.
[0012] In one possible implementation, the step of performing multi-scale feature extraction processing on the fused feature matrix using a target classification model to obtain multiple first feature matrices includes:
[0013] The dimensionality of the fused feature matrix is reduced by the first linear projection layer in the target classification model to obtain the first feature matrix;
[0014] The first feature matrix is obtained by performing average pooling and dimensionality reduction on the fused feature matrix through the average pooling layer and the second linear projection layer in the target classification model.
[0015] The first feature matrix is obtained by performing max pooling and dimensionality reduction on the fused feature matrix through the max pooling layer and the third linear projection layer in the target classification model.
[0016] In one possible implementation, the step of performing discriminative feature extraction processing on the second feature matrix using the target classification model to obtain the third feature matrix includes:
[0017] The second feature matrix is subjected to layer normalization and numerical clipping through the first normalization layer in the target classification model to obtain the fourth feature matrix.
[0018] The fourth feature matrix is reduced in dimensionality by using the fourth linear projection layer in the target classification model to obtain the fifth feature matrix;
[0019] The fifth feature matrix is normalized by the second normalization layer in the target classification model to obtain the sixth feature matrix.
[0020] The sixth feature matrix is smoothed by gradient processing through the activation function layer in the target classification model to obtain the third feature matrix.
[0021] In one possible implementation, the step of classifying the third feature matrix using the target classification model to obtain the probability of occurrence of each type of power defect includes:
[0022] The third feature matrix is classified using the classifier in the target classification model to obtain a score for each type of power defect.
[0023] The third normalization layer in the target classification model is used to perform numerical cropping and normalization on the score of each power defect to obtain the probability of occurrence of each power defect.
[0024] In one possible implementation, before classifying the third feature matrix using the target classification model to obtain the probability of occurrence of each power defect, the method further includes:
[0025] The third feature matrix is enhanced by self-attention layer in the target classification model to obtain the seventh feature matrix.
[0026] The process of classifying the third feature matrix using the target classification model to obtain the probability of occurrence of each type of power defect includes:
[0027] The seventh feature matrix is classified using the target classification model to obtain the probability of occurrence of each type of power defect.
[0028] In one possible implementation, before performing multi-scale feature extraction processing on the fused feature matrix using a target classification model to obtain multiple first feature matrices, the method further includes:
[0029] The eighth feature matrix is obtained by extracting column vectors from the fusion feature matrix through the dimensionality reduction layer in the target classification model.
[0030] The process involves performing multi-scale feature extraction on the fused feature matrix using a target classification model to obtain multiple first feature matrices, including:
[0031] The target classification model is used to perform multi-scale feature extraction processing on the eighth feature matrix to obtain multiple first feature matrices.
[0032] In one possible implementation, before performing column vector extraction processing on the fused feature matrix through the dimensionality reduction layer in the target classification model to obtain the eighth feature matrix, the method further includes:
[0033] The outlier processing layer in the target classification model performs outlier replacement processing on the fused feature matrix to obtain the ninth feature matrix.
[0034] The eighth feature matrix is obtained by extracting column vectors from the fusion feature matrix through the dimensionality reduction layer in the target classification model, including:
[0035] The ninth feature matrix is processed by extracting column vectors through the dimensionality reduction layer in the target classification model to obtain the eighth feature matrix.
[0036] Secondly, embodiments of this application provide a power defect detection device, comprising:
[0037] The first processing module is used to input the acquired power image to be detected and the preset defect detection prompt text into the fusion feature extraction model to obtain the fusion feature matrix. The fusion feature extraction model is a pre-trained visual language model that obtains the feature matrix based on the image and text.
[0038] The second processing module is used for:
[0039] By using a target classification model, multi-scale feature extraction processing is performed on the fused feature matrix to obtain multiple first feature matrices;
[0040] The second feature matrix is obtained by fusing all the first feature matrices using the target classification model.
[0041] The second feature matrix is subjected to discriminative feature extraction processing using the target classification model to obtain the third feature matrix;
[0042] The target classification model is used to classify the third feature matrix to obtain the probability of occurrence of each power defect. The target classification model is a pre-trained deep neural network model used to obtain the probability of occurrence of each power defect based on the fused feature matrix.
[0043] Thirdly, embodiments of this application provide an electronic device, including:
[0044] Processor, memory, communication interface;
[0045] The memory is used to store the executable instructions of the processor;
[0046] The processor is configured to execute the power defect detection method according to any one of the first aspects by executing the executable instructions.
[0047] Fourthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the power defect detection method described in any of the first aspects.
[0048] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the power defect detection method described in any of the first aspects.
[0049] The power defect detection method, apparatus, equipment, medium, and program products provided in this application involve inputting the acquired power image to be detected and the preset defect detection prompt text into a fusion feature extraction model to obtain a fusion feature matrix. Then, a target classification model is used to perform multi-scale feature extraction processing on the fusion feature matrix to obtain multiple first feature matrices; all first feature matrices are fused to obtain a second feature matrix; the second feature matrix is then subjected to discriminative feature extraction processing to obtain a third feature matrix; and the third feature matrix is then classified to obtain the probability of occurrence for each type of power defect. This solution improves the matching degree between features and power defects by processing the fusion features obtained from the fusion feature extraction model using a target classification model, thereby improving the accuracy of the probability of power defect occurrence. Attached Figure Description
[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0051] Figure 1 A flowchart illustrating an embodiment of the power defect detection method provided in this application;
[0052] Figure 2 A schematic diagram of the architecture of the fusion feature extraction model provided in this application;
[0053] Figure 3 Schematic diagram of the target classification model provided in this application Figure 1 ;
[0054] Figure 4 Schematic diagram of the target classification model provided in this application Figure 2 ;
[0055] Figure 5 Schematic diagram of the target classification model provided in this application Figure 3 ;
[0056] Figure 6 Schematic diagram of the target classification model provided in this application Figure 4 ;
[0057] Figure 7 A schematic diagram of the structure of an embodiment of the power defect detection device provided in this application;
[0058] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application.
[0059] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0060] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0061] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0062] In power grids, there are often various electrical defects, such as bird nests on power lines, fires in power equipment, and oil stains on power equipment, which can affect the stable operation of the power grid. Therefore, it is necessary to conduct electrical defect detection.
[0063] In existing technologies, the detection of power defects typically involves inputting the image to be detected into a Vision Language Model (VLM) to obtain a defect description text. Workers then use this description text to determine whether a power defect exists. A Vision Language Model, also known as a Vision Large Model (VLM), is a large language model capable of processing both images and text.
[0064] However, because the visual language model includes many features unrelated to defects in the features obtained from image processing, the resulting defect description text is inaccurate, which leads to inaccurate defect detection.
[0065] To address the problems existing in the prior art, the inventors, during their research on power defect detection methods, discovered that to improve detection accuracy, the feature matrix obtained through a visual language model can be processed to obtain features related to and matching power defects, which can then be classified to obtain the probability of power defect occurrence. Multi-scale feature extraction, feature fusion, discriminative feature extraction, and classification can be performed using a target classification model to obtain the probability of occurrence for each type of power defect. Based on the above inventive concept, the power defect detection scheme in this application was designed.
[0066] The execution subject of the power defect detection method in this application can be a computer, or a server, terminal equipment, etc. This application does not limit it. The following description uses a computer as an example.
[0067] The following provides examples illustrating the application scenarios of the power defect detection method provided in this application.
[0068] For example, in this application scenario, staff members collect multiple images of the power grid to be inspected, and then input these images into a computer.
[0069] After the computer acquires the power image to be detected, it inputs the acquired power image and the preset defect detection prompt text into the fusion feature extraction model to obtain the fusion feature matrix. The fusion feature extraction model is a pre-trained visual language model that obtains the feature matrix based on the image and text.
[0070] Then, through the target classification model, multi-scale feature extraction processing is performed on the fused feature matrix to obtain multiple first feature matrices; through the target classification model, all first feature matrices are fused to obtain second feature matrices; through the target classification model, discriminative feature extraction processing is performed on the second feature matrices to obtain third feature matrices.
[0071] The third feature matrix is classified using a target classification model to obtain the probability of occurrence of each power defect. The target classification model is a pre-trained deep neural network model used to obtain the probability of occurrence of each power defect based on the fused feature matrix.
[0072] The computer displays the probability of each type of power defect, allowing staff to identify which type of power defect has occurred and thus carry out power grid maintenance.
[0073] It should be noted that the above scenario is only an example of an application scenario provided by the embodiments of this application. The embodiments of this application do not limit the actual form of the various devices included in the scenario. In the specific application of the solution, it can be set according to actual needs.
[0074] The technical solution of this application will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0075] Figure 1 This is a flowchart illustrating an embodiment of the power defect detection method provided in this application. This embodiment describes the use of a computer to perform power defect detection by fusing a feature extraction model and a target classification model. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 1 As shown, the power defect detection method specifically includes the following steps:
[0076] S101: Input the acquired power image to be detected and the preset defect detection prompt text into the fusion feature extraction model to obtain the fusion feature matrix.
[0077] In this step, after the computer acquires the power image to be detected, in order to perform power defect detection, the power image to be detected and the preset defect detection prompt text are input into the fusion feature extraction model to obtain the fusion feature matrix.
[0078] The fusion feature extraction model is a pre-trained visual language model that obtains feature matrices from images and text.
[0079] It should be noted that the preset defect detection prompt text is text that indicates the type of defect to be detected. For example, the preset defect detection prompt text may be: "Please check for power defects such as oil stains or bird nests on the power equipment", "Please check for power defects such as fire on the power equipment or damaged lines", "Please check for power defects such as rust on the power equipment or tilted utility poles", etc. This application embodiment does not limit the preset defect detection prompt text, and it can be determined according to the actual situation.
[0080] It should be noted that the number of power images to be detected is at least one.
[0081] For example, Figure 2 A schematic diagram of the architecture of the fusion feature extraction model provided in this application is shown below. Figure 2 As shown, the fusion feature extraction model includes a visual encoder, a text encoder, and a feature fusion layer. The visual encoder encodes the power image to be detected, obtaining an image feature matrix; the text encoder segments and embeds the preset defect detection prompt text, obtaining a text feature matrix. The feature fusion layer then fuses the image feature matrix and the text feature matrix to obtain a fused feature matrix. The feature fusion layer consists of at least one transfer layer.
[0082] It should be noted that the number of power images to be detected is the target number. The fusion feature matrix is a three-dimensional matrix, composed of the target number of two-dimensional matrices. Each two-dimensional matrix corresponds to one power image to be detected. The number of columns in the two-dimensional matrix indicates the number of feature extraction methods. Each column of the two-dimensional matrix is the fusion feature of a power image to be detected and a preset defect detection prompt text.
[0083] S102: Using the target classification model, multi-scale feature extraction is performed on the fused feature matrix to obtain multiple first feature matrices.
[0084] In this step, after the computer obtains the fused feature matrix, in order to improve the correlation and matching degree between the features and power defects, it processes the fused feature matrix through a target classification model. First, it performs multi-scale feature extraction processing on the fused feature matrix to obtain multiple first feature matrices.
[0085] The target classification model is a pre-trained deep neural network model used to obtain the probability of occurrence of each power defect based on the fused feature matrix.
[0086] For example, Figure 3 Schematic diagram of the target classification model provided in this application Figure 1 ,like Figure 3 As shown, the target classification model includes a first linear projection layer, an average pooling layer, a second linear projection layer, a max pooling layer, a third linear projection layer, a feature fusion layer, a first normalization layer, a fourth linear projection layer, a second normalization layer, an activation function layer, a classifier, and a third normalization layer.
[0087] The first feature matrix is obtained by reducing the dimensionality of the fused feature matrix through the first linear projection layer in the target classification model.
[0088] Dimensionality reduction of the fused feature matrix through the first linear projection layer is equivalent to using the formula Calculate the first characteristic matrix. Wherein, Denotes the first characteristic matrix. Represents the fused feature matrix. This represents the weight matrix in the first linear projection layer. This represents the bias term in the first linear projection layer. Dimensionality reduction refers to reducing the number of rows in the two-dimensional matrix corresponding to each power image to be detected.
[0089] By using the first linear projection layer, not only can dimensionality be reduced to decrease computation, but local, detailed features in images and text can also be preserved.
[0090] The first feature matrix is obtained by performing average pooling and dimensionality reduction on the fused feature matrix through the average pooling layer and the second linear projection layer in the target classification model.
[0091] In other words, the average pooling layer averages the value of each row in the two-dimensional matrix corresponding to each power image to be detected, replaces that row, and then inputs it into the second linear projection layer. The second linear projection layer uses the formula... Calculate the first characteristic matrix. Wherein, Denotes the first characteristic matrix. This represents the matrix after processing by the average pooling layer. This represents the weight matrix in the second linear projection layer. This represents the bias term in the second linear projection layer. Dimensionality reduction refers to reducing the number of rows in the two-dimensional matrix corresponding to each power image to be detected.
[0092] By using average pooling layers and a second linear projection layer, we can not only reduce dimensionality to reduce computation, but also preserve global features in images and text.
[0093] The first feature matrix is obtained by performing max pooling and dimensionality reduction on the fused feature matrix through the max pooling layer and the third linear projection layer in the target classification model.
[0094] In other words, the max pooling layer replaces the row with the maximum value in each row of the two-dimensional matrix corresponding to each power image to be detected, and then inputs it to the third linear projection layer. The third linear projection layer uses the formula... Calculate the first characteristic matrix. Wherein, Denotes the first characteristic matrix. This represents the matrix after processing by the max pooling layer. This represents the weight matrix in the third linear projection layer. This represents the bias term in the third linear projection layer. Dimensionality reduction refers to reducing the number of rows in the two-dimensional matrix corresponding to each power image to be detected.
[0095] By using average pooling layers and a third linear projection layer, we can not only reduce dimensionality to reduce computation, but also preserve significant and prominent features in images and text.
[0096] Multi-scale feature extraction processing can preserve multiple features of images and text, thereby improving detection accuracy.
[0097] S103: The second feature matrix is obtained by fusing all the first feature matrices through the target classification model.
[0098] In this step, after the computer obtains multiple first feature matrices, in order to facilitate subsequent processing, all the first feature matrices are fused through the feature fusion layer in the target classification model to obtain the second feature matrix.
[0099] It should be noted that the feature fusion layer can perform fusion in various ways, such as averaging, weighted summation, or attention mechanism fusion. This application does not limit the fusion method of the feature fusion layer, and it can be determined according to the actual situation.
[0100] S104: Using the target classification model, the second feature matrix is processed to extract discriminative features, resulting in the third feature matrix.
[0101] In this step, after the computer obtains the second feature matrix, in order to improve the correlation and matching degree between the features and power defects, the second feature matrix is processed by a target classification model to extract discriminative features, thereby obtaining the third feature matrix.
[0102] Specifically, the second feature matrix is normalized and numerically pruned through the first normalization layer in the target classification model to obtain the fourth feature matrix.
[0103] Layer normalization refers to standardizing the two-dimensional matrix corresponding to each power image to be detected in the third feature matrix.
[0104] For each element in the two-dimensional matrix corresponding to each power image to be detected, the formula is used. The standardized elements are calculated, and then the standardized elements are replaced. Represents the standardized elements. This indicates the element. This represents the mean of the two-dimensional matrix. This represents the variance of the two-dimensional matrix. This represents a preset constant. This represents the scaling parameter in the first normalization layer. This represents the offset parameter in the first normalization layer. The preset constant can be one part per hundred thousand, one part per million, one part per ten million, etc. This application embodiment does not limit the preset constant and can be determined according to the actual situation.
[0105] Numerical pruning refers to the following: for each element in the layer-normalized matrix, if the element is greater than or equal to the first threshold and less than or equal to the second threshold, the element is not updated; if the element is less than the first threshold, the element is updated to the first threshold; if the element is greater than the second threshold, the element is updated to the second threshold.
[0106] The second threshold is greater than the first threshold. The first threshold can be -9, -10, -11, etc., and the second threshold can be 9, 10, 11, etc. This application embodiment does not limit the first threshold and the second threshold, and can be determined according to the actual situation.
[0107] Layer normalization and numerical pruning can stabilize feature distribution and control feature range, thereby stabilizing the training process, accelerating convergence, and improving model stability.
[0108] After obtaining the fourth feature matrix, the fourth feature matrix is reduced in dimensionality through the fourth linear projection layer in the target classification model to obtain the fifth feature matrix.
[0109] Dimensionality reduction of the fourth feature matrix through the fourth linear projection layer is equivalent to using the formula Calculate the fifth characteristic matrix. Wherein, This represents the fifth characteristic matrix. Represents the fourth eigenmoment. This represents the weight matrix in the fourth linear projection layer. This represents the bias term in the fourth linear projection layer.
[0110] After obtaining the fifth feature matrix, the fifth feature matrix is subjected to layer normalization processing through the second normalization layer in the target classification model to obtain the sixth feature matrix.
[0111] Then, the sixth feature matrix is smoothed by gradient processing through the activation function layer in the target classification model to obtain the third feature matrix. In other words, the sixth feature matrix is substituted into the activation function to obtain the third feature matrix.
[0112] It should be noted that the activation function used in the activation function layer can be the Rectified Linear Unit (ReLU) activation function, the Gaussian Error Linear Unit (GELU) activation function, the Sigmoid Linear Unit (SiLU) activation function, etc. This application does not limit the activation function, and it can be determined according to the actual situation.
[0113] It should be noted that the target classification model may also include a dropout layer. After obtaining the third feature matrix, the third feature matrix can be randomly dropped through the dropout layer to update the third feature matrix and improve the robustness of the features.
[0114] The fourth linear projection layer reduces dimensionality to decrease computation; the second normalization layer stabilizes the feature distribution after linear transformation, reduces internal covariate shift, and prevents gradient vanishing or exploding; and the activation function layer enhances feature representation and improves the correlation and matching degree between features and power defects.
[0115] It should be noted that the fourth linear projection layer, the second normalization layer, and the activation function layer can form a discriminative feature extraction layer. The target classification model may include multiple discriminative feature extraction layers. The fourth feature matrix is processed by all discriminative feature extraction layers in sequence to obtain the third feature matrix, which can improve the correlation and matching degree between features and power defects.
[0116] S105: The third feature matrix is classified using a target classification model to obtain the probability of occurrence of each type of power defect.
[0117] In this step, after the computer obtains the third feature matrix, it uses a target classification model to classify the third feature matrix and obtain the probability of occurrence of each type of power defect in each power image to be detected.
[0118] Specifically, the third feature matrix is classified by the classifier in the target classification model to obtain the score of each power defect in each power image to be detected.
[0119] That is, the classifier uses the formula The score for each type of power defect in each power image to be detected is calculated. This represents a matrix representing the probability of occurrence of each type of power defect in each power image to be detected. Represents the third characteristic matrix. This represents the weight matrix in the classifier. This represents the bias term in the classifier. The order of each element in L can be used as an index for the power defect corresponding to that element, with each type of power defect corresponding to a separate index.
[0120] The computer then uses the third normalization layer in the target classification model to perform numerical cropping and normalization on the score of each power defect, thereby obtaining the occurrence probability corresponding to the index of each power defect, which is the probability of occurrence of each power defect.
[0121] For example, the data obtained through the third normalization layer in the target classification model is as follows:
[0122] 1:0.2, 2:0.3, 3:0.1.
[0123] In this diagram, 1, 2, and 3 are indices for power defects; 1 is the index for oil stains on transformer equipment, 2 is the index for bird nests, and 3 is the index for line damage. Therefore, the probability of oil stains on transformer equipment occurring is 0.2, the probability of bird nests occurring is 0.3, and the probability of line damage occurring is 0.1. This application does not limit the types of defects, the indices for power defects, or the probabilities of occurrence in this embodiment; these can be determined based on actual circumstances.
[0124] Numerical cropping refers to the following: for each power defect score in each power image to be detected, if the score is greater than or equal to the third threshold and less than or equal to the fourth threshold, the score is not updated; if the score is less than the third threshold, the score is updated to the third threshold; if the score is greater than the fourth threshold, the score is updated to the fourth threshold.
[0125] The fourth threshold is greater than the third threshold. The third threshold can be -4, -5, -6, etc., and the fourth threshold can be 4, 5, 6, etc. This application does not limit the third threshold and the fourth threshold, and they can be determined according to the actual situation.
[0126] Normalization is performed using the Softmax function.
[0127] The power defect detection method provided in this embodiment inputs the acquired power image to be detected and the preset defect detection prompt text into a fusion feature extraction model to obtain a fusion feature matrix. Then, a target classification model performs multi-scale feature extraction processing on the fusion feature matrix to obtain multiple first feature matrices; all first feature matrices are fused to obtain a second feature matrix; discriminative feature extraction is performed on the second feature matrix to obtain a third feature matrix; and the third feature matrix is classified to obtain the probability of occurrence of each type of power defect. This scheme improves the correlation and matching degree between features and power defects by processing the fusion features obtained from the fusion feature extraction model through a target classification model, thereby improving the accuracy of the probability of occurrence of power defects.
[0128] In addition, this solution uses a visual language model to extract features, which can process different images. It can also obtain good features for images with poor lighting or poor shooting angles, thus improving generalization and robustness.
[0129] Based on the above embodiments, the following describes, through Embodiment 2 of the power defect detection method provided in this application, the process of obtaining the third feature matrix by the computer, performing self-attention enhancement processing, and then classifying to obtain the probability of occurrence of each power defect.
[0130] For example, in Figure 3 On this basis, Figure 4 Schematic diagram of the target classification model provided in this application Figure 2 ,like Figure 4 As shown, the target classification model also includes a self-attention layer.
[0131] After the computer obtains the third feature matrix, the different features within the third feature matrix may not contribute equally to the classification task. To enable the model to dynamically and selectively focus on the most discriminative features, a self-attention enhancement process is applied to the third feature matrix through a self-attention layer in the target classification model, resulting in the seventh feature matrix.
[0132] Then, the seventh feature matrix is classified using a target classification model to obtain the probability of occurrence of each type of power defect.
[0133] It should be noted that the process of classifying the seventh feature matrix is similar to step S105 in Example 1, and will not be repeated here.
[0134] The power defect detection method provided in this embodiment improves detection accuracy by performing self-attention enhancement processing on the third feature matrix through a self-attention layer.
[0135] Based on the above embodiments, the following describes the process of obtaining the first feature matrix by performing column vector extraction processing and multi-scale feature extraction processing after the computer obtains the fused feature matrix, using the third embodiment of the power defect detection method provided in this application.
[0136] For example, in Figure 4 On this basis, Figure 5 Schematic diagram of the target classification model provided in this application Figure 3 ,like Figure 5 As shown, the target classification model also includes a dimensionality reduction layer.
[0137] After obtaining the fused feature matrix, the computer performs column vector extraction on the fused feature matrix through the dimensionality reduction layer in the target classification model to obtain the eighth feature matrix.
[0138] In other words, for each two-dimensional matrix corresponding to the power image to be detected in the fused feature matrix, each column of this two-dimensional matrix has a corresponding correlation coefficient. The correlation coefficient is used to characterize the degree of correlation between the feature of that column and the image and text. The larger the correlation coefficient, the stronger the correlation. The column with the largest correlation coefficient in each two-dimensional matrix is retained, and the other columns are removed to obtain the eighth feature matrix.
[0139] Then, through the target classification model, multi-scale feature extraction processing is performed on the eighth feature matrix to obtain multiple first feature matrices.
[0140] It should be noted that the process of multi-scale feature extraction of the eighth feature matrix is similar to step S102 in Example 1, and will not be repeated here.
[0141] The power defect detection method provided in this embodiment reduces dimensionality by extracting column vectors from the fused feature matrix. This not only reduces computation but also preserves the features most relevant to images and text.
[0142] Based on the above embodiments, the following describes the process of obtaining the eighth feature matrix by performing outlier replacement processing and column vector extraction processing after the computer obtains the fused feature matrix, using the power defect detection method embodiment four provided in this application.
[0143] For example, in Figure 5 On this basis, Figure 6 Schematic diagram of the target classification model provided in this application Figure 4 ,like Figure 6 As shown, the target classification model also includes an outlier handling layer.
[0144] After the computer obtains the fused feature vector, since there may be outliers in the fused feature vector, the outlier replacement process is performed on the fused feature matrix through the outlier processing layer in the target classification model to obtain the ninth feature matrix.
[0145] For each element in the fused feature matrix, if the element is null, replace it with the default value.
[0146] For each element in the fusion feature matrix, if the element is greater than or equal to the fifth threshold and less than or equal to the sixth threshold, the element is not updated; if the element is less than the fifth threshold, the element is updated to the fifth threshold; if the element is greater than the sixth threshold, the element is updated to the sixth threshold.
[0147] It should be noted that the default value can be 100, 150, 200, etc. This application embodiment does not limit the default value, and it can be determined according to the actual situation.
[0148] It should be noted that the sixth threshold is greater than the fifth threshold. The fifth threshold can be -180, -200, or -230, and the sixth threshold can be 180, 200, or 230, etc. This application does not limit the fifth and sixth thresholds, and they can be determined according to the actual situation.
[0149] Then, through the dimensionality reduction layer in the target classification model, column vector extraction is performed on the ninth feature matrix to obtain the eighth feature matrix.
[0150] It should be noted that the process of extracting column vectors from the ninth feature matrix is similar to the process of extracting column vectors from the fused feature matrix in Example 3, and will not be described again here.
[0151] The power defect detection method provided in this embodiment can improve detection accuracy by performing outlier replacement processing on the fused feature matrix.
[0152] Based on the above embodiments, the training process of the target classification model will be described below through Embodiment 5 of the power defect detection method provided in this application.
[0153] First, an initial visual language model is trained using the first training data to obtain a fusion feature extraction model. Each set of first training data includes an electrical image and defect prompt text.
[0154] It should be noted that the process of training the initial visual language model is the same as the training process of the visual language model in the existing technology, and will not be described again here.
[0155] Then, second training data is obtained, and each piece of second training data is sequentially input into the fusion feature extraction model to obtain third training data. The second training data includes power images and defect warning text. The third training data is a feature matrix.
[0156] The initial deep neural network model is then trained using the third training data to obtain the target classification model.
[0157] The architecture of the initial deep neural network model is the same as that of the target classification model in the above embodiments.
[0158] The process of training the initial deep neural network model is similar to the training process of deep neural network models in existing technologies, the difference being the process of calculating the loss value.
[0159] The true probability value corresponding to the third training data needs to be corrected because problems inevitably occur in the process of obtaining the true probability value, resulting in an abnormal true probability value.
[0160] For each type of power defect in each power image, use the formula The corrected probability value of the power defect in the power image is calculated. This represents the corrected probability value for the power defect in the power image. This represents the true probability value of the power defect in the power image. Indicates the number of different types of power defects. This represents the smoothing parameter.
[0161] It should be noted that the smoothing parameter can be 0.05, 0.1, 0.15, etc. This application embodiment does not limit the smoothing parameter, and it can be determined according to the actual situation.
[0162] Then, the corrected probability value of each power defect in each power image, and the occurrence probability of each power defect in each power image output by the model during this training process, are substituted into the loss function to obtain the loss value.
[0163] The power defect detection method provided in this embodiment first trains a fusion feature extraction model, and then keeps the fusion feature extraction model unchanged during the subsequent training of the target classification model. This reduces computational load and avoids overfitting of the fusion feature extraction model. By correcting the true probability values, the accuracy and generalization ability of the target classification model can be improved, and overfitting of the target classification model can be avoided.
[0164] Furthermore, the initial deep neural network model architecture incorporates multiple layers capable of dimensionality reduction, decomposing the detection process into several smooth and gentle subtasks. Gradients are effectively buffered and stabilized during stepwise backpropagation. Simultaneously, built-in layer normalization and numerical pruning mechanisms provide dual guarantees for numerical stability. This reduces training crashes, lowers sensitivity to hyperparameters, and improves robustness and reliability.
[0165] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0166] Figure 7 This is a schematic diagram of the structure of an embodiment of the power defect detection device provided in this application; as shown below. Figure 7 As shown, the power defect detection device 70 includes:
[0167] The first processing module 71 is used to input the acquired power image to be detected and the preset defect detection prompt text into the fusion feature extraction model to obtain the fusion feature matrix. The fusion feature extraction model is a pre-trained visual language model that obtains the feature matrix based on the image and text.
[0168] The second processing module 72 is used for:
[0169] By using a target classification model, multi-scale feature extraction is performed on the fused feature matrix to obtain multiple first feature matrices;
[0170] The second feature matrix is obtained by fusing all the first feature matrices using the target classification model.
[0171] The third feature matrix is obtained by performing discriminative feature extraction on the second feature matrix using a target classification model.
[0172] The third feature matrix is classified using a target classification model to obtain the probability of occurrence of each power defect. The target classification model is a pre-trained deep neural network model used to obtain the probability of occurrence of each power defect based on the fused feature matrix.
[0173] Furthermore, the second processing module 72 is specifically used for:
[0174] The first feature matrix is obtained by reducing the dimensionality of the fused feature matrix through the first linear projection layer in the target classification model.
[0175] The first feature matrix is obtained by performing average pooling and dimensionality reduction on the fused feature matrix through the average pooling layer and the second linear projection layer in the target classification model.
[0176] The first feature matrix is obtained by performing max pooling and dimensionality reduction on the fused feature matrix through the max pooling layer and the third linear projection layer in the target classification model.
[0177] Furthermore, the second processing module 72 is specifically used for:
[0178] The second feature matrix is obtained by performing layer normalization and numerical clipping on the first normalization layer in the target classification model;
[0179] The fourth feature matrix is reduced in dimensionality by using the fourth linear projection layer in the target classification model to obtain the fifth feature matrix;
[0180] The fifth feature matrix is normalized by the second normalization layer in the target classification model to obtain the sixth feature matrix.
[0181] The sixth feature matrix is smoothed by gradient processing through the activation function layer in the target classification model to obtain the third feature matrix.
[0182] Furthermore, the second processing module 72 is specifically used for:
[0183] The third feature matrix is classified using the classifier in the target classification model to obtain a score for each type of power defect.
[0184] By using the third normalization layer in the target classification model, the scores of each power defect are numerically clipped and normalized to obtain the probability of occurrence of each power defect.
[0185] Furthermore, before classifying the third feature matrix using the target classification model to obtain the probability of occurrence for each type of power defect, the second processing module 72 is also used for:
[0186] By using the self-attention layer in the target classification model, the third feature matrix is enhanced with self-attention to obtain the seventh feature matrix;
[0187] The second processing module 72 is specifically used for:
[0188] The seventh feature matrix is classified using a target classification model to obtain the probability of occurrence for each type of power defect.
[0189] Furthermore, before obtaining multiple first feature matrices by performing multi-scale feature extraction on the fused feature matrix through the target classification model, the second processing module 72 is also used for:
[0190] The eighth feature matrix is obtained by extracting column vectors from the fusion feature matrix through the dimensionality reduction layer in the target classification model.
[0191] The second processing module 72 is specifically used for:
[0192] By using a target classification model, multi-scale feature extraction is performed on the eighth feature matrix to obtain multiple first feature matrices.
[0193] Furthermore, before obtaining the eighth feature matrix by extracting column vectors from the fused feature matrix through the dimensionality reduction layer in the target classification model, the second processing module 72 is also used for:
[0194] The outlier handling layer in the target classification model is used to perform outlier replacement on the fused feature matrix to obtain the ninth feature matrix.
[0195] The second processing module 72 is specifically used for:
[0196] The ninth feature matrix is processed by extracting column vectors through the dimensionality reduction layer in the target classification model to obtain the eighth feature matrix.
[0197] The power defect detection device provided in this embodiment is used to execute the technical solution in any of the aforementioned method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.
[0198] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application. Figure 8 As shown, the electronic device 80 includes:
[0199] Processor 81, memory 82, and communication interface 83;
[0200] Memory 82 is used to store executable instructions of processor 81;
[0201] The processor 81 is configured to execute the technical solutions in any of the foregoing method embodiments by executing executable instructions.
[0202] Optionally, the memory 82 can be either standalone or integrated with the processor 81.
[0203] Optionally, when the memory 82 is a device independent of the processor 81, the electronic device 80 may further include:
[0204] Bus 84, memory 82 and communication interface 83 are connected to processor 81 through bus 84 and complete communication with each other. Communication interface 83 is used to communicate with other devices.
[0205] Optionally, the communication interface 83 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk drive.
[0206] Bus 84 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus.
[0207] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0208] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.
[0209] This application also provides a readable storage medium storing a computer program thereon, which, when executed by a processor, implements the technical solutions provided in any of the foregoing method embodiments.
[0210] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the technical solutions provided in any of the foregoing method embodiments.
[0211] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0212] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A method for detecting power defects, characterized in that, include: The acquired power image to be detected and the preset defect detection prompt text are input into the fusion feature extraction model to obtain the fusion feature matrix. The fusion feature extraction model is a pre-trained visual language model that obtains the feature matrix based on the image and text. By using a target classification model, multi-scale feature extraction processing is performed on the fused feature matrix to obtain multiple first feature matrices; The second feature matrix is obtained by fusing all the first feature matrices using the target classification model. The second feature matrix is subjected to discriminative feature extraction processing using the target classification model to obtain the third feature matrix; The target classification model is used to classify the third feature matrix to obtain the probability of occurrence of each power defect. The target classification model is a pre-trained deep neural network model used to obtain the probability of occurrence of each power defect based on the fused feature matrix.
2. The method according to claim 1, characterized in that, The process involves performing multi-scale feature extraction on the fused feature matrix using a target classification model to obtain multiple first feature matrices, including: The dimensionality of the fused feature matrix is reduced by the first linear projection layer in the target classification model to obtain the first feature matrix; The first feature matrix is obtained by performing average pooling and dimensionality reduction on the fused feature matrix through the average pooling layer and the second linear projection layer in the target classification model. The first feature matrix is obtained by performing max pooling and dimensionality reduction on the fused feature matrix through the max pooling layer and the third linear projection layer in the target classification model.
3. The method according to claim 1, characterized in that, The step of performing discriminative feature extraction processing on the second feature matrix using the target classification model to obtain the third feature matrix includes: The second feature matrix is subjected to layer normalization and numerical clipping through the first normalization layer in the target classification model to obtain the fourth feature matrix. The fourth feature matrix is reduced in dimensionality by using the fourth linear projection layer in the target classification model to obtain the fifth feature matrix; The fifth feature matrix is normalized by the second normalization layer in the target classification model to obtain the sixth feature matrix. The sixth feature matrix is smoothed by gradient processing through the activation function layer in the target classification model to obtain the third feature matrix.
4. The method according to claim 1, characterized in that, The process of classifying the third feature matrix using the target classification model to obtain the probability of occurrence for each type of power defect includes: The third feature matrix is classified using the classifier in the target classification model to obtain a score for each type of power defect. The third normalization layer in the target classification model is used to perform numerical cropping and normalization on the score of each power defect to obtain the probability of occurrence of each power defect.
5. The method according to any one of claims 1 to 4, characterized in that, Before classifying the third feature matrix using the target classification model to obtain the probability of occurrence of each type of power defect, the method further includes: The third feature matrix is enhanced by self-attention layer in the target classification model to obtain the seventh feature matrix. The process of classifying the third feature matrix using the target classification model to obtain the probability of occurrence of each type of power defect includes: The seventh feature matrix is classified using the target classification model to obtain the probability of occurrence of each type of power defect.
6. The method according to claim 5, characterized in that, Before performing multi-scale feature extraction processing on the fused feature matrix using the target classification model to obtain multiple first feature matrices, the method further includes: The eighth feature matrix is obtained by extracting column vectors from the fusion feature matrix through the dimensionality reduction layer in the target classification model. The process involves performing multi-scale feature extraction on the fused feature matrix using a target classification model to obtain multiple first feature matrices, including: The target classification model is used to perform multi-scale feature extraction processing on the eighth feature matrix to obtain multiple first feature matrices.
7. The method according to claim 6, characterized in that, Before extracting column vectors from the fused feature matrix through the dimensionality reduction layer in the target classification model to obtain the eighth feature matrix, the method further includes: The outlier processing layer in the target classification model performs outlier replacement processing on the fused feature matrix to obtain the ninth feature matrix. The eighth feature matrix is obtained by extracting column vectors from the fusion feature matrix through the dimensionality reduction layer in the target classification model, including: The ninth feature matrix is processed by extracting column vectors through the dimensionality reduction layer in the target classification model to obtain the eighth feature matrix.
8. A power defect detection device, characterized in that, include: The first processing module is used to input the acquired power image to be detected and the preset defect detection prompt text into the fusion feature extraction model to obtain the fusion feature matrix. The fusion feature extraction model is a pre-trained visual language model that obtains the feature matrix based on the image and text. The second processing module is used for: By using a target classification model, multi-scale feature extraction processing is performed on the fused feature matrix to obtain multiple first feature matrices; The second feature matrix is obtained by fusing all the first feature matrices using the target classification model. The second feature matrix is subjected to discriminative feature extraction processing using the target classification model to obtain the third feature matrix; The target classification model is used to classify the third feature matrix to obtain the probability of occurrence of each power defect. The target classification model is a pre-trained deep neural network model used to obtain the probability of occurrence of each power defect based on the fused feature matrix.
9. An electronic device, characterized in that, include: Processor, memory, communication interface; The memory is used to store the executable instructions of the processor; The processor is configured to execute the power defect detection method according to any one of claims 1 to 7 by executing the executable instructions.
10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the power defect detection method according to any one of claims 1 to 7.
11. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, is used to implement the power defect detection method according to any one of claims 1 to 7.