A method for identifying defects in power components based on machine vision
The hyperspectral denoising module solves the problem of noise interference in hyperspectral images, enabling efficient and accurate identification of defects in power components, and improving detection accuracy and model robustness.
Patent Information
- Application Number
- CN202410890674.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-04
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-04
AI Technical Summary
Traditional power equipment defect detection relies on manual visual inspection, which has problems such as low identification efficiency, strong subjectivity, and high cost. Moreover, hyperspectral images are easily affected by noise, which affects the detection accuracy and efficiency.
A hyperspectral denoising module is employed, which includes a spectral spatial convolution module, a guided spectral module, and a pre-modulation feedback network. This module is used to extract spatial spectral correlation features, perform adaptive processing and information aggregation, and enhance image clarity and feature recognition.
It improves the accuracy and efficiency of power component defect identification, removes hyperspectral image noise, and enhances the accuracy of target identification and the robustness of the model.
Smart Images

Figure CN118608517B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of computer vision, and in particular relates to a method for identifying defects of electric components based on machine vision. Background Art
[0002] Machine vision-based power component defect recognition is the use of machine vision technology to automatically detect and identify defects in power equipment. In the power system, various power equipment such as transformers, switchgear, transmission lines, etc. may have various types of defects, which may lead to equipment performance degradation, safety hazards, and downtime for maintenance. Therefore, early detection and identification of these defects is crucial to maintaining the stable operation of the power system. Traditional power equipment defect detection usually relies on manual visual inspection, which has problems such as low recognition efficiency, strong subjectivity, and high cost. The development of machine vision technology provides an automated, efficient and accurate solution for power component defect recognition. Through machine vision technology, cameras or other imaging devices can be used to monitor and image power equipment in real time, and then machine vision algorithms can be used to analyze and process the images, thereby realizing automatic detection and recognition of power component defects.
[0003] Using hyperspectral images instead of ordinary images can bring richer information to machine vision, because hyperspectral images collect a large amount of spectral information of different wavelengths at each pixel. Such data provides more detailed and accurate object features, which helps to improve the accuracy of object recognition and classification, as well as more detailed surface analysis and object composition detection. In addition, hyperspectral images can also help distinguish different materials and substances, providing more comprehensive scene understanding and environmental monitoring.
[0004] Hyperspectral images are often affected by noise from multiple sources, such as sensor noise, atmospheric noise, ambient light interference, etc. These noises appear as random or periodic changes in the image, reducing the quality and clarity of the image. In machine vision detection, noise can interfere with the boundaries, textures, and detail information of the target, making tasks such as target detection, classification, and segmentation more difficult and unreliable. Therefore, noise removal is crucial for the processing of hyperspectral images and machine vision detection. Noise removal can improve the signal-to-noise ratio of the image, enhance the contrast between the target and the background, make the target more prominent and clear, and thus help the machine vision system to detect and identify the target more accurately. In addition, noise removal can also help reduce the complexity of subsequent image processing and analysis, improve the training effect of machine learning and deep learning models, and enhance the robustness and generalization ability of the model. Summary of the invention
[0005] The present invention provides a method for identifying defects in power components based on machine vision, aiming to propose a hyperspectral denoising module, which is applied to the defect identification scenario of power components. The method comprises a spectral space convolution module, a guided spectrum module and a modulation front feedback network, wherein the spectral space convolution module, as a lightweight alternative to 3D convolution, can extract more spatial spectrum related features; the guided spectrum module can perform adaptive processing, perform self-attention on the entire spectral band, has the ability to identify global spectral correlation, and maintains linear complexity; the modulation front feedback network can strengthen the activation of more information areas, thereby further enhancing the aggregated features.
[0006] The present invention aims to propose a hyperspectral denoising module and provide a method for identifying defects in power components based on machine vision, comprising the following steps:
[0007] S1. Obtain defect images of power components. The acquisition method is to use a hyperspectral camera to take pictures on site. The shooting content is the defect area of the power component. All the acquired defect images of power components are hyperspectral images, and contain 4 types of defects, including insulator defects, transmission line defects, power transformer defects and switchgear defects.
[0008] S2. Create a power component defect training data set, annotate all collected power component defect images, annotate four defect types, divide the annotated images into a training set, a validation set, and a test set, and use the training set and the validation set as the power component defect training data set;
[0009] S3, constructing a spectrum space convolution module SSC, including a spatial branch and a spectral branch;
[0010] S4, construct the guided spectrum module GS, which includes global average pooling, linear projection and matrix multiplication;
[0011] S5, constructing a modulation front feedback network MFFN, including linear projection, activation function, element-by-element addition and element-by-element multiplication;
[0012] S6, constructing a transformer block, including a spectral spatial convolution module SSC, a guided spectrum module GS and a modulation front feedback network MFFN;
[0013] S7, build a hyperspectral denoising module, which includes multiple transformer blocks, skip connections, residual connections and 3D convolutions;
[0014] S8. Build and train a machine vision power component defect recognition model, including an input, a hyperspectral denoising module, a backbone network, a detection head, and an output, and use a power component defect training data set to train the machine vision power component defect recognition model;
[0015] S9, power component defect image recognition, using a hyperspectral shooting device to photograph the defective area of the power component to obtain a power component defect image. The power component defect images are all hyperspectral images. A machine vision power component defect recognition model is used to recognize the power component defect image to obtain a recognition result.
[0016] Preferably, in step S1, for the four types of defects, insulator defects include cracks, stains, surface damage and coating omissions; transmission line defects include transmission line breakage, transmission line corrosion and tower tilting; power transformer defects include loose terminal blocks, cooler blockage and transformer oil leakage; switchgear defects include contact wear, deformation and breakage.
[0017] Preferably, in step S3, for the spectral spatial convolution module SSC, input X to two parallel branches, namely a spatial branch and a spectral branch, In the spatial dimension, the spatial branch uses multiple spatial blocks to extract features. A single spatial block consists of 2D convolution and Leaky ReLU. In the spectral dimension, the spectral branch applies 1×1 convolution to associate the spectral information of all spaces. The outputs of the two branches are combined by element-wise addition, and the output of SSC is obtained. .
[0018] Preferably, in step S4, for the guided spectrum module GS, input , , along the input Perform global average pooling on the spatial dimension of and , , ,right Perform linear projection to obtain , ,Will and Perform matrix multiplication and softmax normalization on the transposed matrix to obtain the transposed attention map , ,Will and Perform matrix multiplication to get output , ,Will After linear projection and input Add element by element to get the output of GS , , the calculation formula of the guided spectrum module GS is expressed as follows: , , is element-wise addition, is matrix multiplication, , Linear projection implemented for linear layers, is the softmax function.
[0019] Preferably, in step S5, for the modulation front feedback network MFFN, the input , , output , , , N is the product of H, W and D, where is element-wise addition, , is a linear projection, , , N is the product of H, W and D, is the GELU activation function, , ;in First calculate , is a linear projection, , , N is the product of H, W and D, then Split into and , , , , , is element-wise multiplication, is the sigmoid function.
[0020] Preferably, in step S6, for the converter block, given the input ,have , , where the input , For element-by-element addition, the intermediate output , the final output , and Indicates width and height, represents the number of spectral bands, represents the number of feature maps, and Indicates the width and height of the final output, Indicates the number of spectral bands in the final output, Represents the number of feature maps of the final output, represents batch normalization, represents the spectral space convolution module, represents the guided spectrum module, represents the pre-modulation feedback network.
[0021] Preferably, in step S7, for the hyperspectral denoising module, the overall architecture is a U-shaped encoder and decoder structure with jump connections. Given an input hyperspectral band noisy image, the hyperspectral band noisy image is passed through multiple transformer blocks, and jump connections are used in each level of transformer. Residual connections are used to add the input noise image element by element before the final output to obtain the final output.
[0022] Compared with the prior art, the present invention has the following technical effects:
[0023] The technical solution provided by the present invention proposes a hyperspectral denoising module, which is applied to the defect recognition scenario of power components. The hyperspectral power component defect image is conducive to the accurate recognition of power component defects. The hyperspectral denoising module can further remove noise in the hyperspectral image, thereby improving the power component defect recognition effect. The hyperspectral denoising module includes a spectral space convolution module, a guided spectrum module and a modulation front feedback network. The spectral space convolution module is a lightweight alternative to 3D convolution and can extract more spatial spectrum related features; the guided spectrum module can perform adaptive processing and perform self-attention on the entire spectral band. It has the ability to identify global spectral correlation and maintains linear complexity; the modulation front feedback network can strengthen the activation of more information areas, thereby further enhancing the aggregation features. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a flow chart of electric component defect identification provided by the present invention.
[0025] Figure 2 It is a structural diagram of the spectral space convolution module SSC provided by the present invention.
[0026] Figure 3 It is a structural diagram of the guided spectrum module GS provided by the present invention.
[0027] Figure 4 It is a structural diagram of the modulation front feedback network MFFN provided by the present invention.
[0028] Figure 5 This is a block diagram of the converter provided by the present invention.
[0029] Figure 6 This is a structural diagram of the hyperspectral denoising module provided by the present invention. DETAILED DESCRIPTION
[0030] The present invention provides a method for identifying defects in power components based on machine vision, aiming to propose a hyperspectral denoising module, which is applied to the defect identification scenario of power components. The method comprises a spectral space convolution module, a guided spectrum module and a modulation front feedback network, wherein the spectral space convolution module, as a lightweight alternative to 3D convolution, can extract more spatial spectrum related features; the guided spectrum module can perform adaptive processing, perform self-attention on the entire spectral band, has the ability to identify global spectral correlation, and maintains linear complexity; the modulation front feedback network can strengthen the activation of more information areas, thereby further enhancing the aggregated features.
[0031] See also Figure 1 As shown, a method for identifying defects in power components based on machine vision in an embodiment of the present application:
[0032] S1. Obtain defect images of power components. The acquisition method is to use a hyperspectral camera for on-site photography. The photographed content is the defect area of the power component. All acquired defect images of power components are hyperspectral images, and contain 4 types of defects, including insulator defects, transmission line defects, power transformer defects and switchgear defects. The number of all acquired defect images of power components is 5000;
[0033] S2. Create a power component defect training data set, annotate all collected power component defect images, and annotate four defect types. Divide the annotated images into a training set, a validation set, and a test set. The training set has 3,500 images, the validation set has 1,000 images, and the test set has 500 images. The training set and the validation set are used as the power component defect training data set.
[0034] S3, constructing a spectrum space convolution module SSC, including a spatial branch and a spectral branch;
[0035] S4, construct the guided spectrum module GS, which includes global average pooling, linear projection and matrix multiplication;
[0036] S5, constructing a modulation front feedback network MFFN, including linear projection, activation function, element-by-element addition and element-by-element multiplication;
[0037] S6, constructing a transformer block, including a spectral spatial convolution module SSC, a guided spectrum module GS and a modulation front feedback network MFFN;
[0038] S7, build a hyperspectral denoising module, which includes 7 transformer blocks, 4 skip connections, 1 residual connection and 1 3D convolution;
[0039] S8. Build and train a machine vision power component defect recognition model, including input, hyperspectral denoising module, ResNet backbone network, ResNet detection head and output. Use the power component defect training dataset to train the machine vision power component defect recognition model. The number of training rounds is 150 and the batch size is 32.
[0040] S9, power component defect image recognition, using a hyperspectral shooting device to photograph the defective area of the power component to obtain a power component defect image. The power component defect images are all hyperspectral images. A machine vision power component defect recognition model is used to recognize the power component defect image to obtain a recognition result.
[0041] Furthermore, in step S1, for the four types of defects, insulator defects include cracks, stains, surface damage and coating omissions; transmission line defects include transmission line breakage, transmission line corrosion and tower tilt; power transformer defects include loose terminal blocks, cooler blockage and transformer oil leakage; switchgear defects include contact wear, deformation and breakage.
[0042] Further, in step S3, for the spectral space convolution module SSC, its structure is as follows Figure 2 As shown, the input X is sent to two parallel branches, namely the spatial branch and the spectral branch. In the spatial dimension, the spatial branch uses multiple spatial blocks to extract features. A single spatial block consists of 2D convolution and Leaky ReLU. In the spectral dimension, the spectral branch applies 1×1 convolution to associate the spectral information of all spaces. Figure 2 In It represents element-by-element addition, which combines the outputs of the two branches by element-by-element addition and obtains the output of SSC. .
[0043] Further, in step S4, for the guided spectrum module GS, its structure is as follows Figure 3 As shown, Figure 3 In represents element-by-element addition, Figure 3 In represents matrix multiplication, Figure 3 The pooling in represents global average pooling, input , , along the input Perform global average pooling on the spatial dimension of and , , ,right Perform linear projection to obtain , the linear projection is through Figure 3 The linear layer implementation in , ,Will and Perform matrix multiplication and softmax normalization on the transposed matrix to obtain the transposed attention map , ,Will and Perform matrix multiplication to get output , ,Will After linear projection and input Add element by element to get the output of GS , , the calculation formula of the guided spectrum module GS is expressed as follows: , , is element-wise addition, and Figure 3 In correspond, is matrix multiplication, and Figure 3 Two of correspond, , Linear projection implemented for linear layers, is the softmax function.
[0044] Further, in step S5, for the modulation front feedback network MFFN, its structure is as follows Figure 4 As shown, Figure 4 In represents element-by-element addition, Figure 4 In represents element-wise multiplication, Figure 4 In Represents split, input , , output , , , N is the product of H, W and D, where is element-wise addition, and Figure 4 In correspond, , is a linear projection, , , N is the product of H, W and D, is the GELU activation function, , ;in First calculate , is a linear projection, , , N is the product of H, W and D, then Split into and , split Figure 4 In , , , , , is element-wise multiplication, and Figure 4 In correspond, is the sigmoid function.
[0045] Further, in step S6, for the converter block, its structure is as follows Figure 5 As shown, Figure 5 In Represents element-by-element addition, given input ,have , , where the input , For element-by-element addition, the intermediate output , the final output , and Indicates width and height, represents the number of spectral bands, represents the number of feature maps, and Indicates the width and height of the final output, Indicates the number of spectral bands in the final output, Represents the number of feature maps of the final output, represents batch normalization, represents the spectral space convolution module, represents the guided spectrum module, represents the pre-modulation feedback network.
[0046] Further, in step S7, for the hyperspectral denoising module, its structure is as follows Figure 6 As shown, the overall architecture is a U-shaped encoder and decoder structure with skip connections. Given an input hyperspectral band noisy image, the hyperspectral band noisy image is passed through multiple transformer blocks, and skip connections are adopted in each level of transformer. The input hyperspectral band noisy image is added element by element before the final output using residual connections to obtain the final output.
[0047] Further, in step S7, for the hyperspectral denoising module, its structure is as follows Figure 6 As shown, 7 transformer blocks are used. For the input feature X, , transformer block 1 outputs features , transformer block 2 outputs features , transformer block 3 outputs features , transformer block 4 outputs features , transformer block 5 outputs features , transformer block 6 outputs the feature , transformer block 7 outputs the feature , After 3D convolution, we get , , Through the residual connection and input X, add element by element to the final output , , It represents element-by-element addition. At the same time, transformer block 1 is connected to 3D convolution using a jump connection, transformer block 2 is connected to transformer block 7 using a jump connection, transformer block 3 is connected to transformer block 6 using a jump connection, and transformer block 4 is connected to transformer block 5 using a jump connection.
[0048] Further, in step S7, for the hyperspectral denoising module, its structure is as follows Figure 6 As shown, using 7 transformer blocks, given an input hyperspectral band noisy image X, , 640 and 640 represent width and height, 200 represents the number of spectral bands, 3 represents the number of feature maps, and transformer block 1 outputs feature , transformer block 2 outputs features , transformer block 3 outputs features , transformer block 4 outputs features , transformer block 5 outputs features , transformer block 6 outputs the feature , transformer block 7 outputs the feature , After 3D convolution, we get , , Through the residual connection and input X, add element by element to the final output , .
[0049] Further, in step S3, for converter block 2, input to the spectral spatial convolution module SSC in transformer block 2, , the spatial branch uses N spatial blocks to extract features. In this embodiment, N is 3. Input to two parallel branches and get the output of SSC , ,Will Input to the guided spectrum module GS in the converter block 2, and get the output of the guided spectrum module GS , ,Will Input to the modulation front feedback network MFFN in the converter block 2, and obtain the output of the modulation front feedback network MFFN , , where 20480000 is the product of 320, 320, and 200. Adjust the dimensions to ,Then It will be used as input to converter block 3.
[0050] The above are only preferred embodiments of the present invention. It should be pointed out that a person skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.
Claims
1. A method for identifying defects in power components based on machine vision, characterized in that: The following steps are involved: S1. Obtain defect images of power components. The acquisition method is to use a hyperspectral camera to take pictures on site. The shooting content is the defect area of the power component. All the acquired defect images of power components are hyperspectral images, and contain 4 types of defects, including insulator defects, transmission line defects, power transformer defects and switchgear defects. S2. Create a power component defect training data set, annotate all collected power component defect images, annotate four defect types, divide the annotated images into a training set, a validation set, and a test set, and use the training set and the validation set as the power component defect training data set; S3, construct the spectrum space convolution module SSC, including the spatial branch and the spectral branch; specifically: input X to two parallel branches, namely the spatial branch and the spectral branch, In the spatial dimension, the spatial branch uses multiple spatial blocks to extract features. A single spatial block consists of 2D convolution and Leaky ReLU. In the spectral dimension, the spectral branch applies 1×1 convolution to associate the spectral information of all spaces. The outputs of the two branches are combined by element-wise addition, and the output of SSC is obtained. ; S4, construct the guided spectrum module GS, which includes global average pooling, linear projection and matrix multiplication; specifically: input , , along the input Perform global average pooling on the spatial dimension of and , , ,right Perform linear projection to obtain , ,Will and Perform matrix multiplication and softmax normalization on the transposed matrix to obtain the transposed attention map , ,Will and Perform matrix multiplication to get output , ,Will After linear projection and input Add element by element to get the output of GS , , the calculation formula of the guided spectrum module GS is expressed as follows: , , is element-wise addition, is matrix multiplication, , Linear projection implemented for linear layers, is the softmax function; S5. Construct the modulation front feedback network MFFN, including linear projection, activation function, element-by-element addition and element-by-element multiplication; specifically: input , , output , , , N is the product of H, W and D, where is element-wise addition, , is a linear projection, , , N is the product of H, W and D, is the GELU activation function, , ;in First calculate , is a linear projection, , , N is the product of H, W and D, then Split into and , , , , , is element-wise multiplication, is the sigmoid function; S6, constructing a transformer block, including a spectral spatial convolution module SSC, a guided spectrum module GS and a modulation front feedback network MFFN; S7, build a hyperspectral denoising module, which includes multiple transformer blocks, skip connections, residual connections and 3D convolutions; S8. Build and train a machine vision power component defect recognition model, including an input, a hyperspectral denoising module, a backbone network, a detection head, and an output, and use a power component defect training data set to train the machine vision power component defect recognition model; S9, power component defect image recognition, using a hyperspectral shooting device to photograph the defective area of the power component to obtain a power component defect image. The power component defect images are all hyperspectral images. A machine vision power component defect recognition model is used to recognize the power component defect image to obtain a recognition result.
2. The method for identifying defects of power components based on machine vision according to claim 1, characterized in that: In step S1, for the four types of defects, insulator defects include cracks, stains, surface damage and coating omissions; transmission line defects include transmission line breakage, transmission line corrosion and tower tilt; power transformer defects include loose terminals, cooler blockage and transformer oil leakage; switchgear defects include contact wear, deformation and breakage.
3. The method for identifying defects of power components based on machine vision according to claim 1, characterized in that: In step S6, for the transformer block, given the input ,have , , where the input , For element-by-element addition, the intermediate output , the final output , and Indicates width and height, represents the number of spectral bands, represents the number of feature maps, and Indicates the width and height of the final output, Indicates the number of spectral bands in the final output, Represents the number of feature maps of the final output, represents batch normalization, represents the spectral space convolution module, represents the guided spectrum module, represents the pre-modulation feedback network.
4. The method for identifying defects of power components based on machine vision according to claim 1, characterized in that: In step S7, for the hyperspectral denoising module, the overall architecture is a U-shaped encoder and decoder structure with skip connections. Given an input hyperspectral band noisy image, the hyperspectral band noisy image is passed through multiple transformer blocks, and skip connections are used in each level of transformer. The input noisy image is added element by element before the final output using residual connections to obtain the final output.
