Power equipment surface defect detection method and system based on image recognition

Abstract

The invention provides a power equipment surface defect detection method and system based on image recognition. The method comprises the following steps: acquiring image data of the surface of power equipment; extracting depth information of the surface of the power equipment from the image data; based on a texture analysis algorithm, extracting texture information of the surface of the power equipment from the image data; performing feature fusion on the depth information and the texture information to obtain a comprehensive feature vector; the comprehensive feature vectors are input into a preset defect detection model, so that the defect detection model classifies the comprehensive feature vectors to generate a plurality of defect types and a plurality of defect positions, and the defect detection model is constructed based on a deep learning model and is obtained through training of a plurality of historical image data; and according to the plurality of defect types and the plurality of defect positions, performing corresponding labeling in the image data to generate defect detection image data, thereby improving the efficiency and accuracy of surface defect detection of the power equipment.

Description

Technical Field

[0001] This application relates to the fields of image recognition and machine learning technology, and in particular to a method and system for detecting surface defects in power equipment based on image recognition. Background Technology

[0002] Currently, the mainstream technologies for surface defect detection in power equipment still rely on manual inspection and single-modal automated inspection. Manual inspection suffers from low efficiency and insufficient coverage, and the results are easily affected by personnel experience and environmental interference. Among automated inspection methods, infrared thermal imaging technology can identify temperature-related defects, but it lacks sensitivity to non-thermal features such as surface corrosion and cracks. While depth information extraction methods based on 3D reconstruction can capture macroscopic deformations, they struggle to identify microscopic texture changes. In recent years, although deep learning technology has made progress in the field of target detection, single-modal models often suffer from incomplete feature representation when handling multiple types and scales of defects, leading to false positives or false negatives. Furthermore, they exhibit poor robustness in environments with complex lighting and dirt occlusion. For example, while lightweight models relied upon for drone inspection have improved detection speed, they face bottlenecks such as inefficient feature fusion strategies and insufficient cross-scene generalization capabilities.

[0003] The limitations of existing research mainly lie in the insufficient collaborative analysis of multimodal data. Traditional fusion methods ignore the nonlinear correlation between depth and texture information, leading to redundant fusion features or loss of key information. In addition, power equipment defects are diverse, and a single modality cannot fully cover the correlation between their three-dimensional deformation and microstructure, resulting in low detection accuracy. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a method and system for detecting surface defects in power equipment based on image recognition, thereby improving the efficiency and accuracy of surface defect detection in power equipment.

[0005] In a first aspect, embodiments of this application provide a method for detecting surface defects in power equipment based on image recognition, including:

[0006] Acquire image data of the surface of electrical equipment;

[0007] Extract depth information of the surface of the power equipment from the image data;

[0008] Based on texture analysis algorithms, texture information of the surface of the power equipment is extracted from the image data;

[0009] The depth information and the texture information are fused to obtain a comprehensive feature vector;

[0010] The comprehensive feature vector is input into a preset defect detection model so that the defect detection model classifies the comprehensive feature vector and generates several defect types and several defect locations.

[0011] Based on the aforementioned defect types and defect locations, corresponding annotations are made in the image data to generate defect detection image data.

[0012] The defect detection model is built based on a deep learning model and trained using several historical image data.

[0013] This application provides a method for detecting surface defects in power equipment based on image recognition. It generates a comprehensive feature vector by fusing depth and texture information, classifies defect types and locations using a deep learning model, and finally labels each defect type at its corresponding location in the image data, outputting visualized defect detection image data. Compared to single-modal recognition in existing technologies, this application simultaneously extracts depth and texture information from the same image data for defect detection. Depth information captures macroscopic deformation, while texture information identifies microscopic details. The two complement each other, enhancing feature representation capabilities and overcoming the limitations of traditional single-modal recognition in terms of sensitivity to multiple defect types, effectively improving the accuracy of surface defect detection in power equipment. Furthermore, this method enables automated identification and labeling, reducing manual identification costs and improving defect detection efficiency. In addition, the final output of labeled defect detection image data allows surface defects in power equipment to be visually displayed in the image, facilitating rapid defect location by users and providing data support for subsequent maintenance work.

[0014] Furthermore, acquiring image data of the surface of the power equipment includes:

[0015] Raw image data of the surface of the power equipment is acquired by a camera or laser scanner at a preset location;

[0016] The original image data is subjected to image denoising, image contrast adjustment, and image brightness adjustment in sequence to obtain the image data.

[0017] This application provides a method for acquiring image data. It utilizes a camera or laser scanner at a preset location to acquire raw image data and preprocesses the raw image data to generate high-quality image data. The camera should have sufficient resolution to capture minute details on the device surface, while the laser scanner provides more accurate three-dimensional morphological information. Preprocessing the acquired image data to generate high-quality image data effectively reduces noise interference, enhances the distinction between defect areas and the background, provides reliable input for subsequent feature extraction, and makes the subsequent processing more efficient and stable, thus contributing to improved accuracy in defect detection.

[0018] In one possible implementation, extracting depth information of the surface of the power equipment from the image data includes:

[0019] Based on 3D reconstruction technology, a 3D model of the surface of the power equipment is generated from the image data;

[0020] Depth information of the surface of the power equipment is extracted from the three-dimensional model.

[0021] This application provides a method for obtaining depth information. By generating a three-dimensional model and extracting depth information through three-dimensional reconstruction technology, a high-precision three-dimensional representation can be obtained, accurately restoring the geometric structure of the surface of power equipment. This helps to identify defects caused by surface depressions, protrusions, etc., and improves the comprehensiveness and accuracy of defect detection.

[0022] In one possible implementation, extracting depth information of the surface of the power equipment from the image data includes:

[0023] The image data is input into a preset deep learning model so that the deep learning model can predict and generate depth information of the surface of the power equipment based on the image features in the image data.

[0024] This application provides another method for obtaining depth information. It directly predicts and generates corresponding depth information based on image features in image data using a pre-defined deep learning model. This avoids complex 3D reconstruction calculations, reduces hardware dependence and processing time, and makes depth information extraction more automated and intelligent, reducing manual intervention and improving the efficiency of surface defect detection in power equipment. Furthermore, the deep learning model can adaptively learn the depth features of different devices, making it suitable for diverse defect detection scenarios.

[0025] Furthermore, the step of fusing the depth information and the texture information to obtain a comprehensive feature vector includes:

[0026] Based on the spatial position in the image data, the depth information and the texture information are spatially aligned to obtain aligned depth information and aligned texture information;

[0027] The initial depth feature vector is extracted from the aligned depth information using the feature descriptor of the 3D point cloud, and the initial depth feature vector is converted into a depth feature vector of a preset length.

[0028] An initial texture feature vector is extracted from the aligned texture information based on a texture analysis algorithm, and the initial texture feature vector is converted into a texture feature vector of a preset length.

[0029] The depth feature vector and the texture feature vector are weighted and fused according to preset weight values ​​to obtain the comprehensive feature vector.

[0030] This application provides a feature fusion method. First, spatial alignment ensures a one-to-one correspondence between depth and texture information in spatial location, facilitating accurate fusion later. Then, the extracted features are converted into fixed-length feature vectors to eliminate dimensional differences between different features and improve the stability of subsequent fusion. Finally, based on the importance of depth and texture features in defect detection, different weights are assigned to them, and a weighted fusion method is used to fuse the depth and texture feature vectors into a comprehensive feature vector. This method can more flexibly adjust the influence of different features on the final classification result, improving the scene adaptability and accuracy of defect detection.

[0031] In one possible implementation, the defect detection model classifies the comprehensive feature vector to generate several defect types and several defect locations, including:

[0032] The comprehensive feature vector is input into a preset convolutional layer, so that the convolutional layer uses a first convolutional kernel to extract the global deformation features of the comprehensive feature vector, and uses a second convolutional kernel to extract the local texture features of the comprehensive feature vector. Finally, the global deformation features and the local texture features are fused based on an attention mechanism to obtain the fused features.

[0033] The fused features are input into a preset pooling layer, so that the pooling layer performs pooling and downsampling operations on the fused features in sequence to obtain a pooled feature vector;

[0034] The pooled feature vector is input into a preset fully connected layer, so that the fully connected layer performs linear combination and nonlinear transformation on the pooled feature vector, thereby generating several defect types and several defect locations.

[0035] This application provides a method for generating defect types and locations using sensory integrated feature vectors. In the convolutional layer, a dual-branch convolutional kernel design uses kernels of different sizes to extract global deformation features and local texture features respectively, capturing both large-scale deformation and microscopic detail changes on the surface of power equipment, thus covering defect types at different scales. Simultaneously, an attention mechanism is used to further achieve dynamic feature weight allocation. For example, in scenarios dominated by macroscopic deformation, the weight of global features is enhanced, or in scenarios sensitive to microscopic details, local texture features are focused. This adaptive weight allocation mechanism can effectively address the multi-scale characteristics of surface defects on power equipment, improving the accuracy of surface defect detection. In the pooling and fully connected layers, pooling and downsampling operations reduce feature dimensions, lowering subsequent computational complexity while preserving the spatial invariance of key defect features. The nonlinear transformation of the fully connected layer maps the fused features to a high-dimensional classification space, enhancing the model's ability to distinguish complex defect patterns, further improving the efficiency and accuracy of surface defect detection on power equipment.

[0036] In one possible implementation, the defect detection model, constructed based on a deep learning model and trained using several historical image data, includes:

[0037] Acquire historical image data of several power devices at different times;

[0038] Extract the corresponding comprehensive feature vectors from each of the historical image data, and manually annotate each of the comprehensive feature vectors according to each of the historical image data to construct a training dataset;

[0039] An initial defect detection model is constructed based on a deep learning model. The initial defect detection model includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer.

[0040] The initial defect detection model is trained using the training dataset, so that the initial defect detection model updates its parameters through backpropagation according to a preset loss function, thereby obtaining the defect detection model.

[0041] This application provides a method for training a defect detection model. A training set is constructed based on historical data to train an initial model, resulting in the final defect detection model. During training, historical data from multiple periods is used to cover different device states and defect types, enhancing the model's cross-scenario adaptability. Simultaneously, automated backpropagation optimizes parameters, shortening the model iteration cycle and improving training efficiency.

[0042] Secondly, embodiments of this application provide an image recognition-based power equipment surface defect detection system, including an acquisition module, a depth extraction module, a texture extraction module, a feature fusion module, a defect detection module, and a detection result generation module;

[0043] The acquisition module is used to acquire image data of the surface of the power equipment;

[0044] The depth extraction module is used to extract depth information of the surface of the power equipment from the image data;

[0045] The texture extraction module is used to extract texture information of the surface of the power equipment from the image data based on the texture analysis algorithm;

[0046] The feature fusion module is used to fuse the depth information and the texture information to obtain a comprehensive feature vector.

[0047] The defect detection module is used to input the comprehensive feature vector into a preset defect detection model, so that the defect detection model can classify the comprehensive feature vector and generate several defect types and several defect locations.

[0048] The detection result generation module is used to annotate the image data according to the several defect types and several defect locations to generate defect detection image data.

[0049] The defect detection model is built based on a deep learning model and trained using several historical image data.

[0050] In one possible implementation, the defect detection model classifies the comprehensive feature vector to generate several defect types and several defect locations, including:

[0051] The comprehensive feature vector is input into a preset convolutional layer, so that the convolutional layer uses a first convolutional kernel to extract the global deformation features of the comprehensive feature vector, and uses a second convolutional kernel to extract the local texture features of the comprehensive feature vector. Finally, the global deformation features and the local texture features are fused based on an attention mechanism to obtain the fused features.

[0052] The fused features are input into a preset pooling layer, so that the pooling layer performs pooling and downsampling operations on the fused features in sequence to obtain a pooled feature vector;

[0053] The pooled feature vector is input into a preset fully connected layer, so that the fully connected layer performs linear combination and nonlinear transformation on the pooled feature vector, thereby generating several defect types and several defect locations.

[0054] In one possible implementation, the power equipment surface defect detection system further includes a training module, which is used to construct a defect detection model based on a deep learning model and train it using several historical image data, including:

[0055] Acquire historical image data of several power devices at different times;

[0056] Extract the corresponding comprehensive feature vectors from each of the historical image data, and manually annotate each of the comprehensive feature vectors according to each of the historical image data to construct a training dataset;

[0057] An initial defect detection model is constructed based on a deep learning model. The initial defect detection model includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer.

[0058] The initial defect detection model is trained using the training dataset, so that the initial defect detection model updates its parameters through backpropagation according to a preset loss function, thereby obtaining the defect detection model. Attached Figure Description

[0059] Figure 1 A schematic flowchart of a method for detecting surface defects in power equipment based on image recognition, provided in an embodiment of this application;

[0060] Figure 2 This is a schematic diagram of the structure of an image recognition-based power equipment surface defect detection system provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0062] It should be noted that the step numbers in this document are only for the convenience of explaining the specific embodiments and are not intended to limit the order in which the steps are performed. In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0063] Example 1:

[0064] like Figure 1As shown, Embodiment 1 provides a method for detecting surface defects in power equipment based on image recognition, including steps S1-S6:

[0065] Step S1: Acquire image data of the surface of the power equipment;

[0066] Step S2: Extract depth information of the surface of the power equipment from the image data;

[0067] Step S3: Based on the texture analysis algorithm, extract the texture information of the surface of the power equipment from the image data;

[0068] Step S4: Perform feature fusion between the depth information and the texture information to obtain a comprehensive feature vector;

[0069] Step S5: Input the comprehensive feature vector into a preset defect detection model so that the defect detection model can classify the comprehensive feature vector and generate several defect types and several defect locations;

[0070] Step S6: According to the aforementioned defect types and defect locations, perform corresponding annotations in the image data to generate defect detection image data;

[0071] The defect detection model is built based on a deep learning model and trained using several historical image data.

[0072] This application provides a method for detecting surface defects in power equipment based on image recognition. It generates a comprehensive feature vector by fusing depth and texture information, classifies defect types and locations using a deep learning model, and finally labels each defect type at its corresponding location in the image data, outputting visualized defect detection image data. Compared to single-modal recognition in existing technologies, this application simultaneously extracts depth and texture information from the same image data for defect detection. Depth information captures macroscopic deformation, while texture information identifies microscopic details. The two complement each other, enhancing feature representation capabilities and overcoming the limitations of traditional single-modal recognition in terms of sensitivity to multiple defect types, effectively improving the accuracy of surface defect detection in power equipment. Furthermore, this method enables automated identification and labeling, reducing manual identification costs and improving defect detection efficiency. In addition, the final output of labeled defect detection image data allows surface defects in power equipment to be visually displayed in the image, facilitating rapid defect location by users and providing data support for subsequent maintenance work.

[0073] Furthermore, in step S1, acquiring image data of the surface of the power equipment includes:

[0074] Raw image data of the surface of the power equipment is acquired by a camera or laser scanner at a preset location;

[0075] The original image data is subjected to image denoising, image contrast adjustment, and image brightness adjustment in sequence to obtain the image data.

[0076] This application provides a method for acquiring image data. It utilizes a camera or laser scanner at a preset location to acquire raw image data and preprocesses the raw image data to generate high-quality image data. The camera should have sufficient resolution to capture minute details on the device surface, while the laser scanner provides more accurate three-dimensional morphological information. Preprocessing the acquired image data to generate high-quality image data effectively reduces noise interference, enhances the distinction between defect areas and the background, provides reliable input for subsequent feature extraction, and makes the subsequent processing more efficient and stable, thus contributing to improved accuracy in defect detection.

[0077] In a preferred embodiment, a high-resolution camera or laser scanner is used to perform a comprehensive scan of the power equipment surface to acquire detailed surface image data. The camera or laser scanner is positioned appropriately to ensure that every detail of the transformer surface is captured. The acquired image data is preprocessed, including noise removal, image contrast adjustment, and brightness adjustment, to improve image quality and provide clear and accurate input for subsequent processing steps.

[0078] In one possible implementation, step S2, extracting depth information of the surface of the power equipment from the image data, includes:

[0079] Based on 3D reconstruction technology, a 3D model of the surface of the power equipment is generated from the image data;

[0080] Depth information of the surface of the power equipment is extracted from the three-dimensional model.

[0081] This application provides a method for obtaining depth information. By generating a three-dimensional model and extracting depth information through three-dimensional reconstruction technology, a high-precision three-dimensional representation can be obtained, accurately restoring the geometric structure of the surface of power equipment. This helps to identify defects caused by surface depressions, protrusions, etc., and improves the comprehensiveness and accuracy of defect detection.

[0082] In one possible implementation, step S2, extracting depth information of the surface of the power equipment from the image data, includes:

[0083] The image data is input into a preset deep learning model so that the deep learning model can predict and generate depth information of the surface of the power equipment based on the image features in the image data.

[0084] This application provides another method for obtaining depth information. It directly predicts and generates corresponding depth information based on image features in image data using a pre-defined deep learning model. This avoids complex 3D reconstruction calculations, reduces hardware dependence and processing time, and makes depth information extraction more automated and intelligent, reducing manual intervention and improving the efficiency of surface defect detection in power equipment. Furthermore, the deep learning model can adaptively learn the depth features of different devices, making it suitable for diverse defect detection scenarios.

[0085] In a preferred embodiment, 3D reconstruction techniques, such as structured light scanning and stereo vision, are used to reconstruct the 3D morphology of the power equipment surface from the preprocessed image, thereby extracting depth information. Alternatively, a deep learning model, such as a convolutional neural network, can be used to train the image, learn the depth features in the image, and thus predict the depth information of the power equipment surface.

[0086] In a preferred embodiment, in step S3, texture analysis algorithms, such as Gray-Level Co-occurrence Matrix (GLCM) and Local Binary Pattern (LBP), are used to extract texture features from the preprocessed image. These algorithms can analyze the gray-level relationships or local texture patterns between pixels in the image, thereby capturing the microstructural features of the power equipment surface. Texture information reflects the microstructural features of the power equipment surface, which helps to identify defects caused by material aging, corrosion, etc., and improves the sensitivity of defect detection. The application of texture analysis algorithms makes the extraction of texture information more objective and accurate, reducing the influence of subjective factors.

[0087] Furthermore, in step S4, the feature fusion of the depth information and the texture information to obtain a comprehensive feature vector includes:

[0088] Based on the spatial position in the image data, the depth information and the texture information are spatially aligned to obtain aligned depth information and aligned texture information;

[0089] The initial depth feature vector is extracted from the aligned depth information using the feature descriptor of the 3D point cloud, and the initial depth feature vector is converted into a depth feature vector of a preset length.

[0090] An initial texture feature vector is extracted from the aligned texture information based on a texture analysis algorithm, and the initial texture feature vector is converted into a texture feature vector of a preset length.

[0091] The depth feature vector and the texture feature vector are weighted and fused according to preset weight values ​​to obtain the comprehensive feature vector.

[0092] This application provides a feature fusion method. First, spatial alignment ensures a one-to-one correspondence between depth and texture information in spatial location, facilitating accurate fusion later. Then, the extracted features are converted into fixed-length feature vectors to eliminate dimensional differences between different features and improve the stability of subsequent fusion. Finally, based on the importance of depth and texture features in defect detection, different weights are assigned to them, and a weighted fusion method is used to fuse the depth and texture feature vectors into a comprehensive feature vector. This method can more flexibly adjust the influence of different features on the final classification result, improving the scene adaptability and accuracy of defect detection.

[0093] In a preferred embodiment, step S4 mainly includes three sub-steps: alignment of depth and texture information, extraction and normalization of feature vectors, and feature vector fusion. Aligning depth and texture information ensures a one-to-one spatial correspondence for accurate subsequent fusion. If the depth and texture information come from different images or data sources (e.g., depth information from 3D reconstruction and texture information from a 2D image), spatial alignment is required. This can be achieved using image registration techniques, such as feature point matching or mutual information, to align the depth map and texture map.

[0094] In the process of feature vector extraction and normalization, depth features are extracted from depth information. Feature descriptors for 3D point clouds, such as FPFH and SHOT, can be used, or features can be extracted directly from the depth map using a convolutional neural network. The extracted depth features are then converted into fixed-length feature vectors. Texture analysis algorithms, such as Gray-Level Co-occurrence Matrix (GLCM) and Local Binary Pattern (LBP), are used to extract texture features from the texture information. Similarly, the extracted texture features are converted into fixed-length feature vectors. Both the depth feature vector and the texture feature vector are then normalized to eliminate dimensional differences between different features and improve the stability of subsequent fusion.

[0095] In the feature vector fusion stage, deep feature vectors and texture feature vectors are fused according to a preset feature fusion strategy. The feature fusion strategy can be simple concatenation, weighted fusion, or deep learning fusion. Simple concatenation directly concatenates the normalized deep feature vectors and texture feature vectors to form a longer composite feature vector. This method is simple and direct, but may ignore the intrinsic relationships between different features. Weighted fusion assigns different weights to deep and texture features based on their importance in defect detection. Using weighted averaging or other weighted fusion methods, the deep and texture feature vectors are fused into a single composite feature vector. This method allows for more flexible adjustment of the impact of different features on the final classification result. Deep learning fusion uses deep learning models (such as multilayer perceptrons, convolutional neural networks, etc.) to automatically learn the optimal fusion method between deep and texture features. The deep and texture feature vectors are used as input to the model, and the feature fusion strategy is automatically optimized through the model's training process. This method fully utilizes the powerful learning capabilities of deep learning models to improve fusion performance. Based on the selected fusion strategy, the deep and texture feature vectors are fused into a single composite feature vector. The comprehensive feature vector should contain enough information to accurately describe the defect characteristics of the power equipment surface, providing a reliable basis for subsequent classification and identification.

[0096] In one possible implementation, in step S5, the defect detection model classifies the comprehensive feature vector to generate several defect types and several defect locations, including:

[0097] The comprehensive feature vector is input into a preset convolutional layer, so that the convolutional layer uses a first convolutional kernel to extract the global deformation features of the comprehensive feature vector, and uses a second convolutional kernel to extract the local texture features of the comprehensive feature vector. Finally, the global deformation features and the local texture features are fused based on an attention mechanism to obtain the fused features.

[0098] The fused features are input into a preset pooling layer, so that the pooling layer performs pooling and downsampling operations on the fused features in sequence to obtain a pooled feature vector;

[0099] The pooled feature vector is input into a preset fully connected layer, so that the fully connected layer performs linear combination and nonlinear transformation on the pooled feature vector, thereby generating several defect types and several defect locations.

[0100] This application provides a method for generating defect types and locations using sensory integrated feature vectors. In the convolutional layer, a dual-branch convolutional kernel design uses kernels of different sizes to extract global deformation features and local texture features respectively, capturing both large-scale deformation and microscopic detail changes on the surface of power equipment, thus covering defect types at different scales. Simultaneously, an attention mechanism is used to further achieve dynamic feature weight allocation. For example, in scenarios dominated by macroscopic deformation, the weight of global features is enhanced, or in scenarios sensitive to microscopic details, local texture features are focused. This adaptive weight allocation mechanism can effectively address the multi-scale characteristics of surface defects on power equipment, improving the accuracy of surface defect detection. In the pooling and fully connected layers, pooling and downsampling operations reduce feature dimensions, lowering subsequent computational complexity while preserving the spatial invariance of key defect features. The nonlinear transformation of the fully connected layer maps the fused features to a high-dimensional classification space, enhancing the model's ability to distinguish complex defect patterns, further improving the efficiency and accuracy of surface defect detection on power equipment.

[0101] In a preferred embodiment, a machine learning model or a deep learning model can be used to construct the defect detection model, thereby classifying the comprehensive feature vector and generating several defect types and several defect locations.

[0102] When choosing a machine learning model, let's take Support Vector Machine (SVM) as an example. The process begins with model initialization, selecting a suitable kernel function (e.g., linear kernel, polynomial kernel, RBF kernel, etc.) and setting its parameters. Other SVM model parameters, such as the regularization parameter C, are then initialized. Next, the combined feature vectors from the training set are input into the SVM model. The SVM model learns the mapping relationship between the combined feature vectors and defect labels by finding the optimal hyperplane to maximize the margin between different classes. The trained SVM model is then validated using a validation set to evaluate its classification performance. Based on the validation results, the model parameters (e.g., kernel function parameters, regularization parameters, etc.) are adjusted to optimize the classification performance. Finally, the optimized SVM model is tested using a test set to evaluate its final classification performance. Finally, the trained SVM model is applied to the detection of surface defects in actual power equipment, classifying the new combined feature vectors.

[0103] When choosing a deep learning model, let's take a Convolutional Neural Network (CNN) as an example. The CNN model structure is designed, including input layers, convolutional layers, pooling layers, fully connected layers, and output layers. Parameters for each layer are set, such as the size, number, and stride of the convolutional kernels; the size and stride of the pooling layers; and the number of neurons in the fully connected layers. The synthesized feature vector used for model training is input into the input layer of the CNN model. The synthesized feature vector is processed sequentially through convolutional layers, pooling layers, and fully connected layers. Each layer performs specific transformations on its input, such as convolution, pooling, and non-linear activation, to extract higher-level feature representations. In the fully connected layers, the extracted features are linearly combined and non-linearly transformed to obtain the final classification result. Based on the classification result and the true defect labels, the model's loss function, such as the cross-entropy loss function, is calculated. The backpropagation algorithm is used to propagate the gradient of the loss function from the output layer to the input layer, while simultaneously updating the parameters of each layer, such as the weights and biases of the convolutional kernels, to minimize the loss function. The trained CNN model is validated using a validation set to evaluate its classification performance. Based on the validation results, the model parameters, such as learning rate, batch size, and number of iterations, were adjusted, or the model structure was modified, such as increasing the number of convolutional layers and fully connected layers, to optimize the model's classification performance. The optimized CNN model was then tested using a test set to evaluate its final classification performance. Finally, the trained CNN model was applied to the detection of surface defects in actual power equipment, classifying the new comprehensive feature vectors.

[0104] In one possible implementation, the defect detection model, constructed based on a deep learning model and trained using several historical image data, includes:

[0105] Acquire historical image data of several power devices at different times;

[0106] Extract the corresponding comprehensive feature vectors from each of the historical image data, and manually annotate each of the comprehensive feature vectors according to each of the historical image data to construct a training dataset;

[0107] An initial defect detection model is constructed based on a deep learning model. The initial defect detection model includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer.

[0108] The initial defect detection model is trained using the training dataset, so that the initial defect detection model updates its parameters through backpropagation according to a preset loss function, thereby obtaining the defect detection model.

[0109] This application provides a method for training a defect detection model. A training set is constructed based on historical data to train an initial model, resulting in the final defect detection model. During training, historical data from multiple periods is used to cover different device states and defect types, enhancing the model's cross-scenario adaptability. Simultaneously, automated backpropagation optimizes parameters, shortening the model iteration cycle and improving training efficiency.

[0110] In a preferred embodiment, the training process of the defect detection model is as follows:

[0111] Dataset Preparation: Collect and label a large amount of surface image data of power equipment, including samples with various types of defects and no defects. Preprocess the image data, including denoising and contrast enhancement, to improve image quality. Extract depth and texture information from each image and fuse them to form a comprehensive feature vector. Organize the comprehensive feature vector and its corresponding defect labels (such as corrosion, cracks, dents, etc.) into a dataset suitable for model training.

[0112] Model selection: Based on task requirements and data characteristics, select appropriate machine learning models (such as support vector machines (SVM), random forests, etc.) or deep learning models (such as convolutional neural networks (CNN), recurrent neural networks (RNN), etc.).

[0113] Environment setup: Set up a suitable hardware and software environment for model training, including a high-performance computer, deep learning frameworks (such as TensorFlow, PyTorch, etc.), and necessary libraries and tools.

[0114] Data partitioning: The dataset is divided into a training set, a validation set, and a test set. The training set is used for learning the model's parameters, the validation set is used for model tuning and selection, and the test set is used to evaluate the model's final performance.

[0115] Model initialization: For machine learning models, set the initial parameters of the model (such as kernel parameters and regularization parameters for SVM). For deep learning models, construct the network structure of the model and initialize the network parameters (such as weights and bias terms of convolutional kernels).

[0116] Forward propagation and loss calculation: The comprehensive feature vector of the training set is input into the model, and forward propagation is performed to obtain the model's predicted output. Based on the predicted output and the true defect labels, the model's loss function (such as cross-entropy loss function, mean squared error loss function, etc.) is calculated.

[0117] Backpropagation and parameter update: The backpropagation algorithm is used to pass the gradient of the loss function from the output layer to the input layer, and the gradient of the parameters in each layer is calculated. The model parameters are then updated according to gradient descent algorithms (such as stochastic gradient descent SGD, Adam, etc.) to minimize the loss function.

[0118] Iterative training: Repeat the forward propagation, loss calculation, backpropagation, and parameter update process until the model's loss function on the training set converges to a small value, or the preset number of iterations is reached. The final defect detection model is then obtained.

[0119] In a preferred embodiment, in step S6, based on the classification results, the specific location of the defect is located and marked on the original image. For example, if the model identifies corrosion defects on the transformer surface, the corrosion area can be marked on the original image with a rectangular or circular frame, and the type of defect (e.g., corrosion, crack, dent, etc.) and severity (e.g., mild, moderate, severe) can be indicated. Finally, a defect detection report is output, including information such as defect type, location, and severity. The report will list all defects present on the transformer surface in detail and provide a basis for subsequent maintenance work.

[0120] Example 2:

[0121] like Figure 2 As shown, Embodiment 2 provides a power equipment surface defect detection system based on image recognition, including an acquisition module 10, a depth extraction module 20, a texture extraction module 30, a feature fusion module 40, a defect detection module 50, and a detection result generation module 60.

[0122] The acquisition module is used to acquire image data of the surface of the power equipment;

[0123] The depth extraction module is used to extract depth information of the surface of the power equipment from the image data;

[0124] The texture extraction module is used to extract texture information of the surface of the power equipment from the image data based on the texture analysis algorithm;

[0125] The feature fusion module is used to fuse the depth information and the texture information to obtain a comprehensive feature vector.

[0126] The defect detection module is used to input the comprehensive feature vector into a preset defect detection model, so that the defect detection model can classify the comprehensive feature vector and generate several defect types and several defect locations.

[0127] The detection result generation module is used to annotate the image data according to the several defect types and several defect locations to generate defect detection image data.

[0128] The defect detection model is built based on a deep learning model and trained using several historical image data.

[0129] Furthermore, the acquisition module 10 acquires image data of the surface of the power equipment, including:

[0130] Raw image data of the surface of the power equipment is acquired by a camera or laser scanner at a preset location;

[0131] The original image data is subjected to image denoising, image contrast adjustment, and image brightness adjustment in sequence to obtain the image data.

[0132] In one possible implementation, the depth extraction module 20 extracts depth information of the surface of the power equipment from the image data, including:

[0133] Based on 3D reconstruction technology, a 3D model of the surface of the power equipment is generated from the image data;

[0134] Depth information of the surface of the power equipment is extracted from the three-dimensional model.

[0135] In one possible implementation, the depth extraction module 20 extracts depth information of the surface of the power equipment from the image data, including:

[0136] The image data is input into a preset deep learning model so that the deep learning model can predict and generate depth information of the surface of the power equipment based on the image features in the image data.

[0137] Furthermore, the feature fusion module 40 fuses the depth information and the texture information to obtain a comprehensive feature vector, including:

[0138] Based on the spatial position in the image data, the depth information and the texture information are spatially aligned to obtain aligned depth information and aligned texture information;

[0139] The initial depth feature vector is extracted from the aligned depth information using the feature descriptor of the 3D point cloud, and the initial depth feature vector is converted into a depth feature vector of a preset length.

[0140] An initial texture feature vector is extracted from the aligned texture information based on a texture analysis algorithm, and the initial texture feature vector is converted into a texture feature vector of a preset length.

[0141] The depth feature vector and the texture feature vector are weighted and fused according to preset weight values ​​to obtain the comprehensive feature vector.

[0142] In one possible implementation, the defect detection model classifies the comprehensive feature vector to generate several defect types and several defect locations, including:

[0143] The comprehensive feature vector is input into a preset convolutional layer, so that the convolutional layer uses a first convolutional kernel to extract the global deformation features of the comprehensive feature vector, and uses a second convolutional kernel to extract the local texture features of the comprehensive feature vector. Finally, the global deformation features and the local texture features are fused based on an attention mechanism to obtain the fused features.

[0144] The fused features are input into a preset pooling layer, so that the pooling layer performs pooling and downsampling operations on the fused features in sequence to obtain a pooled feature vector;

[0145] The pooled feature vector is input into a preset fully connected layer, so that the fully connected layer performs linear combination and nonlinear transformation on the pooled feature vector, thereby generating several defect types and several defect locations.

[0146] In one possible implementation, the power equipment surface defect detection system further includes a training module, which is used to construct a defect detection model based on a deep learning model and train it using several historical image data, including:

[0147] Acquire historical image data of several power devices at different times;

[0148] Extract the corresponding comprehensive feature vectors from each of the historical image data, and manually annotate each of the comprehensive feature vectors according to each of the historical image data to construct a training dataset;

[0149] An initial defect detection model is constructed based on a deep learning model. The initial defect detection model includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer.

[0150] The initial defect detection model is trained using the training dataset, so that the initial defect detection model updates its parameters through backpropagation according to a preset loss function, thereby obtaining the defect detection model.

[0151] This application provides an image recognition-based surface defect detection system for power equipment. It generates a comprehensive feature vector by fusing depth and texture information, classifies defect types and locations using a deep learning model, and finally labels each defect type at its corresponding location in the image data, outputting visualized defect detection image data. Compared to the single-modal recognition in existing technologies, this application simultaneously extracts depth and texture information from the same image data for defect detection. Depth information captures macroscopic deformation, while texture information identifies microscopic details. The two complement each other, enhancing feature representation capabilities and overcoming the limitations of traditional single-modal recognition in terms of sensitivity to multiple defect types, effectively improving the accuracy of power equipment surface defect detection. Furthermore, this method enables automated identification and labeling, reducing manual identification costs and improving defect detection efficiency. In addition, the final output of labeled defect detection image data allows surface defects of power equipment to be visually displayed in the image, facilitating rapid defect location by users and providing data support for subsequent maintenance work.

[0152] For a more detailed explanation of the working principle and procedures of this embodiment, please refer to the relevant description in Embodiment 1.

[0153] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application for those skilled in the art.

Claims

1. A method for detecting surface defects in power equipment based on image recognition, characterized in that, include: Acquire image data of the surface of electrical equipment; Extract depth information of the surface of the power equipment from the image data; Based on texture analysis algorithms, texture information of the surface of the power equipment is extracted from the image data; The depth information and the texture information are fused to obtain a comprehensive feature vector; The comprehensive feature vector is input into a preset defect detection model so that the defect detection model classifies the comprehensive feature vector and generates several defect types and several defect locations. Based on the aforementioned defect types and defect locations, corresponding annotations are made in the image data to generate defect detection image data. The defect detection model is built based on a deep learning model and trained using several historical image data.

2. The method for detecting surface defects in power equipment based on image recognition as described in claim 1, characterized in that, The acquisition of image data of the surface of the power equipment includes: Raw image data of the surface of the power equipment is acquired by a camera or laser scanner at a preset location; The original image data is subjected to image denoising, image contrast adjustment, and image brightness adjustment in sequence to obtain the image data.

3. The method for detecting surface defects in power equipment based on image recognition as described in claim 1, characterized in that, The step of extracting depth information of the surface of the power equipment from the image data includes: Based on 3D reconstruction technology, a 3D model of the surface of the power equipment is generated from the image data; Depth information of the surface of the power equipment is extracted from the three-dimensional model.

4. The method for detecting surface defects in power equipment based on image recognition as described in claim 1, characterized in that, The step of extracting depth information of the surface of the power equipment from the image data includes: The image data is input into a preset deep learning model so that the deep learning model can predict and generate depth information of the surface of the power equipment based on the image features in the image data.

5. The method for detecting surface defects in power equipment based on image recognition as described in claim 1, characterized in that, The step of fusing the depth information and the texture information to obtain a comprehensive feature vector includes: Based on the spatial position in the image data, the depth information and the texture information are spatially aligned to obtain aligned depth information and aligned texture information; The initial depth feature vector is extracted from the aligned depth information using the feature descriptor of the 3D point cloud, and the initial depth feature vector is converted into a depth feature vector of a preset length. An initial texture feature vector is extracted from the aligned texture information based on a texture analysis algorithm, and the initial texture feature vector is converted into a texture feature vector of a preset length. The depth feature vector and the texture feature vector are weighted and fused according to preset weight values ​​to obtain the comprehensive feature vector.

6. The method for detecting surface defects in power equipment based on image recognition as described in claim 1, characterized in that, The defect detection model classifies the comprehensive feature vector to generate several defect types and several defect locations, including: The comprehensive feature vector is input into a preset convolutional layer, so that the convolutional layer uses a first convolutional kernel to extract the global deformation features of the comprehensive feature vector, and uses a second convolutional kernel to extract the local texture features of the comprehensive feature vector. Finally, the global deformation features and the local texture features are fused based on an attention mechanism to obtain the fused features. The fused features are input into a preset pooling layer, so that the pooling layer performs pooling and downsampling operations on the fused features in sequence to obtain a pooled feature vector; The pooled feature vector is input into a preset fully connected layer, so that the fully connected layer performs linear combination and nonlinear transformation on the pooled feature vector, thereby generating several defect types and several defect locations.

7. The method for detecting surface defects in power equipment based on image recognition as described in claim 1, characterized in that, The defect detection model, constructed based on a deep learning model and trained using several historical image data, includes: Acquire historical image data of several power devices at different times; Extract the corresponding comprehensive feature vectors from each of the historical image data, and manually annotate each of the comprehensive feature vectors according to each of the historical image data to construct a training dataset; An initial defect detection model is constructed based on a deep learning model, which includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer. The initial defect detection model is trained using the training dataset, so that the initial defect detection model updates its parameters through backpropagation according to a preset loss function, thereby obtaining the defect detection model.

8. A surface defect detection system for power equipment based on image recognition, characterized in that, It includes an acquisition module, a depth extraction module, a texture extraction module, a feature fusion module, a defect detection module, and a detection result generation module; The acquisition module is used to acquire image data of the surface of the power equipment; The depth extraction module is used to extract depth information of the surface of the power equipment from the image data; The texture extraction module is used to extract texture information of the surface of the power equipment from the image data based on the texture analysis algorithm; The feature fusion module is used to fuse the depth information and the texture information to obtain a comprehensive feature vector. The defect detection module is used to input the comprehensive feature vector into a preset defect detection model, so that the defect detection model can classify the comprehensive feature vector and generate several defect types and several defect locations. The detection result generation module is used to annotate the image data according to the several defect types and several defect locations to generate defect detection image data. The defect detection model is built based on a deep learning model and trained using several historical image data.

9. The power equipment surface defect detection system based on image recognition as described in claim 8, characterized in that, The defect detection model classifies the comprehensive feature vector to generate several defect types and several defect locations, including: The comprehensive feature vector is input into a preset convolutional layer, so that the convolutional layer uses a first convolutional kernel to extract the global deformation features of the comprehensive feature vector, and uses a second convolutional kernel to extract the local texture features of the comprehensive feature vector. Finally, the global deformation features and the local texture features are fused based on an attention mechanism to obtain the fused features. The fused features are input into a preset pooling layer, so that the pooling layer performs pooling and downsampling operations on the fused features in sequence to obtain a pooled feature vector; The pooled feature vector is input into a preset fully connected layer, so that the fully connected layer performs linear combination and nonlinear transformation on the pooled feature vector, thereby generating several defect types and several defect locations.

10. The power equipment surface defect detection system based on image recognition as described in claim 8, characterized in that, The power equipment surface defect detection system further includes a training module, which is used to construct a defect detection model based on a deep learning model and train it using several historical image data, including: Acquire historical image data of several power devices at different times; Extract the corresponding comprehensive feature vectors from each of the historical image data, and manually annotate each of the comprehensive feature vectors according to each of the historical image data to construct a training dataset; An initial defect detection model is constructed based on a deep learning model, which includes an input layer, a convolutional layer, a pooling layer, and a fully connected layer. The initial defect detection model is trained using the training dataset, so that the initial defect detection model updates its parameters through backpropagation according to a preset loss function, thereby obtaining the defect detection model.

Images