Equipment defect positioning and identification method based on attention mechanism

Abstract

The invention discloses a device defect positioning and identification method based on an attention mechanism, and belongs to the field of defect positioning and identification, and the device defect positioning and identification method based on the attention mechanism comprises the following steps: S1, collecting diversified image data of a target device, comprising normal images and images containing various defects of different illumination conditions, view angles and backgrounds, preprocessing the images, and performing data enhancement; s2, designing an attention mechanism model, constructing a convolutional neural network basis, and integrating an attention mechanism module; s3, designing a small sample learning algorithm, and carrying out transfer learning and data enhancement; s4, performing model training, training the model in stages, performing fine adjustment and optimization, and accurately classifying and accurately positioning defects; and S5, performing image reasoning by using the trained model, generating a positioning frame and a category label, visualizing a result and generating a report.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment defect location and identification, and more specifically, to an equipment defect location and identification method based on an attention mechanism. Background Art

[0002] The development of equipment defect recognition technology reflects the progress of image processing and artificial intelligence technology. Initially, traditional image processing technology was widely used. These methods mainly rely on basic image processing methods such as edge detection and morphological processing. Through operations such as filtering, threshold segmentation and vector field calculation, these methods can detect obvious defects in images, such as cracks, stains or deformations. However, these traditional methods are very sensitive to complex backgrounds and noise, have low detection accuracy and robustness, and are difficult to stably identify defects. With the development of computer vision and machine learning technology, image defect recognition methods based on machine learning have begun to be studied and applied. These methods include classification algorithms such as support vector machines (SVM) and random forests, which rely on manually designed feature extraction and classification processes. Although these methods can identify defects in equipment images through trained models, they still have limitations when dealing with complex and changeable actual environments. The process of manual feature extraction is also cumbersome and difficult to capture all important information. In recent years, the emergence of deep learning, especially convolutional neural networks (CNN), has greatly promoted the development of image processing technology. CNN achieves end-to-end image feature extraction and classification through layer-by-layer convolution, pooling and fully connected neural network structures. It performs well in large-scale image classification and target detection tasks and is suitable for a variety of image defect detection tasks. However, although CNN performs well in many tasks, its recognition ability for complex backgrounds and subtle defects is still limited, and it consumes a lot of computing resources, making it difficult to meet the requirements of real-time applications.

[0003] In general, traditional image processing methods, machine learning-based methods, and CNN-based methods each have their own advantages and disadvantages. Traditional methods are simple but not robust enough for complex environments; machine learning methods rely on a large amount of labeled data and the feature extraction process is cumbersome; and although CNN has made breakthroughs in many aspects, it still needs to be improved in terms of computational efficiency and generalization ability. The continuous evolution of these technologies reflects the historical development process of the field of equipment defect recognition from simple to complex, from manual design to automatic learning. Summary of the invention

[0004] The purpose of the present invention is to provide a device defect location and identification method based on an attention mechanism to solve the problems raised in the above background technology: (1) Difficulty in defect detection under complex backgrounds: When processing images with complex backgrounds, the existing technology is often affected by background interference, resulting in reduced recall and accuracy of defect detection. Traditional image processing methods are sensitive to background changes, and although the methods based on traditional machine learning and CNN have certain robustness, they may still be unable to accurately identify and locate defects in the face of complex and diverse actual scenes. (2) Low sensitivity of subtle defect detection: Existing image processing and machine learning technologies often have difficulty in detecting relatively subtle or inconspicuous defects. Traditional methods tend to identify defects with obvious shapes and large sizes, while defects such as subtle cracks and tiny stains are easily ignored or misjudged. (3) Strong dependence on training data: Methods based on machine learning and deep learning require a large amount of high-quality and accurately labeled training data to work effectively. However, data labeling is costly and difficult, and in actual applications, the types and forms of defects on different devices vary, making the acquisition and expansion of training data a major challenge. (4) High computing resource consumption and poor real-time performance: Existing deep learning and machine learning methods often require a lot of computing resources and time for training and reasoning, which makes it difficult to meet the needs of real-time monitoring and prediction. Real-time performance is crucial for equipment status monitoring and fault prediction. Insufficient model generalization ability: In actual applications, the defect forms and backgrounds in equipment images are complex and diverse, and most existing models show insufficient stability when generalizing different scenarios and new data with large changes. This leads to a significant decrease in model performance in different application environments.

[0005] Technical solution: In order to solve the above technical problems, the present invention proposes a device image defect location and recognition method based on the attention mechanism, which is improved and optimized mainly from the following aspects:

[0006] (1) Introducing the attention mechanism to enhance feature extraction: By designing the attention mechanism module, the model can automatically focus on the key areas in the image and improve its sensitivity to defects in complex backgrounds. The attention mechanism can flexibly adjust the model's focus area, reduce the interference of background noise, and improve the detection ability of subtle defects.

[0007] (2) Effective learning based on small samples: Combining attention mechanism with transfer learning and data augmentation technology to reduce dependence on large-scale labeled data; by effectively utilizing a small amount of high-quality data, the model can still significantly improve recognition accuracy even when data is scarce.

[0008] (3) Optimize computing efficiency and real-time performance: Use lightweight neural network structure and efficient attention mechanism design to reduce computing resource consumption; improve the response speed of the model in real-time defect detection tasks to meet the needs of industrial field applications.

[0009] Improve model generalization ability: By integrating a multi-scale attention mechanism, the model can adapt to defects of different scales and shapes, improve adaptability to different devices and diverse image scenes; and enhance the robustness of the model in dealing with complex situations such as transformation, occlusion, and environmental changes.

[0010] A method for locating and identifying equipment defects based on an attention mechanism, the method comprising the following steps:

[0011] S1. Collect diverse image data of the target device, including normal images and images with various defects under different lighting conditions, viewing angles, and backgrounds, preprocess the images, and perform data augmentation operations such as flipping, rotating, scaling, and translating;

[0012] S2. Design an attention mechanism model, build a convolutional neural network foundation, integrate the attention mechanism module, and enhance feature extraction and selection capabilities;

[0013] S3. Design a small sample learning algorithm and use transfer learning and data enhancement to optimize the performance of the model under small sample conditions;

[0014] S4. Conduct model training, train the model in stages, perform fine-tuning and optimization, accurately classify and precisely locate defects;

[0015] S5. Use the trained model to perform image reasoning, generate positioning boxes and category labels, and visualize the results and generate reports.

[0016] Preferably, the S2 design attention mechanism model also includes selecting ResNet as the basic feature extractor, extracting multi-level features of the image through several layers of convolution and pooling operations; adding an attention mechanism module SENet between different convolution layers, weighting the features of the convolution layer output through the attention mechanism, highlighting key areas, and suppressing unimportant background noise; extracting multi-level and multi-scale features of the image through convolution operations of different scales, and fusing the multi-level and multi-scale features through attention weights.

[0017] Preferably, fusing multi-scale features comprises the following steps:

[0018] S2-1. Use convolution kernels of different sizes to extract features at multiple levels of the base network, including extracting low-level, medium-level, and high-level features at the initial layer, intermediate layer, and advanced layer respectively;

[0019] S2-2. Build a feature pyramid network to connect feature maps of different scales;

[0020] FPN combines high-level features with low-level features through lateral connections and top-down upsampling;

[0021] S2-3. Within the same scale, the extracted feature map is further processed through convolution and pooling operations to make it more recognizable;

[0022] Use the attention mechanism to weight feature maps within the same scale to highlight key features and suppress irrelevant noise;

[0023] S2-4. Use the attention mechanism module to perform weighted fusion of feature maps of different scales;

[0024] Through feature weighting, features useful for defect identification are selectively highlighted, and background noise and unimportant information are suppressed;

[0025] S2-5. Dynamically adjust the weight of each feature map according to the importance of features at different scales;

[0026] Use the attention mechanism to calculate the weight of each scale feature and perform weighted fusion to generate a comprehensive feature map;

[0027] S2-6. Decode the fused comprehensive feature map to generate the defect positioning box and category label;

[0028] The decoding process restores the spatial resolution of the image through deconvolution and upsampling operations;

[0029] S2-7. Perform non-maximum suppression and post-processing on the decoding results to optimize the detection results;

[0030] Finally, the defect positioning box and category label are output.

[0031] Preferably, the S3 design of a small sample learning algorithm also includes pre-training a model on a large-scale general data set to obtain basic feature extraction capabilities, migrating the pre-trained model to the device image defect detection task, and fine-tuning it with less labeled data; performing a variety of data enhancement operations on the small sample data, including random cropping, flipping, rotation, scaling, and color jittering; using the attention mechanism to further improve the feature extraction efficiency, extracting key features under small sample conditions, and improving the learning effect of the model under limited data conditions by dynamically adjusting the focus area during training.

[0032] Preferably, the S5 also includes using the trained model to infer the new image, extracting features and weighting them through the attention mechanism to generate a feature map, decoding and post-processing the feature map, generating a positioning box and category label for the defect; further analyzing the area within the positioning box to confirm the specific features of the defect including shape, size, and position; further segmenting the defect area according to specific needs to extract more detailed defect information; visualizing the results of recognition and positioning, generating output images and videos with defect annotations, providing statistical information of image defects, and generating reports.

[0033] Preferably, the defect area analysis comprises the following steps:

[0034] S5-1. Use the trained model to infer the input image and extract the feature map;

[0035] Decode the weighted feature map through the attention mechanism to generate the preliminary positioning box and category label of the defect;

[0036] S5-2. Post-process the decoding results, including non-maximum suppression NMS, and optimize the defect location frame;

[0037] Get the preliminary location frame of the defect and mark the possible area of ​​the defect;

[0038] S5-3. Crop the image area within the preliminary positioning frame as input for refinement analysis;

[0039] The cropped area contains sufficient background information;

[0040] S5-4. Use a finer-grained convolutional network and a specific refinement module to further extract features from the cropped area;

[0041] Extract fine-grained features of defects, including shape, edge, and texture;

[0042] S5-5. Describe and quantify the detailed features, including the shape, size, location, and edge clarity of the defects;

[0043] Generate detailed defect characterization information;

[0044] S5-6. Further classify the defects according to the refined features and identify the specific types of defects;

[0045] Fine-tune defect classification using classifiers including support vector machines, random forests, and neural networks;

[0046] Use segmentation networks including fully convolutional networks (FCN) and U-Net to perform semantic segmentation on the cropped area, accurately segment the defect area, and obtain the defect segmentation mask.

[0047] Preferably, the device defect location and identification method based on the attention mechanism also uses unlabeled data for self-supervised learning to extract common features; the self-supervised learning algorithm includes contrastive learning and automatic encoder training on a large number of unlabeled device images to learn useful image features without labels;

[0048] The features obtained through self-supervised learning are combined with the features of labeled data in other steps as the initial feature representation of the model.

[0049] Preferably, the device defect location and identification method based on the attention mechanism combines data from other sensors including infrared and ultrasonic waves, and performs multimodal fusion with image data;

[0050] Image data from different time points during equipment operation are introduced and combined with time series information to detect dynamic defects.

[0051] Compared with the prior art, the advantages of the present invention are:

[0052] Improve defect detection accuracy and robustness

[0053] (1) Higher detection accuracy: By introducing the attention mechanism, the present invention can automatically focus on the key areas in the image, significantly improving the detection capabilities of complex backgrounds and subtle defects.

[0054] (2) Strong robustness: The attention mechanism enables the model to better adapt to different lighting, noise and background changes, providing stable and reliable defect detection results.

[0055] 2. Reduce reliance on large-scale data

[0056] (1) Effective small sample learning: By combining the attention mechanism and transfer learning technology, the present invention reduces the dependence on large-scale labeled data, improves the learning efficiency under small sample conditions, and reduces the cost of data collection and labeling.

[0057] 3. Improve computing efficiency and support real-time applications

[0058] (1) Improved computing efficiency: Through lightweight model design and optimization strategies, the present invention significantly reduces the consumption of computing resources and improves processing speed.

[0059] (2) Real-time processing capability: The optimized model can achieve real-time detection with limited computing resources and is suitable for scenarios that require rapid response, such as industrial production lines.

[0060] 4. Enhance the generalization and stability of the model

[0061] (1) Strong generalization capability: Multi-scale feature fusion and attention mechanism ensure that the model can maintain efficient recognition in defects of different scales and shapes and in changing environments, thus improving the general adaptability of the model.

[0062] (2) Improved stability: The attention mechanism enables the model to better respond to changes in the external environment and ensure stability in different practical applications.

[0063] 5. Improve feature extraction and positioning accuracy

[0064] (1) Accurate feature extraction: Through attention weighting, the present invention can more precisely identify and extract key features, improving the ability to capture subtle and important defects.

[0065] (2) High-precision positioning: The attention mechanism enhances the accuracy of defect positioning, can accurately calibrate the location and attributes of defects, and provide more detailed defect information. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 The figure is a schematic diagram of the overall process of a method for locating and identifying equipment defects based on an attention mechanism of the present invention.

[0067] (1) Image acquisition and preprocessing: Preprocess the input image, including image input, image enhancement and size normalization.

[0068] (2) Attention mechanism model design: By adding attention mechanism modules and multi-scale feature fusion to the basic network, the image feature selection capability is enhanced and noise is suppressed.

[0069] (3) Small sample learning algorithm design: Combine the attention mechanism with transfer learning and data augmentation technology to further enhance the model's feature extraction efficiency under small sample conditions.

[0070] (4) Model training: Use a segmented training strategy to train the model and optimize the model.

[0071] Defect localization and identification: Use the trained model to infer the image, visualize the results of identification and localization, and generate output images or videos with defect annotations. DETAILED DESCRIPTION

[0072] The present invention proposes an equipment defect recognition algorithm based on the attention mechanism. By introducing the attention mechanism model, the defects in the equipment image are accurately located and identified. The specific implementation method and process of the present invention are described in detail below in conjunction with the drawings in Chapter 3. A method for locating and identifying equipment defects based on the attention mechanism includes the following steps:

[0073] S1. Collect diverse image data of the target device, including normal images and images with various defects under different lighting conditions, viewing angles, and backgrounds, preprocess the images, and perform data augmentation operations such as flipping, rotating, scaling, and translating;

[0074] S2. Design an attention mechanism model, build a convolutional neural network foundation, integrate the attention mechanism module, and enhance feature extraction and selection capabilities;

[0075] S3. Design a small sample learning algorithm and use transfer learning and data enhancement to optimize the performance of the model under small sample conditions;

[0076] S4. Conduct model training, train the model in stages, perform fine-tuning and optimization, accurately classify and precisely locate defects;

[0077] S5. Use the trained model to perform image reasoning, generate positioning boxes and category labels, and visualize the results and generate reports.

[0078] The S2 attention mechanism model design also includes selecting ResNet as the basic feature extractor, extracting multi-level features of the image through several layers of convolution and pooling operations; adding the attention mechanism module SENet between different convolution layers, weighting the features of the convolution layer output through the attention mechanism, highlighting the key areas, and suppressing unimportant background noise; extracting multi-level and multi-scale features of the image through convolution operations of different scales, and fusing multi-level and multi-scale features through attention weights.

[0079] Specifically, ResNet (Residual Network) is a deep convolutional neural network architecture proposed by He Kaiming et al. in 2015. Its core idea is to introduce residual learning to solve the degradation problem (Degradation Problem) that occurs during the training of deep neural networks.

[0080] Residual Learning: As the number of layers of traditional deep neural networks increases, the training error increases and the network performance decreases. ResNet introduces residual blocks so that each layer learns the residual of the input instead of directly learning the mapping relationship between input and output.

[0081] The basic form of the residual block is: Y = F (X) + X, where X is the input, F (X) is the feature obtained through several convolution, BatchNormalization and ReLU operations, and Y is the output. Through this design, the original input information can be directly passed to the subsequent layers through identity mapping, effectively avoiding the problems of gradient vanishing and gradient exploding.

[0082] Network structure: The basic unit of ResNet is the residual block. Each residual block consists of two or three convolutional layers and corresponding BatchNormalization layers and ReLU activation functions.

[0083] The output of the residual block is added to the input through a skip connection and then passed to the next residual block. ResNet can be stacked with multiple residual blocks to form a very deep network.

[0084] Specifically, SENet (Squeeze-and-Excitation Network) is an attention mechanism module used to enhance the feature representation capability of deep convolutional neural networks (CNNs). Its core idea is to dynamically adjust the weight of each channel by learning the relationship between feature channels, thereby enhancing the focus on important features and suppressing the response to unimportant features. SENet mainly consists of two key steps: squeeze and excitation.

[0085] Squeeze: In this step, SENet compresses the feature map of each channel into a single value through a global pooling operation (usually global average pooling). This value reflects the importance of all features in that channel.

[0086] Excitation: In this step, SENet uses a small fully connected network to learn the excitation weights for each channel. This fully connected network usually consists of a hidden layer and a sigmoid activation function to generate a channel attention vector, which is used to adjust the feature response of each channel.

[0087] Fusion of multi-scale features includes the following steps:

[0088] S2-1. Use convolution kernels of different sizes to extract features at multiple levels of the base network, including extracting low-level, medium-level, and high-level features at the initial layer, intermediate layer, and advanced layer respectively;

[0089] S2-2. Build a feature pyramid network to connect feature maps of different scales;

[0090] FPN combines high-level features with low-level features through lateral connections and top-down upsampling;

[0091] S2-3. Within the same scale, the extracted feature map is further processed through convolution and pooling operations to make it more recognizable;

[0092] Use the attention mechanism to weight feature maps within the same scale to highlight key features and suppress irrelevant noise;

[0093] S2-4. Use the attention mechanism module to perform weighted fusion of feature maps of different scales;

[0094] Through feature weighting, features useful for defect identification are selectively highlighted, and background noise and unimportant information are suppressed;

[0095] S2-5. Dynamically adjust the weight of each feature map according to the importance of features at different scales;

[0096] Use the attention mechanism to calculate the weight of each scale feature and perform weighted fusion to generate a comprehensive feature map;

[0097] S2-6. Decode the fused comprehensive feature map to generate the defect positioning box and category label;

[0098] The decoding process restores the spatial resolution of the image through deconvolution and upsampling operations;

[0099] S2-7. Perform non-maximum suppression and post-processing on the decoding results to optimize the detection results;

[0100] Finally, the defect positioning box and category label are output.

[0101] S3's design of a small sample learning algorithm also includes pre-training the model on a large-scale general data set to obtain basic feature extraction capabilities, migrating the pre-trained model to the device image defect detection task, and fine-tuning it with less labeled data; performing a variety of data enhancement operations on small sample data, including random cropping, flipping, rotation, scaling, and color jittering; using the attention mechanism to further improve feature extraction efficiency, extracting key features under small sample conditions, and improving the model's learning effect under limited data conditions by dynamically adjusting the focus area during training.

[0102] S5 also includes using the trained model to infer new images, extract features and weight them through the attention mechanism to generate feature maps, decode and post-process the feature maps, and generate defect positioning boxes and category labels; further analyze the area within the positioning box to confirm the specific features of the defect including shape, size, and position; further segment the defect area according to specific needs and extract more detailed defect information; visualize the results of recognition and positioning, generate output images and videos with defect annotations, provide statistical information on image defects, and generate reports.

[0103] Defective area analysis includes the following steps:

[0104] S5-1. Use the trained model to infer the input image and extract the feature map;

[0105] Decode the weighted feature map through the attention mechanism to generate the preliminary positioning box and category label of the defect;

[0106] S5-2. Post-process the decoding results, including non-maximum suppression NMS, and optimize the defect location frame;

[0107] Get the preliminary location frame of the defect and mark the possible area of ​​the defect;

[0108] S5-3. Crop the image area within the preliminary positioning frame as input for refinement analysis;

[0109] The cropped area contains sufficient background information;

[0110] S5-4. Use a finer-grained convolutional network and a specific refinement module to further extract features from the cropped area;

[0111] Extract fine-grained features of defects, including shape, edge, and texture;

[0112] S5-5. Describe and quantify the detailed features, including the shape, size, location, and edge clarity of the defects;

[0113] Generate detailed defect characterization information;

[0114] S5-6. Further classify the defects according to the refined features and identify the specific types of defects;

[0115] Fine-tune defect classification using classifiers including support vector machines, random forests, and neural networks;

[0116] Use segmentation networks including fully convolutional networks (FCN) and U-Net to perform semantic segmentation on the cropped area, accurately segment the defect area, and obtain the defect segmentation mask.

[0117] A device defect location and identification method based on an attention mechanism also uses unlabeled data for self-supervised learning to extract common features; self-supervised learning algorithms include contrastive learning and autoencoders trained on a large number of unlabeled device images to learn useful image features without labels;

[0118] The features obtained through self-supervised learning are combined with the features of labeled data in other steps as the initial feature representation of the model.

[0119] A method for locating and identifying equipment defects based on an attention mechanism combines data from other sensors, including infrared and ultrasonic waves, with image data for multimodal fusion;

[0120] Image data from different time points during equipment operation are introduced and combined with time series information to detect dynamic defects.

[0121] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for locating and identifying equipment defects based on an attention mechanism, characterized in that: The device defect location and identification method based on the attention mechanism comprises the following steps: S1. Collect diverse image data of the target device, including normal images and images with various defects under different lighting conditions, viewing angles, and backgrounds, preprocess the images, and perform data augmentation operations such as flipping, rotating, scaling, and translating; S2. Design an attention mechanism model, build a convolutional neural network foundation, integrate the attention mechanism module, and enhance feature extraction and selection capabilities; S3. Design a small sample learning algorithm and use transfer learning and data enhancement to optimize the performance of the model under small sample conditions; S4. Conduct model training, train the model in stages, perform fine-tuning and optimization, accurately classify and precisely locate defects; S5. Use the trained model to perform image reasoning, generate positioning boxes and category labels, and visualize the results and generate reports.

2. According to the method of claim 1, the device defect location and identification method based on the attention mechanism is characterized in that: The S2 attention mechanism model also includes selecting ResNet as the basic feature extractor, extracting multi-level features of the image through several layers of convolution and pooling operations; adding an attention mechanism module SENet between different convolution layers, weighting the features of the convolution layer output through the attention mechanism, highlighting key areas, and suppressing unimportant background noise; extracting multi-level and multi-scale features of the image through convolution operations of different scales, and fusing the multi-level and multi-scale features through attention weights.

3. According to the method of claim 2, the device defect location and identification method based on the attention mechanism is characterized in that: Fusion of multi-scale features includes the following steps: S2-1. Use convolution kernels of different sizes to extract features at multiple levels of the base network, including extracting low-level, medium-level, and high-level features at the initial layer, intermediate layer, and advanced layer respectively; S2-2. Build a feature pyramid network to connect feature maps of different scales; FPN combines high-level features with low-level features through lateral connections and top-down upsampling; S2-3. Within the same scale, the extracted feature map is further processed through convolution and pooling operations to make it more recognizable; Use the attention mechanism to weight feature maps within the same scale to highlight key features and suppress irrelevant noise; S2-4. Use the attention mechanism module to perform weighted fusion of feature maps of different scales; Through feature weighting, features useful for defect identification are selectively highlighted, and background noise and unimportant information are suppressed; S2-5. Dynamically adjust the weight of each feature map according to the importance of features at different scales; Use the attention mechanism to calculate the weight of each scale feature and perform weighted fusion to generate a comprehensive feature map; S2-6. Decode the fused comprehensive feature map to generate the defect positioning box and category label; The decoding process restores the spatial resolution of the image through deconvolution and upsampling operations; S2-7. Perform non-maximum suppression and post-processing on the decoding results to optimize the detection results; Finally, the defect positioning box and category label are output.

4. According to the method of claim 1, the method is characterized in that: The S3 small sample learning algorithm also includes pre-training the model on a large-scale general data set to obtain basic feature extraction capabilities, migrating the pre-trained model to the device image defect detection task, and fine-tuning it with less labeled data; performing a variety of data enhancement operations on the small sample data, including random cropping, flipping, rotation, scaling, and color jittering; using the attention mechanism to further improve the feature extraction efficiency, extracting key features under small sample conditions, and improving the learning effect of the model under limited data conditions by dynamically adjusting the focus area during training.

5. According to the method of equipment defect location and identification based on attention mechanism in claim 1, it is characterized in that: The S5 also includes using the trained model to infer new images, extract features and weight them through the attention mechanism to generate a feature map, decode and post-process the feature map, and generate a positioning box and category label for the defect; further analyze the area within the positioning box to confirm the specific features of the defect including shape, size, and position; further segment the defect area according to specific needs and extract more detailed defect information; visualize the results of recognition and positioning, generate output images and videos with defect annotations, provide statistical information of image defects, and generate reports.

6. The method for locating and identifying equipment defects based on the attention mechanism according to claim 5, characterized in that: The defect area analysis comprises the following steps: S5-1. Use the trained model to infer the input image and extract the feature map; Decode the weighted feature map through the attention mechanism to generate the preliminary positioning box and category label of the defect; S5-2. Post-process the decoding results, including non-maximum suppression NMS, and optimize the defect location frame; Get the preliminary location frame of the defect and mark the possible area of ​​the defect; S5-3. Crop the image area within the preliminary positioning frame as input for refinement analysis; The cropped area contains sufficient background information; S5-4. Use a finer-grained convolutional network and a specific refinement module to further extract features from the cropped area; Extract fine-grained features of defects, including shape, edge, and texture; S5-5. Describe and quantify the detailed features, including the shape, size, location, and edge clarity of the defects; Generate detailed defect characterization information; S5-6. Further classify the defects according to the refined features and identify the specific types of defects; Fine-tune defect classification using classifiers including support vector machines, random forests, and neural networks; Use segmentation networks including fully convolutional networks (FCN) and U-Net to perform semantic segmentation on the cropped area, accurately segment the defect area, and obtain the defect segmentation mask.

7. The method for locating and identifying equipment defects based on the attention mechanism according to claim 1, characterized in that: The device defect location and identification method based on the attention mechanism also uses unlabeled data for self-supervised learning to extract common features; the self-supervised learning algorithm includes contrastive learning and autoencoder training on a large number of unlabeled device images to learn useful image features without labels; The features obtained through self-supervised learning are combined with the features of labeled data in other steps as the initial feature representation of the model.

8. The method for locating and identifying equipment defects based on the attention mechanism according to claim 1, characterized in that: The device defect location and identification method based on the attention mechanism combines data from other sensors including infrared and ultrasonic waves, and performs multimodal fusion with image data; Image data from different time points during equipment operation are introduced and combined with time series information to detect dynamic defects.

Images