Defect detection method and device, equipment, storage medium and program product

Abstract

The invention relates to a defect detection method and device, equipment, a storage medium and a program product. The method comprises the following steps: acquiring an annotated image, and determining a defect detection data set based on the annotated image; constructing an initial detection network, and training the initial detection network based on the defect detection data set to obtain an optimized detection network comprising a feature extraction module, a feature fusion module, a multi-receptive-field collaborative refinement module and a detection head module; acquiring a plurality of hierarchical features of the to-be-detected image based on a feature extraction module, a feature fusion module and a multi-receptive-field collaborative refinement module; enabling each level feature to pass through a corresponding multi-receptive-field collaborative refinement module to obtain a refinement feature of the to-be-detected image; and combining the path-enhanced refined features of the to-be-detected image with the same-scale hierarchical features passing through the feature pyramid, and inputting the combined features into a detection head module to obtain a detection result of the to-be-detected image. By adopting the method, the insulator defect detection precision can be effectively improved on the premise of ensuring real-time detection.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a defect detection method, device, equipment, storage medium and program product. Background Art

[0002] Among all the components of the power system, insulators are one of the most important parts of the transmission line. They mainly bear the dual functions of electrical insulation and mechanical support, ensuring the electrical insulation performance between the transmission line, tower and the ground. However, due to their long-term exposure to high-voltage electric fields and harsh environmental conditions, insulators are prone to cracks, damage, insulation failure and other problems, threatening the normal operation of the transmission line, and in severe cases may even cause large-scale power outages.

[0003] The target detection algorithm based on deep learning can automatically analyze and process image data, identify insulators and their defects, and thus improve the efficiency and accuracy of inspections. However, in the detection of insulator defects, the existing deep learning model faces the problem of insufficient feature expression when dealing with diverse defects, resulting in blurred target boundaries and reduced detection accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a defect detection method, device, computer equipment, computer-readable storage medium and computer program product that can effectively improve the accuracy of insulator defect detection while ensuring real-time detection in response to the above technical problems.

[0005] In a first aspect, the present application provides a defect detection method, comprising:

[0006] Acquire an annotated image, and determine a defect detection data set based on the annotated image;

[0007] Constructing an initial detection network, and training the initial detection network based on the defect detection data set to obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0008] Based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0009] Passing each of the hierarchical features through a corresponding multi-receptive field collaborative refinement module to obtain a refined feature of the image to be tested;

[0010] The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those of the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0011] In one embodiment, the acquiring of the annotated image and determining the defect detection data set based on the annotated image includes:

[0012] Acquire the original image including the target to be measured in the power system;

[0013] Annotating the target to be measured and the target defect category in the original image to obtain the annotated image, wherein the target to be measured includes an insulator, and the target defect category includes at least one of insulator breakage, flashover, or insulator segment dropout;

[0014] The set of labeled images is determined as the defect detection dataset.

[0015] In one embodiment, the training of the initial detection network based on the defect detection data set to obtain an optimized detection network includes:

[0016] Training the initial detection network based on the defect detection data set and a preset loss function;

[0017] When the evaluation index of the initial detection network meets the preset index, the current initial detection network is determined as the optimized detection network.

[0018] In one of the embodiments, the preset loss function includes a bounding rectangle loss function, a target classification loss function and a confidence loss function.

[0019] In one embodiment, the method of obtaining multiple levels of features of the image to be tested based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module includes:

[0020] Based on the feature extraction module, the image to be tested is downsampled multiple times to obtain coding layer features of multiple different scales;

[0021] Based on the feature fusion module, each of the coding layer features is decoded to obtain a plurality of the hierarchical features.

[0022] In one embodiment, the step of subjecting each of the hierarchical features to a corresponding multi-receptive field collaborative refinement module to obtain the refined features of the image to be tested includes:

[0023] Inputting the hierarchical features into the collaborative weight allocation submodule in the multi-receptive field collaborative refinement module to obtain the full pixel weights corresponding to the hierarchical features;

[0024] Performing convolution operation, normalization and activation on the hierarchical features to obtain normalized features corresponding to the hierarchical features;

[0025] Perform multi-level extraction and refinement on the hierarchical features based on the full pixel weights to obtain step-by-step refinement features corresponding to the current hierarchical features;

[0026] The hierarchical feature, the normalized feature and the step-by-step refined feature are added together to obtain a refined feature corresponding to the hierarchical feature.

[0027] In a second aspect, the present application also provides a defect detection device, comprising:

[0028] An acquisition module, used for acquiring an annotated image and determining a defect detection data set based on the annotated image;

[0029] A construction module, used to construct an initial detection network, train the initial detection network based on the defect detection data set, and obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0030] A first generation module, used for acquiring multiple levels of features of the image to be tested based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module;

[0031] A second generation module is used to obtain the refined features of the image to be tested by passing each of the hierarchical features through a corresponding multi-receptive field collaborative refinement module;

[0032] The third generation module is used to combine the refined features of the image to be tested after path enhancement with the hierarchical features of the same scale as the feature pyramid and input them into the detection head module to obtain the detection result of the image to be tested.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Acquire an annotated image, and determine a defect detection data set based on the annotated image;

[0035] Constructing an initial detection network, and training the initial detection network based on the defect detection data set to obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0036] Based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0037] Passing each of the hierarchical features through a corresponding multi-receptive field collaborative refinement module to obtain a refined feature of the image to be tested;

[0038] The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those of the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0040] Acquire an annotated image, and determine a defect detection data set based on the annotated image;

[0041] Constructing an initial detection network, and training the initial detection network based on the defect detection data set to obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0042] Based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0043] Passing each of the hierarchical features through a corresponding multi-receptive field collaborative refinement module to obtain a refined feature of the image to be tested;

[0044] The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those of the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0045] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0046] Acquire an annotated image, and determine a defect detection data set based on the annotated image;

[0047] Constructing an initial detection network, and training the initial detection network based on the defect detection data set to obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0048] Based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0049] Passing each of the hierarchical features through a corresponding multi-receptive field collaborative refinement module to obtain a refined feature of the image to be tested;

[0050] The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those of the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0051] The above-mentioned defect detection method, device, computer equipment, computer-readable storage medium and computer program product can enhance the expression ability of different scale features in the feature pyramid by multiple reinforcement of important information in the feature map based on the multi-receptive field collaborative feature refinement module. The method can effectively improve the accuracy of insulator defect detection under the premise of ensuring real-time detection, and is suitable for defect detection tasks of insulators on transmission lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 is a schematic flow chart of a defect detection method in one embodiment;

[0054] Figure 2 is a schematic diagram of a global model of a defect detection method in one embodiment;

[0055] Figure 3 is a schematic diagram of an output result of a defect detection method in an embodiment;

[0056] Figure 4 A schematic diagram of a multi-receptive field collaborative refinement module of a defect detection method in an embodiment;

[0057] Figure 5 A schematic diagram of a collaborative weight allocation submodule of a defect detection method in one embodiment;

[0058] Figure 6 is a structural block diagram of a defect detection device in one embodiment;

[0059] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] In today's society, electricity has become an important pillar to promote the country's economic and social development. With the rapid economic growth and the acceleration of industrialization, the demand for electricity resources in the whole society continues to increase. The power system is a complex network structure, including power plants, transmission and transformation lines, distribution networks and power terminals. Its operating status not only directly affects the steady development of the national economy, but is also closely related to the quality of people's daily life. Among all the components of the power system, insulators, as one of the important components of the transmission line, mainly bear the dual functions of electrical insulation and mechanical support, and ensure the electrical insulation performance between the transmission line, the tower and the ground. However, due to its long-term exposure to high-voltage electric fields and harsh environmental conditions, insulators are prone to cracks, damage, insulation failure and other problems, threatening the normal operation of the transmission line, and in severe cases may even cause large-scale power outages.

[0062] Traditional insulator inspections are mainly done manually, including ground inspections, instrument inspections, and pole climbing. These methods can detect insulator defects to a certain extent, but they are difficult, dangerous, and inefficient. Especially when facing long-distance and large-scale transmission line inspection tasks, they are more likely to miss inspections and cannot meet the needs of modern power systems for efficient and accurate inspections. In recent years, with the rise of drone technology, drone-based inspections have gradually become a new trend in transmission line inspections. Drones are equipped with high-definition camera equipment, which can conduct efficient and low-cost comprehensive inspections of transmission lines from the air. However, a large amount of acquired image data still relies on manual analysis and screening, which is time-consuming and prone to misjudgment. Therefore, more intelligent inspection methods are urgently needed.

[0063] The rise of deep learning technology has provided a new solution for the intelligent inspection of power systems. The target detection algorithm based on deep learning can automatically analyze and process image data, accurately identify insulators and their defects, and thus significantly improve the efficiency and accuracy of inspections. Unlike traditional methods, deep learning technology is free from the reliance on manual feature extraction. Instead, it can achieve fast and accurate identification of complex defect features through large-scale data training models. Currently, researchers are committed to continuously optimizing deep learning models in the fields of feature extraction, feature fusion, and attention mechanisms to cope with insulator defect detection tasks in complex scenarios, and gradually promote the application of this technology in actual power inspections.

[0064] Although the existing deep learning models have made some progress in insulator defect detection, they ignore the fact that the existing feature pyramid network is often limited to simple up-sampling and down-sampling of features at different levels when processing features of different scales, which leads to insufficient feature expression when the model processes diversified defects, resulting in blurred target boundaries and reduced detection accuracy. Therefore, how to effectively enhance the feature expression ability of the feature pyramid and improve the detection performance of the model has become one of the key issues in the current research on insulator defect detection.

[0065] Based on this, in one embodiment, Figure 1 As shown, a defect detection method is provided. This embodiment takes the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0066] Step 101, obtaining a labeled image, and determining a defect detection data set based on the labeled image;

[0067] Step 102, constructing an initial detection network, training the initial detection network based on the defect detection data set to obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module and a detection head module;

[0068] Step 103, based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0069] Step 104, passing each level of features through the corresponding multi-receptive field collaborative refinement module to obtain refined features of the image to be tested;

[0070] Step 105 , the refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those of the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0071] Among them, the annotated image is an image that has been annotated and used as training data; the set of annotated images is determined as a defect detection data set. The initial detection network is an untrained detection network that is initially built; the optimized detection network is a trained detection network that meets the evaluation indicators. In this embodiment, the optimized detection network is used to detect defects in insulators, and each module of the optimized detection network will be described in detail later. Features at each level are features of different scales.

[0072] In the above defect detection method, the important information in the feature map is repeatedly enhanced based on the multi-receptive field collaborative feature refinement module, which can improve the expression ability of different scale features in the feature pyramid. This method can effectively improve the accuracy of insulator defect detection while ensuring real-time detection, and is suitable for defect detection tasks of insulators on transmission lines.

[0073] In an exemplary embodiment, obtaining a labeled image and determining a defect detection dataset based on the labeled image include:

[0074] An original image including a target to be measured in a power system is obtained; the target to be measured and the target defect category in the original image are annotated to obtain an annotated image, wherein the target to be measured includes an insulator, and the target defect category includes at least one of insulator breakage, flashover, or insulator segment dropout; a set of annotated images is determined as a defect detection data set.

[0075] The original images of insulators and insulator defects in the power system are collected and photographed, and the collected images are annotated with data, and the image categories are annotated as insulator breakage, flashover, insulator drop, and insulator, forming a defect detection dataset. For the sake of convenience, the IDDS (Insulator Defect Detection Dataset) dataset is used to refer to the defect detection dataset. For example, the LabelImg tool can be used for data annotation.

[0076] In an exemplary embodiment, an initial detection network is trained based on a defect detection data set to obtain an optimized detection network, including: training the initial detection network based on the defect detection data set and a preset loss function; when an evaluation index of the initial detection network meets the preset index, determining the current initial detection network as the optimized detection network.

[0077] In an exemplary embodiment, the preset loss functions include a bounding rectangle loss function, a target classification loss function, and a confidence loss function.

[0078] Specifically, the preset loss function consists of a bounding rectangle loss function, a target classification loss function, and a confidence loss function, which are described as follows:

[0079] The definition of the bounding rectangle loss function is shown in Formula 1:

[0080] ;

[0081] in, represents the loss value of the bounding rectangle loss function, , , represents the output image through the initial detection network, Represents the Ground-Truth (true value) of the target, Indicates the width of the real target box, Indicates the height of the real target box, Indicates the width of the prediction box, Indicates the height of the prediction box, , where v is used to measure the consistency of the aspect ratio and α is used to balance the value of v.

[0082] The target classification loss function is defined in Formula 2:

[0083] ;

[0084] in, represents the loss value of the target classification loss function, Representative Is there a prediction box Class instances, Representative There is a The probability of a class instance.

[0085] The confidence loss function is shown in Formula 3:

[0086] ;

[0087] in, represents the loss value of the confidence loss function, Represents the prediction box Whether there is a target instance in Represents the prediction box The probability that the target instance exists in .

[0088] The total loss value is calculated based on the following formula 4:

[0089] ;

[0090] in , , is a hyperparameter used to balance the three loss functions. For example, it can be set to , , ; The total loss value is calculated using multiple losses between the output detection image and the true target, and the training process of the overall network is constrained according to the total loss value.

[0091] In an exemplary embodiment, based on a feature extraction module, a feature fusion module and a multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained, including:

[0092] Based on the feature extraction module, the image to be tested is downsampled multiple times to obtain coding layer features of multiple scales; based on the feature fusion module, the features of each coding layer are decoded to obtain multiple levels of features.

[0093] Exemplarily, the feature extraction module is a backbone feature extraction module, and the feature fusion module is a feature pyramid multi-scale feature fusion module; see Figure 2 , Figure 2 Schematic diagram of the global model of the detection network used in this embodiment. For the convenience of description, the detection network used in this embodiment is divided into a backbone, a neck and a head. Figure 2 The CFR in stands for Collaborative Feature Refinement.

[0094] The backbone feature extraction module is used to perform multiple convolution operations on the input labeled image. For ease of description, this embodiment takes 5 times as an example. Five convolution operations are used to downsample the input insulator image by 32 times to obtain coding layer features of different scales. , , , , . Set the lowest resolution The encoded features are obtained after convolution activation After the feature is input, the feature pyramid is Decoding using the bottom-up path , ,in , The feature map sizes correspond to 1 / 16 and 1 / 8 of the original image resolution, respectively, and then The features are obtained after convolution activation operation Features, yes Perform top-down path decoding to obtain , ,in , The feature map sizes correspond to 1 / 16 and 1 / 32 of the original image resolution respectively. , , The feature input corresponds to the detection head of the level. Figure 3 , Figure 3 An output result provided by this embodiment is shown. Figure 3 The prediction boxes in the figure represent: insulator, confidence level 0.71; insulator, confidence level 0.86; insulator segment disconnected, confidence level 0.89.

[0095] In an exemplary embodiment, each level of features is passed through a corresponding multi-receptive field collaborative refinement module to obtain refined features of the image to be tested, including:

[0096] The hierarchical features are input into the collaborative weight allocation submodule in the multi-receptive field collaborative refinement module to obtain the full-pixel weights corresponding to the hierarchical features; the hierarchical features are convolved, normalized and activated to obtain the normalized features corresponding to the hierarchical features; the hierarchical features are multi-level extracted and refined based on the full-pixel weights to obtain the step-by-step refined features corresponding to the current hierarchical features; the hierarchical features, normalized features and step-by-step refined features are added together to obtain the refined features corresponding to the hierarchical features.

[0097] Using the features extracted in the above steps , , , Input the multi-receptive field collaborative refinement module respectively. Figure 4 As shown in Figure 4 An architectural diagram of the multi-receptive field collaborative refinement module provided in this embodiment is shown, where CWD represents the cooperative weight distribution module (Cooperative Weight Distribution module), and CBS represents Conv+BN+Sigmoid, i.e., after the convolution layer, the normalization layer (Batch Normalization), the activation function (Sigmoid), other operations include: element-wise addition (Element-wise Addition), element-wise multiplication (Element-wise multiplication), activation (Sigmoid), and concatenation (concatenate).

[0098] In this embodiment, after multiple CFR modules, for the convenience of description, the hierarchical features input to the CFR module are recorded as features F. The process of feature refinement by the multi-receptive field collaborative refinement module proposed in this embodiment is as follows:

[0099] Input level features (Hereinafter referred to as feature ) to the collaborative weight allocation submodule to obtain the current level features The corresponding full pixel weight ;

[0100] Input Features To the 3*3 convolution layer (Conv1), get the features ; Input features To the 5*5 convolution layer (Conv2), get the features ; Input features To the 7*7 convolution layer (Conv3), the features are obtained ;

[0101] Input Features After passing through the convolution layer, normalization layer (Batch Normalization), activation function (Sigmoid), the normalized features are obtained. ;

[0102] The characteristics With full pixel weight Multiply pixel by pixel to get features ;

[0103] The characteristics With features Perform channel cascade operation, then go through convolution layer, normalization layer, activation function, and get the features ;

[0104] The characteristics With full pixel weight Multiply pixel by pixel to get features ;

[0105] The characteristics With features Perform channel cascade operation, then go through convolution layer, normalization layer, activation function, and get the features ;

[0106] The characteristics With full pixel weight Multiply pixel by pixel to get features ;

[0107] The characteristics , , , add pixel by pixel to obtain the refined features after multi-receptive field collaborative refinement .

[0108] The structure of the above collaborative weight allocation submodule is as follows:

[0109] Input Features , after global average pooling, 1*1 convolution layer, Relu activation function, 1*1 convolution layer, Relu activation function, the characteristics of aggregated channel information are obtained ;

[0110] Input Features , after 1*1 convolution layer, Relu activation function, 1*1 convolution layer, Relu activation function, the features of aggregated spatial information are obtained ;

[0111] Will , After pixel-by-pixel multiplication, the activation function is used to obtain the full pixel weight. .

[0112] Using full pixel weights , guiding the input feature map to be extracted and refined step by step under the feature extraction mode of gradually expanding the receptive field, thereby realizing multiple enhancements of important information in the feature map and effectively suppressing irrelevant features.

[0113] In an exemplary embodiment, the refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those of the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested, including:

[0114] See also Figure 2 ,exist Figure 2 In , there are refined features that have been through the feature pyramid and path enhancement process, for example, , , It is the refined features of different scales in the FPN (Feature Pyramid Networks) process; , , Refinement features of different scales in the PAN (Path Aggregation Network) process; features with the same superscript have the same scale.

[0115] Multi-receptive field collaborative refinement module Figure 2 In the neck part of the image, a feature refinement operation with unchanged input and output dimensions is performed during the up and down sampling process. The refined features obtained by the multi-receptive field collaborative refinement module in the PAN process are concatenated with the hierarchical features of the same scale in the FPN process and then input into the detection head. For example, Figure 2 In the example, and After the combination, input the detection head, and After the combination, input the detection head, and After the combination, input the detection head.

[0116] In order to explain the content of the scheme of this implementation in more detail, the following is a most detailed embodiment for description:

[0117] The images of insulators and insulator defects are collected, and the LabelImg tool is used to annotate the locations of insulators and insulator defects in the images. The image categories are annotated as insulator breakage, flashover, drop, and insulator, forming the Insulator Defect Detection Dataset (IDDS).

[0118] An insulator defect detection network based on multi-receptive field collaborative refinement of a single-stage detector is constructed. Features from each level of the feature pyramid are refined through the multi-receptive field collaborative refinement module of the corresponding level to enhance the expressiveness of the features.

[0119] The insulator image is sent to the insulator defect detection network based on multi-receptive field collaborative refinement of a single-stage detector. After multi-receptive field collaborative refinement of the hierarchical features in the feature pyramid, the detection image of the insulator defect is output through the detection head. Ground-Truth (true target) is used to constrain the network training process through three loss functions to complete the insulator defect detection task.

[0120] Specifically, the construction of the insulator defect detection method based on multi-receptive field collaborative refinement of a single-stage detector includes the following steps:

[0121] An insulator defect detection method based on a single-stage detector is constructed. The network includes: a backbone feature extraction module, a feature pyramid multi-scale feature fusion module, and a detection head module.

[0122] Specifically, Figure 2 As shown in Figure 2, five convolution operations are used in the backbone feature extraction module to downsample the input insulator image by 32 times to obtain coding layer features of different scales. , , , , . Set the lowest resolution The encoded features are obtained after convolution activation After the feature is input, the feature pyramid is Decoding using the bottom-up path , ,in , The feature map sizes correspond to 1 / 16 and 1 / 8 of the original image resolution, respectively, and then The features are obtained after convolution activation operation Features, yes Perform top-down path decoding to obtain , ,in , The feature map sizes correspond to 1 / 16 and 1 / 32 of the original image resolution respectively. , , The feature input corresponds to the detection head of the level.

[0123] A multi-receptive field collaborative refinement module is constructed, and each level output of the feature pyramid multi-scale feature fusion module is used to refine the features through the multi-receptive field collaborative refinement module of the corresponding level.

[0124] Specifically, using the features extracted in the above steps , , , Input the multi-receptive field collaborative refinement module respectively. Input the lowest resolution of the feature pyramid For example, the feature Input to the multi-receptive field collaborative refinement module. The process of feature refinement by the multi-receptive field collaborative refinement module can be seen as follows:

[0125] First, see Figure 5 and formula 5, Figure 5 The architecture diagram of the collaborative weight allocation submodule (CWD module) provided in this embodiment is shown. The full pixel weight is obtained based on formula 5. ;

[0126] Formula 5:

[0127] ;

[0128] in represents the channel feature map, is the spatial feature map, is the global average pooling operation, is a 1×1 convolution kernel, is the Relu activation function, is element-wise multiplication, is the Sigmoid activation function.

[0129] Based on formula 6, the refined feature map is obtained , Formula 6:

[0130] ;

[0131] in It represents a convolution operation with a kernel size of 3, a padding of 1, and a dilation rate of 1. represents the intermediate generated feature map, Represents the feature map after refinement by the CFR module. The full pixel weight is generated by interactive learning of the channel and spatial information of the feature map. The full pixel weight is used to guide the input feature map to be extracted and refined step by step in the feature extraction mode of gradually expanding the receptive field, thereby achieving multiple enhancements of important information in the feature map and effectively suppressing irrelevant features.

[0132] The insulator image is input into the insulator defect detection network based on multi-receptive field collaborative refinement of a single-stage detector. After feature extraction, multi-receptive field collaborative refinement and multi-scale feature fusion, the detection image is input into the detection head of the corresponding level to output the detection image. Finally, the output prediction image and the Ground-Truth loss value are used to constrain the training process of the entire detection network.

[0133] The defect detection method provided in this application is run on the NVIDIA RTX3090 GPU, and the code is implemented based on the Pytorch framework. The proposed network is optimized using the SGD optimizer, and the initial learning rate batch size is set to 0.01 and 8 respectively, and the cosine annealing decay strategy is used to dynamically update the learning rate. During training, the images input to the network are uniformly adjusted to 640×640, and image enhancement strategies such as random rotation are used to improve the generalization ability of the model. In order to verify the effectiveness of the defect detection method provided in this application in improving detection accuracy, the defect detection method provided in this application is compared with the current 6 advanced detection algorithms. The comparison methods are mainly RetinaNet, Faster R-CNN withfpn, SparseR-CNN, YOLOX, RTMDet, and LiteYOLO-ID.

[0134] Please refer to Table 1, which shows the comparison results of the defect detection method proposed in this application and other detection methods on the IDDS data set.

[0135] Table 1:

[0136]

[0137] A quantitative evaluation was conducted on the IDDS dataset with six currently advanced detection algorithms. Table 1 shows the AP detection accuracy of various detection algorithms for four target categories and the overall MAP index. The results show that the defect detection method provided by this application shows the best performance for the detection accuracy of the four categories of Breakage, Pollution-flashover, Drop, and Insulator, as well as the overall MAP index. Compared with the six detection algorithms RetinaNet, Faster R-CNN with fpn, SparseR-CNN, YOLOX, RTMDet, and LiteYOLO-ID, the MAP50 index on the dataset was improved by 6.5%, 9.9%, 7.1%, 7.6%, 13.9%, and 6.2%, respectively, which proves the effectiveness of this method.

[0138] The embodiment of the present application generates a full-pixel weight map by interactively learning the channel and spatial information of the feature map, and uses the weight map to guide the input feature map to extract and refine the features layer by layer in a feature extraction mode that gradually expands the receptive field, thereby achieving multiple enhancements of important information in the feature map while effectively suppressing irrelevant features. The present application can enhance the expression capabilities of features of different scales in the feature pyramid. The present application can effectively improve the accuracy of insulator defect detection while ensuring real-time detection, and is suitable for defect detection tasks of insulators on transmission lines.

[0139] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0140] Based on the same inventive concept, the embodiment of the present application also provides a defect detection device for implementing the defect detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more defect detection device embodiments provided below can refer to the limitations on the defect detection method above, and will not be repeated here.

[0141] In an exemplary embodiment, Figure 6As shown, a defect detection device 600 is provided, comprising: an acquisition module 601, a construction module 602, a first generation module 603, a second generation module 604 and a third generation module 605, wherein:

[0142] An acquisition module 601 is used to acquire a labeled image and determine a defect detection data set based on the labeled image;

[0143] A construction module 602 is used to construct an initial detection network, train the initial detection network based on a defect detection data set, and obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0144] A first generating module 603 is used to obtain multiple levels of features of the image to be tested based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module;

[0145] The second generation module 604 is used to obtain the refined features of the image to be tested by passing the features of each level through the corresponding multi-receptive field collaborative refinement module;

[0146] The third generation module 605 is used to combine the refined features of the image to be tested after path enhancement with the hierarchical features of the same scale as those of the feature pyramid and input them into the detection head module to obtain the detection result of the image to be tested.

[0147] In an exemplary embodiment, the acquisition module 601 is further configured to:

[0148] Acquire the original image including the target to be measured in the power system;

[0149] Annotating the target to be measured and the target defect category in the original image to obtain an annotated image, wherein the target to be measured includes an insulator, and the target defect category includes at least one of insulator breakage, flashover, or insulator segment string loss;

[0150] A collection of annotated images is determined as a defect detection dataset.

[0151] In an exemplary embodiment, the construction module 602 is further configured to:

[0152] Training the initial detection network based on the defect detection dataset and the preset loss function;

[0153] When the evaluation index of the initial detection network meets the preset index, the current initial detection network is determined as the optimized detection network.

[0154] In an exemplary embodiment, the first generating module 603 is further configured to:

[0155] Based on the feature extraction module, the image to be tested is downsampled multiple times to obtain coding layer features of multiple different scales;

[0156] Based on the feature fusion module, the features of each coding layer are decoded to obtain multiple levels of features.

[0157] In an exemplary embodiment, the second generating module 604 is further configured to:

[0158] The hierarchical features are input into the collaborative weight allocation submodule in the multi-receptive field collaborative refinement module to obtain the full pixel weights corresponding to the hierarchical features;

[0159] Perform convolution operation, normalization and activation on the hierarchical features to obtain the normalized features corresponding to the hierarchical features;

[0160] Perform multi-level extraction and refinement of hierarchical features based on full pixel weights to obtain the level-by-level refined features corresponding to the current hierarchical features;

[0161] The hierarchical features, normalized features and step-by-step refined features are added together to obtain the refined features corresponding to the hierarchical features.

[0162] Each module in the above defect detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0163] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store defect detection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a defect detection method is implemented.

[0164] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0165] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0166] Construct an initial detection network, train the initial detection network based on the defect detection data set, and obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0167] Based on the feature extraction module, feature fusion module and multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0168] The features of each level are passed through the corresponding multi-receptive field collaborative refinement module to obtain the refined features of the image to be tested;

[0169] The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those obtained through the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0170] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0171] Construct an initial detection network, train the initial detection network based on the defect detection data set, and obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0172] Based on the feature extraction module, feature fusion module and multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0173] The features of each level are passed through the corresponding multi-receptive field collaborative refinement module to obtain the refined features of the image to be tested;

[0174] The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those obtained through the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0175] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0176] Acquire an annotated image, and determine a defect detection data set based on the annotated image;

[0177] Construct an initial detection network, train the initial detection network based on the defect detection data set, and obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module;

[0178] Based on the feature extraction module, feature fusion module and multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained;

[0179] The features of each level are passed through the corresponding multi-receptive field collaborative refinement module to obtain the refined features of the image to be tested;

[0180] The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those obtained through the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

[0181] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0182] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0183] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A defect detection method, characterized in that: The method comprises: Acquire an annotated image, and determine a defect detection data set based on the annotated image; Constructing an initial detection network, and training the initial detection network based on the defect detection data set to obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module; Based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module, multiple levels of features of the image to be tested are obtained; Passing each of the hierarchical features through a corresponding multi-receptive field collaborative refinement module to obtain a refined feature of the image to be tested; The refined features of the image to be tested that have undergone path enhancement are combined with the hierarchical features of the same scale as those of the feature pyramid and then input into the detection head module to obtain the detection result of the image to be tested.

2. The method according to claim 1, characterized in that The acquiring of the annotated image and determining the defect detection data set based on the annotated image includes: Acquire the original image including the target to be measured in the power system; Annotating the target to be measured and the target defect category in the original image to obtain the annotated image, wherein the target to be measured includes an insulator, and the target defect category includes at least one of insulator breakage, flashover, or insulator segment dropout; The set of labeled images is determined as the defect detection dataset.

3. The method according to claim 1, characterized in that The step of training the initial detection network based on the defect detection data set to obtain an optimized detection network includes: Training the initial detection network based on the defect detection data set and a preset loss function; When the evaluation index of the initial detection network meets the preset index, the current initial detection network is determined as the optimized detection network.

4. The method according to claim 3, characterized in that The preset loss functions include a bounding rectangle loss function, a target classification loss function and a confidence loss function.

5. The method according to claim 1, characterized in that The method of obtaining multiple levels of features of the image to be tested based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module includes: Based on the feature extraction module, the image to be tested is downsampled multiple times to obtain coding layer features of multiple different scales; Based on the feature fusion module, each of the coding layer features is decoded to obtain a plurality of the hierarchical features.

6. The method according to claim 5, characterized in that The step of subjecting each of the hierarchical features to a corresponding multi-receptive field collaborative refinement module to obtain the refined features of the image to be tested includes: Inputting the hierarchical features into the collaborative weight allocation submodule in the multi-receptive field collaborative refinement module to obtain the full pixel weights corresponding to the hierarchical features; Performing convolution operation, normalization and activation on the hierarchical features to obtain normalized features corresponding to the hierarchical features; Perform multi-level extraction and refinement on the hierarchical features based on the full pixel weights to obtain step-by-step refinement features corresponding to the current hierarchical features; The hierarchical feature, the normalized feature and the step-by-step refined feature are added together to obtain a refined feature corresponding to the hierarchical feature.

7. A defect detection device, characterized in that: The device comprises: An acquisition module, used for acquiring an annotated image and determining a defect detection data set based on the annotated image; A construction module, used to construct an initial detection network, train the initial detection network based on the defect detection data set, and obtain an optimized detection network, wherein the optimized detection network includes a feature extraction module, a feature fusion module, a multi-receptive field collaborative refinement module, and a detection head module; A first generation module, used for acquiring multiple levels of features of the image to be tested based on the feature extraction module, the feature fusion module and the multi-receptive field collaborative refinement module; A second generation module is used to obtain the refined features of the image to be tested by passing each of the hierarchical features through a corresponding multi-receptive field collaborative refinement module; The third generation module is used to combine the refined features of the image to be tested after path enhancement with the hierarchical features of the same scale as the feature pyramid and input them into the detection head module to obtain the detection result of the image to be tested.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images