Insulator image segmentation and defect detection method and device, storage medium and equipment

Abstract

The present application relates to a kind of insulator image segmentation and defect detection method, device, storage medium and equipment, insulator image is segmented using sliding window, insulator image dataset is constructed, and data enhancement is carried out to insulator image dataset, insulator defect detection model is trained using enhanced dataset, and insulator defect detection solidification model is obtained, insulator image is input into the insulator defect detection solidification model trained and carried out insulator defect point detection, the present application introduces attention module and weighted bidirectional cross-scale feature pyramid network in insulator defect detection solidification model, the representation ability after feature fusion is enhanced, the interference of complex background is reduced, to obtain the detection result of the insulator defect point with high accuracy, improve the detection efficiency of insulator defect.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to an insulator defect detection method and device, a storage medium and equipment. BACKGROUND

[0002] Insulators are a kind of special insulating control, which mainly realizes electrical insulation and mechanical fixation, and ensures the reliable operation of power transmission lines, and plays an important role in overhead transmission lines. In recent years, with the rapid development of unmanned aerial vehicle technology, a large number of pictures of power equipment and lines are taken by unmanned aerial vehicles for manual annotation to achieve the purpose of inspection. However, due to the large number of pictures taken by unmanned aerial vehicles, manual annotation is slow and laborious, and the manual annotation method is prone to miss or mislabel due to fatigue. Therefore, it is of great research value and significance to use a computer to process insulator images and input the collected images into a target detection model to realize automatic annotation of insulator defects.

[0003] Due to the high image resolution and complex image background of the aerial image taken by the unmanned aerial vehicle, the current existing detection model has poor detection effect on the aerial image. SUMMARY

[0004] In view of the problems in the above background art, the present application provides an insulator image segmentation and defect detection method, device, storage medium and equipment, which segments the insulator image by using a sliding window, constructs an insulator image dataset, and performs data enhancement on the insulator image dataset to enhance the robustness of the insulator defect model. The CA attention mechanism is introduced into the insulator defect detection model, and a weighted bidirectional cross-scale feature pyramid network structure is adopted, which can improve the accuracy and speed of insulator defect detection. The technical solution is as follows:

[0005] In the first aspect, the present application provides an insulator image segmentation and defect detection method, which includes the following steps:

[0006] Obtaining an insulator image;

[0007] Segmenting the insulator image by using a sliding window, constructing an insulator image dataset, and performing data enhancement processing on the insulator image dataset;

[0008] Constructing an insulator defect detection model, wherein the insulator defect detection model includes an attention module and a weighted bidirectional cross-scale feature pyramid network;

[0009] Training the insulator defect detection model by using the data-enhanced insulator image dataset to obtain an insulator defect detection solidification model;

[0010] The insulator defect detection solidification model is used for detecting the insulator defect in the image, and detection results and defect positions of the insulator defect are obtained.

[0011] In a second aspect, an insulator image segmentation and defect detection device is provided, comprising:

[0012] An image acquisition module is configured to acquire an insulator image.

[0013] A sliding window segmentation module is configured to segment the insulator image by using a sliding window, construct an insulator image dataset, and perform data enhancement processing on the insulator image dataset.

[0014] A model construction module is configured to construct an insulator defect detection model, wherein the insulator defect detection model comprises an attention module and a weighted bidirectional cross-scale feature pyramid network.

[0015] A model training module is configured to train the insulator defect detection model by using the data-enhanced insulator image dataset, and obtain an insulator defect detection solidification model.

[0016] A defect recognition module is configured to detect the insulator defect in the image by using the insulator defect detection solidification model, and obtain detection results and defect positions of the insulator defect.

[0017] In a third aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the insulator image segmentation and defect detection method according to any one of the above aspects are implemented.

[0018] In a fourth aspect, a computer device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable by the processor. When the processor executes the computer program, the steps of the insulator image segmentation and defect detection method according to any one of the above aspects are implemented.

[0019] In the embodiments of the present application, the insulator image is segmented by using a sliding window, an insulator image dataset is constructed, the insulator image dataset is data-enhanced, the insulator defect detection model is trained by using the enhanced dataset, the insulator defect detection solidification model is obtained, the insulator image is input into the trained insulator defect detection solidification model for insulator defect point detection, the attention module and the weighted bidirectional cross-scale feature pyramid network are introduced into the insulator defect detection solidification model, the representation ability after feature fusion is enhanced, the interference of a complex background is reduced, and thus the detection results of the insulator defect points with high accuracy are obtained, and the detection efficiency of the insulator defect is improved.

[0020] For better understanding and implementation, the present application is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flow chart of an insulator image segmentation and defect detection method in an embodiment of the present application;

[0022] Figure 2 A structural diagram of a YOLOv5 model in an embodiment of the present application;

[0023] Figure 3 A structural diagram of a weighted bidirectional feature fusion pyramid network in an embodiment of the present application;

[0024] Figure 4 A structural diagram of an attention module in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of a mask image in an embodiment of the present application;

[0026] Figure 6 A structural diagram of a U-net model in an embodiment of the present application;

[0027] Figure 7 A comparison diagram of loss value curves and dice score curves of insulator image segmentation using the method described in the present application and the existing U-net model;

[0028] Figure 8 A structural diagram of an insulator image segmentation and defect detection device in an embodiment of the present application;

[0029] Figure 9 A comparison diagram of mask images obtained by image segmentation of defect insulator original images using the FCN model and the insulator image segmentation and defect detection method described in the present application;

[0030] Figure 10 A comparison diagram of mask images obtained by image segmentation of complex background original images using the FCN model and the insulator image segmentation and defect detection method described in the present application;

[0031] Figure 11 A schematic diagram of identifying defect points of long-range insulator images and close-range insulator images using the insulator image segmentation and defect detection method described in the present application. DETAILED DESCRIPTION

[0032] To make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0033] It should be noted that the embodiments described are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0034] The terms used in the embodiments of the present application are only for the purpose of describing the specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms "a", "an" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0035] The following description refers to the accompanying drawings. In the following description, same numbers in different drawings represent the same or similar elements unless otherwise represented. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not necessarily mean a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0036] In addition, in the description of the present application, "several" means two or more, unless otherwise stated. The "and / or" describes the association between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0037] As Figure 1 shown, the embodiments of the present application provide an insulator image segmentation and defect detection method, comprising the following steps.

[0038] Step S1: acquiring an insulator image;

[0039] Step S2: segmenting the insulator image by using a sliding window, constructing an insulator image dataset, and performing data enhancement processing on the insulator image dataset;

[0040] The size of the sliding window can be set according to the size of the insulator image.

[0041] In one embodiment, the preset sliding window includes a sliding window with a size of 1024*1024 and a sliding window with a size of 2048*2048, and the insulator image is divided by two sliding windows with different sizes. Preferably, the adjacent sliding window overlap rate is set to 0.5 to increase the amount of data in the data set.

[0042] The data enhancement processing includes the steps of performing basic image transformation on the image, such as scaling, translation, folding, rotation, and brightness change, to balance the number of positive and negative samples in the data set.

[0043] Step S3: constructing an insulator defect detection model, wherein the insulator defect detection model introduces an attention module and a weighted bidirectional cross-scale feature pyramid network;

[0044] The insulator defect detection model is constructed based on a YOLOv5 model. Since the aerial insulator pictures have the characteristics of high resolution, small target, and complex background, etc., in order to accurately locate the self-explosion insulator region, the model needs to have high detection and positioning accuracy. At the same time, the number of aerial pictures is large, which requires high speed of image processing. Although the traditional target detection algorithm is intuitive, it has high time complexity and poor robustness, and is not suitable for image detection containing self-explosion insulators and other images containing complex and variable backgrounds. The target detection algorithm based on deep learning has become the mainstream now, and has strong practicability. Considering that the YOLO model has the characteristics of low hardware requirement, simple structure, high detection accuracy, and fast recognition speed, etc., YOLOv5 is selected as the core of the insulator defect detection in the embodiments of the present application.

[0045] The core idea of the YOLO model is to regard target detection as a single regression problem, and directly obtain the bounding box coordinates and class probability from the image pixels.

[0046] Specifically, the network structure of the insulator defect detection model is as shown in Figure 2 The insulator defect detection model is divided into three parts: Backbone, Neck, and Output. The Backbone is used for feature extraction, the Neck is used for feature fusion and enhances the feature representation ability, and the Output outputs the detection result. The Backbone of the insulator defect detection model adopts CSPDarknet53, and a spatial pyramid pool SPP block (SPPBlock) in a deep convolutional network is added to the CSPDarknet53. The neck of the insulator defect detection model adopts a weighted bidirectional cross-scale feature fusion network to replace the original PAN structure as a method of parameter aggregation for different backbone layers. As shown in Figure 3As shown, the weighted bidirectional cross-scale feature fusion network structure is different from the common feature pyramid, and the weighted bidirectional cross-scale feature pyramid network structure sets a weight coefficient for each feature before feature fusion, which can adjust the importance of different scale features to enhance the representation ability after feature fusion. The weighted bidirectional cross-scale feature pyramid network establishes a cross-scale connection between the high-level and the bottom layer network, which can obtain global features containing high-level and bottom layer semantic information, and can improve the insulator defect recognition rate in high-resolution aerial images.

[0047] As shown in Figure 3 , the feature fusion calculation formula of the weighted bidirectional cross-scale feature pyramid network is as follows:

[0048]

[0049] Among them, is the input feature of the fourth layer, is the intermediate feature of the fourth layer, is the input feature of the fifth layer, w1 is the first feature weight coefficient of the input feature of the fourth layer, and w2 is the feature weight coefficient of the input feature of the fifth layer, is the input feature of the fifth layer after adjusting the size of the feature map, the input feature of the fifth layer after adjusting the size of the feature map is consistent with the feature map size of the intermediate feature of the fourth layer; ε is a preset positive number, and ε is used to make the denominator not 0.

[0050]

[0051] Among them, is the output feature of the fourth layer, w1' is the second feature weight coefficient of the input feature of the fourth layer, w2' is the feature weight coefficient of the intermediate feature of the fourth layer, and w3' is the feature weight coefficient of the output feature of the third layer. is the output feature of the third layer, is the output feature of the third layer after adjusting the size of the feature map, and the output feature of the third layer after adjusting the size of the feature map is consistent with the feature map size of the output feature of the fourth layer.

[0052] As shown in Figure 2 , the insulator defect detection model also introduces an attention module CoordAtt to obtain the region of interest, reduce the interference of complex background, and make it have better performance in target detection. Specifically, the structure of the attention module is as Figure 4The attention module uses adaptive average pooling layers in the vertical and horizontal directions to extract features from each feature channel, respectively. The generated feature maps are spliced, then pass through a 1x1 convolution, and the feature maps are divided into two feature maps in the vertical and horizontal directions along the spatial direction. Then, the two 1x1 convolutions are performed respectively to obtain the attention weights in the vertical and horizontal spatial directions. The attention weights are multiplied with the input feature maps to obtain the feature maps with attention weights.

[0053] Step S4: training the insulator defect detection model using the data-augmented insulator image dataset to obtain an insulator defect detection solidification model;

[0054] The training parameters of the insulator defect detection model are set as follows: the training batch size is 16, the initial learning rate is 0.01, the cosine annealing algorithm is used to update the learning rate, the maximum batch number is 150, and the training network resolution is 864*864.

[0055] During the training of the algorithm model, it is easy to fall into a saddle surface, that is, part of the points on the slope surface are rising, and part of the points are descending, such as the minimum value on the z-axis and the maximum value on the x-axis. The first derivative of the loss with respect to the parameter on the saddle surface is 0, and the positive and negative values of the second derivative are different. Since the gradient is 0, the model cannot further update the parameters, so the model training is easy to fall into the saddle surface and no longer update.

[0056] The cosine annealing algorithm is also called the learning rate annealing algorithm. After each batch training, the learning rate is reduced a little, and when it is reduced to a specified value, the learning rate is immediately increased to the initial value, and the process is repeated. The learning rate decays after each training because the amount of parameter adjustment required by the model decreases, so a smaller learning rate is needed. When the model training reaches a certain time, it may fall into a saddle surface with a very small gradient, so the learning rate is increased to the initial value, hoping to use a large learning rate to update the parameters and make the model out of the saddle surface.

[0057] Step S5: detecting the insulator defects in the image through the insulator defect detection solidification model to obtain the detection results and defect positions of the insulator defects;

[0058] Specifically, the step of detecting the insulator defects in the image through the insulator defect detection solidification model specifically includes:

[0059] The insulator image is input into the insulator defect detection solidification model, the insulator image features are extracted and fused by the insulator defect detection solidification model, the insulator defect position prediction box is obtained through the prediction head, and the detection results and defect positions of the insulator defects are obtained.

[0060] In the embodiment of the present application, the insulator image is segmented by using a sliding window, an insulator image dataset is constructed, data augmentation is performed on the insulator image dataset, the insulator defect detection model is trained by using the augmented dataset, and an insulator defect detection solidification model is obtained. The insulator image is input into the trained insulator defect detection solidification model for insulator defect point detection. The present application introduces an attention module and a weighted bidirectional cross-scale feature pyramid network into the insulator defect detection solidification model, enhances the representation ability after feature fusion, reduces the interference of complex background, and thus obtains accurate insulator defect point detection results. The insulator image to be detected is input into the trained insulator defect detection solidification model, and the detection result of the insulator defect point can be automatically output, thereby improving the detection efficiency of the insulator defect.

[0061] As shown in Table 1, which is a comparison table of detection results of the YOLOv5 model, the existing YOLOv5 model, the existing YOLOv4 model and the Faster-RCNN model on the test dataset, it can be seen that the recognition accuracy of the YOLOv5 model is higher and the detection speed is faster.

[0062] Table 1: Comparison table of detection results

[0063] Model Average processing time / ms Recognition accuracy / % The YOLOv5 model described in the embodiments of the present application 4 98.6 The existing YOLOv5 model 4 97.7 The existing YOLOv4 model 53 88.0 Faster-RCNN model 605 93.0

[0064] In one exemplary embodiment, after the step of segmenting the insulator image by using a sliding window, the method further comprises:

[0065] Down-sampling the plurality of sub-images obtained after segmentation according to a set sampling value;

[0066] Inputting the plurality of sub-images after down-sampling into a U-net model for image segmentation to obtain a plurality of sub-mask images;

[0067] Performing morphological filtering on the plurality of sub-mask images and synthesizing a mask image consistent with the size of the insulator image;

[0068] The sampling value can be set according to actual user needs.

[0069] The U-net model is a deep learning-based image segmentation model. The U-net model is composed of a contracting path and an expansive path. The contracting path is a typical convolutional neural network structure. It repeatedly adopts a structure of two convolutional layers and one max-pooling layer. After each pooling operation, the dimension of the feature map is doubled. In the expansive path, a deconvolution operation is first performed to halve the dimension of the feature map. Then, the feature map obtained by cropping the contracting path is spliced to form a 2-fold feature map. Two convolutional layers are then used for feature extraction, and the structure is repeated. In the last output layer, two convolutional layers are used to map a 64-dimensional feature map to a 2-dimensional output map.

[0070] Since the insulator image has the characteristics of small target, complex background, blurred boundary, and complex gradient, more high-resolution information is required. The U-net model combines low-resolution information (providing object category recognition basis) and high-resolution information (providing accurate segmentation positioning basis). The U-net model can realize accurate segmentation of the insulator image.

[0071] Morphological filtering is commonly used for image de-noising and enhancement. Specifically, the morphological filtering can include erosion, dilation, opening operation, closing operation, and the like.

[0072] As shown in Figure 5 The mask image is a binary image composed of 0 and 1. In the embodiment of the present application, the image region value of the insulator is 1, and the value of other regions is 0.

[0073] In the embodiment of the present application, the insulator image is divided into a plurality of sub-images by using a sliding window, and the plurality of sub-images are down-sampled according to a set sampling value. The plurality of down-sampled sub-images are input into the U-net model for image segmentation to obtain each sub-mask image. Morphological filtering is performed on the each sub-mask image, and a mask image with a size consistent with the insulator image is synthesized, thereby realizing accurate recognition and extraction of the insulator.

[0074] As shown in Figure 6As shown, the size of the down-sampled sub-images is 512*512, and the network structure of the U-net model includes two symmetrical parts: a contracting path and an expansive path. The former part of the network is the same as a common convolutional network, using 3*3 convolution and pooling down-sampling to obtain shallow features and deep features, and capture the relationship between pixels in the image. The latter part of the network is basically symmetrical with the former part, using 3*3 convolution and up-sampling. When up-sampling, the deep features and shallow features are combined by cascading to achieve the purpose of accurately positioning the part to be segmented in the image. This feature fusion operation fully utilizes the shallow features and deep features, and can achieve better segmentation results.

[0075] In an exemplary embodiment, the plurality of sub-images include sub-images and their corresponding sub-mask images with artificial annotations;

[0076] Before the step of inputting the down-sampled plurality of sub-images into the U-net model for image segmentation to obtain a plurality of sub-mask images, further comprising:

[0077] According to a preset sampling rate, randomly select part of the sub-images as training samples;

[0078] The training parameters of the U-net model are set in the following manner: the training batch size is 4, the learning rate is 10e -6 , the network resolution is 512*512, the maximum batch number is 50, and the loss probability is 0.5;

[0079] According to the training samples and the training parameters of the U-net model, the U-net model is trained.

[0080] The preset sampling rate can be set according to actual user needs.

[0081] In an exemplary embodiment, before the step of training the U-net model, further comprising:

[0082] Perform connected component analysis on the plurality of sub-mask images with artificial annotations to obtain the connected components of the plurality of sub-mask images;

[0083] According to the connected components of the sub-mask images, determine whether the structure of the insulator in the sub-image is complete, and delete the sub-image whose structure of the insulator is not complete;

[0084] By connected component analysis of each sub-mask image, the image in which the structure of the insulator is damaged due to sliding window cutting is removed.

[0085] Preferably, the training samples include subgraphs containing insulators and subgraphs not containing insulators. Inputting the above training samples into the U-net model for training can effectively improve the error detection capability of the U-net model.

[0086] By adding a dropout mechanism to the U-net model, where dropout refers to temporarily deactivating neurons from the network with a certain probability during network training, overfitting of the U-net model can be avoided.

[0087] Because insulator images are small targets with complex backgrounds and blurred boundaries, they often require a large amount of high-resolution information. In a preferred embodiment, the U-net model uses the binary cross-entropy loss function (BCEWithLogitsLoss) as the loss function. The steps for training the U-net model based on the training samples and the training parameters of the U-net model include:

[0088] Batch normalization is performed on the features obtained after each convolution calculation;

[0089] The loss function of the U-net model is optimized using the RMSprop optimizer;

[0090] Batch normalization refers to normalizing the output results so that the mean of the output signal (each dimension) is 0 and the variance is 1. By performing batch normalization after each convolution calculation, gradient vanishing can be prevented, the generalization ability of the U-net model can be improved, and the network can be allowed to learn at a higher rate, thereby accelerating convergence.

[0091] The RMSprop optimizer uses the weighted average of the squared derivatives to calculate the gradient. When the gradient values ​​of the weights and biases are large, the update amplitude is reduced by dividing by the square root of the accumulated gradient, thereby correcting the oscillation amplitude and improving the convergence speed of the network function.

[0092] In an exemplary embodiment, before the step of inputting the downsampled sub-images into the U-net model for image segmentation, the method further includes:

[0093] The aforementioned subgraphs are then normalized.

[0094] Normalization involves using the image's invariant matrix to find a set of parameters that can eliminate the influence of other transformation functions on the image transformation, converting it into a unique standard form to resist affine transformations, and summarizing the statistical distribution of samples to avoid overfitting of the U-net model.

[0095] In an exemplary embodiment, the Dice coefficient is used to evaluate the mask image obtained in this application embodiment. The Dice coefficient is a set similarity metric function; a value closer to 1 indicates a stronger match between the predicted result and the expert annotation result. Specifically, after the step of synthesizing a mask image with the same size as the insulator image, the following steps are included:

[0096] Obtain manually annotated images of insulators;

[0097] The precision of the mask image is obtained in the following manner:

[0098]

[0099] Where Dice(A,B) represents the precision of the mask image, A represents the labeled area in the manually annotated insulator image, and B represents the region of interest in the mask image.

[0100] like Figure 7 As shown, these are comparison curves of the loss value and Dice score obtained by image segmentation of an insulator image using the method described in the embodiments of this application and the existing U-net model, respectively. Figure (a) is a schematic diagram of the loss value and Dice score obtained by image segmentation of an insulator image using the method described in the embodiments of this application, and Figure (b) is a schematic diagram of the loss value and Dice score obtained by image segmentation of an insulator image using the existing U-net model. Figure 7 As can be seen, the method described in this application embodiment has less oscillation in the loss curve compared to the existing U-net model, and has a better convergence effect.

[0101] like Figure 8 As shown in the figure, this application embodiment also provides an insulator image segmentation and defect detection device, including:

[0102] Image acquisition module 1 is used to acquire images of insulators;

[0103] The sliding window segmentation module 2 is used to segment the insulator image using a sliding window, construct an insulator image dataset, and perform data augmentation processing on the insulator image dataset;

[0104] Model building module 3 is used to build an insulator defect detection model, wherein the insulator defect detection model includes an attention module and a weighted bidirectional cross-scale feature pyramid network;

[0105] Model training module 4 is used to train the insulator defect detection model using the data-enhanced insulator image dataset to obtain a fixed insulator defect detection model.

[0106] The defect identification module 5 is used to detect insulator defects in the image through the insulator defect detection solidification model, and obtain the detection results and defect locations of the insulator defects.

[0107] It should be noted that the insulator image segmentation and defect detection device provided in the above embodiments is only illustrated by the division of the above functional modules when performing the insulator image segmentation and defect detection method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the insulator image segmentation and defect detection device and the insulator image segmentation and defect detection method provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0108] This application also provides a computer-readable storage medium storing a computer program thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of an insulator image segmentation and defect detection method as described in any of the above claims.

[0109] The embodiments of this application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0110] This application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable by the processor. When the processor executes the computer program, it implements the steps of an insulator image segmentation and defect detection method as described in any of the above claims.

[0111] Table 2 below shows a comparison of the segmentation results of image segmentation of 126 insulator images using the FCN model and the U-net model described in this application in an exemplary embodiment.

[0112] Table 2 Comparison of Segmentation Results

[0113]

[0114] like Figure 9-10 As shown, this is a comparison of mask images obtained by segmenting the original image of a defective insulator and the original image of a complex background using the FCN model and the insulator image segmentation and defect detection method described in this application in an exemplary embodiment. Figure (a) is the original image containing the defective insulator, Figure (b) is the mask image segmented by the FCN model, Figure (c) is the mask image segmented by U-net, Figure (d) is the original image of the complex background, Figure (e) is the mask image segmented by the FCN model, and Figure (f) is the mask image segmented by U-net.

[0115] From the images above and Table 2, it can be concluded that: when the defect area and lighting conditions are complex, the FCN model does not respond well to the insulator area, while the U-net model in this embodiment can accurately complete the segmentation; when the image has complex foreground and background, the U-net model can segment the insulator well, while the FCN model performs poorly in segmenting the boundaries of complex areas. Compared to FCN, the U-net model in this embodiment can better cope with different lighting conditions and complex foreground and background, and has a stronger generalization ability.

[0116] like Figure 11 As shown, this is a schematic diagram illustrating the identification of defect points in distant and close-up insulator images using the insulator image segmentation and defect detection method described in this application. Figure (g) shows the identification diagram of the close-up insulator image, and Figure (h) shows the identification diagram of the distant insulator image. As can be seen from the figures, the insulator image segmentation and defect detection method described in this application can accurately identify defect points in both close-up and distant insulator images.

[0117] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.

Claims

1. An insulator image segmentation and defect detection method, characterized in that, The method comprises the following steps: An insulator image is acquired; The insulator image is divided into two different sizes by two sliding windows to construct an insulator image dataset, and data augmentation is performed on the insulator image dataset; An insulator defect detection model is constructed, wherein the insulator defect detection model introduces an attention module and a weighted bidirectional cross-scale feature pyramid network, and the insulator defect detection model is constructed based on a YOLOv5 target detection model; The insulator defect detection model is trained by using the data-augmented insulator image dataset to obtain an insulator defect detection solidification model; The insulator defect detection solidification model is used to detect insulator defects in an image to obtain a detection result and a defect position of the insulator defects; The step of detecting insulator defects in an image by using the insulator defect detection solidification model comprises the following steps: The insulator image is input into the insulator defect detection solidification model, the insulator image features are extracted and fused by the insulator defect detection solidification model, an insulator defect position prediction box is obtained by a prediction head, and a detection result and a defect position of the insulator defects are obtained.

2. The insulator image segmentation and defect detection method of claim 1, wherein The sliding window comprises a sliding window with a size of 1024*1024 and a sliding window with a size of 2048*2048.

3. The insulator image segmentation and defect detection method of claim 1, wherein The data augmentation on the insulator image dataset comprises the following steps: scaling, translation, folding, rotation, and brightness change of the image.

4. The insulator image segmentation and defect detection method of claim 1, wherein After the insulator image is divided by using the sliding window, the following steps are further included: According to a set sampling value, the plurality of sub-images obtained after division are down-sampled; The down-sampled sub-images are input into a U-net model for image segmentation to obtain a plurality of sub-mask images; The plurality of sub-mask images are subjected to morphological filtering, and a mask image with a size consistent with the insulator image is synthesized.

5. The insulator image segmentation and defect detection method of claim 4, wherein The plurality of sub-images comprise sub-images and corresponding sub-mask images with artificial annotations; Before the step of inputting the down-sampled sub-images into the U-net model for image segmentation to obtain a plurality of sub-mask images, the following steps are further included: According to a preset sampling rate, a part of the sub-images are randomly extracted as training samples; The training parameters of the U-net model are set in the following manner: the training batch size is 4, the learning rate is 10e -6 , the network resolution is 512x512, the maximum batch number is 50, and the loss probability is 0.

5. According to the training samples and training parameters of the U-net model, the U-net model is trained.

6. The insulator image segmentation and defect detection method of claim 4, wherein After the step of synthesizing the mask image with a size consistent with the insulator image, the following steps are further included: An insulator image with artificial annotations is acquired; The accuracy of the mask image is acquired in the following manner: wherein Dice(A, B) is the accuracy of the mask image, A is a labeled region in the insulator image with artificial annotations, and B is a region of interest in the mask image.

7. An insulator image segmentation and defect detection apparatus, characterized by comprising: The method comprises the following steps: An image acquisition module is configured to acquire an insulator image; A sliding window division module is configured to divide the insulator image into two different sizes by two sliding windows to construct an insulator image dataset, and perform data augmentation on the insulator image dataset; The model construction module is configured to construct an insulator defect detection model, wherein the insulator defect detection model comprises an attention module and a weighted bidirectional cross-scale feature pyramid network; and the insulator defect detection model is constructed based on a YOLOv5 target detection model. The model training module is configured to train the insulator defect detection model by using the data-enhanced insulator image dataset, to obtain an insulator defect detection solidified model. The defect identification module is configured to detect insulator defects in an image by using the insulator defect detection solidified model, to obtain a detection result and a defect position of the insulator defects; the defect identification module is configured to input the insulator image into the insulator defect detection solidified model, to extract and fuse features of the insulator image by using the insulator defect detection solidified model, to obtain an insulator defect position prediction box by using a prediction head, and to obtain the detection result and the defect position of the insulator defects.

8. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by a processor to implement the steps of the insulator image segmentation and defect detection method according to any one of claims 1-6.

9. A computer device, comprising: The computer program is executed by a processor to implement the steps of the insulator image segmentation and defect detection method according to any one of claims 1-6.

Images