Insulator fault detection method and device based on class cosine mapping

Abstract

The present invention discloses an insulator fault detection method and device based on class cosine mapping, which relates to the fields of artificial intelligence and machine vision, including: obtaining image data, fault labels and class labels from real data sets and synthetic data sets; using class cosine bias coding to encode class bias into image data, and using feature extraction network to extract spatial features in the encoding result; using class cosine bias decoding to alleviate bias information in spatial features to obtain class features, and using multi-label classifier to obtain classification loss of class features and class labels; using segmentation decoder to obtain mask loss of spatial features and fault labels; completing model training based on classification loss and mask loss. The present invention encodes the class differences of normal, faulty insulators and synthetic images into image data through cosine bias, guides the model to understand the distribution differences between synthetic images and real scenes, and finally improves the effectiveness of synthetic data in practice.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and machine vision, and in particular to an insulator fault detection method and device based on class cosine mapping. Background Art

[0002] Insulator fault detection is a key step in the automated maintenance of power systems. The insulators of the towers are located at high altitudes, and the faulty insulators will be repaired in time after being discovered, which makes it difficult to obtain images of faulty insulators on high-altitude towers. In order to solve the difficulty in data acquisition, synthesizing the images of faulty insulators with the images of high-altitude towers is a feasible solution. However, there are distribution differences between the synthetic images and the real scenes. Therefore, alleviating the distribution differences between the synthetic images and the real scenes and improving the availability of the synthetic images are issues that need to be addressed. Summary of the invention

[0003] In order to alleviate the problem of distribution differences between synthetic data and real scenes, the present invention provides an insulator fault detection method and device based on class cosine mapping, which uses class cosine bias to encode class bias information into image data, and uses an auxiliary multi-label classification optimization model to perceive class differences, thereby alleviating the distribution differences between synthetic images and real scenes, improving the usability of synthetic images, and making detection based on synthetic images more accurate.

[0004] The present invention adopts the following technical solution:

[0005] On the one hand, an insulator fault detection method based on class cosine mapping includes:

[0006] The training steps of the insulator fault detection model are as follows: image data, fault labels and category labels are obtained from real data sets and synthetic data sets; the category bias is encoded into the image data using category cosine bias encoding to obtain the encoding result, and the spatial features in the encoding result are extracted using the feature extraction network; the bias information in the spatial features is alleviated using category cosine bias decoding to obtain the category features, and the classification loss of the category features and category labels is obtained using a multi-label classifier; the mask loss of the spatial features and fault labels is obtained using a segmentation decoder; the insulator fault detection model is optimized based on the classification loss and mask loss to complete the model training;

[0007] In the insulator fault detection step, the trained insulator fault detection model is used to perform fault detection on the image data to be detected.

[0008] Preferably, image data, fault labels, and category labels are obtained from the real data set and the synthetic data set as follows:

[0009] ;

[0010] Among them, S1 and S2 are the real data set and the synthetic data set respectively. The real data set S1 includes background pictures without insulators, normal insulator pictures and faulty insulator pictures, and the synthetic data set S2 is the faulty insulator pictures under the background of high-altitude power towers; RS is the random sampling method, ~RS means sampling using the RS method, It means randomly sampling B samples from the union of S1 and S2 as training data for a training batch, and the sampling result is (x, s, t); is the image data of B samples; is the fault label; is the category label; the height of image data and fault label are both H, and the width is W; fault label It is a binary mask map. When the mask value of the spatial position is 0, it means that the position does not belong to the fault insulator. When the mask value of the spatial position is 1, it means that the position belongs to the fault insulator. Category label Three channels are used to indicate whether the sample has normal insulators, abnormal insulators, and whether it is a synthetic image.

[0011] Preferably, the category bias is encoded into the image data using category cosine bias encoding to obtain an encoding result, and the spatial features in the encoding result are extracted using a feature extraction network, as shown below:

[0012] ;

[0013] in, The category cosine weights obtained by cosine initialization, the three category cosine weight maps with height H and width W belong to the three categories of category label t respectively. Indicates the weight value of the class cosine weight map of the dth channel whose height is numbered i and width is numbered j; is the category indicated by the dth channel of the bth sample, B represents the number of samples; Norm is the regularization function; is the value of the spatial position of the image data x in the bth sample, on the cth channel, with the height number i and the width number j; is the encoding result after class cosine bias encoding of image data x using class cosine weight q. is the spatial position value of the mapping data in the bth sample, on the cth channel, with the height number i and the width number j; Net is the feature extraction network, and the result of feature extraction on the mapping data is the spatial feature , the number of channels of the spatial feature is C, the height is h, and the width is w.

[0014] Preferably, class cosine bias decoding is used to alleviate the bias information in the spatial features to obtain class features, and a multi-label classifier is used to obtain the classification loss of the class features and the class labels, as shown below:

[0015] ;

[0016] in, The restored cosine weight obtained by cosine initialization is equivalent to the scaled version of the category cosine weight q, that is, the height of the category cosine weight is scaled from H to h, and the width is scaled from W to w; Indicates the weight value of the restored cosine weight map of the dth channel with a height number of i and a width number of j; the restored cosine weight p is used to decode the result of the class cosine bias encoding; is the value of the spatial feature z at the spatial position with height number i and width number j in the cth channel of the bth sample; is the decoded category feature, is the value of the cth channel of the bth sample in the category feature; the multi-label classifier includes three parts: the cross entropy loss function CEL, the multi-label probability mapping method Sigmoid and the classifier Classifier. The classifier Classifier is used to align the category feature with the category label, the multi-label probability mapping method Sigmoid is used to independently map the category feature to the category probability, and the cross entropy loss function CEL is used to calculate the loss value of the category probability and the category label; L cls Classification loss calculated for the multi-label classifier.

[0017] Preferably, a segmentation decoder is used to obtain the mask loss of spatial features and fault labels as follows:

[0018] ;

[0019] Among them, the segmentation decoder contains the mask cross entropy loss function CEL m , probability mapping method Softmax and decoder Decoder are three parts. The decoder Decoder is used to align the spatial features with the fault labels. The probability mapping method Softmax is used to map the spatial features to the spatial category probabilities. The mask cross entropy loss function CEL m Used to calculate the loss value of spatial category probability and fault label; L mask The mask loss computed for the segmentation decoder.

[0020] Preferably, the method for optimizing the insulator fault detection model based on classification loss and mask loss is a stochastic gradient descent method.

[0021] Preferably, using a trained insulator fault detection model to perform fault detection on the image data to be detected specifically includes:

[0022] The image data to be detected is input into the trained feature extraction network to obtain spatial features, and the spatial features are input into the segmentation decoder. When the segmentation decoder calculates the spatial category probability, it outputs it, and sets the position where the spatial category probability value is greater than or equal to the preset probability value to 1, and sets the position where the spatial category probability value is less than the preset probability value to 0. The obtained binary mask image is the insulator fault detection result, and a value of 1 indicates that there is an insulator fault at that position.

[0023] On the other hand, an insulator fault detection device based on class cosine mapping includes:

[0024] The insulator fault detection model training module is used to obtain image data, fault labels and category labels from real data sets and synthetic data sets; class cosine bias coding is used to encode the class bias into the image data to obtain the encoding result, and the feature extraction network is used to extract the spatial features in the encoding result; class cosine bias decoding is used to alleviate the bias information in the spatial features to obtain the class features, and the multi-label classifier is used to obtain the classification loss of the class features and the class labels; the segmentation decoder is used to obtain the mask loss of the spatial features and the fault labels; the insulator fault detection model is optimized based on the classification loss and the mask loss to complete the model training;

[0025] The insulator fault detection module is used to perform fault detection on the image data to be detected using the trained insulator fault detection model.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention uses category cosine bias to encode category bias information into image data, and uses an auxiliary multi-label classification optimization model to enhance the perception of category differences, thereby alleviating the distribution difference between synthetic images and real scenes, improving the usability of synthetic images, and making detection based on synthetic images more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the insulator fault detection method based on class cosine mapping of the present invention;

[0029] Figure 2 A flow chart of the insulator fault detection model training steps of the present invention;

[0030] Figure 3 It is a structural block diagram of the insulator fault detection device based on class cosine mapping of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0032] See also Figure 1 As shown, the present invention provides an insulator fault detection method based on class cosine mapping, comprising: an insulator fault detection model training step S1 and an insulator fault detection step S2.

[0033] Specifically, the implementation of the insulator fault detection model training steps is as follows.

[0034] S11, obtain real insulator images, real tower background images and synthetic insulator images as training sets; each data has two types of labels, the first type of label is the category label, which indicates whether there is a normal insulator, whether there is a faulty insulator and whether it is a synthetic image; the second type of label is the fault label, which is a binary mask image of the same size as the image. When the value is 1, it indicates that the spatial position belongs to a faulty insulator, and when the value is 0, it indicates that the position does not belong to a faulty insulator.

[0035] S12, select the ResNet-50 model as the feature extraction network to build the insulator fault detection model.

[0036] S13, unify the height of the image data to H, the width to W, and use random flipping for data enhancement.

[0037] S14, training an insulator fault detection model using an insulator fault detection method based on class cosine offset coding, including the following steps.

[0038] S141, image data, fault labels and category labels are obtained from the real data set and the synthetic data set, as shown in formula (1):

[0039] ; (1)

[0040] Among them, S1 and S2 are the real data set and the synthetic data set respectively. The real data set S1 includes background pictures without insulators, normal insulator pictures and faulty insulator pictures, and the synthetic data set S2 is the faulty insulator pictures under the background of high-altitude power towers; RS is the random sampling method, ~RS means sampling using the RS method, It means randomly sampling B samples from the union of S1 and S2 as training data for a training batch, and the sampling result is (x, s, t); , and They are the image data, fault labels and category labels of B samples, the height of image data and fault labels are both H, and the width is both W; fault labels It is a binary mask map. When the mask value of the spatial position is 0, it means that the position does not belong to the fault insulator. When the mask value of the spatial position is 1, it means that the position belongs to the fault insulator. Category label Three channels are used to indicate whether the sample has normal insulators, abnormal insulators, and whether it is a synthetic image.

[0041] S142, using the category cosine offset coding to encode the category offset into the image data, and using the feature extraction network to extract the spatial features in the encoding result, as shown in formula (2):

[0042] ; (2)

[0043] in, The category cosine weights obtained by cosine initialization, the three category cosine weight maps with height H and width W belong to the three categories of category label t respectively. Indicates the weight value of the class cosine weight map of the dth channel whose height is numbered i and width is numbered j; is the category indicated by the dth channel of the bth sample; the category cosine weight q has two characteristics. First, in a single category cosine weight map, the closer the spatial position of two weight values ​​is, the smaller the difference between the two weight values ​​is, which ensures the continuity of adjacent data. Second, the weight distribution of category cosine weight maps of different categories is different, which ensures the differentiation of different categories. Norm is the regularization function. is the value of the spatial position of the image data x in the bth sample, on the cth channel, with the height number i and the width number j; is the encoding result after class cosine bias encoding of image data x using class cosine weight q. is the spatial position value of the mapping data in the bth sample, on the cth channel, with the height number i and the width number j; Net is the feature extraction network, and the result of feature extraction on the mapping data is the spatial feature , the number of channels of the spatial feature is C, the height is h, and the width is w.

[0044] S143, using category cosine bias decoding to alleviate the bias information in the spatial features to obtain category features, and using a multi-label classifier to obtain the classification loss of the category features and the category labels, as shown in formula (3):

[0045] ; (3)

[0046] in, The restored cosine weight obtained by cosine initialization is equivalent to the scaled version of the category cosine weight q (that is, the height of the category cosine weight is scaled from H to h, and the width is scaled from W to w). Indicates the weight value of the restored cosine weight map of the dth channel with a height number of i and a width number of j; the restored cosine weight p is used to decode the result of the class cosine bias encoding, remove the influence of the class cosine weight, and thus alleviate the influence of the class bias information on the classification in the multi-label classifier; is the value of the spatial feature z at the spatial position with height number i and width number j in the cth channel of the bth sample; is the decoded category feature, is the value of the cth channel of the bth sample in the category feature; the multi-label classifier consists of three parts, the cross entropy loss function CEL, the multi-label probability mapping method Sigmoid and the classifier Classifier, where the classifier Classifier is used to align the category feature with the category label, the multi-label probability mapping method Sigmoid is used to independently map the category feature to the category probability (that is, the mapping process of the category probability is not affected by the category features other than the category), and the cross entropy loss function CEL is used to calculate the loss value of the category probability and the category label; L cls Classification loss calculated for the multi-label classifier.

[0047] S144, using the segmentation decoder to obtain the mask loss of spatial features and fault labels, as shown in formula (4):

[0048] ; (4)

[0049] Among them, the segmentation decoder consists of three parts, the mask cross entropy loss function CEL m , probability mapping method Softmax and decoder Decoder, where the decoder Decoder is used to align the spatial features with the fault labels, the probability mapping method Softmax is used to map the spatial features to the spatial category probability, the larger the spatial category probability, the higher the probability that the spatial position belongs to the fault insulator, the mask cross entropy loss function CEL m Used to calculate the loss value of spatial category probability and fault label; L mask The mask loss computed for the segmentation decoder.

[0050] S5, stochastic gradient descent, according to the classification loss L cls With the mask loss L mask Stochastic gradient descent is performed on the insulator fault detection model to complete the model training.

[0051] Furthermore, in this implementation case, an insulator fault detection method based on category cosine bias coding, the reasoning process of the insulator fault detection step is: input the image data into a trained feature extraction network to obtain spatial features, input the spatial features into a segmentation decoder, and output the segmentation decoder when it calculates the spatial category probability, setting the position where the spatial category probability value is greater than or equal to 0.5 to 1, and the position where the spatial category probability value is less than 0.5 to 0. The obtained binary mask image is the insulator fault detection result, and a value of 1 indicates that there is an insulator fault at that position.

[0052] The key of the present invention is to use the class cosine bias to encode the class bias information into the image data, and use the auxiliary multi-label classification optimization model to perceive the class difference, thereby alleviating the distribution difference between the synthetic image and the real scene and improving the usability of the synthetic image. Therefore, the present invention can be widely used in automated maintenance applications in power systems.

[0053] See also Figure 3 As shown, the present invention also discloses an insulator fault detection device based on class cosine mapping, comprising:

[0054] The insulator fault detection model training module 301 is used to obtain image data, fault labels and category labels from real data sets and synthetic data sets; use category cosine offset coding to encode category offsets into image data to obtain encoding results, and use feature extraction networks to extract spatial features in the encoding results; use category cosine offset decoding to alleviate the bias information in spatial features to obtain category features, and use a multi-label classifier to obtain the classification loss of category features and category labels; use a segmentation decoder to obtain the mask loss of spatial features and fault labels; optimize the insulator fault detection model based on the classification loss and mask loss to complete model training;

[0055] The insulator fault detection module 302 is used to perform fault detection on the image data to be detected using the trained insulator fault detection model.

[0056] The specific implementation of each module of an insulator fault detection device based on class cosine mapping is the same as the insulator fault detection method based on class cosine mapping, and the present embodiment will not repeat the description.

[0057] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described in this patent are only illustrative and not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. An insulator fault detection method based on class cosine mapping, characterized in that: include: Insulator fault detection model training steps, obtain image data, fault labels and category labels from real data sets and synthetic data sets; The category bias is encoded into the image data using category cosine bias coding to obtain the encoding result, and the spatial features in the encoding result are extracted using a feature extraction network; The class cosine bias decoding is used to alleviate the bias information in the spatial features to obtain the class features, and the multi-label classifier is used to obtain the classification loss of the class features and the class labels; the segmentation decoder is used to obtain the mask loss of the spatial features and the fault labels; the insulator fault detection model is optimized based on the classification loss and mask loss to complete the model training; Insulator fault detection step, using the trained insulator fault detection model to perform fault detection on the image data to be detected; The category bias is encoded into the image data using category cosine bias encoding to obtain the encoding result, and the spatial features in the encoding result are extracted using the feature extraction network, as shown below: ； in, The category cosine weights obtained by cosine initialization, the three category cosine weight maps with height H and width W belong to the three categories of category label t, q d,i,j Indicates the weight value of the class cosine weight map of the dth channel with the height number i and the width number j; t b,d is the category indicated by the dth channel of the bth sample, B represents the number of samples; Norm is the regularization function; x b,c,i,j is the value of the spatial position of the image data x in the bth sample, on the cth channel, with the height number i and the width number j; is the encoding result after class cosine bias encoding of image data x using class cosine weight q, y b,c,i,j is the spatial position value of the mapping data in the bth sample, on the cth channel, with the height number i and the width number j; Net is the feature extraction network, and the result of feature extraction on the mapping data is the spatial feature The number of channels of the spatial feature is C, the height is h, and the width is w; The category cosine bias decoding is used to alleviate the bias information in the spatial features to obtain the category features, and the multi-label classifier is used to obtain the classification loss of the category features and the category labels, as shown below: ； in, is the restored cosine weight obtained by cosine initialization, which is equivalent to the scaled version of the category cosine weight q, that is, the height of the category cosine weight is scaled from H to h, and the width is scaled from W to w; p d,i,j Indicates the weight value of the restored cosine weight map of the dth channel with the height number i and the width number j; the restored cosine weight p is used to decode the result of the class cosine bias encoding; z b,c,i,j is the value of the spatial feature z at the spatial position with height number i and width number j in the cth channel of the bth sample; is the decoded category feature, r b,c is the value of the cth channel of the bth sample in the category feature; the multi-label classifier includes three parts: the cross entropy loss function CEL, the multi-label probability mapping method Sigmoid and the classifier Classifier. The classifier Classifier is used to align the category feature with the category label, the multi-label probability mapping method Sigmoid is used to independently map the category feature to the category probability, and the cross entropy loss function CEL is used to calculate the loss value of the category probability and the category label; L cls The classification loss calculated for the multi-label classifier; is the category label; The segmentation decoder is used to obtain the mask loss of spatial features and fault labels as follows: L mask =CEL m (Softmax(Decoder(z)),s); Among them, the segmentation decoder contains the mask cross entropy loss function CEL m , probability mapping method Softmax and decoder Decoder are three parts. The decoder Decoder is used to align the spatial features with the fault labels. The probability mapping method Softmax is used to map the spatial features to the spatial category probabilities. The mask cross entropy loss function CEL m Used to calculate the loss value of spatial category probability and fault label; L mask The mask loss calculated for the segmentation decoder; Fault label.

2. The insulator fault detection method based on class cosine mapping according to claim 1 is characterized in that: Get image data, fault labels, and category labels from the real data set and synthetic data set as follows: (x,s,t)~RS(S1∪S2,B); Among them, S1 and S2 are the real data set and the synthetic data set respectively. The real data set S1 includes background pictures without insulators, normal insulator pictures and faulty insulator pictures, and the synthetic data set S2 is the faulty insulator pictures under the background of high-altitude power towers; RS is the random sampling method, ~RS means sampling using the RS method, (x, s, t)~RS(S1∪S2, B) means randomly sampling B samples from the union of S1 and S2 as training data for a training batch, and the sampling result is (x, s, t); is the image data of B samples; the height of the image data and the fault label are both H, and the width is W; the fault label It is a binary mask map. When the mask value of the spatial position is 0, it means that the position does not belong to the fault insulator. When the mask value of the spatial position is 1, it means that the position belongs to the fault insulator. Category label Three channels are used to indicate whether the sample has normal insulators, abnormal insulators, and whether it is a synthetic image.

3. The insulator fault detection method based on class cosine mapping according to claim 1, characterized in that: The method for optimizing the insulator fault detection model based on classification loss and mask loss is the stochastic gradient descent method.

4. The insulator fault detection method based on class cosine mapping according to claim 1, characterized in that: Use the trained insulator fault detection model to perform fault detection on the image data to be detected, including: The image data to be detected is input into the trained feature extraction network to obtain spatial features, and the spatial features are input into the segmentation decoder. When the segmentation decoder calculates the spatial category probability, it outputs it, and sets the position where the spatial category probability value is greater than or equal to the preset probability value to 1, and sets the position where the spatial category probability value is less than the preset probability value to 0. The obtained binary mask image is the insulator fault detection result, and a value of 1 indicates that there is an insulator fault at that position.

5. An insulator fault detection device based on class cosine mapping, characterized in that: include: Insulator fault detection model training module, used to obtain image data, fault labels and category labels from real data sets and synthetic data sets; The category bias is encoded into the image data using category cosine bias coding to obtain the encoding result, and the spatial features in the encoding result are extracted using a feature extraction network; The class cosine bias decoding is used to alleviate the bias information in the spatial features to obtain the class features, and the multi-label classifier is used to obtain the classification loss of the class features and the class labels; the segmentation decoder is used to obtain the mask loss of the spatial features and the fault labels; the insulator fault detection model is optimized based on the classification loss and mask loss to complete the model training; An insulator fault detection module is used to perform fault detection on the image data to be detected using a trained insulator fault detection model; The category bias is encoded into the image data using category cosine bias encoding to obtain the encoding result, and the spatial features in the encoding result are extracted using the feature extraction network, as shown below: ； in, The category cosine weights obtained by cosine initialization, the three category cosine weight maps with height H and width W belong to the three categories of category label t, q d,i,j Indicates the weight value of the class cosine weight map of the dth channel whose height number is i and width number is j; t b,d is the category indicated by the dth channel of the bth sample, B represents the number of samples; Norm is the regularization function; x b,c,i,j is the value of the spatial position of the image data x in the bth sample, on the cth channel, with the height number i and the width number j; is the encoding result after class cosine bias encoding of image data x using class cosine weight q, y b,c,i,j is the spatial position value of the mapping data in the bth sample, on the cth channel, with the height number i and the width number j; Net is the feature extraction network, and the result of feature extraction on the mapping data is the spatial feature The number of channels of the spatial feature is C, the height is h, and the width is w; The category cosine bias decoding is used to alleviate the bias information in the spatial features to obtain the category features, and the multi-label classifier is used to obtain the classification loss of the category features and the category labels, as shown below: ； in, is the restored cosine weight obtained by cosine initialization, which is equivalent to the scaled version of the category cosine weight q, that is, the height of the category cosine weight is scaled from H to h, and the width is scaled from W to w; p d,i,j Indicates the weight value of the restored cosine weight map of the dth channel with the height number i and the width number j; the restored cosine weight p is used to decode the result of the class cosine bias encoding; z b,c,i,j is the value of the spatial feature z at the spatial position with height number i and width number j in the cth channel of the bth sample; is the decoded category feature, r b,c is the value of the cth channel of the bth sample in the category feature; the multi-label classifier includes three parts: the cross entropy loss function CEL, the multi-label probability mapping method Sigmoid and the classifier Classifier. The classifier Classifier is used to align the category feature with the category label, the multi-label probability mapping method Sigmoid is used to independently map the category feature to the category probability, and the cross entropy loss function CEL is used to calculate the loss value of the category probability and the category label; L cls The classification loss calculated for the multi-label classifier; is the category label; The segmentation decoder is used to obtain the mask loss of spatial features and fault labels as follows: L mask =CEL m (Softmax(Decoder(z)),s); Among them, the segmentation decoder contains the mask cross entropy loss function CEL m , probability mapping method Softmax and decoder Decoder are three parts. The decoder Decoder is used to align the spatial features with the fault labels. The probability mapping method Softmax is used to map the spatial features to the spatial category probabilities. The mask cross entropy loss function CEL m Used to calculate the loss value of spatial category probability and fault label; L mask The mask loss calculated for the segmentation decoder; Fault label.

Images