Distribution line power target segmentation method and device based on spatial difference feature enhancement, terminal equipment and storage medium

Abstract

The invention discloses a distribution line electric power target segmentation method and device based on spatial difference feature enhancement, terminal equipment and a storage medium, and belongs to the technical field of distribution line electric power target segmentation, and the method comprises the steps: obtaining a distribution line infrared image containing an electric power target, and inputting the image into a preset electric power target segmentation model, obtaining a multi-scale image feature sequence; then, according to the multi-scale image feature sequence and a preset image enhancement decoder, generating an enhanced distribution line infrared image, and then performing difference to obtain difference features; performing feature enhancement on the multi-scale image feature sequence according to the differential features to obtain an enhanced multi-scale image feature sequence; and finally, inputting the enhanced multi-scale image feature sequence into a preset instance segmentation decoder, and performing power target segmentation. According to the invention, the problem of high possibility of missing detection and false detection of the power target of the distribution line can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of distribution line power target segmentation, and in particular to a distribution line power target segmentation method, device, terminal equipment and storage medium based on spatial difference feature enhancement. Background Art

[0002] In recent years, the rapid development of technologies such as drones, infrared sensors, and deep learning has significantly improved the intelligence and automation of distribution line inspections. During routine distribution line inspections, identifying and spatially locating electrical targets such as insulators, lightning arresters, and wire clamps (i.e., instance segmentation) is a prerequisite for determining whether they have defects such as overheating and damage. This is also essential for ensuring the safe and stable operation of the power grid.

[0003] In existing technology, infrared sensors are typically used to capture images of distribution lines, which are then processed to extract power targets. However, due to hardware limitations, infrared sensors generally have low resolution and a small number of pixels. This low-resolution imaging reduces the texture and details of power targets, resulting in frequent missed detections (i.e., failure to detect) and false detections (misidentification of background images). Summary of the Invention

[0004] The present invention provides a method, apparatus, terminal device and storage medium for segmenting power targets in distribution lines based on spatial differential feature enhancement, which can solve the problem in the prior art of missed detection and false detection of power targets during the power target segmentation process due to the lack of texture and details of power targets on the acquired infrared images of distribution lines.

[0005] An embodiment of the present invention provides a method for segmenting power targets of distribution lines based on spatial difference feature enhancement, comprising:

[0006] Acquire an infrared image of a power distribution line containing a power target, and input the image into a preset power target segmentation model, so that a preset image encoder built into the preset power target segmentation model performs multi-scale feature extraction on the infrared image of the power distribution line to obtain a multi-scale image feature sequence;

[0007] Inputting the multi-scale image feature sequence into a preset image enhancement decoder built into the preset power target segmentation model, so that the preset image enhancement decoder reconstructs the distribution line infrared image according to the multi-scale image feature sequence to generate an enhanced distribution line infrared image;

[0008] performing a differential operation on the power distribution line infrared image and the enhanced power distribution line infrared image to obtain differential features;

[0009] Performing feature enhancement on the multi-scale image feature sequence according to the differential features to obtain an enhanced multi-scale image feature sequence;

[0010] The enhanced multi-scale image feature sequence is input into a preset instance segmentation decoder built into the preset power target segmentation model, so that the preset instance segmentation decoder performs power target segmentation on the distribution line infrared image according to the enhanced multi-scale image feature sequence.

[0011] Furthermore, performing a differential operation on the power distribution line infrared image and the enhanced power distribution line infrared image to obtain differential features includes:

[0012] Calculating the absolute value of the difference between the enhanced infrared image of the power distribution line and the infrared image of the power distribution line to obtain a differential image;

[0013] A projection operation is performed on the difference image to obtain the difference feature.

[0014] Furthermore, the performing feature enhancement based on the differential feature and the multi-scale image feature sequence to obtain an enhanced multi-scale image feature sequence includes:

[0015] Convolving the differential features to generate a query matrix;

[0016] Convolving the multi-scale image feature sequence to generate a key matrix and a value matrix;

[0017] Performing an activation operation on the query matrix and the key matrix to generate a cross attention graph;

[0018] An enhanced multi-scale image feature sequence is obtained according to the product of the cross attention map and the value matrix.

[0019] Furthermore, the training of the preset power target segmentation model includes:

[0020] Acquire a plurality of infrared sample images of power distribution lines, an infrared image truth value of each infrared sample image of the power distribution lines, a category truth value of the power target, a spatial coordinate frame truth value of the power target, and a boundary truth value of the power target;

[0021] Inputting the plurality of distribution line infrared sample images, infrared image true values, category true values, spatial coordinate frame true values, and boundary true values ​​into the power target segmentation model to be trained for iterative training until the number of iterations is not less than a preset iteration number threshold, thereby generating a trained preset power target segmentation model;

[0022] In each iteration, it is determined whether the current number of iterations is less than a preset iteration number threshold;

[0023] If the current number of iterations is less than a preset iteration threshold, multi-scale feature extraction is performed on the current distribution line infrared sample image through a preset image encoder built into the current power target segmentation model to obtain a current multi-scale sample feature sequence; the current multi-scale sample feature sequence is input into an image enhancement decoder built into the current power target segmentation model to generate a current enhanced distribution line infrared sample image; a difference operation is performed on the current distribution line infrared sample image and the current enhanced distribution line infrared sample image to generate a current differential sample feature;

[0024] The current multi-scale sample feature sequence is enhanced according to the current differential feature to obtain the current enhanced multi-scale sample feature sequence; the current enhanced multi-scale sample feature sequence is input into the instance segmentation decoder built into the current power target segmentation model to perform power target segmentation, and the current predicted power target category, predicted power target spatial coordinate frame, and predicted power target boundary are obtained;

[0025] According to the current enhanced distribution line infrared sample image, the current predicted power target category, the predicted power target spatial coordinate frame, the predicted power target boundary, the corresponding infrared image true value, the corresponding category true value, the corresponding spatial coordinate frame true value and the corresponding boundary true value, the current total loss function is calculated, and after adjusting the parameters of the current power target segmentation model according to the current total loss function, iterative training continues;

[0026] Otherwise, the current power target segmentation model is used as the trained preset power target segmentation model.

[0027] Furthermore, the current total loss function is calculated based on the current enhanced distribution line infrared sample image, the current predicted power target category, the predicted power target spatial coordinate frame, the predicted power target boundary, the corresponding infrared image true value, the corresponding category true value, the corresponding spatial coordinate frame true value, and the corresponding boundary true value, including:

[0028] Calculate a current first mean absolute error loss function based on the current enhanced first distribution line infrared sample image and the corresponding infrared image true value;

[0029] Calculate the current first cross entropy loss function according to the current predicted power target category and the corresponding category true value, and use the current first cross entropy loss function as the current classification loss function of the power target;

[0030] Calculate the current second mean absolute error function based on the current predicted power target spatial coordinate frame and the corresponding spatial coordinate frame true value, and use the current second mean absolute error function as the current regression loss function of the power target;

[0031] Calculate the current second cross entropy loss function according to the current predicted power target boundary and the corresponding boundary true value, and use the current second cross entropy loss function as the current segmentation loss function of the power target;

[0032] The current total loss function is calculated based on the current first mean absolute error loss function, classification loss function, current regression loss function, current segmentation loss function, preset image enhancement weight factor, preset regression loss weight factor and preset segmentation loss weight factor.

[0033] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0034] The present invention provides a distribution line power target segmentation device based on spatial difference feature enhancement, comprising:

[0035] Multi-scale image feature sequence extraction module, image reconstruction module, differential feature extraction module, feature enhancement module and power target segmentation module;

[0036] The multi-scale image feature sequence extraction module is used to obtain an infrared image of a distribution line containing an electric power target and input it into a preset electric power target segmentation model, so that a preset image encoder built into the preset electric power target segmentation model performs multi-scale feature extraction on the infrared image of the distribution line to obtain a multi-scale image feature sequence;

[0037] The image reconstruction module is configured to input the multi-scale image feature sequence into a preset image enhancement decoder built into the preset power target segmentation model, so that the preset image enhancement decoder reconstructs the infrared image of the distribution line according to the multi-scale image feature sequence to generate an enhanced infrared image of the distribution line;

[0038] The differential feature extraction module is used to perform a differential operation on the power distribution line infrared image and the enhanced power distribution line infrared image to obtain differential features;

[0039] The feature enhancement module is used to perform feature enhancement on the multi-scale image feature sequence according to the differential feature to obtain an enhanced multi-scale image feature sequence;

[0040] The power target segmentation module is used to input the enhanced multi-scale image feature sequence into the preset instance segmentation decoder built into the preset power target segmentation model, so that the preset instance segmentation decoder performs power target segmentation on the distribution line infrared image based on the enhanced multi-scale image feature sequence.

[0041] Furthermore, the differential feature extraction module includes:

[0042] Image calculation unit and image projection unit;

[0043] The image calculation unit is used to calculate the absolute value of the difference between the enhanced distribution line infrared image and the distribution line infrared image to obtain a differential image;

[0044] The image projection unit is used to perform a projection operation on the differential image to obtain the differential feature.

[0045] Furthermore, the feature enhancement module includes:

[0046] Differential feature convolution unit, multi-scale image feature convolution unit, cross attention map generation unit and multi-scale image feature enhancement unit;

[0047] The differential feature convolution unit is used to perform convolution on the differential features to generate a query matrix;

[0048] The multi-scale image feature convolution unit is used to convolve the multi-scale image feature sequence to generate a key matrix and a value matrix;

[0049] The cross-attention map generating unit is configured to perform an activation operation on the query matrix and the key matrix to generate a cross-attention map;

[0050] The multi-scale image feature enhancement unit is used to obtain an enhanced multi-scale image feature sequence according to the product of the cross attention map and the value matrix.

[0051] Based on the above method embodiment, the present invention provides a corresponding terminal device embodiment;

[0052] The present invention provides a terminal device, including a processor, a memory, and a computer program stored in the above memory and configured to be executed by the above processor. When the above processor executes the above computer program, it implements the above-mentioned distribution line power target segmentation method based on spatial differential feature enhancement in any embodiment of the present invention.

[0053] Based on the above method embodiment, the present invention provides a storage medium embodiment;

[0054] The present invention provides a storage medium comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the method for power target segmentation of distribution lines based on spatial differential feature enhancement according to any embodiment of the present invention is implemented.

[0055] The embodiments of the present invention have the following beneficial effects:

[0056] The present invention provides a method, apparatus, terminal device and storage medium for segmenting power targets of distribution lines based on spatial differential feature enhancement. The method comprises: firstly obtaining an infrared image of a distribution line containing power targets and inputting the image into a preset power target segmentation model, so that a preset image encoder built into the preset power target segmentation model performs multi-scale feature extraction on the infrared image of the distribution line to obtain a multi-scale image feature sequence; then inputting the multi-scale image feature sequence into a preset image enhancement decoder built into the preset power target segmentation model, so that the preset image enhancement decoder extracts the multi-scale image feature sequence according to the multi-scale image feature sequence. The infrared image of the power distribution line is reconstructed to generate an enhanced infrared image of the power distribution line; then, a differential operation is performed on the infrared image of the power distribution line and the enhanced infrared image of the power distribution line to obtain differential features; then, the multi-scale image feature sequence is feature enhanced according to the differential features to obtain an enhanced multi-scale image feature sequence; finally, the enhanced multi-scale image feature sequence is input into the preset instance segmentation decoder built into the preset power target segmentation model, so that the preset instance segmentation decoder performs power target segmentation on the infrared image of the power distribution line according to the enhanced multi-scale image feature sequence. Therefore, in the preset power target segmentation model of the present invention, the extracted multi-scale image feature sequence is used to reconstruct the infrared image of the power distribution line, so that the features on the reconstructed infrared image of the power distribution line are enhanced, and the differential features obtained by the differential operation can highlight the difference between the power target and the surrounding background, thereby enhancing the saliency of the area where the power target is located. Therefore, by enhancing the infrared image of the power distribution line and highlighting the difference between the power target and the surrounding background, the texture and details of the power target on the image are increased, and then in the final power target segmentation process, the problems of missed detection and false detection of the power target are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the implementation. Obviously, the drawings described below are only some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0058] Figure 1 This is a flow chart of a method for power target segmentation of distribution lines based on spatial difference feature enhancement provided by one embodiment of the present invention.

[0059] Figure 2It is a structural schematic diagram of a distribution line power target segmentation device based on spatial difference feature enhancement provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0062] In the description of the embodiments of this application, the technical terms "first" and "second" are used only to distinguish different objects and should not be understood to indicate or imply relative importance or implicitly specify the quantity, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, the meaning of "plurality" is more than two, unless otherwise clearly and specifically defined.

[0063] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] In the description of the embodiments of this application, the term "and / or" is simply a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0065] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0066] In the description of the embodiments of the present application, unless otherwise expressly specified or limited, technical terms such as "installed," "connected," "connected," and "fixed" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integration; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; internal connections between two components or interactions between two components. Those skilled in the art can understand the specific meanings of the above terms in the embodiments of the present application based on specific circumstances.

[0067] See also Figure 1 To address the problem of missed detection and false detection of power targets in the prior art, which is caused by the generally low resolution and small number of pixels of infrared sensors. This low-resolution imaging results in less texture and details of power targets. An embodiment of the present invention provides a method for segmenting power targets on distribution lines based on spatial differential feature enhancement, comprising:

[0068] Step S101: Acquire an infrared image of a power distribution line containing a power target and input it into a preset power target segmentation model, so that a preset image encoder built into the preset power target segmentation model performs multi-scale feature extraction on the infrared image of the power distribution line to obtain a multi-scale image feature sequence;

[0069] Specifically, the above-mentioned power targets refer to the hardware in the distribution lines, including: lightning arresters, transformer bushings, knife switches, insulators, circuit breakers, drop-out fuses and cable terminal heads. The infrared images of the above-mentioned distribution lines can be obtained through infrared sensors.

[0070] Specifically, the preset image encoder is provided with several convolutional layers. The infrared image of the distribution line is input into the preset image encoder and multi-scale image feature sequence extraction is performed through the convolutional layers. The context information in different ranges can be captured. The extracted multi-scale image feature sequence can be obtained by the following formula:

[0071]

[0072] Where, f represents the multi-scale image feature sequence, f p represents the p-th multi-scale image feature, P represents the total number of multi-scale image features, the value of P depends on the architectural design of the image encoder, Enc represents the image encoder, and I represents the infrared image of the distribution line.

[0073] Specifically, the image encoder is trained in advance on a large-scale image dataset to obtain a trained preset image encoder. The large-scale image dataset can be ImageNet (Image Network) or COCO (Microsoft Common Objects in Context). The preset image encoder can be a commonly used classification network, such as VGG (Visual Geometry Group Image Encoder), ResNet-50 (Residual Network-50 Image Encoder), or DenseNet (Densely Connected Convolutional Network Image Encoder).

[0074] Step S102: Inputting the multi-scale image feature sequence into a preset image enhancement decoder built into the preset power target segmentation model, so that the preset image enhancement decoder reconstructs the infrared image of the power distribution line according to the multi-scale image feature sequence to generate an enhanced infrared image of the power distribution line;

[0075] Specifically, the preset image enhancement decoder and the above-mentioned preset image encoder have a symmetrical structure. The feature extraction operations of the two are the same and are both implemented by a large number of convolutional layers. The upsampling operation in the preset image enhancement decoder can be implemented by deconvolution layers, pixel shuffles, and bicubic linear interpolation. The preset image enhancement decoder learns the spatial details and contextual information of power targets in the multi-scale image feature sequence through feature extraction and upsampling operations, and generates an enhanced distribution line infrared image. This generation process can be expressed as follows:

[0076] I E =Dec E (f)

[0077] Where, I E represents the enhanced infrared image of the power distribution line, Dec E Indicates the preset image enhancement decoder.

[0078] Optimally, a pre-set image enhancement decoder enables deep extraction and reconstruction of multi-scale spatial details and contextual information in infrared images. This mechanism uses hierarchical feature reconstruction, capturing feature information from local details to global semantics layer by layer, significantly improving the ability to distinguish areas where power targets are located.

[0079] Step S103: performing a differential operation on the infrared image of the power distribution line and the enhanced infrared image of the power distribution line to obtain differential features;

[0080] In a preferred embodiment, performing a differential operation on the infrared image of the power distribution line and the enhanced infrared image of the power distribution line to obtain differential features includes:

[0081] Calculating the absolute value of the difference between the enhanced infrared image of the power distribution line and the infrared image of the power distribution line to obtain a differential image;

[0082] Specifically, the differential image is calculated according to the following formula:

[0083] D=|I E -I|

[0084] Where D represents the difference image.

[0085] The above differential image is projected to obtain the above differential features.

[0086] Specifically, the differential features are calculated according to the following formula:

[0087] d=Proj(D)

[0088] Where d represents the differential feature and Proj represents the projection operation.

[0089] Specifically, the projection operation aims to project the differential image from a low-dimensional linear space to a high-dimensional feature space. It consists of two convolutional layers and a Sigmoid activation function. The convolutional kernel size of each convolutional layer is 3×3, the stride length is 1, and the number of input and output channels is 3 and 256, respectively. The Sigmoid activation function suppresses noise activation by weighting and calibrating the differential features.

[0090] In this preferred embodiment, differential features are obtained by performing differential analysis on the infrared image of the power distribution line and the enhanced infrared image of the power distribution line.

[0091] Step S104: performing feature enhancement based on the differential features and the multi-scale image feature sequence to obtain an enhanced multi-scale image feature sequence;

[0092] In a preferred embodiment, the multi-scale image feature sequence is enhanced according to the differential features to obtain an enhanced multi-scale image feature sequence, including:

[0093] Convolve the above differential features to generate a query matrix;

[0094] Specifically, the query matrix is ​​obtained according to the following formula:

[0095] q=Conv(d)

[0096] In the formula, q represents the query matrix and Conv represents the convolution operation.

[0097] Convolve the above multi-scale image feature sequence to generate a key matrix and a value matrix;

[0098] Specifically, the bond matrix is ​​obtained according to the following formula:

[0099] k=Conv(f)

[0100] Where k represents the bond matrix.

[0101] Specifically, the value matrix is ​​obtained according to the following formula:

[0102] v=Conv(f)

[0103] Where v represents the value matrix.

[0104] Perform activation operations on the query matrix and the key matrix to generate a cross attention map;

[0105] Specifically, the cross attention map is generated according to the following formula:

[0106] Att=Softmax(q*k T / s)

[0107] Where Att represents the cross attention map, Softmax represents the activation function, and k T represents the transpose operation of the key matrix, and s represents the scaling factor.

[0108] According to the product of the above cross attention map and the above value matrix, the enhanced multi-scale image feature sequence is obtained.

[0109] Specifically, the enhanced multi-scale image feature sequence is obtained according to the following formula:

[0110] f e =Att*v

[0111] Where, f e Represents the enhanced multi-scale image feature sequence.

[0112] Preferably, when convolving the differential features and the multi-scale image feature sequence, the convolution kernel sizes in the three convolution layers used are all 1×1, the stride lengths are all 1, the number of input and output channels are all 256, and the value of the scaling factor s is set to 0.5.

[0113] Preferably, the query matrix is ​​generated by differential features, while the key matrix and value matrix are generated by multi-scale image features. The interaction and enhancement between different features are achieved through the self-attention operation between the query, key and value matrices.

[0114] Preferably, by enhancing the differential features and multi-scale image feature sequences, an adaptive fusion of the differential features and multi-scale image features in the global dimension is achieved. The differential features, as spatial prior knowledge of the power target, are dynamically adjusted through learnable attention weights, effectively suppressing the feature degradation of the power target due to spatial degradation.

[0115] In this preferred embodiment, by performing feature enhancement on the differential features and the multi-scale image feature sequence, an enhanced multi-scale image feature sequence is obtained.

[0116] Step S105: Input the enhanced multi-scale image feature sequence into the preset instance segmentation decoder built into the preset power target segmentation model, so that the preset instance segmentation decoder performs power target segmentation on the distribution line infrared image according to the enhanced multi-scale image feature sequence.

[0117] Specifically, after performing power target segmentation on the infrared image of the distribution line, the predicted category, predicted spatial coordinates, and predicted boundaries of the power target are obtained. The enhanced multi-scale image feature sequence is input into the preset instance segmentation decoder for power target classification, regression, and segmentation. Classification determines the category of the power target, regression determines the spatial coordinate frame of the power target, and segmentation determines the boundary of the power target. The predicted category, predicted spatial coordinate frame, and predicted boundary of the power target can be obtained by the following formula:

[0118] cls,box,mask = Dec S (f e )

[0119] In the formula, cls represents the predicted category, box represents the predicted space coordinate box, mask represents the predicted boundary, and Dec S Represents the preset instance segmentation decoder.

[0120] In a preferred embodiment, the training of the preset power target segmentation model includes:

[0121] Acquire a plurality of infrared sample images of power distribution lines, an infrared image truth value of each infrared sample image of the power distribution lines, a category truth value of the power target, a spatial coordinate frame truth value of the power target, and a boundary truth value of the power target;

[0122] Input the above-mentioned several distribution line infrared sample images, infrared image true values, category true values, spatial coordinate frame true values ​​and boundary true values ​​into the power target segmentation model to be trained for iterative training until the number of iterations is not less than a preset iteration number threshold, thereby generating a trained preset power target segmentation model;

[0123] In each iteration, it is determined whether the current number of iterations is less than a preset iteration number threshold;

[0124] If the current number of iterations is less than a preset iteration threshold, multi-scale feature extraction is performed on the current distribution line infrared sample image through a preset image encoder built into the current power target segmentation model to obtain a current multi-scale sample feature sequence; the current multi-scale sample feature sequence is input into an image enhancement decoder built into the current power target segmentation model to generate a current enhanced distribution line infrared sample image; a difference operation is performed on the current distribution line infrared sample image and the current enhanced distribution line infrared sample image to generate a current differential sample feature;

[0125] The current multi-scale sample feature sequence is enhanced according to the current differential feature to obtain the current enhanced multi-scale sample feature sequence; the current enhanced multi-scale sample feature sequence is input into the instance segmentation decoder built into the current power target segmentation model to perform power target segmentation, and the current predicted power target category, predicted power target spatial coordinate frame, and predicted power target boundary are obtained;

[0126] According to the current enhanced distribution line infrared sample image, the current predicted power target category, the predicted power target spatial coordinate frame, the predicted power target boundary, the corresponding infrared image true value, the corresponding category true value, the corresponding spatial coordinate frame true value and the corresponding boundary true value, the current total loss function is calculated, and after adjusting the parameters of the current power target segmentation model according to the current total loss function, iterative training continues;

[0127] Otherwise, the current power target segmentation model is used as the trained preset power target segmentation model.

[0128] Preferably, the true value of the infrared image can be generated by an existing image enhancement algorithm, such as Real-ESRGAN (Real-World Enhanced Super-Resolution Generative Adversarial Network).

[0129] Preferably, a data augmentation strategy is used to enhance the plurality of second distribution line infrared sample images and their corresponding category truth values, spatial coordinate frame truth values, and boundary truth values, thereby increasing the diversity of the sample data and avoiding the problem of model overfitting. The data augmentation strategy includes random cropping and random flipping with a probability of 0.5, as well as random Gaussian noise.

[0130] Preferably, the power target segmentation model is optimized under supervised conditions using a total loss function. Generally speaking, there are two conditions for model convergence. The first is that the variation in the total loss function between consecutive model iterations is within 1%; the second is that the current number of iterations of the power target segmentation model reaches a preset iteration threshold. In the present invention, the second method is used to determine whether the power target segmentation model has reached convergence, and the number of iterations is set to 100.

[0131] In this preferred embodiment, the power target segmentation model is iteratively trained using several distribution line infrared sample images, infrared image true values, category true values, spatial coordinate frame true values, and boundary true values ​​to generate a trained preset power target segmentation model.

[0132] In another preferred embodiment, the current total loss function is calculated based on the current enhanced distribution line infrared sample image, the current predicted power target category, the predicted power target spatial coordinate frame, the predicted power target boundary, the corresponding infrared image true value, the corresponding category true value, the corresponding spatial coordinate frame true value, and the corresponding boundary true value, including:

[0133] Calculate a current first mean absolute error loss function based on the current enhanced first distribution line infrared sample image and the corresponding infrared image true value;

[0134] Specifically, the value of the first mean absolute error loss function is calculated by the following formula:

[0135]

[0136] Where, L E Represents the value of the first mean absolute error loss function, MAE represents the first mean absolute error function, Indicates the true value of the infrared image, I E represents the enhanced infrared sample image of the first power distribution line, H represents the length of the enhanced infrared sample image of the first power distribution line, W represents the width of the enhanced infrared sample image of the first power distribution line, i and j both represent pixel indices on the enhanced infrared sample image of the first power distribution line, and ||·||1 represents the l1 norm.

[0137] Calculate the current first cross entropy loss function according to the current predicted power target category and the corresponding category true value, and use the current first cross entropy loss function as the current classification loss function of the power target;

[0138] Specifically, the value of the classification loss function is calculated according to the following formula:

[0139]

[0140] Where, L cls Represents the value of the classification loss function, CE represents the first cross entropy loss function, cls * Represents the true value of the category, cls' represents the predicted power target category, K represents the number of power target categories, k represents the index of the power target category, Represents the true value of the kth category, cls′ k represents the kth predicted power target category.

[0141] Calculate the current second mean absolute error function based on the current predicted power target spatial coordinate frame and the corresponding spatial coordinate frame true value, and use the current second mean absolute error function as the current regression loss function of the power target;

[0142] Specifically, the value of the regression loss function is calculated according to the following formula:

[0143]

[0144] Where, L box Represents the value of the regression loss function, MAE' represents the second mean absolute error function, box * represents the true value of the spatial coordinate frame, box′ represents the predicted power target spatial coordinate frame, u represents the index of the spatial coordinate frame, Represents the true value of the u-th spatial coordinate box, box′ u Represents the u-th predicted power target space coordinate frame.

[0145] Calculate the current second cross entropy loss function according to the current predicted power target boundary and the corresponding boundary true value, and use the current second cross entropy loss function as the current segmentation loss function of the power target;

[0146] The value of the segmentation loss function is calculated according to the following formula:

[0147]

[0148] Where, L mask Represents the value of the segmentation loss function, CE' represents the second cross entropy loss function, mask *represents the boundary truth value, mask′ represents the predicted power target boundary, H′ represents the length of the second distribution line infrared sample image, W′ represents the width of the second distribution line infrared sample image, m and n are both pixel indices on the second distribution line infrared sample image, Indicates the true value of the boundary at a length of m pixels and a width of n pixels, mask′ m,n Indicates the predicted power target boundary at a length of m pixels and a width of n pixels.

[0149] The current total loss function is calculated based on the current first mean absolute error loss function, classification loss function, current regression loss function, current segmentation loss function, preset image enhancement weight factor, preset regression loss weight factor and preset segmentation loss weight factor.

[0150] Specifically, the value of the total loss function is calculated by the following formula:

[0151] L total =L cls +αL box +βL mask +γL E

[0152] Where, L total represents the value of the total loss function, α represents the preset regression loss weight factor, β represents the preset segmentation loss weight factor, γ represents the preset image enhancement weight factor, and the values ​​of α, β and γ are all set to 0.5.

[0153] Specifically, the preset image encoder, the preset image enhancement decoder, and the preset instance segmentation decoder share structural parameters and weights.

[0154] Preferably, an end-to-end supervised learning framework is constructed by introducing mean absolute error loss as a constraint for the preset image enhancement decoder. This mechanism embeds supervisory signals at the intermediate layers of the model, forming a residual gradient propagation path, effectively guiding the optimization direction of model parameters and enhancing the model's learning ability.

[0155] In a preferred embodiment, the current total loss function is calculated through the current enhanced distribution line infrared sample image, the current predicted power target category, the predicted power target spatial coordinate frame, the predicted power target boundary, the corresponding infrared image true value, the corresponding category true value, the corresponding spatial coordinate frame true value and the corresponding boundary true value.

[0156] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0157] like Figure 2As shown, an embodiment of the present invention provides a distribution line power target segmentation device based on spatial difference feature enhancement, comprising:

[0158] Multi-scale image feature sequence extraction module, image reconstruction module, differential feature extraction module, feature enhancement module and power target segmentation module;

[0159] The multi-scale image feature sequence extraction module is used to obtain an infrared image of a distribution line containing a power target and input it into a preset power target segmentation model, so that a preset image encoder built into the preset power target segmentation model performs multi-scale feature extraction on the infrared image of the distribution line to obtain a multi-scale image feature sequence;

[0160] The image reconstruction module is configured to input the multi-scale image feature sequence into a preset image enhancement decoder built into the preset power target segmentation model, so that the preset image enhancement decoder reconstructs the infrared image of the power distribution line based on the multi-scale image feature sequence to generate an enhanced infrared image of the power distribution line;

[0161] The differential feature extraction module is used to perform a differential operation on the infrared image of the power distribution line and the enhanced infrared image of the power distribution line to obtain differential features;

[0162] The feature enhancement module is used to perform feature enhancement on the multi-scale image feature sequence according to the differential feature to obtain an enhanced multi-scale image feature sequence;

[0163] The above-mentioned power target segmentation module is used to input the above-mentioned enhanced multi-scale image feature sequence into the preset instance segmentation decoder built into the above-mentioned preset power target segmentation model, so that the above-mentioned preset instance segmentation decoder performs power target segmentation on the above-mentioned distribution line infrared image according to the above-mentioned enhanced multi-scale image feature sequence.

[0164] In a preferred embodiment, the differential feature extraction module includes:

[0165] Image calculation unit and image projection unit;

[0166] The image calculation unit is used to calculate the absolute value of the difference between the enhanced infrared image of the power distribution line and the infrared image of the power distribution line to obtain a differential image;

[0167] The image projection unit is used to perform a projection operation on the difference image to obtain the difference feature.

[0168] In another preferred embodiment, the feature enhancement module includes:

[0169] Differential feature convolution unit, multi-scale image feature convolution unit, cross attention map generation unit and multi-scale image feature enhancement unit;

[0170] The differential feature convolution unit is used to convolve the differential features to generate a query matrix;

[0171] The multi-scale image feature convolution unit is used to convolve the multi-scale image feature sequence to generate a key matrix and a value matrix;

[0172] The cross-attention map generating unit is used to perform an activation operation on the query matrix and the key matrix to generate a cross-attention map;

[0173] The multi-scale image feature enhancement unit is used to obtain an enhanced multi-scale image feature sequence according to the product of the cross-attention map and the value matrix.

[0174] It should be noted that the device embodiment described above is merely illustrative, wherein the modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiment provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement it without paying any creative work. The above schematic diagram is merely an example of a distribution line power target segmentation device based on spatial differential feature enhancement, and does not constitute a limitation on a distribution line power target segmentation device based on spatial differential feature enhancement, and may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0175] Based on the above method embodiment, the present invention provides a corresponding terminal device embodiment.

[0176] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the above memory and configured to be executed by the above processor. When the above processor executes the above computer program, it implements the above-mentioned distribution line power target segmentation method based on spatial differential feature enhancement in any embodiment of the present invention.

[0177] For example, in this embodiment, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the device.

[0178] The terminal device may be a computing device such as a desktop computer, a notebook computer, a PDA, or a cloud server. The device may include, but is not limited to, a processor and a memory;

[0179] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the device, connecting the various parts of the device using various interfaces and lines.

[0180] The above-mentioned memory can be used to store the above-mentioned computer programs and / or modules. The above-mentioned processor realizes various functions of the above-mentioned device by running or executing the computer programs and / or modules stored in the above-mentioned memory, and calling the data stored in the memory. The above-mentioned memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; in addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0181] Based on the above method embodiment, the present invention provides a corresponding storage medium embodiment.

[0182] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the distribution line power target segmentation method based on spatial differential feature enhancement described in any embodiment of the present invention.

[0183] In this embodiment, the storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in source code form, object code form, an executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium.

[0184] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A distribution line power target segmentation method based on spatial difference feature enhancement, characterized in that: include: Acquire an infrared image of a power distribution line containing a power target, and input the image into a preset power target segmentation model, so that a preset image encoder built into the preset power target segmentation model performs multi-scale feature extraction on the infrared image of the power distribution line to obtain a multi-scale image feature sequence; Inputting the multi-scale image feature sequence into a preset image enhancement decoder built into the preset power target segmentation model, so that the preset image enhancement decoder reconstructs the distribution line infrared image according to the multi-scale image feature sequence to generate an enhanced distribution line infrared image; performing a differential operation on the power distribution line infrared image and the enhanced power distribution line infrared image to obtain differential features; Performing feature enhancement on the multi-scale image feature sequence according to the differential features to obtain an enhanced multi-scale image feature sequence; The enhanced multi-scale image feature sequence is input into a preset instance segmentation decoder built into the preset power target segmentation model, so that the preset instance segmentation decoder performs power target segmentation on the distribution line infrared image according to the enhanced multi-scale image feature sequence.

2. The method for power target segmentation of distribution lines based on spatial difference feature enhancement according to claim 1, characterized in that: The differential operation is performed on the infrared image of the power distribution line and the enhanced infrared image of the power distribution line to obtain differential features, including: Calculating the absolute value of the difference between the enhanced infrared image of the power distribution line and the infrared image of the power distribution line to obtain a differential image; A projection operation is performed on the difference image to obtain the difference feature.

3. The method for power target segmentation of distribution lines based on spatial difference feature enhancement according to claim 2, characterized in that: The performing feature enhancement according to the differential feature and the multi-scale image feature sequence to obtain an enhanced multi-scale image feature sequence includes: Convolving the differential features to generate a query matrix; Convolving the multi-scale image feature sequence to generate a key matrix and a value matrix; Performing an activation operation on the query matrix and the key matrix to generate a cross attention graph; An enhanced multi-scale image feature sequence is obtained according to the product of the cross attention map and the value matrix.

4. The method for power target segmentation of distribution lines based on spatial difference feature enhancement according to claim 3 is characterized in that: The training of the preset power target segmentation model includes: Acquire a plurality of infrared sample images of power distribution lines, an infrared image truth value of each infrared sample image of the power distribution lines, a category truth value of the power target, a spatial coordinate frame truth value of the power target, and a boundary truth value of the power target; Inputting the plurality of distribution line infrared sample images, infrared image true values, category true values, spatial coordinate frame true values, and boundary true values ​​into the power target segmentation model to be trained for iterative training until the number of iterations is not less than a preset iteration number threshold, thereby generating a trained preset power target segmentation model; In each iteration, it is determined whether the current number of iterations is less than a preset iteration number threshold; If the current number of iterations is less than a preset iteration threshold, multi-scale feature extraction is performed on the current distribution line infrared sample image through a preset image encoder built into the current power target segmentation model to obtain a current multi-scale sample feature sequence; the current multi-scale sample feature sequence is input into an image enhancement decoder built into the current power target segmentation model to generate a current enhanced distribution line infrared sample image; a difference operation is performed on the current distribution line infrared sample image and the current enhanced distribution line infrared sample image to generate a current differential sample feature; The current multi-scale sample feature sequence is enhanced according to the current differential feature to obtain the current enhanced multi-scale sample feature sequence; the current enhanced multi-scale sample feature sequence is input into the instance segmentation decoder built into the current power target segmentation model to perform power target segmentation, and the current predicted power target category, predicted power target spatial coordinate frame, and predicted power target boundary are obtained; According to the current enhanced distribution line infrared sample image, the current predicted power target category, the predicted power target spatial coordinate frame, the predicted power target boundary, the corresponding infrared image true value, the corresponding category true value, the corresponding spatial coordinate frame true value and the corresponding boundary true value, the current total loss function is calculated, and after adjusting the parameters of the current power target segmentation model according to the current total loss function, iterative training continues; Otherwise, the current power target segmentation model is used as the trained preset power target segmentation model.

5. The method for power target segmentation of distribution lines based on spatial difference feature enhancement according to claim 4, characterized in that: The current total loss function is calculated based on the current enhanced distribution line infrared sample image, the current predicted power target category, the predicted power target spatial coordinate frame, the predicted power target boundary, the corresponding infrared image true value, the corresponding category true value, the corresponding spatial coordinate frame true value, and the corresponding boundary true value, including: Calculate a current first mean absolute error loss function based on the current enhanced first distribution line infrared sample image and the corresponding infrared image true value; Calculate the current first cross entropy loss function according to the current predicted power target category and the corresponding category true value, and use the current first cross entropy loss function as the current classification loss function of the power target; Calculate the current second mean absolute error function based on the current predicted power target spatial coordinate frame and the corresponding spatial coordinate frame true value, and use the current second mean absolute error function as the current regression loss function of the power target; Calculate the current second cross entropy loss function according to the current predicted power target boundary and the corresponding boundary true value, and use the current second cross entropy loss function as the current segmentation loss function of the power target; The current total loss function is calculated based on the current first mean absolute error loss function, classification loss function, current regression loss function, current segmentation loss function, preset image enhancement weight factor, preset regression loss weight factor and preset segmentation loss weight factor.

6. A distribution line power target segmentation device based on spatial difference feature enhancement, characterized in that: include: Multi-scale image feature sequence extraction module, image reconstruction module, differential feature extraction module, feature enhancement module and power target segmentation module; The multi-scale image feature sequence extraction module is used to obtain an infrared image of a distribution line containing an electric power target and input it into a preset electric power target segmentation model, so that a preset image encoder built into the preset electric power target segmentation model performs multi-scale feature extraction on the infrared image of the distribution line to obtain a multi-scale image feature sequence; The image reconstruction module is configured to input the multi-scale image feature sequence into a preset image enhancement decoder built into the preset power target segmentation model, so that the preset image enhancement decoder reconstructs the infrared image of the distribution line according to the multi-scale image feature sequence to generate an enhanced infrared image of the distribution line; The differential feature extraction module is used to perform a differential operation on the power distribution line infrared image and the enhanced power distribution line infrared image to obtain differential features; The feature enhancement module is used to perform feature enhancement on the multi-scale image feature sequence according to the differential feature to obtain an enhanced multi-scale image feature sequence; The power target segmentation module is used to input the enhanced multi-scale image feature sequence into the preset instance segmentation decoder built into the preset power target segmentation model, so that the preset instance segmentation decoder performs power target segmentation on the distribution line infrared image based on the enhanced multi-scale image feature sequence.

7. The power target segmentation device for distribution lines based on spatial difference feature enhancement according to claim 6, characterized in that: The differential feature extraction module includes: Image calculation unit and image projection unit; The image calculation unit is used to calculate the absolute value of the difference between the enhanced distribution line infrared image and the distribution line infrared image to obtain a differential image; The image projection unit is used to perform a projection operation on the differential image to obtain the differential feature.

8. The power target segmentation device for distribution lines based on spatial difference feature enhancement according to claim 7, characterized in that: The feature enhancement module includes: Differential feature convolution unit, multi-scale image feature convolution unit, cross attention map generation unit and multi-scale image feature enhancement unit; The differential feature convolution unit is used to convolve the differential features to generate a query matrix; The multi-scale image feature convolution unit is used to convolve the multi-scale image feature sequence to generate a key matrix and a value matrix; The cross-attention map generating unit is used to perform an activation operation on the query matrix and the key matrix to generate a cross-attention map; The multi-scale image feature enhancement unit is used to obtain an enhanced multi-scale image feature sequence according to the product of the cross attention map and the value matrix.

9. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for power target segmentation of distribution lines based on spatial differential feature enhancement as described in any one of claims 1 to 5 is implemented.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the distribution line power target segmentation method based on spatial difference feature enhancement as described in any one of claims 1 to 5.

Images