Overhead power transmission line image segmentation identification method based on electric power vision large model

Abstract

The embodiment of the invention provides an overhead transmission line image segmentation identification method based on an electric power visual large model, and relates to the technical field of transmission line image segmentation identification. The image segmentation and identification method comprises the following steps: inputting an image of an overhead transmission line to be detected into an encoder in an image segmentation and identification network, and outputting a feature map of the image of the overhead transmission line to be detected; and respectively inputting the characteristic pattern output by the encoder into a segmentation decoder and an identification decoder in the image segmentation and identification network, outputting a segmentation result of the image of the overhead transmission line to be detected through the segmentation decoder, and outputting an identification result of the image of the overhead transmission line to be detected through the identification decoder. According to the image segmentation recognition method, the overhead transmission line image is processed and recognized through the encoder and the decoder, elements such as wires, insulators and towers in the image are recognized and detected, the accuracy is high, and defects and hidden dangers in the line can be found in time.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line image segmentation and recognition, and in particular to an overhead power transmission line image segmentation and recognition method based on a large electric power vision model. Background Art

[0002] Overhead transmission lines are an important part of the power system, responsible for transmitting electricity from power plants or substations to the user end. They are usually erected above the ground and are composed of towers, wires, insulators and other components. Overhead transmission lines have the advantages of long transmission distance, large transmission capacity and low operating cost, and are an indispensable transmission method in the power system. In the existing technology, the segmentation and recognition results are not good enough and the errors are large. Summary of the invention

[0003] The purpose of an embodiment of the present invention is to provide an overhead transmission line image segmentation and recognition method based on a large electric power vision model. The image segmentation and recognition method processes and recognizes the overhead transmission line image through an encoder and a decoder, and recognizes and detects the wires, insulators, towers and other components in the image with high accuracy, and can promptly discover defects and hidden dangers in the line.

[0004] In order to achieve the above object, an embodiment of the present invention provides an overhead transmission line image segmentation and recognition method based on a large electric power visual model, comprising:

[0005] Inputting the image of the overhead power transmission line to be tested into an encoder in the image segmentation and recognition network, and outputting a feature map of the image of the overhead power transmission line to be tested;

[0006] Inputting the feature map output by the encoder into the segmentation decoder and the recognition decoder in the image segmentation and recognition network respectively, outputting the segmentation result of the image of the overhead power transmission line to be tested through the segmentation decoder, and outputting the recognition result of the image of the overhead power transmission line to be tested through the recognition decoder;

[0007] Wherein, the image segmentation and recognition network is obtained after training based on overhead power line image samples, predetermined segmentation results of each overhead power line image sample and predetermined image type labels;

[0008] The step of outputting the recognition result of the image of the overhead power transmission line to be tested by the recognition decoder specifically comprises:

[0009] For any layer of the recognition decoder, obtaining a feature map output by the layer and a feature map output by any layer of the segmentation decoder having the same scale as the feature map output by the layer;

[0010] The feature map output by the layer and the feature map output by any layer of the segmentation decoder having the same scale are used as inputs of the next layer of the layer until the next layer of the layer outputs the recognition result of the overhead power transmission line image to be tested.

[0011] Optionally, the encoder and the segmentation decoder form a Unet structure, and the encoder and the recognition decoder form a Unet structure.

[0012] Optionally, the encoder establishes a skip connection with the segmentation decoder and the recognition decoder.

[0013] Optionally, outputting the segmentation result of the overhead power transmission line image to be tested through the segmentation decoder further includes:

[0014] The image of the overhead power transmission line to be tested is changed into an image with a size of 512×512 or 256×256 pixels.

[0015] Optionally, outputting the recognition result of the image of the overhead power transmission line to be tested through the recognition decoder further includes:

[0016] According to the recognition result of the overhead power transmission line image to be tested, a probability map of each image type label is generated.

[0017] Optionally, the image segmentation and recognition method further includes:

[0018] Acquire a set of sample images of the overhead power transmission line;

[0019] Inputting an overhead power transmission line image sample into the encoder, and outputting a feature map of the overhead power transmission line image sample;

[0020] Inputting the feature graph of the overhead power line image sample output by the encoder to the segmentation encoder and the recognition encoder respectively, outputting the segmentation result of the overhead power line image sample through the segmentation encoder, and outputting the recognition result of the overhead power line image sample through the recognition encoder;

[0021] Clustering the feature graph of the overhead power transmission line image sample finally output by the recognition decoder to obtain a clustering result of the feature graph;

[0022] The image segmentation and recognition network is trained according to the segmentation results, recognition results and clustering results of the overhead power transmission line image samples until a preset termination condition is met.

[0023] Optionally, outputting the recognition result of the overhead power transmission line image sample by the recognition encoder further includes:

[0024] For any pixel in the segmentation result, the image label of the pixel closest to the pixel is obtained from the recognition result, and the image label of the pixel closest to the pixel is used as the category of the pixel to obtain an optimized recognition result.

[0025] Optionally, training the image segmentation and recognition network according to the segmentation results, recognition results and clustering results of the overhead power line image samples further includes:

[0026] According to formula (1), the objective function is determined:

[0027] loss = αL 1 +βL 2 +γL 3 , (1)

[0028] Among them, loss is the target loss function, α, β and γ are weight coefficients, L 1 is the loss function of the recognition result, L 2 is the loss function of the segmentation result, L 3 is the loss function of the clustering result.

[0029] Through the above technical solution, the embodiment of the present invention provides an overhead transmission line image segmentation and recognition method based on the electric power vision large model. By inputting the image of the overhead transmission line to be tested into the encoder in the image segmentation and recognition network, outputting the feature map of the image of the overhead transmission line to be tested, and inputting the feature map output by the encoder into the segmentation decoder and the recognition decoder in the image segmentation and recognition network respectively, the segmentation result of the image of the overhead transmission line to be tested is output by the segmentation decoder, and the recognition result of the image of the overhead transmission line to be tested is output by the recognition decoder. The image segmentation and recognition method processes and recognizes the image of the overhead transmission line through the encoder and the decoder, and recognizes and detects the components such as the conductors, insulators, and pole towers in the image, with high accuracy, and can timely discover defects and hidden dangers in the line.

[0030] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0032] Figure 1 is a flow chart of an image segmentation and recognition method according to an embodiment of the present invention;

[0033] Figure 2It is a flowchart of a training image segmentation and recognition network according to one embodiment of the present invention. DETAILED DESCRIPTION

[0034] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0035] like Figure 1 The figure is a flow chart of an image segmentation and recognition method according to an embodiment of the present invention. Figure 1 In the method, the image segmentation and recognition method comprises:

[0036] In step S11, the image of the overhead power transmission line to be tested is input into an encoder in the image segmentation and recognition network, and a feature map of the image of the overhead power transmission line to be tested is output;

[0037] In step S12, the feature map output by the encoder is respectively input into the segmentation decoder and the recognition decoder in the image segmentation and recognition network, the segmentation result of the overhead power line image to be tested is output by the segmentation decoder, and the recognition result of the overhead power line image to be tested is output by the recognition decoder.

[0038] The image segmentation and recognition network is obtained after training based on overhead power line image samples, predetermined segmentation results of each overhead power line image sample and predetermined image type labels.

[0039] The step of outputting the recognition result of the image of the overhead transmission line to be tested by the recognition decoder specifically includes:

[0040] For any layer of the recognition decoder, obtain the feature map output by the layer and the feature map output by any layer of the segmentation decoder having the same scale as the feature map output by the layer;

[0041] The feature map output by this layer and the feature map output by any layer of the segmentation decoder with the same scale are used as the input of the next layer of this layer until the next layer of this layer outputs the recognition result of the overhead transmission line image to be tested.

[0042] In this embodiment, the encoder and the segmentation decoder form a Unet structure, and the encoder and the recognition decoder form a Unet structure. The encoder is responsible for extracting features from the input image, and through a series of convolutional layers and maximum pooling operations, gradually reducing the spatial dimension of the image, while increasing the number of channels to extract higher-level features, and the decoder is responsible for restoring the spatial resolution of the image, and through a series of deconvolutional layers, gradually increasing the spatial dimension of the image, while combining the feature maps of the corresponding layers of the encoder to restore the details of the input image.

[0043] In this embodiment, the encoder establishes a jump connection with the segmentation decoder and the recognition decoder, and the jump connection helps to retain more detail information and improve the accuracy of segmentation. The segmentation decoder outputs the segmentation result of the overhead power line image to be tested, and the overhead power line image to be tested needs to be changed into an image of 512×512 or 256×256 pixel size, and a probability map of each image type label is generated according to the recognition result of the overhead power line image to be tested.

[0044] In this Figure 2 In the method for image segmentation and recognition, the method further includes:

[0045] In step S21, a sample image set of overhead power transmission lines is obtained;

[0046] In step S22, the overhead power transmission line image sample is input into the encoder, and a feature map of the overhead power transmission line image sample is output;

[0047] In step S23, the feature graph of the overhead power line image sample output by the encoder is input to the segmentation encoder and the recognition encoder respectively, the segmentation result of the overhead power line image sample is output by the segmentation encoder, and the recognition result of the overhead power line image sample is output by the recognition encoder;

[0048] In step S24, clustering is performed on the feature graph of the overhead power line image sample finally output by the recognition decoder to obtain a clustering result of the feature graph;

[0049] In step S25, the image segmentation and recognition network is trained according to the segmentation results, recognition results and clustering results of the overhead power line image samples until a preset termination condition is met.

[0050] In this embodiment, the recognition result of the overhead transmission line image sample output by the recognition encoder also includes, for any pixel in the segmentation result, obtaining the image label of the pixel closest to the position of the pixel from the recognition result, using the image label of the nearest pixel as the category of the pixel, and obtaining an optimized recognition result.

[0051] In this embodiment, the image segmentation and recognition network is trained according to the segmentation results, recognition results and clustering results of the overhead transmission line image samples. The specific form of network training can be various forms known to those skilled in the art. In one example of the present invention, the objective function can be determined by formula (1):

[0052] loss = αL 1 +βL 2 +γL 3 , (1)

[0053] Among them, loss is the target loss function, α, β and γ are weight coefficients, L 1 is the loss function of the recognition result, L 2 is the loss function of the segmentation result, L 3 is the loss function of the clustering result.

[0054] Through the above technical solution, the embodiment of the present invention provides an overhead transmission line image segmentation and recognition method based on the electric power vision large model. By inputting the image of the overhead transmission line to be tested into the encoder in the image segmentation and recognition network, outputting the feature map of the image of the overhead transmission line to be tested, and inputting the feature map output by the encoder into the segmentation decoder and the recognition decoder in the image segmentation and recognition network respectively, the segmentation result of the image of the overhead transmission line to be tested is output by the segmentation decoder, and the recognition result of the image of the overhead transmission line to be tested is output by the recognition decoder. The image segmentation and recognition method processes and recognizes the image of the overhead transmission line through the encoder and the decoder, and recognizes and detects the components such as the conductors, insulators, and pole towers in the image, with high accuracy, and can timely discover defects and hidden dangers in the line.

[0055] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0056] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0059] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0060] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0061] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0062] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0063] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for segmenting and recognizing overhead power line images based on a large electric power visual model, characterized in that: The image segmentation and recognition method comprises: Inputting the image of the overhead power transmission line to be tested into an encoder in the image segmentation and recognition network, and outputting a feature map of the image of the overhead power transmission line to be tested; Inputting the feature map output by the encoder into the segmentation decoder and the recognition decoder in the image segmentation and recognition network respectively, outputting the segmentation result of the image of the overhead power transmission line to be tested through the segmentation decoder, and outputting the recognition result of the image of the overhead power transmission line to be tested through the recognition decoder; Wherein, the image segmentation and recognition network is obtained after training based on overhead power line image samples, predetermined segmentation results of each overhead power line image sample and predetermined image type labels; The step of outputting the recognition result of the image of the overhead power transmission line to be tested by the recognition decoder specifically comprises: For any layer of the recognition decoder, obtaining a feature map output by the layer and a feature map output by any layer of the segmentation decoder having the same scale as the feature map output by the layer; The feature map output by the layer and the feature map output by any layer of the segmentation decoder having the same scale are used as inputs of the next layer of the layer until the next layer of the layer outputs the recognition result of the overhead power transmission line image to be tested.

2. The image segmentation and recognition method according to claim 1, characterized in that: The encoder and the segmentation decoder form a Unet structure, and the encoder and the recognition decoder form a Unet structure.

3. The image segmentation and recognition method according to claim 1, characterized in that: The encoder establishes a skip connection with the segmentation decoder and the recognition decoder.

4. The image segmentation and recognition method according to claim 1, characterized in that: Outputting the segmentation result of the overhead power transmission line image to be tested through the segmentation decoder also includes: The image of the overhead power transmission line to be tested is changed into an image with a size of 512×512 or 256×256 pixels.

5. The image segmentation and recognition method according to claim 1, characterized in that: Outputting the recognition result of the image of the overhead power transmission line to be tested through the recognition decoder also includes: According to the recognition result of the overhead power transmission line image to be tested, a probability map of each image type label is generated.

6. The image segmentation and recognition method according to claim 1, characterized in that: The image segmentation and recognition method also includes: Acquire a set of sample images of the overhead power transmission line; Inputting an overhead power transmission line image sample into the encoder, and outputting a feature map of the overhead power transmission line image sample; Inputting the feature graph of the overhead power line image sample output by the encoder to the segmentation encoder and the recognition encoder respectively, outputting the segmentation result of the overhead power line image sample through the segmentation encoder, and outputting the recognition result of the overhead power line image sample through the recognition encoder; Clustering the feature graph of the overhead power transmission line image sample finally output by the recognition decoder to obtain a clustering result of the feature graph; The image segmentation and recognition network is trained according to the segmentation results, recognition results and clustering results of the overhead power transmission line image samples until a preset termination condition is met.

7. The image segmentation and recognition method according to claim 6, characterized in that: Outputting the recognition result of the overhead power transmission line image sample through the recognition encoder also includes: For any pixel in the segmentation result, the image label of the pixel closest to the pixel is obtained from the recognition result, and the image label of the pixel closest to the pixel is used as the category of the pixel to obtain an optimized recognition result.

8. The image segmentation and recognition method according to claim 6, characterized in that: The image segmentation and recognition network is trained according to the segmentation results, recognition results and clustering results of the overhead power line image samples, and further includes: According to formula (1), the objective function is determined: loss = αL1 + βL2 + γL3, (1) Among them, loss is the target loss function, α, β and γ are weight coefficients, L1 is the loss function of the recognition result, L2 is the loss function of the segmentation result, and L3 is the loss function of the clustering result.

Images