Power line identification method and device based on image fusion, equipment and medium

Abstract

The invention discloses a power line identification method, device and equipment based on image fusion and a medium, and the method comprises the steps: obtaining an infrared image and a visible light image of a power line of a power distribution network, and carrying out the registration of the infrared image and the visible light image through a preset image registration algorithm, and generating a first image; wherein the infrared image and the visible light image are derived from the same picture; performing feature extraction and feature fusion on the first image and the visible light image through a preset image fusion algorithm to generate a second image; and performing power line identification on the second image through a preset power line identification algorithm. According to the method, the infrared image and the visible light image which are derived from the same power line picture are fused to obtain the fused image which contains more source information and has more obvious power line characteristics, so that power line recognition is performed, the difficulty of power line recognition is greatly reduced, and the accuracy of power line recognition of the power distribution network is further improved.

Description

Technical Field

[0001] The present invention relates to the field of power Internet of Things, and in particular to a power line identification method, device, equipment and medium based on image fusion. Background Art

[0002] As a vital public infrastructure, distribution networks play a crucial role in ensuring power supply, supporting economic and social development, and improving people's livelihoods. In recent years, unmanned power inspection technologies have been increasingly used to improve the efficiency and accuracy of distribution network inspections. Power line identification is a key technology for unmanned power inspections. Intelligent technologies such as autonomous navigation of inspection drones, abnormal target detection on power lines, external damage identification, ice detection, and defect identification of transmission equipment all rely on power line identification.

[0003] However, current power line identification technologies often rely on single-type images (such as infrared images and visible light images) obtained through unmanned inspections to identify and detect power lines. However, power lines in distribution networks are thin and lack rich features. The environment and climate are more complex and changeable, and obstruction of some power lines is more common. Therefore, accurately identifying power lines in unmanned inspection images of distribution networks faces significant challenges. In particular, identification based on a single piece of information from a single type of image significantly reduces the accuracy of power line identification. As can be seen from the above, how to improve the accuracy of power line identification in complex environments by obtaining multi-source information is an urgent problem to be solved in the field of intelligent power line identification in distribution networks. Summary of the Invention

[0004] The present invention provides a power line identification method, device, equipment and medium based on image fusion, which can improve the accuracy of power line identification in a distribution network.

[0005] In a first aspect, an embodiment of the present invention provides a method for identifying power lines based on image fusion, comprising:

[0006] Acquire an infrared image and a visible light image of power lines in a distribution network, and register the infrared image and the visible light image using a preset image registration algorithm to generate a first image; wherein the infrared image and the visible light image are from the same picture;

[0007] Performing feature extraction and feature fusion on the first image and the visible light image using a preset image fusion algorithm to generate a second image;

[0008] Power line recognition is performed on the second image using a preset power line recognition algorithm.

[0009] The embodiment of the present invention obtains an infrared image and a visible light image from the same distribution network power line image, aligns the two images, and generates a first image obtained by combining the two images, thereby preparing the image for subsequent image fusion. Further feature extraction and feature fusion are performed on the first image and the visible light image to obtain a second image with more distinct features and richer information, further preparing the image for subsequent power line identification. Power line identification is performed based on the second image using a power line identification algorithm. The power line features of the second image are now significantly enhanced, reducing the difficulty of power line identification and thereby improving the accuracy of power line identification. Compared with the prior art, the present invention can improve the accuracy of power line identification in the distribution network.

[0010] Furthermore, the registering of the infrared image and the visible light image based on a preset image registration algorithm to generate the first image is specifically as follows:

[0011] Inputting the visible light image into a cross-modal image generation module of the image registration algorithm to generate a cross-modal image based on the visible light image;

[0012] The cross-modal image and the infrared image are input into a single-modality transfer registration model of the image registration algorithm to extract temperature field information from the infrared image, and the temperature field information is transferred to the cross-modal image to generate a first image.

[0013] The embodiment of the present invention first generates a cross-modal image with a clear geometric structure and consistent picture content with the visible light image based on the visible light image, and then extracts the temperature field information of the infrared image through a single-modal migration registration model to migrate the temperature field information to the cross-modal image, thereby realizing the registration of the cross-modal image and the infrared image, and generating a first image that integrates the geometric structure information of the visible light image and the temperature field information of the infrared image, providing image support for subsequent power line recognition, and greatly reducing the difficulty of subsequent power line recognition.

[0014] Furthermore, before inputting the visible light image into the cross-modal image generation module of the image registration algorithm, the method further includes:

[0015] Construct the first generator and the second generator, and embed several residual blocks at the bottom of each generator;

[0016] establishing two loop generation paths between the first generator and the second generator, and establishing a correlation between the two paths;

[0017] Training the first generator and the second generator according to a preset perceptual loss function and a preset style loss function;

[0018] A first discriminator and a second discriminator are constructed to be combined with the first generator and the second generator to obtain the cross-modal image generation module.

[0019] The embodiment of the present invention provides algorithmic support for the subsequent generation of cross-modal images with complete and clear geometric structures that are completely consistent with the input visible light image by constructing a cross-modal generation module that includes a cyclic generation path and loss constraints.

[0020] Furthermore, generating a cross-modal image based on the visible light image is specifically:

[0021] Performing several layers of convolution sampling on the visible light image to extract geometric features of the visible light image, and enhancing the geometric features to generate a pseudo infrared image;

[0022] Structural alignment and ghost suppression are performed on the pseudo infrared image according to the visible light image to generate a cross-modal image.

[0023] The embodiment of the present invention generates a cross-modal image with a complete and clear geometric structure and a picture that is completely consistent with the input visible light image based on multi-layer convolution, structural alignment and ghosting suppression, thereby preparing the image for the subsequent migration of temperature field information and generation of the first image.

[0024] Furthermore, the cross-modal image and the infrared image are input into the unimodal transfer registration model of the image registration algorithm to extract the temperature field information in the infrared image, and the temperature field information is transferred to the cross-modal image to generate the first image, specifically:

[0025] Inputting the cross-modal image and the infrared image into the unimodal transfer registration model, so that the unimodal transfer registration model extracts first features of the cross-modal image and the infrared image respectively, and calculates a global displacement field based on the first features to obtain a first displacement field;

[0026] extracting second features of the cross-modal image and the infrared image, calculating a local correction amount based on the second features, and superimposing the local correction amount on the first displacement field to obtain a second displacement field;

[0027] The infrared image is spatially transformed according to the second displacement field to extract temperature field information from the infrared image, and the temperature field information is transferred to the cross-modal image to generate a first image.

[0028] The embodiment of the present invention solves the problem of overall alignment between images by extracting a first feature containing global large-scale features to calculate a global displacement field, thereby obtaining a first displacement field. Furthermore, a second feature containing local details of the power lines is extracted to calculate a local correction value. By superimposing the local correction value on the first displacement field, the local structure of the power lines is precisely aligned to obtain a second displacement field. Finally, the infrared image is spatially transformed according to the second displacement field to extract the temperature field information in the infrared image, thereby achieving registration of the cross-modal image and the infrared image, thereby generating a first image. It can be seen that the final first image has a clear geometric structure and retains true temperature field information, greatly reducing the difficulty of subsequent power line identification.

[0029] Furthermore, the feature extraction and feature fusion of the first image and the visible light image are performed by a preset image fusion algorithm to generate a second image, specifically:

[0030] extracting, by the image fusion algorithm, a first thermal radiation feature from the first image and a first power line feature from the visible light image;

[0031] Based on a preset attention mechanism, adaptively correcting the first thermal radiation feature and the first power line feature to obtain a second thermal radiation feature and a second power line feature;

[0032] The second thermal radiation feature and the second power line feature are fused to generate a second image.

[0033] The embodiment of the present invention generates a second image by further extracting and fusing features of the first image, so that the fused image retains as many thermal radiation features and power line detail features as possible, thereby providing an image with significantly enhanced power line features for subsequent power line identification.

[0034] Furthermore, the power line recognition is performed on the second image using a preset power line recognition algorithm, specifically:

[0035] processing the global features of the second image according to the power line recognition algorithm and the long-range attention mechanism;

[0036] Power line detail features are extracted from the second image after global feature processing to perform power line recognition.

[0037] An embodiment of the present invention processes the global features in the second image by providing a long-range attention algorithm, reduces the impact of environmental features on subsequent power line recognition, and then focuses on extracting the power line detail features in the second image, thereby improving the reliability of power line recognition.

[0038] In a second aspect, an embodiment of the present invention provides a power line identification device based on image fusion, characterized in that it includes a preliminary registration module, an image fusion module and a power line identification module, wherein:

[0039] The preliminary registration module is used to obtain an infrared image and a visible light image of the power lines of the distribution network, and register the infrared image and the visible light image using a preset image registration algorithm to generate a first image; wherein the infrared image and the visible light image are from the same picture;

[0040] The image fusion module is used to extract and fuse features of the first image and the visible light image using a preset image fusion algorithm to generate a second image;

[0041] The power line identification module is configured to perform power line identification on the second image using a preset power line identification algorithm.

[0042] In an embodiment of the present invention, an infrared image and a visible light image originating from the same power line image of a power distribution network are registered through a preliminary registration module to obtain a first image containing information of both types of images; an image fusion module is used to extract and fuse features of the first image and the visible light image to further enhance the power line features in the image, thereby generating a second image; and a power line identification module is used to identify power lines based on the second image. At this time, the power line features in the second image have been significantly enhanced, greatly reducing the difficulty of power line identification, thereby improving the accuracy of power line identification in the distribution network.

[0043] In a third aspect, an embodiment of the present invention provides a terminal device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0044] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of any one of the above-mentioned power line identification methods based on image fusion.

[0045] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device / apparatus where the computer-readable storage medium is located is controlled to execute the power line identification method based on image fusion as described in any one of the above items.

[0046] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of a power line identification method based on image fusion provided by an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the structure of an image recognition algorithm model exemplified by an embodiment of the present invention;

[0049] Figure 3 A schematic flow chart of a method for identifying power lines based on image fusion according to an embodiment of the present invention;

[0050] Figure 4 This is a structural diagram of a power line identification device based on image fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] Example 1:

[0053] like Figure 1 As shown, a method for identifying power lines based on image fusion provided by an embodiment of the present invention includes the following steps:

[0054] S11, acquiring an infrared image and a visible light image of power lines in a power distribution network, and registering the infrared image and the visible light image using a preset image registration algorithm to generate a first image; wherein the infrared image and the visible light image are from the same picture;

[0055] S12, performing feature extraction and feature fusion on the first image and the visible light image using a preset image fusion algorithm to generate a second image;

[0056] S13: Perform power line recognition on the second image using a preset power line recognition algorithm.

[0057] In specific implementations, infrared images and visible light images of distribution network power lines are usually captured by power drones using their onboard thermal imaging cameras and visible light cameras during distribution network inspections. The two images are roughly the same, with infrared images mainly containing thermal features, while visible light images contain rich color and texture details.

[0058] In actual operation, due to the differences in shooting and spatial deformation, direct image fusion of infrared images and visible light images obtained by drones will produce serious power line ghosting.

[0059] To address the above issues, an embodiment of the present invention uses a preset image registration algorithm to reconstruct the visible light image, and then performs pixel-level registration with the infrared image to obtain a reconstructed image that is completely consistent and free of ghosting. This process specifically includes: inputting the visible light image into the cross-modal image generation module of the image registration algorithm to generate a cross-modal image based on the visible light image; inputting the cross-modal image and the infrared image into the unimodal transfer registration model of the image registration algorithm to extract the temperature field information from the infrared image, and then transferring the temperature field information to the cross-modal image to generate a first image.

[0060] In this embodiment, before the visible light image is input into the cross-modal image generation module of the image registration algorithm, it also includes: constructing a first generator and a second generator, and embedding a plurality of residual blocks at the bottom of each generator; establishing two cyclic generation paths between the first generator and the second generator, and establishing a correlation between the two paths; training the first generator and the second generator according to a preset perceptual loss function and a preset style loss function; constructing a first discriminator and a second discriminator to combine with the first generator and the second generator to obtain the cross-modal image generation module.

[0061] In a specific implementation, the embodiment of the present invention adopts an image registration algorithm based on generative AI, which includes a cross-modal image generation module based on generative adversarial networks and a single-modal migration registration model. The cross-modal image generation module based on generative adversarial networks consists of two generators G(A) and G(B) and two discriminators D ir and D vis The generator network is a U-shaped structure with 9 residual blocks at the bottom. It executes a specific learning strategy controlled by perceptual style transfer constraints and establishes inter-path correlation between the two cyclic generation paths to further optimize the generation of clear structures in pseudo infrared images.

[0062] In this embodiment, generating a cross-modal image based on the visible light image specifically includes: performing several layers of convolution sampling on the visible light image to extract geometric features of the visible light image, and strengthening the geometric features to generate a pseudo infrared image; and performing structural alignment and ghosting suppression on the pseudo infrared image based on the visible light image to generate a cross-modal image.

[0063] In a specific implementation, the above cross-modal image generation process is implemented in a cross-modal image generation module based on a generative adversarial network. The loss function of the cross-modal image generation module consists of two terms. The first term is the perceptual loss function ζ pcp , the second term is the style loss function ζ sty First, the perceptual loss function is used to perform structural alignment of the pseudo infrared image based on the visible light image. The calculation principle is as follows:

[0064]

[0065] Among them, pcp Refers to the perceptual loss function, ψ j Refers to the features of the jth layer in the VGG-19 model, which is a multidimensional matrix. G(A) and G(B) refer to two generators. I ir Refers to infrared image, I vis Refers to visible light images.

[0066] The style loss function is used to calculate (I vis , ) and (I ir , ) The statistical error between the image pairs is used to suppress the potential ghosting of power lines in the image. The calculation principle is as follows:

[0067]

[0068] Among them, sty refers to the style loss function, ψ j Refers to the features of the j-th layer in the VGG-19 model, which is a multidimensional matrix. wj refers to the weight of the j-th layer in the VGG-19 model. G(A) and G(B) refer to two generators. I ir Refers to infrared image, I vis Refers to visible light images, Refers to the visible light image generated by the model, Refers to the infrared image generated by the model.

[0069] In this embodiment, the cross-modal image and the infrared image are input into the unimodal transfer registration model of the image registration algorithm to extract the temperature field information in the infrared image, and the temperature field information is transferred to the cross-modal image to generate a first image. Specifically, the cross-modal image and the infrared image are input into the unimodal transfer registration model so that the unimodal transfer registration model extracts the first features of the cross-modal image and the infrared image respectively, and calculates the global displacement field based on the first features to obtain a first displacement field; extracts the second features of the cross-modal image and the infrared image, and calculates the local correction amount based on the second feature, and superimposes the local correction amount on the first displacement field to obtain a second displacement field; and spatially transforms the infrared image according to the second displacement field to extract the temperature field information in the infrared image, and transfers the temperature field information to the cross-modal image to generate a first image.

[0070] In the specific implementation, the unimodal transfer registration model of real infrared images and cross-modal images adopts a multi-level refinement registration method to predict the displacement vector field between the real and cross-modal images, and reconstruct the registered infrared image in a unimodal setting. The model consists of a shared multi-scale feature extractor (SM-FE) f k , two coarse-to-fine deformation field prediction modules and a resampling layer. In each deformation field prediction module, a coarse-grained feature extraction module M is included. c and a fine-grained feature extraction module M R The calculation principle of the rough deformation field is as follows:

[0071]

[0072] Among them, M c refers to the coarse-grained feature extraction module, k means that the feature extraction module contains k scales, φc refers to the coarse deformation field, f k refers to the feature extractor of the kth scale, I ir Refers to infrared image, I vis Refers to visible light images.

[0073] The calculation principle of the fine deformation field is as follows:

[0074]

[0075] Among them, M R Refers to the fine-grained feature extraction module, φr refers to the fine deformation field, φc refers to the coarse deformation field, Refers to the exclusive OR operation.

[0076] Assuming that the feature extractor contains K scales, when k = K, the final deformation field Subsequently, the resampling layer is used to reconstruct the registered infrared image as shown below:

[0077]

[0078] in, Refer to the infrared image after registration, operation Refers to the spatial transformation used for registration.

[0079] In this embodiment, the first image and the visible light image are subjected to feature extraction and feature fusion by a preset image fusion algorithm to generate a second image. Specifically, the first thermal radiation feature in the first image and the first power line feature of the visible light image are extracted respectively by the image fusion algorithm; based on a preset attention mechanism, the first thermal radiation feature and the first power line feature are subjected to feature correction respectively to obtain a second thermal radiation feature and a second power line feature; the second thermal radiation feature and the second power line feature are fused to generate a second image.

[0080] In the specific implementation, a Transformer-based feature interaction extraction module was constructed to realize the generation process of the above-mentioned second image. This module adaptively selects more meaningful features through dual-path interactive extraction, avoiding feature smoothing caused by immature extraction rules, and based on this, upsampling and fusion are performed to obtain a fused image containing infrared and visible light multi-source features.

[0081] Since infrared and visible light images come from the same scene and are fully registered, the low-frequency information of the two modalities, including background and large-scale environmental features, can mostly be directly shared. However, the high-frequency information of the two modalities is independent of each other, including the texture and detail information of power lines in visible light images and the thermal radiation information of power equipment in infrared images, and therefore needs to be retained as much as possible.

[0082] In order to solve the problem of loss of required high-frequency input information, a Transformer-based dual-path interactive fusion module is proposed. The dual-path interactive fusion module consists of a feature extraction module and a feature interactive fusion module. The main principles of the feature extraction module are as follows:

[0083]

[0084] Among them, f ir eg Refers to the features extracted from the infrared image after registration, f vis Refers to the features extracted from the visible light image, I ir eg Refers to the infrared image after registration, I vis Refers to visible light images, represents the feature extraction module, θ E are the feature extraction module parameters.

[0085] The feature interaction fusion module can adaptively select features from infrared and visible light images for fusion. In order to make it focus on more high-frequency features, it is necessary to recalibrate the feature response mode. The principle is as follows:

[0086]

[0087] Among them, A tt Refers to the output of the attention mechanism, S is the Sigmoid function, Refers to the multiplication operation between pixels, f ir eg Refers to the input infrared image features, f vis Refers to the input visible light image features, conv 1×1 Refers to a one-dimensional convolution operation.

[0088] The corrected infrared and visible light image features can be expressed as

[0089]

[0090] Among them, f ir ATT refers to the infrared image features after attention enhancement, f ir eg Refers to the input infrared image features, f vis Refers to the input visible light image features, A tt refers to the output of the attention mechanism, Refers to the multiplication operation between pixels.

[0091] Finally, the infrared and visible light image features are fused. The principle is as follows:

[0092]

[0093] Among them, f ir ATT refers to the infrared image features after attention enhancement, f vi ATT Refers to the visible light image features after attention enhancement, conv 3×3 Refers to the three-dimensional convolution operation, concat refers to the matrix splicing operation, I fus Refers to the fused image.

[0094] In this embodiment, the power line recognition is performed on the second image through a preset power line recognition algorithm, specifically: according to the power line recognition algorithm and the long-range attention mechanism, the global features of the second image are processed; and the power line detail features of the second image after the global feature processing are extracted to perform power line recognition.

[0095] In the specific implementation, in order to more reliably extract power lines, the embodiment of the present invention proposes a power line detection algorithm based on CNN-Transformer to identify the fused image, where Transformer mainly uses long-range attention to process low-frequency global features, while CNN is mainly used to extract high-frequency local features. The model framework can be found in Figure 2 An example of Figure 2 As shown, the example includes: inputting the fused image into the image feature extraction module to perform low-frequency global feature processing and high-frequency local feature extraction; then inputting the feature-extracted image into the convolution block to perform multi-layer convolution; finally, the power lines are identified and classified through the mask decoder, and the recognition result is finally output.

[0096] To better illustrate the working principle and steps of this embodiment, see Figure 3An example. This example is a flow chart of a method for power line recognition based on image fusion according to an embodiment of the present invention, and the specific process is as follows: first, the visible light image a is input into a cross-modal image generation module based on a generative adversarial network. The network has been fully trained on a public visible light-infrared image dataset to generate and output a pseudo-infrared image c of the visible light image a. The pseudo-infrared image c is completely aligned with the infrared thermal imaging image b, but lacks the real temperature field information; wherein, the visible light image a and the infrared thermal imaging image b are images taken during the inspection of the power drone, and the two are imaged almost at the same time, and the pictures contained are roughly the same, but have not been aligned; further, a single-modal transfer registration model for infrared images is used to align the infrared thermal imaging image b and the pseudo-infrared image c. The model inputs are the infrared thermal imaging image b and the pseudo-infrared image c, and the infrared thermal imaging image b is input. The temperature field information of image b is migrated to the pseudo infrared image c, and the registered infrared image d is output. Image d contains the real temperature field information and is completely registered with the visible light image b; then, the registered infrared image d is fused with the visible light image a, and image d is completely registered with image a. They are respectively input into the image fusion algorithm based on the Transformer structure. The algorithm adopts the fusion rule of "high-frequency independence, low-frequency sharing" and outputs the infrared-visible fusion image e after the fusion of the infrared image d and the visible light image a; finally, the power line recognition algorithm based on CNN-Transformer is used to perform power line detection on the infrared-visible light fusion image e, and the detection result is output. At this point, all steps of the power line detection algorithm for distribution network in complex environment based on infrared and visible light image fusion proposed in the present invention are completed, and the final result is obtained.

[0097] The embodiment of the present invention obtains an infrared image and a visible light image from the same distribution network power line image, aligns the two images, and generates a first image obtained by combining the two images, thereby preparing the image for subsequent image fusion. Further feature extraction and feature fusion are performed on the first image and the visible light image to obtain a second image with more distinct features and richer information, further preparing the image for subsequent power line identification. Power line identification is performed based on the second image using a power line identification algorithm. The power line features of the second image are now significantly enhanced, reducing the difficulty of power line identification and thereby improving the accuracy of power line identification. Compared with the prior art, the present invention can improve the accuracy of power line identification in the distribution network.

[0098] Example 2:

[0099] like Figure 4 As shown, this embodiment provides a power line identification device based on image fusion, including a preliminary registration module 001, an image fusion module 002 and a power line identification module 003, wherein:

[0100] The preliminary registration module 001 is used to obtain an infrared image and a visible light image of the power lines of the power distribution network, and register the infrared image and the visible light image using a preset image registration algorithm to generate a first image; wherein the infrared image and the visible light image are from the same picture;

[0101] The image fusion module 002 is used to extract and fuse features of the first image and the visible light image using a preset image fusion algorithm to generate a second image;

[0102] The power line identification module 003 is configured to perform power line identification on the second image using a preset power line identification algorithm.

[0103] In this embodiment, the preliminary registration module 001 generates a first image by aligning the infrared image and the visible light image based on a preset image registration algorithm, specifically: inputting the visible light image into the cross-modal image generation module of the image registration algorithm to generate a cross-modal image based on the visible light image; inputting the cross-modal image and the infrared image into the single-modal migration registration model of the image registration algorithm to extract the temperature field information in the infrared image, and migrating the temperature field information to the cross-modal image to generate the first image.

[0104] In this embodiment, the image fusion module 002 uses a preset image fusion algorithm to perform feature extraction and feature fusion on the first image and the visible light image to generate a second image. Specifically, the image fusion module 002 uses the image fusion algorithm to extract the first thermal radiation feature in the first image and the first power line feature of the visible light image; based on a preset attention mechanism, the first thermal radiation feature and the first power line feature are feature corrected to obtain a second thermal radiation feature and a second power line feature; the second thermal radiation feature and the second power line feature are fused to generate a second image.

[0105] In this embodiment, the power line recognition module 003 performs power line recognition on the second image through a preset power line recognition algorithm. Specifically, the power line recognition module 003 processes the global features of the second image according to the power line recognition algorithm and the long-range attention mechanism; and extracts power line detail features from the second image after the global feature processing to perform power line recognition.

[0106] For more detailed working principles and process steps of this embodiment, please refer to, but not limited to, the relevant records of Embodiment 1.

[0107] In the embodiment of the present invention, a preliminary registration module 001 is used to register an infrared image and a visible light image originating from the same power line picture of a power distribution network, thereby obtaining a first image containing information of both types of images; an image fusion module 002 is used to extract and fuse features of the first image and the visible light image, thereby further enhancing the power line features in the image and generating a second image; and a power line identification module 003 is used to identify power lines based on the second image. At this time, the power line features in the second image have been significantly enhanced, thereby greatly reducing the difficulty of power line identification and improving the accuracy of power line identification in the distribution network.

[0108] Example 3:

[0109] This embodiment provides a terminal device, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0110] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of any one of the above-mentioned power line identification methods based on image fusion.

[0111] Example 4:

[0112] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device / apparatus where the computer-readable storage medium is located is controlled to execute any one of the power line identification methods based on image fusion as described above.

[0113] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-monitorable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0114] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A power line identification method based on image fusion, characterized in that: include: Acquire an infrared image and a visible light image of power lines in a distribution network, and register the infrared image and the visible light image using a preset image registration algorithm to generate a first image; wherein the infrared image and the visible light image are from the same picture; Performing feature extraction and feature fusion on the first image and the visible light image using a preset image fusion algorithm to generate a second image; Power line recognition is performed on the second image using a preset power line recognition algorithm.

2. The method for identifying power lines based on image fusion according to claim 1, characterized in that: The registering the infrared image and the visible light image based on a preset image registration algorithm to generate the first image is specifically as follows: Inputting the visible light image into a cross-modal image generation module of the image registration algorithm to generate a cross-modal image based on the visible light image; The cross-modal image and the infrared image are input into a single-modality transfer registration model of the image registration algorithm to extract temperature field information from the infrared image, and the temperature field information is transferred to the cross-modal image to generate a first image.

3. The method for identifying power lines based on image fusion according to claim 2, wherein: Before inputting the visible light image into the cross-modal image generation module of the image registration algorithm, the method further includes: Construct the first generator and the second generator, and embed several residual blocks at the bottom of each generator; establishing two loop generation paths between the first generator and the second generator, and establishing a correlation between the two paths; Training the first generator and the second generator according to a preset perceptual loss function and a preset style loss function; A first discriminator and a second discriminator are constructed to be combined with the first generator and the second generator to obtain the cross-modal image generation module.

4. The method for identifying power lines based on image fusion according to claim 2, wherein: Generating a cross-modal image according to the visible light image is specifically: performing a plurality of layers of convolution sampling on the visible light image to extract geometric features of the visible light image, and enhancing the geometric features to generate a pseudo infrared image; Structural alignment and ghost suppression are performed on the pseudo infrared image according to the visible light image to generate a cross-modal image.

5. The method for identifying power lines based on image fusion according to claim 2, wherein: The step of inputting the cross-modal image and the infrared image into the single-modal transfer registration model of the image registration algorithm to extract temperature field information from the infrared image and transferring the temperature field information to the cross-modal image to generate a first image is specifically as follows: Inputting the cross-modal image and the infrared image into the unimodal transfer registration model, so that the unimodal transfer registration model extracts first features of the cross-modal image and the infrared image respectively, and calculates a global displacement field based on the first features to obtain a first displacement field; extracting second features of the cross-modal image and the infrared image, calculating a local correction amount based on the second features, and superimposing the local correction amount on the first displacement field to obtain a second displacement field; The infrared image is spatially transformed according to the second displacement field to extract temperature field information from the infrared image, and the temperature field information is transferred to the cross-modal image to generate a first image.

6. The method for identifying power lines based on image fusion according to claim 1, characterized in that: The feature extraction and feature fusion of the first image and the visible light image are performed by a preset image fusion algorithm to generate a second image, specifically: extracting, by the image fusion algorithm, a first thermal radiation feature from the first image and a first power line feature from the visible light image; Based on a preset attention mechanism, feature correction is performed on the first thermal radiation feature and the first power line feature to obtain a second thermal radiation feature and a second power line feature; The second thermal radiation feature and the second power line feature are fused to generate a second image.

7. The method for identifying power lines based on image fusion according to claim 1, characterized in that: The performing power line recognition on the second image by using a preset power line recognition algorithm is specifically as follows: processing the global features of the second image according to the power line recognition algorithm and the long-range attention mechanism; Power line detail features are extracted from the second image after global feature processing to perform power line recognition.

8. A power line identification device based on image fusion, characterized in that: It includes preliminary registration module, image fusion module and power line recognition module, among which, The preliminary registration module is used to obtain an infrared image and a visible light image of the power lines of the distribution network, and register the infrared image and the visible light image using a preset image registration algorithm to generate a first image; wherein the infrared image and the visible light image are from the same picture; The image fusion module is used to extract and fuse features of the first image and the visible light image using a preset image fusion algorithm to generate a second image; The power line identification module is configured to perform power line identification on the second image using a preset power line identification algorithm.

9. A terminal device, characterized in that: include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the power line identification method based on image fusion according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device / apparatus where the computer-readable storage medium is located is controlled to execute the power line identification method based on image fusion according to any one of claims 1 to 7.

Images