Partial discharge detection method and device based on fusion of infrared and ultraviolet images

By integrating infrared and ultraviolet images to detect partial discharge, the problem of low detection accuracy and efficiency in traditional inspection methods has been solved, enabling efficient and accurate detection of power transmission lines and improving safety and stability.

CN119024109APending Publication Date: 2024-11-26MAOMING POWER SUPPLY BUREAU GUANGDONG POWER GRID CORP
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411034220.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Traditional manual inspection methods for partial discharge detection are costly, inefficient, and pose safety risks. Furthermore, ultraviolet imaging or ultrasonic detection are easily affected by environmental factors, resulting in low accuracy and efficiency, and the possibility of missed detections.

Method used

A partial discharge detection method based on fused infrared and ultraviolet images is adopted. By acquiring the target region in the infrared image, image segmentation and temperature detection are performed. The ultraviolet image is then combined for registration and feature point matching. A weight map is extracted and the images are fused to finally detect partial discharge.

Benefits of technology

It enables comprehensive, multi-angle detection of partial discharge, improving detection accuracy and efficiency, reducing missed detections, and enhancing the safety and stability of transmission lines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119024109A_ABST
    Figure CN119024109A_ABST
Patent Text Reader

Abstract

The invention discloses a partial discharge detection method and device based on fusion of infrared and ultraviolet images. The method comprises the steps that an infrared image is acquired, a target area in the infrared image is detected, an ultraviolet image corresponding to the infrared image is acquired under the condition that the target area is detected, and the target area is an image area with the corresponding temperature difference meeting the preset requirement in the infrared image; performing registration processing on the infrared image and the ultraviolet image according to the feature points in the infrared image and the feature points in the ultraviolet image; extracting a weight map of the infrared image and a weight map of the ultraviolet image, and performing fusion processing on the infrared image, the ultraviolet image, the weight map of the infrared image and the weight map of the ultraviolet image to obtain a fused image; and performing partial discharge detection according to the fused image, and determining a partial discharge detection result. The technical problem of low detection accuracy and efficiency caused by missing detection in a traditional inspection mode is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of power system detection, and more specifically, to a method and device for detecting partial discharge based on the fusion of infrared and ultraviolet images. Background Art

[0002] With the development of the power industry, the length of overhead transmission lines continues to extend, resulting in an increasing inspection and maintenance workload for the power industry. Due to long-term effects of light, humidity, pollution, and bad weather, power equipment on transmission lines, such as conductors and insulators, often experience problems such as breakage, looseness, and cracks, resulting in reduced insulation performance and even various partial discharge phenomena, which have a serious impact on the implementation of power services and residents' lives. However, the cost of partial discharge detection through traditional manual inspection methods is high, the efficiency is low, there is a high safety risk, and operation and maintenance decisions are often affected by subjective factors.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a partial discharge detection method and device based on the fusion of infrared and ultraviolet images, so as to at least solve the technical problems of low detection accuracy and efficiency caused by missed detection in the traditional inspection method.

[0005] According to one aspect of an embodiment of the present application, a local discharge detection method and device based on fusion of infrared and ultraviolet images are provided, including: acquiring an infrared image and detecting a target area in the infrared image, and acquiring an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets preset requirements; registering the infrared image with the ultraviolet image based on feature points in the infrared image and feature points in the ultraviolet image; extracting a weight map of the infrared image and a weight map of the ultraviolet image, and fusing the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weights of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weights of each feature in the ultraviolet image; performing local discharge detection based on the fused image to determine the local discharge detection result.

[0006] Optionally, acquiring an infrared image and detecting a target area in the infrared image, and acquiring an ultraviolet image corresponding to the infrared image when the target area is detected includes: segmenting the infrared image according to the grayscale of the infrared image to obtain a plurality of infrared image areas; performing temperature detection on the plurality of infrared image areas after the segmentation process to determine a maximum temperature and a minimum temperature corresponding to each infrared image area, and determining a maximum relative temperature difference corresponding to each infrared image area based on the maximum temperature and the minimum temperature; determining an infrared image area whose corresponding maximum relative temperature difference is greater than a first threshold as a target area; and acquiring an ultraviolet image corresponding to the infrared image by an ultraviolet imager when the target area is detected. Optionally, registering the infrared image with the ultraviolet image based on the feature points in the infrared image and the feature points in the ultraviolet image includes: determining the autocorrelation matrix of each pixel point based on the directional gradient of each pixel point in the ultraviolet image and the infrared image, wherein the autocorrelation matrix is ​​used to represent the local change of the image; determining the response function of each pixel point based on the autocorrelation matrix of each pixel point, wherein the response function is used to represent the difference in the degree of change in different directions around the pixel point, and determining the pixel point as a feature point when the value of the response function is greater than a second threshold; determining the feature vector of each feature point based on the gradient and direction of the area around each feature point; overlapping the ultraviolet image with the corresponding infrared image, and after overlapping the ultraviolet image and the infrared image, determining the Euclidean distance between each feature point based on the feature vector composed of the feature points in the ultraviolet image and each feature point in the corresponding infrared image; determining the feature point pair with the smallest Euclidean distance between each feature point in the ultraviolet image and the corresponding infrared image as the matching point, and performing image matching.

[0007] Optionally, extracting the weight map of the infrared image and the weight map of the ultraviolet image, fusing the infrared image, the ultraviolet image, the weight map of the infrared image and the weight map of the ultraviolet image to obtain the fused image includes: extracting the feature map of the infrared image and multiple feature maps of the ultraviolet image through multiple convolutional layers and pooling layers, calculating the importance weight of each feature map, weighting each feature map according to the importance weight, and obtaining the weight map of the infrared image and the weight map of the ultraviolet image; performing Gaussian smoothing and downsampling processing on the infrared image multiple times according to the weight map of the infrared image to obtain the first component map and the second component map of the infrared image, performing Gaussian smoothing and downsampling processing on the ultraviolet image multiple times according to the weight map of the ultraviolet image to obtain the first component map and the second component map of the ultraviolet image; fusing the first component map of the infrared image with the first component map of the registered ultraviolet image to obtain the first component fused map, fusing the second component map of the infrared image with the second component map of the registered ultraviolet image to obtain the second component fused map; fusing the first component fused map with the second component fused map to obtain the fused image.

[0008] Optionally, the first component image of the infrared image and the first component image of the ultraviolet image are fused according to the weighted average rule using the following formula: f =α·L UV +β·L IR Among them, L UV and L IR They are the first component images of the ultraviolet image and the infrared image respectively, α and β are weighting coefficients, and α+β=1.

[0009] Optionally, fusing the second component map of the infrared image with the second component map of the registered ultraviolet image to obtain the second component fusion map includes: extracting deep detail information from the second component map using a neural network, and fusing the deep features according to the following formula: Among them, d kUV and d kIR are the second components of the ultraviolet image and infrared image, respectively, f is the second component after fusion.

[0010] Optionally, performing local discharge detection based on the fused image and determining the local discharge detection result includes: establishing a local discharge detection sample library based on the fused image and its corresponding discharge severity information; training a local discharge detection model based on the local discharge detection sample library, and determining that the training of the local discharge detection model is completed when the detection accuracy of the local discharge detection model reaches a preset value; and determining that the local discharge detection result of the fused image is that local discharge exists when the local discharge probability output by the trained local discharge detection model is greater than a third threshold.

[0011] According to another aspect of an embodiment of the present application, a local discharge detection device based on fusion of infrared and ultraviolet images is also provided, including: an acquisition module, used to acquire an infrared image and detect a target area in the infrared image, and acquire an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets preset requirements; an alignment module, which aligns the infrared image with the ultraviolet image based on feature points in the infrared image and feature points in the ultraviolet image; a fusion module, which extracts a weight map of the infrared image and a weight map of the ultraviolet image, and fuses the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weights of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weights of each feature in the ultraviolet image; a detection module, which performs local discharge detection based on the fused image and determines the local discharge detection result.

[0012] According to another aspect of an embodiment of the present application, a non-volatile storage medium is provided, in which a program is stored, wherein when the program is running, a device where the non-volatile storage medium is located is controlled to execute a local discharge detection method based on the fusion of infrared and ultraviolet images.

[0013] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including: a memory and a processor, the processor being used to run a program stored in the memory, wherein the program executes a local discharge detection method based on fusion of infrared and ultraviolet images when running.

[0014] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, which implements a partial discharge detection method based on fusion of infrared and ultraviolet images when executed by a processor.

[0015] In an embodiment of the present application, an infrared image is acquired and a target area in the infrared image is detected, and an ultraviolet image corresponding to the infrared image is acquired when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets preset requirements; the infrared image and the ultraviolet image are registered based on feature points in the infrared image and feature points in the ultraviolet image; a weight map of the infrared image and a weight map of the ultraviolet image are extracted, and the infrared image, the ultraviolet image, the weight map of the infrared image and the weight map of the ultraviolet image are fused to obtain a fused image, wherein the weight map of the infrared image is used to record the weights of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weights of each feature in the ultraviolet image; local discharge detection is performed based on the fused image to determine the local discharge detection result. By fusion detection of the infrared image and the ultraviolet image, the purpose of comprehensively evaluating the local discharge situation is achieved, thereby achieving the technical effect of improving the accuracy of local discharge detection, and further solving the technical problems of low detection accuracy and efficiency caused by missed detection using traditional inspection methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 It is a structural schematic diagram of a computer terminal (mobile device) provided according to an embodiment of the present application;

[0018] Figure 2 It is a flow chart of a partial discharge detection method based on fusion of infrared and ultraviolet images provided in an embodiment of the present application;

[0019] Figure 3It is a structural schematic diagram of a partial discharge detection device based on the fusion of infrared and ultraviolet images provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0023] HARRIS corner detection algorithm: HARRIS corner detection algorithm is a method used in the field of computer vision to detect corners in images. Its goal is to find significant feature points in the image, which have obvious texture changes in local areas of the image.

[0024] OTSU maximum inter-class variance segmentation method: a threshold segmentation method based on pixel gray value, which determines the optimal threshold by maximizing the inter-class variance.

[0025] In the related technologies, partial discharge detection technology mostly adopts a single technology. The cost and efficiency of partial discharge detection through traditional manual inspection methods are high, and there are high safety risks. In addition, operation and maintenance decisions are often affected by subjective factors. Ultraviolet imaging or ultrasonic detection can often only detect partial partial discharges, and are often affected by objective conditions such as environmental factors. Some types of partial discharges may not be accurately identified, which is time-consuming and labor-intensive. In addition, there are blind spots and omissions in large-scale transmission lines. Therefore, there are problems of low detection accuracy and efficiency caused by missed detection using traditional inspection methods. In order to solve this problem, a relevant solution is provided in the embodiments of the present application, which is described in detail below.

[0026] According to an embodiment of the present application, a method embodiment of a partial discharge detection method based on the fusion of infrared and ultraviolet images is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a partial discharge detection method based on the fusion of infrared and ultraviolet images is shown. Figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0028] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0029] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the partial discharge detection method based on the fusion of infrared and ultraviolet images in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, to implement the vulnerability detection method of the above-mentioned application. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0032] Under the above operating environment, the embodiment of the present application provides a partial discharge detection method based on the fusion of infrared and ultraviolet images, such as Figure 2 As shown, the method comprises the following steps:

[0033] Step S202, acquiring an infrared image and detecting a target area in the infrared image, and acquiring an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets a preset requirement;

[0034] In some embodiments of the present application, infrared image acquisition and detection include capturing infrared images of transmission lines, segmenting infrared images, infrared temperature detection; image denoising, Canny edge detection, single-frame discharge count statistics, and overall discharge count statistics.

[0035] Specifically, firstly, the infrared imager is installed on the fuselage of the UAV and fixed by a fixing rod, and the infrared imager parameters are set before the inspection to capture the infrared image of the transmission line. After acquiring the infrared image, the OTSU maximum inter-class variance segmentation method is used to segment the image, and the average grayscale of the image is used as the grayscale threshold to determine the segmentation threshold of the thermal imaging of the transmission line. Among them, the average grayscale is determined by the following formula:

[0036]

[0037] Where h(x,y) represents the average grayscale value at the coordinate (x,y). n represents the neighborhood width, which refers to the size of the area used to calculate the average grayscale value. f(x,y) represents the grayscale value of the original image at the coordinate (x,y). ∑f(x,y) represents the sum of the grayscale values ​​of all pixels within the neighborhood width n.

[0038] After determining the average grayscale, the segmentation threshold is determined by the following formula:

[0039]

[0040] in, is the segmentation threshold, which determines whether a pixel in the image belongs to a specific area. g(x,y) is the characteristic function at the coordinate (x,y), which is used to describe the properties of the image at that point, such as whether the grayscale of the pixel exceeds the grayscale threshold. ij Represents the weighting coefficient. It is used to adjust the weights of different regions or different features in segmentation to make the segmentation more accurate. M×N is the total number of pixels in the image, and M and N are the width and height of the image respectively. This value is used for standardization so that the segmentation threshold is consistent in images of different sizes. δ represents a small adjustment amount, which is used to fine-tune the segmentation threshold to make it more accurate

[0041] After the infrared image is segmented, the infrared temperature is detected on the infrared image, and the relative temperature difference judgment method is used to detect the operation status of the transmission line. The maximum relative temperature difference of the transmission line is calculated according to the temperature information of the transmission line area. The calculation formula is:

[0042]

[0043] Among them, T max To detect the maximum temperature value in the infrared image area, T min It is the minimum temperature value detected in the infrared image area.

[0044] Through the infrared images returned by the drone inspection, the platform determines that the rest of the areas where the maximum relative temperature difference exceeds the first threshold are overheated areas, that is, target areas.

[0045] When the target area is detected, the transmission line with the target area is inspected by using a UAV equipped with a highly sensitive UV imager, and the UV image is collected and recorded under the same environmental conditions and observation distance as the corresponding infrared image. These data will be transmitted to the ground platform for subsequent processing.

[0046] Step S204, registering the infrared image with the ultraviolet image according to the feature points in the infrared image and the feature points in the ultraviolet image;

[0047] In some embodiments of the present application, registering the infrared image with the ultraviolet image includes performing HARRIS corner point detection, feature point matching, and final image registration on the infrared image and the ultraviolet image.

[0048] Specifically, Harris corner detection algorithm is used to extract feature points in ultraviolet images and infrared images. Harris corner points have high stability and significance and can accurately locate key features in images.

[0049] First, the gradients of the ultraviolet image and infrared image in the x and y directions are calculated respectively. Secondly, at each pixel point, the autocorrelation matrix M is calculated to represent the local changes of the image. Finally, the response function R of each pixel point is calculated using the autocorrelation matrix M to determine whether the point is a corner point.

[0050] R = det(M) - k (trace(M)) 2

[0051] Where det(M) is the determinant of the autocorrelation matrix M, trace(M) is the trace of the autocorrelation matrix M, and k is an empirical parameter, usually ranging from 0.04 to 0.06. When the value of the response function R is greater than the set threshold, the point is considered to be a corner point.

[0052] Specifically, the feature point matching of the ultraviolet image and the infrared image includes using the SIFT descriptor to describe each HARRIS corner point in the ultraviolet image and the infrared image for subsequent matching. The specific method is as follows: for each HARRIS corner point in the ultraviolet image, the gradient and direction of the local area around it are calculated to form a feature vector. For each HARRIS corner point in the infrared image, the feature vector is calculated using the same method. By calculating the Euclidean distance between the feature vectors, the feature point pairs in the ultraviolet image and the infrared image are matched, and the point pair with the smallest distance is selected as the matching point.

[0053] Specifically, the registration of ultraviolet images and infrared images includes selecting several sets of optimal matching point pairs from the matching feature point pairs, and using these point pairs to calculate the projection transformation matrix from the ultraviolet image to the infrared image. The projection transformation matrix is ​​estimated using the least squares method to minimize the registration error. The mathematical representation of the projection transformation model is:

[0054]

[0055] Among them, H is the projection transformation matrix, (x, y) and (x′, y′) are the pixel coordinates before and after the transformation. Through the projection transformation, each pixel in the ultraviolet image is mapped to the corresponding position of the infrared image, so that the two images are aligned in space.

[0056] Step S206, extracting the weight map of the infrared image and the weight map of the ultraviolet image, fusing the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weight of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weight of each feature in the ultraviolet image;

[0057] In some embodiments of the present application, SCNN is used to extract features of ultraviolet images and infrared images respectively. The SCNN network consists of two branches with the same structure and parameters, and learns and extracts important information in the image by comparing the features of the two input images. The ultraviolet image and the infrared image are input to the two branches of SCNN, and high-level feature maps are extracted after being processed by multiple convolutional layers and pooling layers. An SE Net module is added to each SCNN branch. The SE Net module calculates the importance weight of each feature map channel through a global average pooling layer and two fully connected layers, and re-weights the feature map to obtain a weight map W containing the clarity information corresponding to all pixels of the ultraviolet and infrared images.

[0058] Next, pyramid Laplacian decomposition is performed. The ultraviolet image, infrared image and weight map are decomposed into a Gaussian pyramid. Each layer of the Gaussian pyramid is obtained by Gaussian smoothing and downsampling the image of the previous layer. The Laplacian pyramid is obtained by subtracting each layer of the Gaussian pyramid from the Gaussian expansion image of the previous layer. The image is then interpolated and enlarged. In this way, the Laplacian pyramid can capture the multi-scale features of the image and provide rich detail information for subsequent image fusion.

[0059] Then, image fusion is performed. The low-frequency component mainly represents the overall structure of the image and a large range of brightness changes, and is usually obtained through Gaussian pyramid decomposition. This process includes Gaussian smoothing of the original image, generating images with gradually decreasing resolution layer by layer, and the bottom layer of the image is used as the low-frequency component. For the low-frequency component, the weighted average rule is used for fusion:

[0060] L f =α·L UV +β·L IR

[0061] Among them, L UV and L IR are the low-frequency components of the ultraviolet image and the infrared image respectively, α and β are weighting coefficients, satisfying α+β=1.

[0062] The high-frequency components of the image contain the details and edge information of the image, which are obtained through Laplacian pyramid decomposition. The specific steps are: interpolate and amplify the images of each layer of the Gaussian pyramid, subtract the interpolated and amplified image from the original image, and obtain a Laplacian pyramid image containing details and edge information. For high-frequency components, the VGG-19 network is used to extract deep detail information from the high-frequency components, and the deep features are fused using the L1 regularization fusion rule:

[0063]

[0064] Among them, d kUV and d kIR are the high frequency components of the ultraviolet image and infrared image respectively, d f is the high-frequency component feature after fusion. Finally, the low-frequency component fusion image and the high-frequency component fusion image are subjected to inverse Laplace transform to construct a complete fusion image.

[0065] Step S208, performing partial discharge detection based on the fused image to determine a partial discharge detection result.

[0066] In some embodiments of the present application, a local discharge detection sample library is established based on the fused image and its corresponding discharge severity information, and a local discharge detection model is trained based on the local discharge detection sample library. When the detection accuracy of the local discharge detection model reaches a preset value, it is determined that the training of the local discharge detection model is completed so that it can accurately identify and analyze the characteristics of local discharge. When the local discharge probability output by the trained local discharge detection model is greater than a third threshold, the local discharge detection result of the fused image is determined to be the presence of local discharge.

[0067] Specifically, through the deep learning algorithm, a partial discharge fault feature extraction network is established to extract target features, and then through the feature fusion network, the extracted shallow position information is fused with the deep information to obtain the feature image containing both information, and then the recognition and classification are established and trained. According to the probability or classification result output by the model, the third threshold is set to determine whether there is partial discharge in the image. When the probability output by the model is higher than the set third threshold, it is considered that there is partial discharge in the image. If partial discharge is found, the system can issue an alarm to remind the operator to conduct further inspection and processing.

[0068] Through the above steps, a partial discharge detection method based on the fusion of infrared and ultraviolet images can be realized, which can comprehensively and multi-angle detect the possible partial discharge problems of the transmission line. Combined with infrared temperature measurement and ultraviolet detection technology, the transmission line is comprehensively monitored. Through image processing technology, infrared and ultraviolet images are processed and fused, an image sample library is established, and intelligent identification of partial discharge is realized, thereby improving the accuracy and efficiency of detection. This method can not only detect abnormal situations in time, but also greatly improve the safety and stability of the transmission line, and solve the technical problems of low detection accuracy and efficiency caused by missed detection using traditional inspection methods.

[0069] The embodiment of the present application provides a partial discharge detection device based on the fusion of infrared and ultraviolet images. Figure 3 is a schematic diagram of the structure of the device, such as Figure 3As shown, the device includes: an acquisition module 30, which is used to acquire an infrared image and detect a target area in the infrared image, and acquire an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets the preset requirements; a registration module 32, which performs registration processing on the infrared image and the ultraviolet image based on feature points in the infrared image and feature points in the ultraviolet image; a fusion module 34, which extracts a weight map of the infrared image and a weight map of the ultraviolet image, and performs fusion processing on the infrared image, the ultraviolet image, the weight map of the infrared image and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weights of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weights of each feature in the ultraviolet image; a detection module 36, which performs local discharge detection based on the fused image and determines the local discharge detection result.

[0070] In some embodiments of the present application, the acquisition module 30 is used to acquire an infrared image and detect a target area in the infrared image, and to acquire an ultraviolet image corresponding to the infrared image when the target area is detected, including: segmenting the infrared image according to the grayscale of the infrared image to obtain a plurality of infrared image areas; performing temperature detection on the plurality of infrared image areas after the segmentation process to determine the maximum temperature and the minimum temperature corresponding to each infrared image area, and determining the maximum relative temperature difference corresponding to each infrared image area based on the maximum temperature and the minimum temperature; determining the infrared image area whose corresponding maximum relative temperature difference is greater than a first threshold as the target area; and acquiring the ultraviolet image corresponding to the infrared image through an ultraviolet imager when the target area is detected.

[0071] In some embodiments of the present application, the registration module 32 is used to perform registration processing on the infrared image and the ultraviolet image based on the feature points in the infrared image and the feature points in the ultraviolet image, including: determining the autocorrelation matrix of each pixel point based on the directional gradient of each pixel point in the ultraviolet image and the infrared image, wherein the autocorrelation matrix is ​​used to represent the local change of the image; determining the response function of each pixel point based on the autocorrelation matrix of each pixel point, wherein the response function is used to represent the difference in the degree of change in different directions around the pixel point, and when the value of the response function is greater than the second threshold, determining the pixel point as a feature point; determining the feature vector of each feature point based on the gradient and direction of the area around each feature point; overlapping the ultraviolet image with the corresponding infrared image, and after overlapping the ultraviolet image and the infrared image, determining the Euclidean distance between each feature point based on the feature vector composed of the feature points in the ultraviolet image and each feature point in the corresponding infrared image; determining the feature point pair with the smallest Euclidean distance between each feature point in the ultraviolet image and the corresponding infrared image as the matching point, and performing image matching.

[0072] In some embodiments of the present application, the fusion module 34 is used to extract the weight map of the infrared image and the weight map of the ultraviolet image, and fuse the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain the fused image, including: extracting the feature map of the infrared image and the multiple feature maps of the ultraviolet image through multiple convolution layers and pooling layers, calculating the importance weight of each feature map, weighting each feature map according to the importance weight, and obtaining the weight map of the infrared image and the weight map of the ultraviolet image; performing multiple Gaussian smoothing and downsampling processes on the infrared image according to the weight map of the infrared image to obtain the first component map and the second component map of the infrared image, performing multiple Gaussian smoothing and downsampling processes on the ultraviolet image according to the weight map of the ultraviolet image to obtain the first component map and the second component map of the ultraviolet image; fusing the first component map of the infrared image with the first component map of the registered ultraviolet image to obtain the first component fused map, fusing the second component map of the infrared image with the second component map of the registered ultraviolet image to obtain the second component fused map; fusing the first component fused map with the second component fused map to obtain the fused image.

[0073] In some embodiments of the present application, the fusion module 34 is used to fuse the first component map of the infrared image and the first component map of the registered ultraviolet image, and obtaining the first component fusion map includes: fusing the first component map of the infrared image and the first component map of the ultraviolet image according to the weighted average rule using the following formula: L f =α·L UV +β·L IR Among them, L UV and L IR They are the first component images of the ultraviolet image and the infrared image respectively, α and β are weighting coefficients, and α+β=1.

[0074] In some embodiments of the present application, the fusion module 34 is used to fuse the second component map of the infrared image and the second component map of the registered ultraviolet image, and obtaining the second component fusion map includes: extracting deep detail information from the second component map using a neural network, and fusing the deep features according to the following formula: Among them, d kUV and d kIR are the second components of the ultraviolet image and infrared image, respectively, f is the second component after fusion.

[0075] In some embodiments of the present application, the detection module 36 is used to perform local discharge detection based on the fused image, and determining the local discharge detection result includes: establishing a local discharge detection sample library based on the fused image and its corresponding discharge severity information; training a local discharge detection model based on the local discharge detection sample library, and determining that the training of the local discharge detection model is completed when the detection accuracy of the local discharge detection model reaches a preset value; and determining that the local discharge detection result of the fused image is that local discharge exists when the local discharge probability output by the trained local discharge detection model is greater than a third threshold.

[0076] It should be noted that the various modules in the above-mentioned local discharge detection device based on the fusion of infrared and ultraviolet images can be program modules (for example, a set of program instructions for implementing a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0077] An embodiment of the present application provides a non-volatile storage medium, in which a program is stored, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the following local discharge detection method based on the fusion of infrared and ultraviolet images: acquiring an infrared image and detecting a target area in the infrared image, and acquiring an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets preset requirements; registering the infrared image and the ultraviolet image based on feature points in the infrared image and feature points in the ultraviolet image; extracting a weight map of the infrared image and a weight map of the ultraviolet image, and fusing the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weights of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weights of each feature in the ultraviolet image; performing local discharge detection based on the fused image to determine the local discharge detection result.

[0078] An embodiment of the present application provides an electronic device, including: a memory and a processor, the processor is used to run a program stored in the memory, wherein when the program is run, the following local discharge detection method based on the fusion of infrared and ultraviolet images is executed: an infrared image is acquired and a target area in the infrared image is detected, and an ultraviolet image corresponding to the infrared image is acquired when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets a preset requirement; the infrared image and the ultraviolet image are registered based on feature points in the infrared image and feature points in the ultraviolet image; a weight map of the infrared image and a weight map of the ultraviolet image are extracted, and the infrared image, the ultraviolet image, the weight map of the infrared image and the weight map of the ultraviolet image are fused to obtain a fused image, wherein the weight map of the infrared image is used to record the weight of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weight of each feature in the ultraviolet image; local discharge detection is performed based on the fused image to determine the local discharge detection result.

[0079] The embodiment of the present application provides a computer program product, including a computer program, which implements the following local discharge detection method based on the fusion of infrared and ultraviolet images when executed by a processor: acquiring an infrared image and detecting a target area in the infrared image, and acquiring an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets preset requirements; registering the infrared image and the ultraviolet image based on feature points in the infrared image and feature points in the ultraviolet image; extracting a weight map of the infrared image and a weight map of the ultraviolet image, and fusing the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weights of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weights of each feature in the ultraviolet image; performing local discharge detection based on the fused image to determine the local discharge detection result.

[0080] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0081] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0084] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0086] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A partial discharge detection method based on the fusion of infrared and ultraviolet images, characterized in that: include: Acquire an infrared image and detect a target area in the infrared image, and acquire an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets a preset requirement; Performing registration processing on the infrared image and the ultraviolet image according to the feature points in the infrared image and the feature points in the ultraviolet image; Extracting the weight map of the infrared image and the weight map of the ultraviolet image, fusing the infrared image, the ultraviolet image, the weight map of the infrared image and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weights of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weights of each feature in the ultraviolet image; Partial discharge detection is performed based on the fused image to determine a partial discharge detection result.

2. The partial discharge detection method based on fusion of infrared and ultraviolet images according to claim 1 is characterized in that: Acquiring an infrared image and detecting a target area in the infrared image, and acquiring an ultraviolet image corresponding to the infrared image when the target area is detected, comprises: Segmenting the infrared image according to the grayscale of the infrared image to obtain a plurality of infrared image regions; Performing temperature detection on the plurality of infrared image regions after segmentation processing, determining a maximum temperature and a minimum temperature corresponding to each of the infrared image regions, and determining a maximum relative temperature difference corresponding to each of the infrared image regions according to the maximum temperature and the minimum temperature; Determine the infrared image area whose corresponding maximum relative temperature difference is greater than a first threshold as a target area; When the target area is detected, an ultraviolet image corresponding to the infrared image is acquired by an ultraviolet imager.

3. The partial discharge detection method based on fusion of infrared and ultraviolet images according to claim 1 is characterized in that: The registering process of the infrared image and the ultraviolet image according to the feature points in the infrared image and the feature points in the ultraviolet image includes: Determining the autocorrelation matrix of each pixel point according to the directional gradient of each pixel point in the ultraviolet image and the infrared image, wherein the autocorrelation matrix is ​​used to represent the local change of the image; Determine a response function of each pixel point according to the autocorrelation matrix of each pixel point, wherein the response function is used to represent the difference of the degree of change in different directions around the pixel point, and determine the pixel point as a feature point when the value of the response function is greater than a second threshold value; Determine a feature vector of each feature point according to the gradient and direction of the area around each feature point; Overlapping the ultraviolet image with the corresponding infrared image, and after overlapping the ultraviolet image and the infrared image, determining the Euclidean distance between each of the feature points according to a feature vector composed of a feature point in the ultraviolet image and each of the feature points in the corresponding infrared image; A pair of feature points with the smallest Euclidean distance between the feature points in the ultraviolet image and the corresponding feature points in the infrared image is determined as a matching point to perform image matching.

4. The partial discharge detection method based on fusion of infrared and ultraviolet images according to claim 1 is characterized in that: Extracting the weight map of the infrared image and the weight map of the ultraviolet image, fusing the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain a fused image includes: Extracting a feature map of the infrared image and a plurality of feature maps of the ultraviolet image through a plurality of convolutional layers and a pooling layer, calculating an importance weight of each feature map, weighting each feature map according to the importance weight, and obtaining a weight map of the infrared image and a weight map of the ultraviolet image; Performing multiple Gaussian smoothing and downsampling processes on the infrared image according to the weight map of the infrared image to obtain a first component map and a second component map of the infrared image, and performing multiple Gaussian smoothing and downsampling processes on the ultraviolet image according to the weight map of the ultraviolet image to obtain a first component map and a second component map of the ultraviolet image; The first component map of the infrared image and the first component map of the registered ultraviolet image are fused to obtain a first component fused map, and the second component map of the infrared image and the second component map of the registered ultraviolet image are fused to obtain a second component fused map; The first component fusion map is fused with the second component fusion map to obtain the fused image.

5. The partial discharge detection method based on fusion of infrared and ultraviolet images according to claim 4 is characterized in that: Fusing the first component map of the infrared image with the first component map of the registered ultraviolet image to obtain a first component fused map includes: The first component image of the infrared image and the first component image of the ultraviolet image are fused using the following formula according to the weighted average rule: L f =α·L UV +β·L IR Among them, L UV and L IR are the first component images of the ultraviolet image and the infrared image respectively, α and β are weighting coefficients, and α+β=1.

6. The partial discharge detection method based on fusion of infrared and ultraviolet images according to claim 4 is characterized in that: Fusing the second component map of the infrared image with the second component map of the registered ultraviolet image to obtain a second component fused map includes: A neural network is used to extract deep detail information from the second component image, and the deep features are fused according to the following formula: Among them, d kUV and d kIR are the second components of the ultraviolet image and the infrared image, respectively, f is the second component after fusion.

7. The partial discharge detection method based on fusion of infrared and ultraviolet images according to claim 1 is characterized in that: Performing partial discharge detection according to the fused image and determining a partial discharge detection result includes: Establishing a partial discharge detection sample library based on the fused image and its corresponding discharge severity information; Training a partial discharge detection model according to the partial discharge detection sample library, and determining that the training of the partial discharge detection model is completed when the detection accuracy of the partial discharge detection model reaches a preset value; When the partial discharge probability output by the trained partial discharge detection model is greater than a third threshold, it is determined that the partial discharge detection result of the fused image is that partial discharge exists.

8. A partial discharge detection device based on the fusion of infrared and ultraviolet images, characterized in that: include: An acquisition module, used to acquire an infrared image and detect a target area in the infrared image, and acquire an ultraviolet image corresponding to the infrared image when the target area is detected, wherein the target area is an image area in the infrared image whose corresponding temperature difference meets a preset requirement; A registration module, performing registration processing on the infrared image and the ultraviolet image according to the feature points in the infrared image and the feature points in the ultraviolet image; a fusion module, extracting the weight map of the infrared image and the weight map of the ultraviolet image, and fusing the infrared image, the ultraviolet image, the weight map of the infrared image, and the weight map of the ultraviolet image to obtain a fused image, wherein the weight map of the infrared image is used to record the weight of each feature in the infrared image, and the weight map of the ultraviolet image is used to record the weight of each feature in the ultraviolet image; The detection module is used to perform partial discharge detection based on the fused image and determine a partial discharge detection result.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute the partial discharge detection method based on fusion of infrared and ultraviolet images as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the partial discharge detection method based on fusion of infrared and ultraviolet images as described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Electrical equipment defect detection method based on fusion of ultraviolet video and infrared video

    CN103487729A

  • Infrared light image and visible light image fusion method based on interactive non-local average filtering

    CN104268847A

  • Soft tissue opto-acoustic / ultrasonic multi-modal image fusion method based on deep learning

    CN118212495A

  • Image fusion method and apparatus and training method and apparatus for image fusion model

    US20230214976A1