Ultraviolet light image and visible light image fusion method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of image fusion, and provides an ultraviolet light image and visible light image fusion method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring a visible light image obtained by performing image acquisition on a discharge area of electrical equipment of the same power transmission line and a first ultraviolet light image under a first gain voltage; adjusting the first gain voltage according to a relative size relationship between a first ratio of a first radius of a visible light image discharge region to a second radius of a maximum connected domain of the first ultraviolet light image and a preset ratio range to obtain a second gain voltage; obtaining a second ultraviolet light image obtained by carrying out image acquisition on the discharge area by the ultraviolet camera under the second gain voltage; and according to a second ratio of the first radius to a third radius of the maximum connected domain of the second ultraviolet image, superposing the second ultraviolet image and the visible light image to obtain a fused image. By adopting the method, the fusion precision of the ultraviolet light image and the visible light image can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of image fusion, and in particular to a method, apparatus, computer equipment, computer-readable storage medium and computer program product for fusing ultraviolet light images and visible light images. Background Art

[0002] In power inspection, the UV imaging corona monitoring method usually adopts dual-band imaging, that is, the discharge area is imaged by UV light through a day-blind UV imaging system, and the discharge area is imaged by visible light through a visible light imaging system. The UV image and the visible light image are superimposed and fused, so that a clear UV image can be obtained under daylight.

[0003] Since the optical paths of the ultraviolet camera and the visible light camera do not overlap, parallax will occur when the two observe the same discharge area at different distances. The existing method for fusing ultraviolet images and visible light images is to manually control the gain voltage of the ultraviolet camera or distinguish the gain voltage of the ultraviolet camera with a fixed gear, adjust the gain voltage of the ultraviolet camera to an appropriate gear, calculate the offset of the discharge area in the ultraviolet image and the visible light image, and perform offset correction when the ultraviolet image and the visible light image are superimposed and fused.

[0004] However, when the gain voltage value of the UV camera is manually controlled, if the gain voltage value is set too large, it will affect the offset calculation, and thus affect the fusion accuracy of the UV image and the visible light image; when the gain voltage value of the UV camera is distinguished by fixed gears, it is possible that the optimal gain voltage is not in the settable gear options, affecting the fusion accuracy of the UV image and the visible light image. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for fusing ultraviolet light images and visible light images in order to solve the above technical problems.

[0006] In a first aspect, the present application provides a method for fusing an ultraviolet light image and a visible light image, comprising:

[0007] Acquire a visible light image and a first ultraviolet light image obtained by collecting images of a discharge area of ​​electrical equipment on the same power transmission line; the first ultraviolet light image is an image collected by an ultraviolet camera at a first gain voltage;

[0008] Determining a first radius of a discharge region in the visible light image and a second radius of a maximum connected domain in the first ultraviolet light image;

[0009] According to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range, the first gain voltage is adjusted to obtain a second gain voltage;

[0010] Acquire a second ultraviolet image obtained by the ultraviolet camera capturing an image of the discharge area at a second gain voltage;

[0011] The second ultraviolet image and the visible light image are superimposed according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image to obtain a fused image.

[0012] In one embodiment, the step of acquiring a visible light image and a first ultraviolet light image obtained by collecting images of a discharge area of ​​an electrical device on the same power transmission line includes:

[0013] Acquire an original visible light image and a first original ultraviolet light image obtained by performing image acquisition on a discharge area of ​​electrical equipment on the same transmission line;

[0014] A denoising operation is performed on the original visible light image and the first original ultraviolet light image to obtain a visible light image and a first ultraviolet light image.

[0015] In one embodiment, determining the discharge area in the visible light image includes:

[0016] Get pixel threshold;

[0017] When a pixel value of the visible light image is greater than or equal to a pixel threshold, taking the pixel value as a discharge pixel;

[0018] The area formed by the discharge pixels in the visible light image is used as the discharge area in the visible light image.

[0019] In one embodiment, the first gain voltage is adjusted according to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range to obtain a second gain voltage, including:

[0020] When the first ratio is not within a preset ratio range, the first gain voltage is adjusted until the ratio of the radius of the ultraviolet image acquired by the ultraviolet camera when capturing an image of the discharge area under the adjusted first gain voltage to the first radius is equal to an end value of the preset ratio range, thereby obtaining an intermediate gain voltage;

[0021] The intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when capturing the image of the discharge area under the adjusted intermediate gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage;

[0022] When the first ratio is within a preset ratio range, the first gain voltage is adjusted until the average grayscale value of the ultraviolet image captured by the ultraviolet camera on the discharge area under the adjusted first gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage.

[0023] In one embodiment, the intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when capturing the image of the discharge area under the adjusted intermediate gain voltage is within a preset standard grayscale value range, to obtain a second gain voltage, including:

[0024] When the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring the image of the discharge area at the intermediate gain voltage is less than the minimum value within the preset standard grayscale value range, the intermediate gain voltage is sequentially increased by one step gain until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring the image of the discharge area at the intermediate gain voltage after the step gain is increased is within the preset standard grayscale value range, thereby obtaining a second gain voltage;

[0025] Alternatively, when the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring image of the discharge area at the intermediate gain voltage is greater than the maximum value within the preset standard grayscale value range, the intermediate gain voltage is reduced by one step gain in sequence until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring image of the discharge area at the intermediate gain voltage after the step gain is reduced is within the preset standard grayscale value range, thereby obtaining a second gain voltage.

[0026] In one embodiment, the step of superimposing the second ultraviolet image and the visible light image to obtain a fused image according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image includes:

[0027] A conversion relationship constant between a first viewing angle and a second viewing angle is obtained according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image; the first viewing angle is the viewing angle when the visible light camera collects the visible light image; the second viewing angle is the viewing angle when the ultraviolet camera collects the second ultraviolet image;

[0028] scaling the second ultraviolet light image according to the conversion relationship constant to obtain a third ultraviolet light image;

[0029] When the distance between the first center point coordinates of the discharge area in the visible light image and the third center point coordinates of the maximum connected domain in the third ultraviolet light image is less than a preset distance threshold, the maximum connected domain of the third ultraviolet light image is superimposed on the visible light image to obtain a fused image.

[0030] In a second aspect, the present application further provides a device for fusing ultraviolet light images and visible light images, comprising:

[0031] An image acquisition module, used to acquire a visible light image and a first ultraviolet light image obtained by acquiring images of a discharge area of ​​an electrical device of the same transmission line; the first ultraviolet light image is an image acquired by an ultraviolet camera at a first gain voltage;

[0032] a radius determination module, used to determine a first radius of the discharge area in the visible light image and a second radius of the largest connected domain in the first ultraviolet light image;

[0033] a gain voltage adjustment module, configured to adjust the first gain voltage according to a relative magnitude relationship between a first ratio of the first radius to the second radius and a preset ratio range, so as to obtain a second gain voltage;

[0034] The image acquisition module is further used to acquire a second ultraviolet image obtained by the ultraviolet camera collecting an image of the discharge area at a second gain voltage;

[0035] The image fusion module is used to superimpose the second ultraviolet image and the visible light image according to a second ratio of the first radius to a third radius of the maximum connected domain in the second ultraviolet image to obtain a fused image.

[0036] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the above method.

[0038] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and the computer program is executed by a processor to execute the above method.

[0039] The above-mentioned method, device, computer equipment, computer-readable storage medium and computer program product for fusing ultraviolet light images and visible light images obtain a visible light image and a first ultraviolet light image obtained by image acquisition of a discharge area of ​​electrical equipment on the same transmission line; the first ultraviolet light image is an image acquired by an ultraviolet camera at a first gain voltage; a first radius of the discharge area in the visible light image and a second radius of the largest connected domain in the first ultraviolet light image are determined; the first gain voltage is adjusted according to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range to obtain a second gain voltage; a second ultraviolet light image is obtained by image acquisition of the discharge area by an ultraviolet camera at the second gain voltage; and the second ultraviolet light image is superimposed with the visible light image according to a second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet light image to obtain a fused image. The present application adjusts the first gain voltage according to the relative size relationship between the first ratio of the first radius of the discharge area in the visible light image and the second radius of the largest connected domain in the first ultraviolet light image and the preset ratio range to obtain the second gain voltage, i.e., the optimal gain voltage, so as to avoid the fusion error caused by the light spot in the ultraviolet light image exceeding the light spot size in the visible light image too much when the gain voltage value is set too large, which affects the offset calculation; and avoids the fusion error caused by the optimal gain voltage not being in the settable gear option when the gain voltage of the ultraviolet camera is distinguished by a fixed gear, thereby improving the fusion accuracy of the ultraviolet image and the visible light image. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A diagram of an application environment of a method for fusing ultraviolet light images and visible light images in an embodiment;

[0042] Figure 2 is a schematic flow chart of a method for fusing an ultraviolet light image and a visible light image in one embodiment;

[0043] Figure 3 is a first ultraviolet image in one embodiment;

[0044] Figure 4 is a fused image in one embodiment;

[0045] Figure 5 is a structural block diagram of a device for fusing ultraviolet light images and visible light images in one embodiment;

[0046] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] The present application embodiment provides a method for fusing an ultraviolet image and a visible light image. The present application embodiment can be executed by a computer device, such as Figure 1 As shown, the computer device can obtain the visible light image and the first ultraviolet light image obtained by image acquisition of the discharge area of ​​the same transmission line electrical equipment, and then obtain a fused image. It can be understood that the computer device can be implemented by a server, or by a terminal, or by an interactive system between a terminal and a server. In this embodiment, the method includes Figure 2 The steps shown are:

[0049] Step S201, obtaining a visible light image and a first ultraviolet light image obtained by capturing images of a discharge area of ​​electrical equipment on the same transmission line; the first ultraviolet light image is an image captured by an ultraviolet camera at a first gain voltage.

[0050] A visible light image can be obtained by acquiring an image of a discharge area of ​​an electrical device on a transmission line by a visible light camera, and a first ultraviolet light image can also be simultaneously acquired by acquiring an image of a discharge area of ​​the electrical device on the same transmission line by an ultraviolet camera at a first gain voltage, wherein the first ultraviolet light image is as follows: Figure 3 shown.

[0051] Step S202 , determining a first radius of the discharge area in the visible light image and a second radius of the largest connected region in the first ultraviolet light image.

[0052] The discharge area in the visible light image can be determined according to a machine vision algorithm such as the YOLOv8 (You Only Look Once Version 8) algorithm. Among all the coordinates of the discharge area, the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate, and the maximum vertical coordinate of the discharge area are determined; the two coordinates with the largest distance in the discharge area are determined according to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate, and the maximum vertical coordinate; half of the distance between the two coordinates with the largest distance in the discharge area is taken as the first radius, and the first radius can be recorded as R1.

[0053] The maximum connected domain in the first ultraviolet image can be determined according to a two-pass scanning method or a seed filling method. Among all the coordinates of the maximum connected domain in the first ultraviolet image, the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate of the maximum connected domain in the first ultraviolet image are determined; according to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate, the two coordinates with the largest distance in the maximum connected domain in the first ultraviolet image are determined; half of the distance between the two coordinates with the largest distance in the maximum connected domain in the first ultraviolet image is used as the second radius, and the second radius can be recorded as R2.

[0054] Step S203: adjusting the first gain voltage according to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range to obtain a second gain voltage.

[0055] The preset ratio range can be determined through multiple experiments based on the viewing angle conversion relationship between the visible light camera and the ultraviolet camera actually used.

[0056] The ratio of the first radius to the second radius can be used as the first ratio. When the first ratio is within the preset ratio range, it indicates that the gain voltage suitability at this time is high, and the fusion accuracy of the ultraviolet light image and the visible light image is high; when the first ratio is not within the preset ratio range, it indicates that the gain voltage suitability at this time is low, and the fusion accuracy of the ultraviolet light image and the visible light image is low, and the first gain voltage needs to be adjusted so that the first ratio is within the preset ratio range to obtain the second gain voltage. In practical applications, automatic gain voltage control can be used to continuously fine-tune the first gain voltage.

[0057] Step S204 , obtaining a second ultraviolet image obtained by capturing an image of the discharge area by an ultraviolet camera at a second gain voltage.

[0058] After the second gain voltage is obtained, a second ultraviolet light image obtained by the ultraviolet camera capturing an image of the discharge area under the second gain voltage can be acquired.

[0059] Step S205 , according to a second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet image, superimpose the second ultraviolet image and the visible light image to obtain a fused image.

[0060] The maximum connected domain in the second ultraviolet image can be determined according to a two-pass scanning method or a seed filling method. Among all the coordinates of the maximum connected domain in the second ultraviolet image, determine the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate and maximum vertical coordinate of the maximum connected domain in the second ultraviolet image; determine the two coordinates with the largest distance in the maximum connected domain in the second ultraviolet image according to the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate and maximum vertical coordinate; take half of the distance between the two coordinates with the largest distance in the maximum connected domain in the second ultraviolet image as the third radius. The ratio of the first radius to the third radius can be taken as the second ratio. According to the second ratio, the second ultraviolet image is superimposed with the visible light image to obtain a fused image, wherein the fused image is as shown in Figure 4 shown.

[0061] In the above-mentioned method for fusing ultraviolet light images and visible light images, the first gain voltage is adjusted according to the relative size relationship between the first ratio of the first radius of the discharge area in the visible light image and the second radius of the largest connected domain in the first ultraviolet light image and the preset ratio range to obtain the second gain voltage, i.e., the optimal gain voltage, so as to avoid the fusion error caused by the light spot in the ultraviolet light image exceeding the light spot size in the visible light image too much when the gain voltage value is set too large, which affects the offset calculation; and avoid the fusion error caused by the optimal gain voltage not being in the settable gear option when the gain voltage of the ultraviolet camera is distinguished by a fixed gear, thereby improving the fusion accuracy of the ultraviolet image and the visible light image.

[0062] In one of the embodiments, a visible light image and a first ultraviolet light image are obtained by image acquisition of a discharge area of ​​electrical equipment on the same transmission line, and the specific steps are as follows: an original visible light image and a first original ultraviolet light image are obtained by image acquisition of a discharge area of ​​electrical equipment on the same transmission line; and a denoising operation is performed on the original visible light image and the first original ultraviolet light image to obtain a visible light image and a first ultraviolet light image.

[0063] An original visible light image and a first original ultraviolet light image are obtained by collecting images of a discharge area of ​​electrical equipment on the same transmission line. The first original ultraviolet light image is processed by a binarization method to obtain a binarized first original ultraviolet light image, and a median filter and a quadratic Wiener filter are used to remove multiplicative noise, salt and pepper noise, and Gaussian noise from the original visible light image and the binarized first original ultraviolet light image to obtain a visible light image and a first ultraviolet light image.

[0064] In this embodiment, a denoising operation is performed on the original visible light image and the first original ultraviolet light image to obtain a more realistic visible light image and first ultraviolet light image.

[0065] In one of the embodiments, the discharge area in the visible light image is determined by the following specific steps: obtaining a pixel threshold; when the pixel value of the visible light image is greater than or equal to the pixel threshold, taking the pixel value as a discharge pixel; and taking the area formed by the discharge pixels in the visible light image as the discharge area in the visible light image.

[0066] The pixel threshold for distinguishing the discharge area and the background area can be obtained according to the maximum inter-class variance method. When the pixel value of the visible light image is less than the pixel threshold, the pixel value is regarded as the discharge pixel; when the pixel value of the visible light image is greater than or equal to the pixel threshold, the pixel value is regarded as the discharge pixel; the area formed by the discharge pixels in the visible light image is regarded as the discharge area in the visible light image.

[0067] In this embodiment, the discharge pixel is determined according to the relative size relationship between the pixel value of the visible light image and the pixel threshold; the area formed by the discharge pixels in the visible light image is used as the discharge area in the visible light image, thereby obtaining a more accurate discharge area.

[0068] In one of the embodiments, the first gain voltage is adjusted according to the relative size relationship between the first ratio of the first radius to the second radius and the preset ratio range to obtain the second gain voltage, and the specific steps are as follows: when the first ratio is not within the preset ratio range, the first gain voltage is adjusted until the ratio of the radius of the ultraviolet image acquired by the ultraviolet camera when acquiring the image of the discharge area under the adjusted first gain voltage to the first radius is equal to the end value of the preset ratio range, thereby obtaining an intermediate gain voltage; the intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring the image of the discharge area under the adjusted intermediate gain voltage is within the preset standard grayscale value range, thereby obtaining the second gain voltage; when the first ratio is within the preset ratio range, the first gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring the image of the discharge area under the adjusted first gain voltage is within the preset standard grayscale value range, thereby obtaining the second gain voltage.

[0069] When the first ratio is not within the preset ratio range, it is determined whether the first ratio is less than a minimum value of the preset ratio range or greater than a maximum value of the preset ratio range.

[0070] When the first ratio is less than the minimum value of the preset ratio range, the first gain voltage is increased until the ratio of the radius of the ultraviolet light image captured by the ultraviolet camera on the discharge area under the increased first gain voltage to the first radius is equal to the minimum value of the preset ratio range, thereby obtaining an intermediate gain voltage.

[0071] Alternatively, when the first ratio is greater than the maximum value of the preset ratio range, the first gain voltage is reduced until the ratio of the radius of the ultraviolet light image captured by the ultraviolet camera at the reduced first gain voltage for capturing an image of the discharge area to the first radius is equal to the maximum value of the preset ratio range, thereby obtaining an intermediate gain voltage.

[0072] The gain voltage of the ultraviolet camera can be divided into ranges according to preset rules, so that each gain voltage has a corresponding gain voltage interval. In practical applications, the gain voltage intervals can be divided according to actual experience, or the gain voltage intervals can be divided evenly. For example, each gain voltage interval includes the same number of step gains, and each step gain can be the minimum value when the gain voltage is adjusted each time, which can be determined according to the parameters of the ultraviolet camera actually used. The preset standard grayscale value range can be determined according to the maximum average grayscale value and the minimum average grayscale value of the gain voltage interval corresponding to the intermediate gain voltage.

[0073] The intermediate gain voltage may be adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when capturing an image of the discharge area under the adjusted intermediate gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage.

[0074] When the first ratio is within the preset ratio range, the preset standard grayscale value range can be determined based on the maximum average grayscale value and the minimum average grayscale value of the gain voltage interval corresponding to the first gain voltage, and the first gain voltage can be adjusted until the average grayscale value of the ultraviolet image captured by the ultraviolet camera on the discharge area under the adjusted first gain voltage is within the preset standard grayscale value range, thereby obtaining the second gain voltage.

[0075] In this embodiment, the first gain voltage is adjusted according to the relative size relationship between the first ratio and the preset ratio range to obtain the second gain voltage, thereby ensuring that the ratio of the radius of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area under the second gain voltage to the first radius is within the preset ratio range, and the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area under the second gain voltage is within the preset standard grayscale value range, thereby making the suitability of the second gain voltage higher and improving the fusion accuracy of the ultraviolet image and the visible light image.

[0076] In one of the embodiments, the intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the adjusted intermediate gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage. The specific steps are as follows: when the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the intermediate gain voltage is less than the minimum value within the preset standard grayscale value range, the intermediate gain voltage is increased by one step gain in sequence until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the intermediate gain voltage after the step gain is increased is within the preset standard grayscale value range, thereby obtaining the second gain voltage; or when the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the intermediate gain voltage is greater than the maximum value within the preset standard grayscale value range, the intermediate gain voltage is reduced by one step gain in sequence until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the intermediate gain voltage after the step gain is reduced is within the preset standard grayscale value range, thereby obtaining the second gain voltage.

[0077] The step gain can be determined according to the parameters of the UV camera actually used.

[0078] When the average grayscale value of the ultraviolet image captured by the ultraviolet camera on the discharge area at an intermediate gain voltage is less than the minimum value within the preset standard grayscale value range, the intermediate gain voltage is increased by a step gain in sequence until the average grayscale value of the ultraviolet image captured by the ultraviolet camera on the discharge area at the intermediate gain voltage after the step gain is increased is within the preset standard grayscale value range, thereby obtaining a second gain voltage.

[0079] Alternatively, when the average grayscale value of the ultraviolet image captured by the ultraviolet camera on the discharge area at an intermediate gain voltage is greater than the maximum value within a preset standard grayscale value range, the intermediate gain voltage is reduced by one step gain in sequence until the average grayscale value of the ultraviolet image captured by the ultraviolet camera on the discharge area at the intermediate gain voltage after the step gain is reduced is within the preset standard grayscale value range, thereby obtaining a second gain voltage.

[0080] In this embodiment, the intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the adjusted intermediate gain voltage is within a preset standard grayscale value range, and a second gain voltage is obtained, so that when the ratio of the radius of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the second gain voltage to the first radius is within the preset ratio range, the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring images of the discharge area at the second gain voltage is also within the preset standard grayscale value range, thereby making the second gain voltage more suitable and improving the fusion accuracy of the ultraviolet image and the visible light image.

[0081] In one of the embodiments, the second ultraviolet light image is superimposed with the visible light image according to a second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet light image to obtain a fused image, and the specific steps are as follows: a conversion relationship constant between the first viewing angle and the second viewing angle is obtained according to the second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet light image; the first viewing angle is the viewing angle when the visible light camera collects the visible light image; the second viewing angle is the viewing angle when the ultraviolet camera collects the second ultraviolet light image; according to the conversion relationship constant, the second ultraviolet light image is scaled to obtain a third ultraviolet light image; when the distance between the first center point coordinates of the discharge area in the visible light image and the third center point coordinates of the largest connected domain in the third ultraviolet light image is less than a preset distance threshold, the largest connected domain of the third ultraviolet light image is superimposed with the visible light image to obtain a fused image.

[0082] In practical applications, before the dual fields of view (ultraviolet light field of view and visible light field of view) are superimposed, the field of view angles need to be converted and matched to reduce the deformation and distortion caused by image superposition.

[0083] Since the field of view of the UV camera is smaller than that of the visible light, the UV image needs to be scaled proportionally and then fused into the visible light image when performing image superposition and fusion. Image scaling means scaling the image while keeping the aspect ratio unchanged, thus maintaining the aspect ratio of the image and preventing distortion.

[0084] According to the second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet image, the conversion relationship constant between the first viewing angle and the second viewing angle is obtained, and the conversion relationship constant can be recorded as C; the first viewing angle is the viewing angle when the visible light camera captures the visible light image; the second viewing angle is the viewing angle when the ultraviolet camera captures the second ultraviolet image.

[0085] The second ultraviolet light image is scaled according to the conversion relationship constant to obtain a third ultraviolet light image.

[0086] Specifically, the image processing library can be used to scale the second ultraviolet image at the same ratio. For example, the cv.resize function in the cross-platform computer vision library (OpenCV) can be used to scale the second ultraviolet image at the same ratio. It is also possible to first calculate the ratio to be scaled based on the conversion relationship constant, and then use a for loop or vector operation to scale each pixel. The calculation formula is: new image size = old image size × scaling ratio. For example, a 500×300 image is reduced by 1 / 2, and the new image size is 250×150. The reverse process can also be used to scale up at the same ratio. The scaling algorithm can include an interpolation algorithm, and a bilinear interpolation algorithm or a cubic interpolation algorithm can be selected according to actual needs.

[0087] Unlike directly scaling the length or width, scaling at the same ratio can well preserve the structural information of the original image. For example, the resolution of the second ultraviolet image is 720×576, corresponding to a horizontal second field of view of 19.2 degrees and a vertical second field of view of 14.4 degrees. The resolution of the visible light image is 1920×1080, corresponding to a horizontal first field of view of 58.07 degrees and a vertical first field of view of 34.8 degrees. Therefore, before the calculation, the second field of view of the second ultraviolet image corresponding to the 16:9 area is calculated first, that is, the horizontal second field of view and the vertical second field of view of the second ultraviolet image with a resolution of 720×405. The horizontal second field of view does not need to be converted, while the vertical second field of view is converted as follows: (405 / 576)×14.4=10.125; therefore, the field of view of the second ultraviolet image with a resolution of 720×405 is 10.125 degrees. During superposition, the scaled size of the second ultraviolet image with a resolution of 720×405 is first calculated, and then restored to the scaled size of the second ultraviolet image with a resolution of 720×576 according to the ratio.

[0088] The discharge area in the visible light image can be determined according to the machine vision algorithm. Among all the coordinates of the discharge area, the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate of the discharge area are determined; the two coordinates with the largest distance in the discharge area are determined according to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate; the center point coordinate between the two coordinates with the largest distance in the discharge area is used as the first center point coordinate.

[0089] The maximum connected domain in the third ultraviolet image can be determined according to a two-pass scanning method or a seed filling method. Among all the coordinates of the maximum connected domain in the third ultraviolet image, the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate of the maximum connected domain in the third ultraviolet image are determined; according to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate, the two coordinates with the largest distance in the maximum connected domain in the third ultraviolet image are determined; and the coordinates of the center point between the two coordinates with the largest distance in the maximum connected domain in the third ultraviolet image are used as the third center point coordinates.

[0090] The preset distance threshold can be determined according to actual conditions. When the distance between the coordinates of the first center point and the coordinates of the third center point is less than the preset distance threshold, it indicates that the overlap between the first center point and the third center point is high, and the accuracy of fusing the third ultraviolet image and the visible light image is high at this time. Therefore, the maximum connected domain of the third ultraviolet image can be superimposed on the visible light image to obtain a fused image. The distance is the Euclidean distance.

[0091] Specifically, when the visible light image is RGB24 type data and the third ultraviolet image is a single-channel image of U8 type, after the third ultraviolet image is binarized, the pixel value of the third ultraviolet image is 0 or 255, and there is no other third value. In order to speed up the superposition speed, the Compute Unified Device Architecture (CUDA) can be used to accelerate the superposition of the maximum connected domain of the third ultraviolet image and the visible light image in parallel to obtain a fused image.

[0092] In this embodiment, according to the second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet image, a conversion relationship constant between the first viewing angle and the second viewing angle is obtained to scale the second ultraviolet image to obtain a third ultraviolet image matching the first field of view. When the distance between the first center point coordinates of the discharge area in the visible light image and the third center point coordinates of the largest connected domain in the third ultraviolet image is less than a preset distance threshold, the largest connected domain of the third ultraviolet image is superimposed on the visible light image to obtain a fused image with higher accuracy.

[0093] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0094] Based on the same inventive concept, the embodiment of the present application also provides a fusion device of ultraviolet light images and visible light images for implementing the above-mentioned fusion method of ultraviolet light images and visible light images. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of one or more fusion devices of ultraviolet light images and visible light images provided below can refer to the limitations of the fusion method of ultraviolet light images and visible light images above, and will not be repeated here.

[0095] In an exemplary embodiment, Figure 5 As shown, a device for fusing ultraviolet light images and visible light images is provided, wherein:

[0096] The image acquisition module 501 is used to acquire a visible light image and a first ultraviolet light image obtained by acquiring images of a discharge area of ​​an electrical device of the same transmission line; the first ultraviolet light image is an image acquired by an ultraviolet camera at a first gain voltage;

[0097] A radius determination module 502, configured to determine a first radius of a discharge region in the visible light image and a second radius of a maximum connected region in the first ultraviolet light image;

[0098] A gain voltage adjustment module 503, configured to adjust the first gain voltage according to a relative magnitude relationship between a first ratio of the first radius to the second radius and a preset ratio range, to obtain a second gain voltage;

[0099] The image acquisition module 501 is further used to acquire a second ultraviolet image obtained by the ultraviolet camera capturing an image of the discharge area at a second gain voltage;

[0100] The image fusion module 504 is configured to superimpose the second ultraviolet image and the visible light image according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image to obtain a fused image.

[0101] In one embodiment, the image acquisition module 501 is further used to: acquire an original visible light image and a first original ultraviolet light image obtained by image acquisition of a discharge area of ​​electrical equipment on the same transmission line; and perform denoising operations on the original visible light image and the first original ultraviolet light image to obtain a visible light image and a first ultraviolet light image.

[0102] In one of the embodiments, the device also includes a discharge area acquisition module, which is used to: obtain a pixel threshold; when the pixel value of the visible light image is greater than or equal to the pixel threshold, use the pixel value as a discharge pixel; and use the area formed by the discharge pixels in the visible light image as the discharge area in the visible light image.

[0103] In one embodiment, the gain voltage adjustment module 503 is further used to: when the first ratio is not within a preset ratio range, adjust the first gain voltage until the ratio of the radius of the ultraviolet image acquired by the ultraviolet camera when acquiring an image of the discharge area under the adjusted first gain voltage to the first radius is equal to the end value of the preset ratio range, thereby obtaining an intermediate gain voltage; adjust the intermediate gain voltage until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring an image of the discharge area under the adjusted intermediate gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage; when the first ratio is within the preset ratio range, adjust the first gain voltage until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring an image of the discharge area under the adjusted first gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage.

[0104] In one embodiment, the gain voltage adjustment module 503 is further used to: when the average grayscale value of the ultraviolet image acquired by the ultraviolet camera at the intermediate gain voltage for image acquisition of the discharge area is less than the minimum value within the preset standard grayscale value range, increase the intermediate gain voltage by one step gain in sequence until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera at the intermediate gain voltage after the step gain is increased is within the preset standard grayscale value range, thereby obtaining a second gain voltage; or when the average grayscale value of the ultraviolet image acquired by the ultraviolet camera at the intermediate gain voltage for image acquisition of the discharge area is greater than the maximum value within the preset standard grayscale value range, reduce the intermediate gain voltage by one step gain in sequence until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera at the intermediate gain voltage after the step gain is reduced is within the preset standard grayscale value range, thereby obtaining a second gain voltage.

[0105] In one embodiment, the image fusion module 504 is further used to: obtain a conversion relationship constant between the first viewing angle and the second viewing angle according to a second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet light image; the first viewing angle is the viewing angle when the visible light camera collects the visible light image; the second viewing angle is the viewing angle when the ultraviolet camera collects the second ultraviolet light image; according to the conversion relationship constant, scale the second ultraviolet light image to obtain a third ultraviolet light image; when the distance between the first center point coordinates of the discharge area in the visible light image and the third center point coordinates of the largest connected domain in the third ultraviolet light image is less than a preset distance threshold, superimpose the largest connected domain of the third ultraviolet light image and the visible light image to obtain a fused image.

[0106] Each module in the above-mentioned ultraviolet image and visible light image fusion device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0107] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of an embodiment of the method for fusing ultraviolet light images and visible light images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for fusing ultraviolet light images and visible light images is implemented.

[0108] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0111] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0113] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0115] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for fusing ultraviolet light images and visible light images, characterized in that: The method comprises: Acquire a visible light image and a first ultraviolet light image obtained by collecting images of a discharge area of ​​electrical equipment on the same power transmission line; the first ultraviolet light image is an image collected by an ultraviolet camera at a first gain voltage; Determining a first radius of a discharge region in the visible light image and a second radius of a maximum connected domain in the first ultraviolet light image; According to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range, the first gain voltage is adjusted to obtain a second gain voltage; Acquire a second ultraviolet image obtained by the ultraviolet camera capturing an image of the discharge area at a second gain voltage; The second ultraviolet image and the visible light image are superimposed according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image to obtain a fused image.

2. The method according to claim 1, characterized in that The method of acquiring a visible light image and a first ultraviolet light image obtained by collecting images of a discharge area of ​​electrical equipment on the same power transmission line includes: Acquire an original visible light image and a first original ultraviolet light image obtained by performing image acquisition on a discharge area of ​​electrical equipment on the same transmission line; A denoising operation is performed on the original visible light image and the first original ultraviolet light image to obtain a visible light image and a first ultraviolet light image.

3. The method according to claim 1, characterized in that Determining a discharge area in the visible light image includes: Get pixel threshold; When a pixel value of the visible light image is greater than or equal to a pixel threshold, taking the pixel value as a discharge pixel; The area formed by the discharge pixels in the visible light image is used as the discharge area in the visible light image.

4. The method according to claim 1, characterized in that: The first gain voltage is adjusted according to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range to obtain a second gain voltage, comprising: When the first ratio is not within a preset ratio range, the first gain voltage is adjusted until the ratio of the radius of the ultraviolet image acquired by the ultraviolet camera when capturing an image of the discharge area under the adjusted first gain voltage to the first radius is equal to an end value of the preset ratio range, thereby obtaining an intermediate gain voltage; The intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when capturing the image of the discharge area under the adjusted intermediate gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage; When the first ratio is within a preset ratio range, the first gain voltage is adjusted until the average grayscale value of the ultraviolet image captured by the ultraviolet camera on the discharge area under the adjusted first gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage.

5. The method according to claim 4, characterized in that The intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when capturing the image of the discharge area under the adjusted intermediate gain voltage is within a preset standard grayscale value range, to obtain a second gain voltage, including: When the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring the image of the discharge area at the intermediate gain voltage is less than the minimum value within the preset standard grayscale value range, the intermediate gain voltage is sequentially increased by one step gain until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring the image of the discharge area at the intermediate gain voltage after the step gain is increased is within the preset standard grayscale value range, thereby obtaining a second gain voltage; Alternatively, when the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring image of the discharge area at the intermediate gain voltage is greater than the maximum value within the preset standard grayscale value range, the intermediate gain voltage is reduced by one step gain in sequence until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when acquiring image of the discharge area at the intermediate gain voltage after the step gain is reduced is within the preset standard grayscale value range, thereby obtaining a second gain voltage.

6. The method according to claim 1, characterized in that The step of superimposing the second ultraviolet image and the visible light image to obtain a fused image according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image includes: A conversion relationship constant between a first viewing angle and a second viewing angle is obtained according to a second ratio of the first radius to a third radius of the maximum connected domain in the second ultraviolet image; the first viewing angle is the viewing angle when the visible light camera collects the visible light image; the second viewing angle is the viewing angle when the ultraviolet camera collects the second ultraviolet image; scaling the second ultraviolet light image according to the conversion relationship constant to obtain a third ultraviolet light image; When the distance between the first center point coordinates of the discharge area in the visible light image and the third center point coordinates of the maximum connected domain in the third ultraviolet light image is less than a preset distance threshold, the maximum connected domain of the third ultraviolet light image is superimposed on the visible light image to obtain a fused image.

7. A device for fusing ultraviolet light images and visible light images, characterized in that: The device comprises: An image acquisition module, used to acquire a visible light image and a first ultraviolet light image obtained by acquiring images of a discharge area of ​​an electrical device on the same power transmission line; the first ultraviolet light image is an image acquired by an ultraviolet camera at a first gain voltage; a radius determination module, used to determine a first radius of the discharge area in the visible light image and a second radius of the largest connected domain in the first ultraviolet light image; a gain voltage adjustment module, configured to adjust the first gain voltage according to a relative magnitude relationship between a first ratio of the first radius to the second radius and a preset ratio range, so as to obtain a second gain voltage; The image acquisition module is further used to acquire a second ultraviolet image obtained by the ultraviolet camera collecting an image of the discharge area at a second gain voltage; The image fusion module is used to superimpose the second ultraviolet image and the visible light image according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image to obtain a fused image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images