Method and device for fusing ultraviolet light image and visible light image, computer equipment, readable storage medium and program product

Abstract

The application relates to the technical field of image fusion, and provides a fusion method and device for ultraviolet light images and visible light images, computer equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring a visible light image and a first ultraviolet light image under a first gain voltage, which are obtained by image collection on a discharge region of the same power transmission line electrical equipment; adjusting the first gain voltage according to the relative size relationship between a first ratio of a first radius of the discharge region in the visible light image and a second radius of a maximum connected domain in the first ultraviolet light image and a preset ratio range, to obtain a second gain voltage; acquiring a second ultraviolet light image obtained by image collection on the discharge region by an ultraviolet camera under the second gain voltage; and superimposing the second ultraviolet light image and the visible light image according to a second ratio of the first radius and a third radius of a maximum connected domain in the second ultraviolet light image, to obtain a fusion image. The method can improve the fusion precision of the ultraviolet light image and the visible light image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image fusion, and in particular to a fusion method and device for ultraviolet light images and visible light images, a computer device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] In power inspection, an ultraviolet imaging corona monitoring method usually adopts dual-band imaging, that is, an ultraviolet light image of a discharge region is obtained by a solar-blind ultraviolet imaging system, a visible light image of the discharge region is obtained by a visible light imaging system, and the ultraviolet light image and the visible light image are superimposed and fused, so that a clear ultraviolet light image can be obtained under sunlight.

[0003] Since the optical paths of the ultraviolet camera and the visible light camera do not coincide, when the same discharge region is observed at different distances, a parallax is generated. The existing fusion method for ultraviolet light images and visible light images is to manually control the gain voltage of the ultraviolet camera or to distinguish the gain voltage of the ultraviolet camera in fixed gears, adjust the gain voltage of the ultraviolet camera to a suitable gear, calculate the offset of the discharge region in the ultraviolet light image and the visible light image, and correct the offset when the ultraviolet light image and the visible light image are superimposed and fused.

[0004] However, when the gain voltage value of the ultraviolet camera is manually controlled, when the gain voltage value is set too large, the calculation of the offset is affected, and then the fusion accuracy of the ultraviolet light image and the visible light image is affected. When the gain voltage value of the ultraviolet camera is distinguished in fixed gears, the best gain voltage may not be in the set gear options, which affects the fusion accuracy of the ultraviolet light image and the visible light image. SUMMARY

[0005] Therefore, it is necessary to provide a fusion method, device, computer device, computer readable storage medium, and computer program product for ultraviolet light images and visible light images in view of the above technical problems.

[0006] In a first aspect, the present application provides a fusion method for ultraviolet light images and visible light images, comprising:

[0007] obtaining a visible light image and a first ultraviolet light image obtained by image acquisition of a discharge region of the same power transmission line electrical equipment; the first ultraviolet light image is an image acquired by an ultraviolet camera under a first gain voltage;

[0008] determining a first radius of the discharge region in the visible light image and a second radius of a maximum connected domain in the first ultraviolet light image;

[0009] adjust the first gain voltage according to a relative size relationship between a first ratio of the first radius and the second radius and a preset ratio range, to obtain a second gain voltage;

[0010] obtain a second ultraviolet light image by collecting an image of the discharge region under the second gain voltage by the ultraviolet camera;

[0011] superimpose the second ultraviolet light image and the visible light image according to a second ratio of the first radius and a third radius of a largest connected domain in the second ultraviolet light image, to obtain a fusion image.

[0012] In one of the embodiments, the visible light image and the first ultraviolet light image collected by collecting images of the discharge region of the same power transmission line electrical equipment include:

[0013] obtain a second ultraviolet light image by collecting an image of the discharge region under the second gain voltage by the ultraviolet camera;

[0014] perform a denoising operation on the original visible light image and the first original ultraviolet light image, to obtain the visible light image and the first ultraviolet light image.

[0015] In one of the embodiments, the discharge region in the visible light image is determined by:

[0016] obtain a pixel threshold value;

[0017] when the pixel value of the visible light image is greater than or equal to the pixel threshold value, the pixel value is taken as a discharge pixel;

[0018] take a region formed by the discharge pixels in the visible light image as the discharge region in the visible light image.

[0019] In one of the embodiments, the first gain voltage is adjusted according to a relative size relationship between a first ratio of the first radius and the second radius and a preset ratio range, to obtain a second gain voltage, including:

[0020] when the first ratio is not in the preset ratio range, the first gain voltage is adjusted until a ratio of a radius of an ultraviolet light image collected by the ultraviolet camera under the adjusted first gain voltage and the first radius is equal to an end value of the preset ratio range, to obtain an intermediate gain voltage;

[0021] the intermediate gain voltage is adjusted until an average gray value of an ultraviolet image collected by the ultraviolet camera under the adjusted intermediate gain voltage is in a preset standard gray value range, to obtain the second gain voltage;

[0022] adjusting the first gain voltage until the average gray value of the ultraviolet image captured by the ultraviolet camera under the adjusted first gain voltage is within the preset standard gray value range, to obtain the second gain voltage.

[0023] In one of the embodiments, the adjusting the intermediate gain voltage until the average gray value of the ultraviolet image captured by the ultraviolet camera under the adjusted intermediate gain voltage is within the preset standard gray value range, to obtain the second gain voltage, comprises:

[0024] when the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is less than the minimum value within the preset standard gray value range, sequentially increasing the intermediate gain voltage by a step gain until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the increase of the step gain is within the preset standard gray value range, to obtain the second gain voltage;

[0025] or when the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is greater than the maximum value within the preset standard gray value range, sequentially decreasing the intermediate gain voltage by a step gain until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the decrease of the step gain is within the preset standard gray value range, to obtain the second gain voltage.

[0026] In one of the embodiments, the superimposing the second ultraviolet image and the visible light image according to the second ratio of the first radius and the third radius of the largest connected domain in the second ultraviolet image, to obtain a fusion image, comprises:

[0027] obtaining a conversion relationship constant between the first view angle and the second view angle according to the second ratio of the first radius and the third radius of the largest connected domain in the second ultraviolet image; the first view angle is the view angle of the visible light camera when capturing the visible light image; and the second view angle is the view angle of the ultraviolet camera when capturing the second ultraviolet image;

[0028] scaling the second ultraviolet image according to the conversion relationship constant, to obtain a third ultraviolet image;

[0029] When a distance between the first center point coordinate of the discharge region in the visible light image and the third center point coordinate of the maximum connected domain in the third ultraviolet light image is less than a preset distance threshold, the maximum connected domain of the third ultraviolet light image is superimposed on the visible light image to obtain a fusion image.

[0030] In a second aspect, the present application further provides a fusion device of ultraviolet light images and visible light images, comprising:

[0031] An image acquisition module is configured to acquire a visible light image and a first ultraviolet light image obtained by image collection on a discharge region of the same power transmission line electrical equipment; the first ultraviolet light image is an image collected by an ultraviolet camera under a first gain voltage;

[0032] A radius determination module is configured to determine a first radius of the discharge region in the visible light image and a second radius of the maximum connected domain in the first ultraviolet light image;

[0033] A gain voltage adjustment module is configured to adjust the first gain voltage according to a relative size relationship between a first ratio of the first radius and the second radius and a preset ratio range to obtain a second gain voltage;

[0034] The image acquisition module is further configured to acquire a second ultraviolet light image obtained by image collection on the discharge region by the ultraviolet camera under the second gain voltage;

[0035] An image fusion module is configured to superimpose the second ultraviolet light image on the visible light image according to a second ratio of the first radius and a third radius of the maximum connected domain in the second ultraviolet light image to obtain a fusion image.

[0036] In a third aspect, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor executes the above method.

[0037] In a fourth aspect, the present application further provides a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to execute the above method.

[0038] In a fifth aspect, the present application further provides a computer program product. The computer program product comprises a computer program, and the computer program is executed by a processor to execute the above method.

[0039] The fusion method, device, computer device, computer readable storage medium and computer program product of the ultraviolet light image and the visible light image obtain a visible light image and a first ultraviolet light image collected by image collection on a discharge region of the same power transmission line electrical equipment; the first ultraviolet light image is an image collected by an ultraviolet camera under a first gain voltage; a first radius of the discharge region in the visible light image and a second radius of a maximum connected domain in the first ultraviolet light image are determined; the first gain voltage is adjusted according to a first ratio of the first radius and the second radius and a relative size relationship of a preset ratio range, to obtain a second gain voltage; a second ultraviolet light image collected by image collection of the discharge region by the ultraviolet camera under the second gain voltage is obtained; and the second ultraviolet light image is superimposed with the visible light image according to a second ratio of the first radius and a third radius of a maximum connected domain in the second ultraviolet light image, to obtain a fusion image. According to the relative size relationship of the first ratio of the first radius of the discharge region in the visible light image and the second radius of the maximum connected domain in the first ultraviolet light image and the preset ratio range, the first gain voltage is adjusted to obtain the second gain voltage, that is, the optimal gain voltage, so that the fusion error caused by the influence of the calculation of the offset amount when the gain voltage value is set too large and the fusion error caused by the fact that the optimal gain voltage is not in the set gear option when the gain voltage of the ultraviolet camera is divided into fixed gears are avoided, and the fusion precision of the ultraviolet light image and the visible light image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other related drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 An application environment diagram of the fusion method of the ultraviolet light image and the visible light image in an embodiment;

[0042] Figure 2 A flowchart of the fusion method of the ultraviolet light image and the visible light image in an embodiment;

[0043] Figure 3 A first ultraviolet light image in an embodiment;

[0044] Figure 4 A fusion image in an embodiment;

[0045] Figure 5 A structural block diagram of the fusion device of the ultraviolet light image and the visible light image in an embodiment;

[0046] Figure 6 Figure 1 is a schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0048] An embodiment of the present application provides a fusion method of ultraviolet light images and visible light images. The embodiment can be executed by a computer device, as shown in Figure 1 The computer device can obtain a visible light image and a first ultraviolet light image obtained by image collection on a discharge region of the same power line electrical equipment, and then obtain a fusion image. It can be understood that the computer device can be implemented by a server, or by a terminal, or by an interactive system of a terminal and a server. In the embodiment, the method comprises the steps shown in Figure 2

[0049] In step S201, a visible light image obtained by image collection on a discharge region of the same power line electrical equipment and a first ultraviolet light image are obtained. The first ultraviolet light image is an image collected by an ultraviolet camera under a first gain voltage.

[0050] A visible light image obtained by image collection on a discharge region of a power line electrical equipment by a visible light camera can be obtained, and a first ultraviolet light image obtained by image collection on the discharge region of the same power line electrical equipment by an ultraviolet camera under a first gain voltage can also be obtained, as shown in Figure 3

[0051] In step S202, a first radius of the discharge region in the visible light image and a second radius of the largest connected domain in the first ultraviolet light image are determined.

[0052] The discharge region in the visible light image can be determined according to a machine vision algorithm, such as a YOLOv8 (You Only Look Once Version 8) algorithm. Among all the coordinates of the discharge region, the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate of the discharge region are determined. According to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate, the two coordinates farthest from each other in the discharge region are determined. Half of the distance between the two coordinates farthest from each other in the discharge region is taken as the first radius, which can be denoted as R1.

[0053] ​​The maximum connected domain in the first ultraviolet light image can be determined according to a two-pass scanning method or a seed filling method. Among all the coordinates of the maximum connected domain in the first ultraviolet light image, the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate of the maximum connected domain in the first ultraviolet light image are determined; the two coordinates farthest apart in the maximum connected domain in the first ultraviolet light image are determined according to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate; and one half of the distance between the two coordinates farthest apart in the maximum connected domain in the first ultraviolet light image is taken as the second radius, which can be denoted as R2.

[0054] In step S203, the first gain voltage is adjusted according to the relative size relationship between the first ratio of the first radius to the second radius and the preset ratio range, to obtain a second gain voltage.

[0055] The preset ratio range can be determined through multiple experiments according to the angle conversion relationship between the visible light camera and the ultraviolet camera actually used.

[0056] The ratio of the first radius to the second radius can be taken as the first ratio. When the first ratio is within the preset ratio range, it indicates that the gain voltage is suitable at this time, and the fusion accuracy of the ultraviolet light image and the visible light image is high; when the first ratio is not within the preset ratio range, it indicates that the gain voltage is not suitable at this time, and the fusion accuracy of the ultraviolet light image and the visible light image is low, and the first gain voltage needs to be adjusted so that the first ratio is within the preset ratio range, to obtain the second gain voltage. In actual application, the first gain voltage can be continuously fine-tuned by using automatic gain voltage control.

[0057] In step S204, a second ultraviolet light image obtained by image acquisition of the discharge region by the ultraviolet camera under the second gain voltage is acquired.

[0058] After the second gain voltage is obtained, the second ultraviolet light image obtained by image acquisition of the discharge region by the ultraviolet camera under the second gain voltage can be acquired.

[0059] In step S205, the second ultraviolet light image is superimposed with the visible light image according to the second ratio of the first radius to the third radius of the maximum connected domain in the second ultraviolet light image, to obtain a fusion image.

[0060] The maximum connected domain in the second ultraviolet light image can be determined according to a two-pass scanning method or a seed filling method. In all coordinates of the maximum connected domain in the second ultraviolet light image, the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate of the maximum connected domain in the second ultraviolet light image are determined. According to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate, the two coordinates farthest apart in the maximum connected domain in the second ultraviolet light image are determined. Half of the distance between the two coordinates farthest apart in the maximum connected domain in the second ultraviolet light image is taken as the third radius. The ratio of the first radius and the third radius can be taken as the second ratio. According to the second ratio, the second ultraviolet light image is superimposed with the visible light image to obtain a fusion image, wherein the fusion image is as shown in Figure 4

[0061] In the fusion method of the ultraviolet light image and the visible light image, the first gain voltage is adjusted according to the relative size relationship between the first ratio of the first radius of the discharge region in the visible light image and the second radius of the maximum connected domain in the first ultraviolet light image and the preset ratio range, to obtain the second gain voltage, i.e., the optimal gain voltage, thereby avoiding the fusion error caused by the influence of the size of the spot in the ultraviolet light image exceeding the size of the spot in the visible light image too much when the gain voltage value is set too large, and avoiding the fusion error caused by the optimal gain voltage not being in the set gear option when the gain voltage of the ultraviolet camera is divided into fixed gears, thereby improving the fusion accuracy of the ultraviolet light image and the visible light image.

[0062] In one of the embodiments, the visible light image and the first ultraviolet light image obtained by image acquisition on the discharge region of the same power line electrical equipment are acquired, and the specific steps are as follows: an original visible light image and a first original ultraviolet light image obtained by image acquisition on the discharge region of the same power line electrical equipment are acquired; the original visible light image and the first original ultraviolet light image are subjected to a denoising operation to obtain the visible light image and the first ultraviolet light image.

[0063] The original visible light image and the first original ultraviolet light image obtained by image acquisition on the discharge region of the same power line electrical equipment are acquired. The first original ultraviolet light image is processed by a binarization method to obtain a binarized first original ultraviolet light image, and the original visible light image and the binarized first original ultraviolet light image are subjected to median filtering and secondary Wiener filtering to remove multiplicative noise, salt and pepper noise and Gaussian noise, to obtain the visible light image and the first ultraviolet light image.

[0064] In the embodiment, the original visible light image and the first original ultraviolet light image are subjected to a denoising operation, so that more real visible light image and first ultraviolet light image can be obtained.

[0065] ​In one of the embodiments, the discharge region in the visible light image is determined, and the specific steps are as follows: obtaining a pixel threshold value; when the pixel value of the visible light image is greater than or equal to the pixel threshold value, the pixel value is taken as a discharge pixel; the region formed by the discharge pixels in the visible light image is taken as the discharge region in the visible light image.

[0066] The pixel threshold value for distinguishing the discharge region and the background region can be obtained according to the maximum inter-class variance method. When the pixel value of the visible light image is less than the pixel threshold value, the pixel value is taken as a discharge pixel; when the pixel value of the visible light image is greater than or equal to the pixel threshold value, the pixel value is taken as a discharge pixel; the region formed by the discharge pixels in the visible light image is taken as the discharge region in the visible light image.

[0067] In the embodiment, the discharge pixels are determined according to the relative size relationship between the pixel value of the visible light image and the pixel threshold value; the region formed by the discharge pixels in the visible light image is taken as the discharge region in the visible light image, so that a more accurate discharge region is obtained.

[0068] In one of the embodiments, the first gain voltage is adjusted according to the relative size relationship between the first ratio of the first radius and the second radius and the preset ratio range, to obtain the second gain voltage, and the specific steps are as follows: when the first ratio is not in the preset ratio range, the first gain voltage is adjusted until the ratio of the radius of the ultraviolet light image obtained by the ultraviolet camera under the adjusted first gain voltage to the first radius is equal to the end value of the preset ratio range, to obtain an intermediate gain voltage; the intermediate gain voltage is adjusted until the average gray value of the ultraviolet image obtained by the ultraviolet camera under the adjusted intermediate gain voltage is in the preset standard gray value range, to obtain the second gain voltage; when the first ratio is in the preset ratio range, the first gain voltage is adjusted until the average gray value of the ultraviolet image obtained by the ultraviolet camera under the adjusted first gain voltage is in the preset standard gray value range, to obtain the second gain voltage.

[0069] When the first ratio is not in the preset ratio range, it is determined whether the first ratio is less than the minimum value of the preset ratio range or greater than the maximum value of the preset ratio range.

[0070] When the first ratio is less than the minimum value of the preset ratio range, the first gain voltage is increased until the ratio of the radius of the ultraviolet light image obtained by the ultraviolet camera under the increased first gain voltage to the first radius is equal to the minimum value of the preset ratio range, to obtain an intermediate gain voltage.

[0071] Or when the first ratio is greater than the maximum value of the preset ratio range, the first gain voltage is reduced until the ratio of the radius of the ultraviolet light image collected by the ultraviolet camera under the first gain voltage after the reduction to the first radius is equal to the maximum value of the preset ratio range, and an intermediate gain voltage is obtained.

[0072] The gain voltage of the ultraviolet camera can be divided into ranges according to a preset rule, so that each gain voltage has a corresponding gain voltage interval. In actual application, the gain voltage interval can be divided according to actual experience, or an average gain voltage interval can be adopted, for example, each gain voltage interval includes the same number of step gains, and each step gain can be the minimum value of the gain voltage adjustment each time, which can be determined according to the parameters of the actually used ultraviolet camera. The preset standard gray value range can be determined according to the maximum average gray value and the minimum average gray value of the gain voltage interval corresponding to the intermediate gain voltage.

[0073] The intermediate gain voltage can be adjusted until the average gray value of the ultraviolet image collected by the ultraviolet camera under the adjusted intermediate gain voltage is within the preset standard gray value range, and a second gain voltage is obtained.

[0074] When the first ratio is within the preset ratio range, the preset standard gray value range can be determined according to the maximum average gray value and the minimum average gray value of the gain voltage interval corresponding to the first gain voltage, and the first gain voltage can be adjusted until the average gray value of the ultraviolet image collected by the ultraviolet camera under the adjusted first gain voltage is within the preset standard gray value range, and a second gain voltage is obtained.

[0075] In this embodiment, the first gain voltage is adjusted according to the relative size relationship between the first ratio and the preset ratio range to obtain the second gain voltage, so that the ratio of the radius of the ultraviolet light image collected by the ultraviolet camera under the second gain voltage to the first radius is within the preset ratio range, and the average gray value of the ultraviolet image collected by the ultraviolet camera under the second gain voltage is within the preset standard gray value range, so that the second gain voltage has a high degree of appropriateness, and the fusion accuracy of the ultraviolet light image and the visible light image is improved.

[0076] In one of the embodiments, the intermediate gain voltage is adjusted until the average gray value of the ultraviolet image captured by the ultraviolet camera under the adjusted intermediate gain voltage is within the preset standard gray value range, and the second gain voltage is obtained. The specific steps are as follows: when the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is less than the minimum value in the preset standard gray value range, the intermediate gain voltage is increased by a step gain in turn until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the step gain is increased is within the preset standard gray value range, and the second gain voltage is obtained; or when the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is greater than the maximum value in the preset standard gray value range, the intermediate gain voltage is reduced by a step gain in turn until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the step gain is reduced is within the preset standard gray value range, and the second gain voltage is obtained.

[0077] The step gain can be determined according to the parameters of the actually used ultraviolet camera.

[0078] When the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is less than the minimum value in the preset standard gray value range, the intermediate gain voltage is increased by a step gain in turn until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the step gain is increased is within the preset standard gray value range, and the second gain voltage is obtained.

[0079] Or when the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is greater than the maximum value in the preset standard gray value range, the intermediate gain voltage is reduced by a step gain in turn until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the step gain is reduced is within the preset standard gray value range, and the second gain voltage is obtained.

[0080] In this embodiment, the intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image captured by the ultraviolet camera when imaging the discharge area at the adjusted intermediate gain voltage is within a preset standard grayscale value range, and a second gain voltage is obtained, so that when the ratio of the radius of the ultraviolet image captured by the ultraviolet camera when imaging the discharge area at the second gain voltage to the first radius is within the preset ratio range, the average grayscale value of the ultraviolet image captured by the ultraviolet camera when imaging the discharge area at the second gain voltage is also within the preset standard grayscale value range, thereby making the suitability of the second gain voltage higher and improving the fusion accuracy of the ultraviolet image and the visible light image.

[0081] In one embodiment, the second ultraviolet image and the visible light image are superimposed on each other to obtain a fused image according to a second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet image. The specific steps are as follows: a conversion relationship constant between the first perspective and the second perspective is obtained according to the second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet image; the first perspective is the perspective when the visible light camera captures the visible light image; the second perspective is the perspective when the ultraviolet camera captures the second ultraviolet image; the second ultraviolet image is scaled according to the conversion relationship constant to obtain a third ultraviolet image; when the distance between the first center point coordinates of the discharge area in the visible light image and the third center point coordinates of the largest connected domain in the third ultraviolet image is less than a preset distance threshold, the largest connected domain of the third ultraviolet image is superimposed on the visible light image to obtain a fused image.

[0082] In practical applications, before the dual fields of view (ultraviolet light field of view and visible light field of view) are superimposed, the field of view angles need to be converted and matched to reduce the deformation and distortion caused by image superposition.

[0083] Because the field of view of a UV camera is smaller than that of visible light, the UV image must be scaled proportionally before being fused into the visible light image. Image scaling means scaling the image while maintaining its aspect ratio, thus preserving the image's aspect ratio and preventing distortion.

[0084] According to the second ratio of the first radius to the third radius of the largest connected domain in the second ultraviolet image, a conversion relationship constant between the first perspective and the second perspective is obtained, and the conversion relationship constant can be recorded as C; the first perspective is the perspective when the visible light camera captures the visible light image; the second perspective is the perspective when the ultraviolet camera captures the second ultraviolet image.

[0085] The second ultraviolet light image is scaled according to the conversion relationship constant to obtain a third ultraviolet light image.

[0086] Specifically, the second ultraviolet light image can be scaled by using an image processing library. For example, the second ultraviolet light image can be scaled by using a cv.resize function in an Open Computer Vision Library (OpenCV). Alternatively, a scaling ratio can be calculated according to a conversion relationship constant, and then each pixel point can be scaled by using a for loop or vector operation. A calculation formula is: new image size = old image size x scaling ratio. For example, a 500x300 picture is scaled by 1 / 2, and a new image size is 250x150. Alternatively, the scaling can be performed in a reverse process. The scaling algorithm can include an interpolation algorithm, and a bilinear interpolation algorithm or a cubic interpolation algorithm can be selected according to actual requirements.

[0087] Different from directly scaling a length or a width, the scaling can well preserve structural information of the original image. For example, a resolution of the second ultraviolet image is 720x576, and a horizontal second field of view angle is 19.2 degrees and a vertical second field of view angle is 14.4 degrees. A resolution of the visible light image is 1920x1080, and a horizontal first field of view angle is 58.07 degrees and a vertical first field of view angle is 34.8 degrees. Therefore, before calculation, a second field of view angle of a 16:9 region of the second ultraviolet image is calculated, that is, a horizontal second field of view angle and a vertical second field of view angle of the second ultraviolet image with a resolution of 720x405. The horizontal second field of view angle does not need to be converted, and the vertical second field of view angle is converted as follows: (405 / 576) x 14.4 = 10.125; therefore, the second ultraviolet image with the resolution of 720x405 has a field of view angle of 10.125 degrees. When superimposition is performed, a size of the second ultraviolet image with the resolution of 720x405 that needs to be scaled is calculated, and then the size is restored to a size of the second ultraviolet image with the resolution of 720x576 according to the scaling ratio.

[0088] A discharge region in the visible light image can be determined according to a machine vision algorithm. In all coordinates of the discharge region, a minimum horizontal coordinate, a maximum horizontal coordinate, a minimum vertical coordinate and a maximum vertical coordinate of the discharge region are determined; two coordinates farthest from each other in the discharge region are determined according to the minimum horizontal coordinate, the maximum horizontal coordinate, the minimum vertical coordinate and the maximum vertical coordinate; and a center point coordinate between the two coordinates farthest from each other in the discharge region is taken as a first center point coordinate.

[0089] The maximum connected domain in the third ultraviolet light image can be determined according to a two-pass scanning method or a seed filling method. In all coordinates of the maximum connected domain in the third ultraviolet light image, a minimum abscissa coordinate, a maximum abscissa coordinate, a minimum ordinate coordinate and a maximum ordinate coordinate of the maximum connected domain in the third ultraviolet light image are determined; according to the minimum abscissa coordinate, the maximum abscissa coordinate, the minimum ordinate coordinate and the maximum ordinate coordinate, two coordinates farthest apart in the maximum connected domain in the third ultraviolet light image are determined; and a center point coordinate between the two coordinates farthest apart in the maximum connected domain in the third ultraviolet light image is taken as a third center point coordinate.

[0090] The preset distance threshold can be determined according to actual conditions. When the distance between the first center point coordinate and the third center point coordinate is less than the preset distance threshold, it indicates that the coincidence degree between the first center point and the third center point is high, and at this time, the accuracy of fusing the third ultraviolet light image and the visible light image is high, so at this time, the maximum connected domain of the third ultraviolet light image can be superimposed on the visible light image to obtain a fused image. The distance is the Euclidean distance.

[0091] Specifically, when the visible light image is RGB24 type data and the third ultraviolet image is a single-channel image of U8 type, after the third ultraviolet image is binarized, the pixel value of the third ultraviolet image is 0 or 255, and there is no other third value. In order to speed up the superimposition speed, a Compute Unified Device Architecture (CUDA) parallel acceleration can be used to superimpose the maximum connected domain of the third ultraviolet light image and the visible light image to obtain a fused image.

[0092] In this embodiment, according to a second ratio of the first radius to a third radius of the maximum connected domain in the second ultraviolet light image, a conversion relationship constant between the first view angle and the second view angle is obtained to scale the second ultraviolet light image to obtain a third ultraviolet light image matching the first field of view. When the distance between the first center point coordinate of the discharge region in the visible light image and the third center point coordinate of the maximum connected domain in the third ultraviolet light image is less than a preset distance threshold, the maximum connected domain of the third ultraviolet light image is superimposed on the visible light image to obtain a fused image with high accuracy.

[0093] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0094] Based on the same inventive concept, the embodiments of the present application also provide an ultraviolet image and visible image fusion device for implementing the above-mentioned ultraviolet image and visible image fusion method. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more ultraviolet image and visible image fusion device embodiments provided below can refer to the limitations of the ultraviolet image and visible image fusion method described above, and will not be repeated here.

[0095] In an exemplary embodiment, as shown in Figure 5 An ultraviolet image and visible image fusion device is provided, wherein:

[0096] The image acquisition module 501 is configured to acquire a visible light image and a first ultraviolet image obtained by image acquisition of a discharge region of the same power line electrical equipment; the first ultraviolet image is an image acquired by an ultraviolet camera under a first gain voltage;

[0097] The radius determination module 502 is configured to determine a first radius of the discharge region in the visible light image and a second radius of the largest connected domain in the first ultraviolet image;

[0098] The gain voltage adjustment module 503 is configured to adjust the first gain voltage according to the relative size relationship between the first ratio of the first radius and the second radius and the preset ratio range, to obtain a second gain voltage;

[0099] The image acquisition module 501 is further configured to acquire a second ultraviolet image obtained by image acquisition of the discharge region by the ultraviolet camera under the second gain voltage;

[0100] The image fusion module 504 is configured to superimpose the second ultraviolet light image and the visible light image according to a second ratio of the first radius to a third radius of a largest connected domain in the second ultraviolet light image, to obtain a fused image.

[0101] In one of the embodiments, the image acquisition module 501 is further configured to acquire a raw visible light image and a first raw ultraviolet light image obtained by image acquisition of the discharge region of the same power transmission line electrical equipment, and to perform a denoising operation on the raw visible light image and the first raw ultraviolet light image to obtain the visible light image and the first ultraviolet light image.

[0102] In one of the embodiments, the device further comprises a discharge region acquisition module configured to acquire a pixel threshold value, to regard a pixel value of the visible light image as a discharge pixel when the pixel value is greater than or equal to the pixel threshold value, and to regard a region formed by the discharge pixels in the visible light image as a discharge region in the visible light image.

[0103] In one of the embodiments, the gain voltage adjustment module 503 is further configured to adjust the first gain voltage until a ratio of a radius of an ultraviolet light image obtained by image acquisition of the discharge region by the ultraviolet camera under the adjusted first gain voltage to the first radius is equal to an end value of the preset ratio range, to obtain an intermediate gain voltage, when the first ratio is not within the preset ratio range, to adjust the intermediate gain voltage until an average gray value of an ultraviolet image obtained by image acquisition of the discharge region by the ultraviolet camera under the adjusted intermediate gain voltage is within a preset standard gray value range, to obtain a second gain voltage, and to adjust the first gain voltage until an average gray value of an ultraviolet image obtained by image acquisition of the discharge region by the ultraviolet camera under the adjusted first gain voltage is within the preset standard gray value range, to obtain the second gain voltage, when the first ratio is within the preset ratio range.

[0104] In one of the embodiments, the gain voltage adjustment module 503 is further configured to: when the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is less than the minimum value in the preset standard gray value range, sequentially increase the intermediate gain voltage by a step gain until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the increase of the step gain is within the preset standard gray value range, to obtain a second gain voltage; or when the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage is greater than the maximum value in the preset standard gray value range, sequentially decrease the intermediate gain voltage by a step gain until the average gray value of the ultraviolet image captured by the ultraviolet camera under the intermediate gain voltage after the decrease of the step gain is within the preset standard gray value range, to obtain a second gain voltage.

[0105] In one of the embodiments, the image fusion module 504 is further configured to: obtain a conversion relationship constant between the first view angle and the second view angle according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image; the first view angle is the view angle of the visible light camera when capturing the visible light image; the second view angle is the view angle of the ultraviolet camera when capturing the second ultraviolet image; scale the second ultraviolet image according to the conversion relationship constant to obtain a third ultraviolet image; when the distance between the first center point coordinate of the discharge region in the visible light image and the third center point coordinate of the largest connected domain in the third ultraviolet image is less than a preset distance threshold, superimpose the largest connected domain of the third ultraviolet image and the visible light image to obtain a fusion image.

[0106] The above-mentioned modules in the fusion device of the ultraviolet image and the visible light image can be realized by software, hardware and combinations thereof in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above-mentioned modules.

[0107] In one of the embodiments, a computer device is provided, which can be a server, and the internal structure diagram of the computer device can be as shown in FIG. 1. Figure 6As shown in the figure. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data of the embodiment of the fusion method of ultraviolet light image and visible light image. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to realize a fusion method of ultraviolet light image and visible light image.

[0108] Those skilled in the art can understand that, Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0109] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to realize the steps in the above method embodiments.

[0110] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by the processor to realize the steps in the above method embodiments.

[0111] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by the processor to realize the steps in the above method embodiments.

[0112] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0114] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0115] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent of the present application. It should be noted that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for fusing ultraviolet light images and visible light images, characterized in that: The method comprises: Acquire a visible light image and a first ultraviolet light image obtained by capturing an image of a discharge area of ​​electrical equipment on the same transmission line; the first ultraviolet light image is an image captured by an ultraviolet camera at a first gain voltage; determining a first radius of a discharge region in the visible light image and a second radius of a maximum connected domain in the first ultraviolet light image; Adjusting the first gain voltage according to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range to obtain a second gain voltage; Acquiring a second ultraviolet image obtained by capturing an image of the discharge area by the ultraviolet camera at a second gain voltage; The second ultraviolet image and the visible light image are superimposed according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image to obtain a fused image.

2. The method according to claim 1, characterized in that The obtaining of the visible light image and the first ultraviolet light image obtained by collecting images of the discharge area of ​​the electrical equipment on the same transmission line includes: Acquire an original visible light image and a first original ultraviolet light image obtained by collecting images of a discharge area of ​​electrical equipment on the same transmission line; A denoising operation is performed on the original visible light image and the first original ultraviolet light image to obtain a visible light image and a first ultraviolet light image.

3. The method according to claim 1, characterized in that Determining a discharge area in the visible light image includes: Get pixel threshold; When a pixel value of the visible light image is greater than or equal to a pixel threshold, the pixel value is used as a discharge pixel; The area formed by the discharge pixels in the visible light image is used as the discharge area in the visible light image.

4. The method according to claim 1, wherein The adjusting the first gain voltage according to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range to obtain a second gain voltage includes: When the first ratio is not within a preset ratio range, adjusting the first gain voltage until a ratio of a radius of an ultraviolet image captured by the ultraviolet camera at the adjusted first gain voltage and the first radius of the ultraviolet image of the discharge area is equal to an end value of the preset ratio range, thereby obtaining an intermediate gain voltage; The intermediate gain voltage is adjusted until the average grayscale value of the ultraviolet image captured by the ultraviolet camera at the adjusted intermediate gain voltage for the discharge area is within a preset standard grayscale value range, thereby obtaining a second gain voltage; When the first ratio is within a preset ratio range, the first gain voltage is adjusted until the average grayscale value of the ultraviolet image captured by the ultraviolet camera of the discharge area under the adjusted first gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage.

5. The method according to claim 4, characterized in that The intermediate gain voltage is adjusted until an average grayscale value of an ultraviolet image acquired by the ultraviolet camera when capturing an image of the discharge area at the adjusted intermediate gain voltage is within a preset standard grayscale value range, thereby obtaining a second gain voltage, comprising: When the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when capturing the image of the discharge area at the intermediate gain voltage is less than the minimum value within the preset standard grayscale value range, the intermediate gain voltage is sequentially increased by one step gain until the average grayscale value of the ultraviolet image acquired by the ultraviolet camera when capturing the image of the discharge area at the intermediate gain voltage after the step gain is increased is within the preset standard grayscale value range, thereby obtaining a second gain voltage; Alternatively, when the average grayscale value of the ultraviolet image captured by the ultraviolet camera when capturing the image of the discharge area at the intermediate gain voltage is greater than the maximum value within the preset standard grayscale value range, the intermediate gain voltage is reduced by one step gain in sequence until the average grayscale value of the ultraviolet image captured by the ultraviolet camera when capturing the image of the discharge area at the intermediate gain voltage after the step gain is reduced is within the preset standard grayscale value range, thereby obtaining a second gain voltage.

6. The method according to claim 1, characterized in that The step of superimposing the second ultraviolet image and the visible light image to obtain a fused image based on a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image includes: a conversion constant between a first viewing angle and a second viewing angle is obtained based on a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image; the first viewing angle is the viewing angle of the visible light camera when capturing the visible light image; and the second viewing angle is the viewing angle of the ultraviolet camera when capturing the second ultraviolet image; scaling the second ultraviolet image according to the conversion relationship constant to obtain a third ultraviolet image; When the distance between the first center point coordinates of the discharge area in the visible light image and the third center point coordinates of the largest connected domain in the third ultraviolet light image is less than a preset distance threshold, the largest connected domain of the third ultraviolet light image is superimposed on the visible light image to obtain a fused image.

7. A device for fusing ultraviolet light images and visible light images, characterized in that: The device comprises: An image acquisition module, configured to acquire a visible light image and a first ultraviolet light image obtained by capturing an image of a discharge area of ​​electrical equipment on the same transmission line; the first ultraviolet light image is an image captured by an ultraviolet camera at a first gain voltage; a radius determination module, configured to determine a first radius of the discharge area in the visible light image and a second radius of the largest connected domain in the first ultraviolet light image; a gain voltage adjustment module, configured to adjust the first gain voltage according to a relative size relationship between a first ratio of the first radius to the second radius and a preset ratio range to obtain a second gain voltage; The image acquisition module is further configured to acquire a second ultraviolet image obtained by the ultraviolet camera capturing an image of the discharge area at a second gain voltage; An image fusion module is configured to superimpose the second ultraviolet image and the visible light image according to a second ratio of the first radius to a third radius of the largest connected domain in the second ultraviolet image to obtain a fused image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images