Visual fusion method and device of ultraviolet image and visible light image, computer equipment, readable storage medium and program product

Abstract

The invention relates to a visual fusion method and device of an ultraviolet image and a visible light image, equipment, a storage medium and a program product, and relates to the technical field of electric power safety detection. By adopting the method, the accuracy of image registration can be improved. The method comprises the following steps: extracting an ultraviolet discharge corona region from an ultraviolet image, and calculating the pixel area of the ultraviolet discharge corona region; inputting the detector gain, the detection distance and the pixel area into a trained neural network model to obtain the discharge pulse current intensity, and determining the ultraviolet discharge intensity grade; performing registration on the ultraviolet image and the visible light image according to the imaging view field offset, and performing pixel-level image fusion based on local content weighting on the registered ultraviolet image and visible light image to obtain an initial fusion image; and superposing the detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity grade on the initial fusion image to obtain a target fusion image, and visually displaying the target fusion image.

Description

Technical Field

[0001] The present application relates to the technical field of power safety detection, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for visual fusion of ultraviolet images and visible light images. Background Art

[0002] In the insulator discharge detection of power systems, using discharge corona for ultraviolet detection is a common method. Since the arc or corona caused by insulator discharge will generate ultraviolet light during the discharge process, the detection of ultraviolet light can identify whether the insulator has discharged, thereby achieving non-contact safety detection of insulators.

[0003] In traditional technology, the corona discharge phenomenon is mainly detected by a dual-channel ultraviolet / visible light detection system. One channel detects visible light signals for imaging, while the other channel detects ultraviolet signals to image the corona discharge area. The discharge luminous area image is superimposed on the visible light channel image to obtain a fused ultraviolet image, and then the degree of damage caused by corona discharge is evaluated through subsequent image processing. However, in the image registration process of ultraviolet detectors and visible light detectors, since ultraviolet detectors and visible light detectors usually have different spatial resolutions, there will be differences in the detail presentation of the two images, making direct image registration difficult to achieve, and the alignment of image details and contour areas is inaccurate, so there is a problem of low registration accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for visual fusion of ultraviolet images and visible light images in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for visual fusion of ultraviolet images and visible light images, comprising:

[0006] Acquire an ultraviolet image and a visible light image detected by a preset dual-channel detector, extract an ultraviolet discharge corona region from the ultraviolet image, and calculate a pixel area of ​​the ultraviolet discharge corona region;

[0007] Inputting the detector gain, detection distance and pixel area of ​​the dual-channel detector into a trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and determining the ultraviolet discharge intensity level according to the discharge pulse current intensity;

[0008] According to a preset imaging field of view offset, the ultraviolet image and the visible light image are registered, and pixel-level image fusion based on local content weighting is performed on the registered ultraviolet image and visible light image to obtain an initial fused image;

[0009] The detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity level are superimposed on the initial fusion image to obtain a target fusion image, and the target fusion image is visually displayed.

[0010] In one embodiment, extracting the ultraviolet discharge corona region from the ultraviolet image and calculating the pixel area of ​​the ultraviolet discharge corona region includes:

[0011] According to a preset edge detection algorithm, the bright spot edge contour coordinate points in the ultraviolet image are extracted by gradient strength and direction; the bright spot edge contour coordinate points are connected to obtain the bright spot image edge contour; and the number of target pixel points within the range of the bright spot image edge contour is counted to obtain the pixel area.

[0012] In one embodiment, before registering the ultraviolet image and the visible light image according to the preset imaging field of view offset, the method further includes:

[0013] Acquire imaging array parameters of the visible light sensor and the ultraviolet imaging sensor, and acquire target parameters of the observed object, and determine the imaging area of ​​the observed object on the visible light sensor and the ultraviolet imaging sensor according to the imaging array parameters and the target parameters; determine the spacing baseline between the visible light camera and the ultraviolet camera, and calculate the imaging field of view offset of the visible light camera according to the spacing baseline and with the field of view center of the ultraviolet camera as the origin, as the preset imaging field of view offset.

[0014] In one embodiment, the registering the ultraviolet image and the visible light image according to a preset imaging field of view offset includes:

[0015] Based on the preset imaging field of view offset, the ultraviolet image and the visible light image are offset calibrated; according to the imaging area, the ultraviolet image and the visible light image after the offset calibration are registered and aligned to obtain the registered ultraviolet image and visible light image.

[0016] In one embodiment, performing pixel-level image fusion based on local content weighting on the registered ultraviolet image and visible light image to obtain an initial fused image includes:

[0017] Perform edge detection on the ultraviolet image and the visible light image respectively to obtain corresponding edge maps; based on the edge maps, determine the current pixel value of each pixel and the corresponding pixel weighting coefficient, perform weighted calculation according to the pixel weighting coefficient and the current pixel value to obtain a fused target pixel value; determine the initial fused image according to the target pixel value.

[0018] In one embodiment, determining the current pixel value of each pixel and the corresponding pixel weighting coefficient based on the edge map includes:

[0019] Obtain a local mean value of each of the pixel points, determine a local standard deviation based on the local mean value, and calculate a local contrast map for the edge map through the local standard deviation; combine the edge map and the local contrast map to obtain a fusion weight map corresponding to the ultraviolet image and the visible light image; in the fusion weight map, determine the current pixel value of each of the pixel points and the corresponding pixel weighting coefficient.

[0020] In a second aspect, the present application further provides a device for visually fusing ultraviolet images and visible light images, comprising:

[0021] An image acquisition module is used to acquire an ultraviolet image and a visible light image detected by a preset dual-channel detector, extract an ultraviolet discharge corona area from the ultraviolet image, and calculate a pixel area of ​​the ultraviolet discharge corona area;

[0022] A model processing module, used for inputting the detector gain, detection distance and pixel area of ​​the dual-channel detector into a trained neural network model, obtaining the discharge pulse current intensity output by the neural network model, and determining the ultraviolet discharge intensity level according to the discharge pulse current intensity;

[0023] An image registration module, used to register the ultraviolet image and the visible light image according to a preset imaging field of view offset, and perform pixel-level image fusion based on local content weighting on the registered ultraviolet image and visible light image to obtain an initial fused image;

[0024] An image display module is used to superimpose the detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity level on the initial fusion image to obtain a target fusion image, and visually display the target fusion image.

[0025] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0026] The ultraviolet image and the visible light image detected by the preset dual-channel detector are obtained, the ultraviolet discharge corona area is extracted from the ultraviolet image, and the pixel area of ​​the ultraviolet discharge corona area is calculated; the detector gain, the detection distance and the pixel area of ​​the dual-channel detector are input into the trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and the ultraviolet discharge intensity level is determined according to the discharge pulse current intensity; the ultraviolet image and the visible light image are registered according to the preset imaging field of view offset, and the registered ultraviolet image and visible light image are subjected to pixel-level image fusion based on local content weighting to obtain an initial fused image; the detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity level are superimposed on the initial fused image to obtain a target fused image, and the target fused image is visually displayed.

[0027] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0028] The ultraviolet image and the visible light image detected by the preset dual-channel detector are obtained, the ultraviolet discharge corona area is extracted from the ultraviolet image, and the pixel area of ​​the ultraviolet discharge corona area is calculated; the detector gain, the detection distance and the pixel area of ​​the dual-channel detector are input into the trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and the ultraviolet discharge intensity level is determined according to the discharge pulse current intensity; the ultraviolet image and the visible light image are registered according to the preset imaging field of view offset, and the registered ultraviolet image and visible light image are subjected to pixel-level image fusion based on local content weighting to obtain an initial fused image; the detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity level are superimposed on the initial fused image to obtain a target fused image, and the target fused image is visually displayed.

[0029] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0030] The ultraviolet image and the visible light image detected by the preset dual-channel detector are obtained, the ultraviolet discharge corona area is extracted from the ultraviolet image, and the pixel area of ​​the ultraviolet discharge corona area is calculated; the detector gain, the detection distance and the pixel area of ​​the dual-channel detector are input into the trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and the ultraviolet discharge intensity level is determined according to the discharge pulse current intensity; the ultraviolet image and the visible light image are registered according to the preset imaging field of view offset, and the registered ultraviolet image and visible light image are subjected to pixel-level image fusion based on local content weighting to obtain an initial fused image; the detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity level are superimposed on the initial fused image to obtain a target fused image, and the target fused image is visually displayed.

[0031] The above-mentioned visualization fusion method, device, computer equipment, computer-readable storage medium and computer program product of ultraviolet image and visible light image extract the ultraviolet discharge corona area from the detected ultraviolet image, calculate the pixel area of ​​the ultraviolet discharge corona area, and then determine the discharge pulse current intensity and the ultraviolet discharge intensity level in combination with the detector gain, detection distance and pixel area, and then quickly align the visible light image and the ultraviolet image according to the preset imaging field of view offset, and obtain the initial fused image through pixel-level image fusion based on local content weighting, and finally superimpose relevant parameter information on the initial fused image to obtain the target fused image, and at the same time achieve the effect of visually displaying the target fused image, thereby reasonably eliminating the difference between the visible light image and the ultraviolet image, reducing the error generated when the two images are aligned, and thus improving the accuracy of image alignment.

[0032] In combination with the implementation scheme of the present application, compared with the prior art, its significant advantages are:

[0033] (1) In view of the defects of large consumption of iterative computing resources and long time in the traditional registration and alignment of visible light channel and ultraviolet channel imaging data, such as wavelet transform, SIFT feature matching, mutual information matching, etc., this application proposes to quickly align the imaging area based on the imaging field of view offset moment. The alignment operation can be directly applied in the image fusion process without complex iterative calculation or feature extraction and matching process. The amount of calculation is greatly reduced, the efficiency is improved, and the image alignment can be completed quickly to achieve real-time detection and output.

[0034] (2) In order to address the defects of fixed fusion coefficient (such as 0.5) in traditional pixel fusion and the computational complexity and speed problems caused by the fusion rule of high-frequency sub-band, a pixel-level image fusion based on local content weighting is proposed to obtain a fused image. On the one hand, it reduces the computational complexity of traditional transform-domain-based image fusion and the distortion problem caused by fixed threshold spatial domain fusion algorithm (such as pixel arithmetic mean). On the other hand, it retains local details that are highly sensitive to human vision and maintains the key information in the original image, thereby improving the accuracy and quality of image fusion and facilitating more accurate analysis of the target through subsequent image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0036] FIG1 is a diagram showing an application environment of a method for visually fusing ultraviolet images and visible light images in one embodiment;

[0037] Figure 2 1 is a flow chart of a method for visually fusing an ultraviolet image and a visible light image in one embodiment;

[0038] Figure 3 is a schematic flow chart of a pixel area calculation step in one embodiment;

[0039] Figure 4 is a schematic flow chart of an image fusion step in one embodiment;

[0040] Figure 5 It is a schematic diagram of a flow chart of a method for visual fusion of ultraviolet image and visible light image in a specific embodiment;

[0041] Figure 6 is a structural block diagram of a visual fusion device for ultraviolet images and visible light images in one embodiment;

[0042] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] The visual fusion method of ultraviolet image and visible light image provided in the embodiment of the present application can be applied in the application environment as shown in FIG1. ​​The terminal communicates with the server through the network. The data storage system can store the data that the server needs to process. The data storage system can be integrated on the server, or it can be placed on the cloud or other network servers.

[0045] Specifically, the visual fusion method of the ultraviolet image and the visible light image provided in the embodiment of the present application can be executed by a terminal.

[0046] Exemplarily, the terminal obtains an ultraviolet image and a visible light image detected by a preset dual-channel detector, extracts an ultraviolet discharge corona area in the ultraviolet image, and calculates the pixel area of ​​the ultraviolet discharge corona area; the terminal inputs the detector gain, detection distance and pixel area of ​​the dual-channel detector into a trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and determines the ultraviolet discharge intensity level according to the discharge pulse current intensity; the terminal aligns the ultraviolet image and the visible light image according to a preset imaging field of view offset, and performs pixel-level image fusion of the aligned ultraviolet image and visible light image based on local content weighting to obtain an initial fused image; the terminal superimposes the detector gain, detection distance, discharge pulse current intensity and ultraviolet discharge intensity level on the initial fused image to obtain a target fused image, and visually displays the target fused image.

[0047] In the application environment shown in FIG1 , the terminal may be, but is not limited to, various personal computers, notebook computers, smart phones, and tablet computers. The server may be implemented as an independent server or a server cluster consisting of multiple servers.

[0048] In one embodiment, Figure 2 As shown, a method for visual fusion of ultraviolet images and visible light images is provided, and the method is applied to the terminal in FIG1 as an example for explanation, including the following steps:

[0049] Step S201 , obtaining an ultraviolet image and a visible light image detected by a preset dual-channel detector, extracting an ultraviolet discharge corona region from the ultraviolet image, and calculating a pixel area of ​​the ultraviolet discharge corona region.

[0050] The dual-channel detector may be a detector having a visible light imaging channel and an ultraviolet imaging channel, and the ultraviolet discharge corona region may be a bright spot region of corona or arc discharge in an ultraviolet image.

[0051] It should be noted that the preset dual-channel detector can be set up at a position with a predetermined distance d from the insulator to be detected, and can determine an equivalent triangular relationship according to the height h of the insulator.

[0052] Specifically, the terminal detects insulator discharge by controlling the visible light imaging channel and ultraviolet imaging channel of the dual-channel detector, and obtains a visible light image and an ultraviolet image respectively; the visible light image has a first resolution, and the ultraviolet image has a second resolution; the ultraviolet image is then preprocessed (grayscale and noise reduction), and then the ultraviolet discharge corona area is extracted, and the pixel area of ​​the ultraviolet discharge corona area is calculated.

[0053] Step S202, input the detector gain, detection distance and pixel area of ​​the dual-channel detector into the trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and determine the ultraviolet discharge intensity level according to the discharge pulse current intensity.

[0054] Among them, the neural network model can be a CNN (Convolutional Neural Network) model, with detector gain, detection distance, corona discharge image characteristics (such as pixel area), and discharge pulse current intensity as feature variables, detector gain, detection distance and pixel area as input features (x1, x2, ..., xn), and discharge pulse current intensity as output (y). Through CNN network training, a prediction model is obtained, and in subsequent use, the discharge pulse current intensity (such as pulse current peak) can be determined through known detection parameters.

[0055] Specifically, the discharge level intensity can be determined based on the correspondence between the discharge pulse current intensity and the ultraviolet discharge intensity level (indicating the degree of danger of partial discharge) formulated by the power system or enterprise. That is, according to the impact of corona and arc generated by insulator discharge on the safety of insulator electrical equipment, the pulse current peak value of the corresponding stage is divided into different danger levels, corresponding to the ultraviolet discharge intensity level.

[0056] For example, when the pulse current peak is within the first setting range [Int1~Int2), corona discharge begins to occur, corresponding to the danger level 1, and the ultraviolet discharge intensity level output is level 1; when the pulse current peak is within the second setting range [Int1~Int2), the ultraviolet discharge intensity level output is level 2; if it exceeds the Int2 standard, it is defined as level 3 (the highest level).

[0057] Step S203, registering the ultraviolet image and the visible light image according to the preset imaging field of view offset, and performing pixel-level image fusion based on local content weighting on the registered ultraviolet image and visible light image to obtain an initial fused image.

[0058] The imaging field of view offset can be used to indicate the deviation between the field of view and the actual imaging system.

[0059] Specifically, the terminal performs offset calibration on the ultraviolet image and the visible light image based on a preset imaging field of view offset, and aligns the ultraviolet image and the visible light image after the offset calibration according to the imaging area to obtain the registered ultraviolet image and visible light image, and then, based on the visible light image after rapid registration, the corresponding discharge area bright spot in the ultraviolet image is fused with it, and the initial fused image is obtained by pixel-level image fusion based on local content weighting.

[0060] Step S204 , superimposing the detector gain, detection distance, discharge pulse current intensity, and ultraviolet discharge intensity level on the initial fusion image to obtain a target fusion image, and visually displaying the target fusion image.

[0061] Specifically, the terminal superimposes parameters such as detector gain, detection distance, discharge pulse current intensity, and ultraviolet discharge intensity level on the initial fusion image to obtain the final target fusion image and visually displays the target fusion image.

[0062] In the above-mentioned visualization fusion method of ultraviolet image and visible light image, the ultraviolet discharge corona area is extracted from the detected ultraviolet image, the pixel area of ​​the ultraviolet discharge corona area is calculated, and then the detector gain, detection distance and pixel area are combined to determine the discharge pulse current intensity and the ultraviolet discharge intensity level. Then, the visible light image and the ultraviolet image are quickly aligned according to the preset imaging field of view offset, and the initial fused image is obtained by pixel-level image fusion based on local content weighting. Finally, relevant parameter information is superimposed on the initial fused image to obtain the target fused image, and the effect of visually displaying the target fused image is achieved, thereby reasonably eliminating the difference between the visible light image and the ultraviolet image, reducing the error generated when the two images are aligned, and thereby improving the accuracy of image alignment.

[0063] In one embodiment, if Figure 3 As shown, in the above step S201, the ultraviolet discharge corona region is extracted from the ultraviolet image, and the pixel area of ​​the ultraviolet discharge corona region is calculated, which specifically includes the following steps:

[0064] Step S301 , according to a preset edge detection algorithm, the coordinate points of the bright spot edge contour in the ultraviolet image are extracted by gradient intensity and direction.

[0065] Step S302, connecting the bright spot edge contour coordinate points to obtain the bright spot image edge contour.

[0066] Step S303: Count the number of target pixel points within the range of the edge contour of the bright spot image to obtain the pixel area.

[0067] Specifically, the terminal extracts the ultraviolet discharge corona area (i.e., the bright spot area of ​​the corona or arc discharge in the ultraviolet image) through the canny algorithm, including: extracting the bright spot edge contour coordinate points of the discharge area through gradient strength and direction, and then refining the edge contour (for example, through non-maximum suppression), and finally further suppressing the noise through the double threshold method, filtering out the noise lines, and obtaining a continuous and clear discharge bright spot image edge contour. Among them, calculating the pixel area of ​​the ultraviolet discharge corona area refers to the number of pixels with a pixel value of 1 within the statistical contour range. Further, the terminal combines the detector gain, detection distance and pixel area, and determines the discharge pulse current intensity and the ultraviolet discharge level intensity in the trained neural network model.

[0068] In one embodiment, before registering the ultraviolet image and the visible light image according to the preset imaging field of view offset, the following steps are also included:

[0069] The imaging array parameters of the visible light sensor and the ultraviolet imaging sensor are obtained, and the target parameters of the observed object are obtained, and the imaging areas of the observed object on the visible light sensor and the ultraviolet imaging sensor are determined according to the imaging array parameters and the target parameters; the spacing baseline between the visible light camera and the ultraviolet camera is determined, and the imaging field of view offset of the visible light camera is calculated according to the spacing baseline and with the field of view center of the ultraviolet camera as the origin, as the preset imaging field of view offset.

[0070] Specifically, the terminal obtains the imaging array parameters of the photosensitive surfaces of the two visible light sensors and the ultraviolet imaging sensor, including the imaging arrays mu*nu (representing the number of rows and columns of the pixel array of the ultraviolet sensor, respectively) and mr*nr (representing the number of rows and columns of the pixel array of the visible light sensor, respectively) of the ultraviolet sensor and the visible light sensor, in pixels; the respective pixel spacing I, in um, and the focal length f, in mm; then determines the relevant parameters of the observed object, including the target distance d of the observed object, in m, and the area length L*width W of the observed object; then, according to the imaging array parameters of the ultraviolet sensor and the visible light sensor and the relevant parameters of the observed object, determines the imaging areas Su and Sr of the observed object on the ultraviolet sensor and the visible light sensor, and the calculation formula of the imaging area is as follows:

[0071] Su1=fu / (Iu*10 -3 )*(L / d);Su2=fu / (Iu*10 -3 )*(W / d)

[0072] Sr1=fr / (Ir*10 -3 )*(L / d); Sr2=fr / (Ir*10 -3 )*(W / d)

[0073] Finally, according to the spacing baseline dl between the UV camera and the visible light camera, the imaging field of view offset of the visible light camera is calculated with the center of the field of view of the UV camera as the origin. The specific calculation formula is as follows:

[0074] ΔF=fr*(dl / 2)*(d / Ir)

[0075] In the above formula, ΔF is the imaging field of view offset, f is the focal length, d is the target distance, and I is the pixel pitch.

[0076] In one embodiment, in the above step S203, the ultraviolet image and the visible light image are registered according to the preset imaging field of view offset, which specifically includes the following steps:

[0077] Based on a preset imaging field of view offset, the ultraviolet image and the visible light image are offset calibrated; according to the imaging area, the ultraviolet image and the visible light image after the offset calibration are registered and aligned to obtain the registered ultraviolet image and visible light image.

[0078] Specifically, the terminal obtains the visible light image after registration with the ultraviolet image through the imaging field of view offset of the visible light camera. In the specific registration process, since both sensors are based on parallel light, the terminal first calibrates the offset of the two imaging images based on the imaging field of view offset before registration, and then performs registration and alignment according to the imaging areas Su and Sr of the observed object on the ultraviolet sensor and the visible light sensor respectively.

[0079] In one embodiment, if Figure 4 As shown, in the above step S203, pixel-level image fusion based on local content weighting is performed on the registered ultraviolet image and visible light image to obtain an initial fused image, which specifically includes the following steps:

[0080] Step S401 : performing edge detection on the ultraviolet image and the visible light image respectively to obtain corresponding edge maps.

[0081] Step S402, based on the edge map, determine the current pixel value of each pixel and the corresponding pixel weighting coefficient, perform weighted calculation according to the pixel weighting coefficient and the current pixel value, and obtain the fused target pixel value.

[0082] Step S403: determining an initial fused image according to the target pixel value.

[0083] In the above step S402, based on the edge map, the current pixel value of each pixel and the corresponding pixel weighting coefficient are determined, which specifically includes the following steps:

[0084] The local mean of each pixel is obtained, and the local standard deviation is determined based on the local mean. The local contrast map is calculated for the edge map through the local standard deviation. The edge map and the local contrast map are combined to obtain the fusion weight map corresponding to the ultraviolet image and the visible light image. In the fusion weight map, the current pixel value of each pixel and the corresponding pixel weighting coefficient are determined.

[0085] Specifically, based on the visible light image after rapid registration, the terminal performs pixel-level image fusion on the ultraviolet image and the visible light image, that is, weighted calculation is performed on the pixels of the two images to obtain the fused pixel values, thereby determining the fused image. The specific calculation method is as follows:

[0086] IF(x,y)=α*rgb(x,y)+β*uv(x,y)

[0087] In the above formula, α and β represent the pixel weighting coefficients of the visible light image and the ultraviolet image respectively, and rgb(x,y) and uv(x,y) represent the current pixel values ​​of the corresponding pixel point (x,y) in the visible light image and the ultraviolet image respectively.

[0088] In this embodiment, in view of the defects of the fixed fusion coefficient (such as a value of 0.5) in traditional pixel fusion and the computational complexity and speed problems caused by the fusion rule of the high-frequency sub-band, a pixel-level image fusion based on local content weighting is proposed to obtain a fused image. On the one hand, the computational complexity of traditional transform-domain-based image fusion and the distortion problem caused by the fixed threshold spatial domain fusion algorithm (such as pixel arithmetic mean) are reduced; on the other hand, local details that are highly sensitive to human vision are retained, key information in the original image is maintained, the accuracy and quality of image fusion are improved, and it is conducive to a more accurate analysis of the target through subsequent image processing.

[0089] First, the terminal processes the ultraviolet image and the visible light image based on the edge detector to obtain the edge map. For the ultraviolet image, as mentioned above, the bright spot of the discharge area is obtained as the edge map, and the edge contour coordinate points are determined. For the visible light image, a contour map characterized by the surface structure of the insulator is obtained, and the edge contour coordinate points are determined. Then, the edge map (binary image, pixel value 0 or 255) generated by the edge detection is adjusted to its range [0, 1] by normalization. By highlighting the edge area with higher weight and the non-edge area with lower weight, important structural features are highlighted in the subsequent fusion process.

[0090] In the preprocessed edge map, the terminal calculates the local mean of each pixel separately, which is done by using a sliding window such as 5x5 (or 3*3). The window slides across the entire image, and for each pixel, the mean of the pixel and its neighborhood is calculated; then the local standard deviation is calculated, and the local standard deviation is calculated using the same sliding window (including calculating the square of the difference between each pixel and the local mean and then calculating the local standard deviation). The local standard deviation here is a measure of the difference between the pixel value and its local mean. A high standard deviation means that there is high contrast around the pixel, which is related to local details and edges. Using the local contrast measurement method, the contrast map is calculated for each image through the local standard deviation, and the contrast map is normalized to the range of [0, 1].

[0091] Finally, the terminal combines the edge map and local contrast map corresponding to the ultraviolet image and the visible light image, and generates the final weight map by multiplying the two, ensuring that the value of the weight map is between [0, 1] and the sum of the weights of the two weight maps is 1 at each pixel position, that is, the fusion weight map corresponding to the visible light image and the ultraviolet image is obtained, which is the same size as the original image (visible light image after registration), where the value of each pixel represents the importance weight of the pixel position. Therefore, the final weight map is used to weightedly calculate each pixel of each image, such as the above formula, the pixel value of the corresponding position is multiplied by its weight, and then the results of the two images are added to obtain the final fused image, thereby realizing the fusion of the bright spot image of the ultraviolet image and the visible light image, and then superimposing the detection distance, detection gain, discharge pulse current intensity and ultraviolet discharge intensity level to achieve visual output.

[0092] In one embodiment, Figure 5 As shown, a method for visual fusion of ultraviolet images and visible light images in a specific embodiment is provided, which specifically includes the following steps:

[0093] Step S501, obtaining an ultraviolet image and a visible light image detected by a preset dual-channel detector, extracting the bright spot edge contour coordinate points in the ultraviolet image by gradient strength and direction according to a preset edge detection algorithm; connecting the bright spot edge contour coordinate points to obtain the bright spot image edge contour; and counting the number of target pixel points within the range of the bright spot image edge contour to obtain the pixel area.

[0094] Step S502, input the detector gain, detection distance and pixel area of ​​the dual-channel detector into the trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and determine the ultraviolet discharge intensity level according to the discharge pulse current intensity.

[0095] Step S503, obtaining imaging array parameters of the visible light sensor and the ultraviolet imaging sensor, and obtaining target parameters of the observed object, and determining the imaging area of ​​the observed object on the visible light sensor and the ultraviolet imaging sensor according to the imaging array parameters and the target parameters; determining the spacing baseline between the visible light camera and the ultraviolet camera, and calculating the imaging field of view offset of the visible light camera according to the spacing baseline and taking the field of view center of the ultraviolet camera as the origin, as a preset imaging field of view offset.

[0096] Step S504, based on the preset imaging field offset, the ultraviolet image and the visible light image are offset calibrated; according to the imaging area, the ultraviolet image and the visible light image after the offset calibration are registered and aligned to obtain the registered ultraviolet image and visible light image.

[0097] Step S505, perform edge detection on the ultraviolet image and the visible light image respectively to obtain corresponding edge maps; obtain the local mean of each pixel point, determine the local standard deviation based on the local mean, and calculate the local contrast map of the edge map through the local standard deviation; combine the edge map and the local contrast map to obtain the fusion weight map corresponding to the ultraviolet image and the visible light image.

[0098] Step S506, in the fusion weight map, determine the current pixel value of each pixel and the corresponding pixel weight coefficient; perform weighted calculation based on the pixel weight coefficient and the current pixel value to obtain the fused target pixel value; and determine the initial fused image based on the target pixel value.

[0099] Step S507 , superimposing the detector gain, detection distance, discharge pulse current intensity, and ultraviolet discharge intensity level on the initial fused image to obtain a target fused image, and visually displaying the target fused image.

[0100] The beneficial effects brought by the above embodiments are as follows:

[0101] The present application extracts the ultraviolet discharge corona area from the detected ultraviolet image, calculates the pixel area of ​​the ultraviolet discharge corona area, and then determines the discharge pulse current intensity and the ultraviolet discharge intensity level in combination with the detector gain, the detection distance and the pixel area. Then, the visible light image and the ultraviolet image are quickly registered according to the preset imaging field of view offset, and an initial fused image is obtained by pixel-level image fusion based on local content weighting. Finally, relevant parameter information is superimposed on the initial fused image to obtain a target fused image, and the effect of visually displaying the target fused image is achieved, thereby reasonably eliminating the difference between the visible light image and the ultraviolet image, reducing the error generated when the two images are registered, and thus improving the accuracy of image registration.

[0102] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0103] Based on the same inventive concept, the embodiment of the present application also provides a visual fusion device of ultraviolet images and visible light images for implementing the visual fusion method of ultraviolet images and visible light images involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more visual fusion devices of ultraviolet images and visible light images provided below can refer to the limitations of the visual fusion method of ultraviolet images and visible light images above, and will not be repeated here.

[0104] In an exemplary embodiment, Figure 6 As shown, a visual fusion device for ultraviolet images and visible light images is provided, comprising:

[0105] The image acquisition module 601 is used to acquire the ultraviolet image and the visible light image detected by the preset dual-channel detector, extract the ultraviolet discharge corona area in the ultraviolet image, and calculate the pixel area of ​​the ultraviolet discharge corona area;

[0106] The model processing module 602 is used to input the detector gain, detection distance and pixel area of ​​the dual-channel detector into the trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and determine the ultraviolet discharge intensity level according to the discharge pulse current intensity;

[0107] An image registration module 603 is used to register the ultraviolet image and the visible light image according to a preset imaging field of view offset, and perform pixel-level image fusion based on local content weighting on the registered ultraviolet image and visible light image to obtain an initial fused image;

[0108] The image display module 604 is used to superimpose the detector gain, detection distance, discharge pulse current intensity and ultraviolet discharge intensity level on the initial fusion image to obtain a target fusion image, and visually display the target fusion image.

[0109] In one embodiment, the image acquisition module 601 is also used to extract the bright spot edge contour coordinate points in the ultraviolet image according to a preset edge detection algorithm by gradient strength and direction; connect the bright spot edge contour coordinate points to obtain the bright spot image edge contour; count the number of target pixel points within the range of the bright spot image edge contour to obtain the pixel area.

[0110] In one embodiment, the visual fusion device of the ultraviolet image and the visible light image also includes an information preparation module, which is used to obtain imaging array parameters of the visible light sensor and the ultraviolet imaging sensor, and obtain target parameters of the observed object, and determine the imaging area of ​​the observed object on the visible light sensor and the ultraviolet imaging sensor according to the imaging array parameters and the target parameters; determine the spacing baseline between the visible light camera and the ultraviolet camera, and calculate the imaging field of view offset of the visible light camera according to the spacing baseline and with the field of view center of the ultraviolet camera as the origin, as a preset imaging field of view offset.

[0111] In one embodiment, the image registration module 603 is also used to perform offset calibration on the ultraviolet image and the visible light image based on a preset imaging field of view offset; and to align the ultraviolet image and the visible light image after the offset calibration according to the imaging area to obtain the registered ultraviolet image and visible light image.

[0112] In one embodiment, the image registration module 603 is also used to perform edge detection on the ultraviolet image and the visible light image respectively to obtain corresponding edge maps; based on the edge map, determine the current pixel value of each pixel point and the corresponding pixel weighting coefficient, perform weighted calculation according to the pixel weighting coefficient and the current pixel value to obtain the fused target pixel value; and determine the initial fused image according to the target pixel value.

[0113] In one embodiment, the image registration module 603 is also used to obtain the local mean of each pixel point, determine the local standard deviation based on the local mean, and calculate the local contrast map of the edge map through the local standard deviation; combine the edge map and the local contrast map to obtain a fusion weight map corresponding to the ultraviolet image and the visible light image; in the fusion weight map, determine the current pixel value of each pixel point and the corresponding pixel weighting coefficient.

[0114] Each module in the above-mentioned ultraviolet image and visible light image visualization fusion device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0115] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for visual fusion of ultraviolet images and visible light images is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0116] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0118] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0119] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0120] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0121] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0122] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0123] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for visual fusion of ultraviolet image and visible light image, characterized in that: The method comprises: Acquire an ultraviolet image and a visible light image detected by a preset dual-channel detector, extract an ultraviolet discharge corona region from the ultraviolet image, and calculate a pixel area of ​​the ultraviolet discharge corona region; Inputting the detector gain, detection distance and pixel area of ​​the dual-channel detector into a trained neural network model to obtain the discharge pulse current intensity output by the neural network model, and determining the ultraviolet discharge intensity level according to the discharge pulse current intensity; According to a preset imaging field of view offset, the ultraviolet image and the visible light image are registered, and pixel-level image fusion based on local content weighting is performed on the registered ultraviolet image and visible light image to obtain an initial fused image; The detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity level are superimposed on the initial fusion image to obtain a target fusion image, and the target fusion image is visually displayed.

2. The method according to claim 1, characterized in that The step of extracting the ultraviolet discharge corona region from the ultraviolet image and calculating the pixel area of ​​the ultraviolet discharge corona region includes: According to a preset edge detection algorithm, the coordinate points of the bright spot edge contour in the ultraviolet image are extracted by gradient intensity and direction; Connecting the bright spot edge contour coordinate points to obtain the bright spot image edge contour; The number of target pixel points within the edge contour of the bright spot image is counted to obtain the pixel area.

3. The method according to claim 1, characterized in that Before registering the ultraviolet image and the visible light image according to the preset imaging field of view offset, the method further includes: Acquire imaging array parameters of a visible light sensor and an ultraviolet imaging sensor, and acquire target parameters of an observed object, and determine imaging areas of the observed object on the visible light sensor and the ultraviolet imaging sensor according to the imaging array parameters and the target parameters; A spacing baseline between the visible light camera and the ultraviolet camera is determined, and an imaging field of view offset of the visible light camera is calculated based on the spacing baseline and with the field of view center of the ultraviolet camera as the origin, as the preset imaging field of view offset.

4. The method according to claim 3, characterized in that The registering the ultraviolet image and the visible light image according to the preset imaging field of view offset includes: Based on the preset imaging field of view offset, performing offset calibration on the ultraviolet image and the visible light image; According to the imaging area, the ultraviolet image and the visible light image after the offset calibration are registered and aligned to obtain the registered ultraviolet image and visible light image.

5. The method according to any one of claims 1 to 4, characterized in that: The method of performing pixel-level image fusion based on local content weighting on the registered ultraviolet image and visible light image to obtain an initial fused image includes: Performing edge detection on the ultraviolet image and the visible light image respectively to obtain corresponding edge maps; Based on the edge map, determine the current pixel value of each pixel and the corresponding pixel weighting coefficient, and perform weighted calculation according to the pixel weighting coefficient and the current pixel value to obtain a fused target pixel value; The initial fused image is determined according to the target pixel value.

6. The method according to claim 5, characterized in that The determining of the current pixel value of each pixel and the corresponding pixel weighting coefficient based on the edge map includes: Obtaining a local mean value of each of the pixel points, determining a local standard deviation according to the local mean value, and calculating a local contrast map for the edge map by using the local standard deviation; Combining the edge map and the local contrast map to obtain a fusion weight map corresponding to the ultraviolet image and the visible light image; In the fusion weight map, the current pixel value of each pixel point and the corresponding pixel weighting coefficient are determined.

7. A visual fusion device for ultraviolet images and visible light images, characterized in that: The device comprises: An image acquisition module is used to acquire an ultraviolet image and a visible light image detected by a preset dual-channel detector, extract an ultraviolet discharge corona area from the ultraviolet image, and calculate a pixel area of ​​the ultraviolet discharge corona area; A model processing module, used for inputting the detector gain, detection distance and pixel area of ​​the dual-channel detector into a trained neural network model, obtaining the discharge pulse current intensity output by the neural network model, and determining the ultraviolet discharge intensity level according to the discharge pulse current intensity; An image registration module, used to register the ultraviolet image and the visible light image according to a preset imaging field of view offset, and perform pixel-level image fusion based on local content weighting on the registered ultraviolet image and visible light image to obtain an initial fused image; An image display module is used to superimpose the detector gain, the detection distance, the discharge pulse current intensity and the ultraviolet discharge intensity level on the initial fusion image to obtain a target fusion image, and visually display the target fusion image.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images