An improved Retinex-Net-based infrared image enhancement method for power equipment

Abstract

The application discloses an infrared image enhancement method for power equipment based on an improved Retinex-Net, and belongs to the fields of image processing and power equipment detection.The application comprises the following steps: performing HSV color space conversion on an infrared image of power equipment to obtain H, S and V three-channel components; based on the improved Retinex-Net network, a preliminary enhanced infrared image is obtained for the H, S and V three-channel components; performing superpixel segmentation and reconstruction on the preliminary enhanced infrared image to obtain a final enhanced infrared image.Through the method of the application, color distortion and image blurring can be avoided, image edge details can be better maintained, the influence of uneven illumination can be reduced, the contrast between image targets and backgrounds can be improved, the comprehensive quality of the infrared image is effectively improved, the target detection model precision can be improved, favorable conditions are provided for realizing power equipment situation awareness, an important foundation is laid for subsequent power equipment fault diagnosis, and the application has good application value for realizing intelligent operation and maintenance of a substation.

Description

TECHNICAL FIELD

[0001] The present application relates to an improved Retinex-Net-based infrared image enhancement method for power equipment, belonging to the field of image processing and power equipment detection. BACKGROUND

[0002] Power equipment is an important part of the power grid, and its safe operation will directly affect the stability of the entire power system. Power equipment is easily affected by external environmental factors, leading to equipment failure, and most of the failures caused by abnormal heating must be prevented by timely detection of the running state of the power equipment to prevent power grid accidents caused by equipment defects.

[0003] Compared with the traditional maintenance method, the infrared thermal imaging technology has the advantages of non-contact, simple operation, high sensitivity, etc., and can timely and effectively find the abnormal heating defects of the equipment. However, due to the complex background of the substation, the infrared image is disturbed by factors such as shooting angle, irrelevant equipment obstruction, poor lighting conditions, etc. during the collection process, resulting in poor overall visual effect, low contrast, uneven illumination, and insufficiently clear target feature details of the power equipment infrared image, increasing the detection difficulty, and unable to effectively distinguish the detection equipment from the background, affecting the automatic recognition accuracy of the equipment infrared image and further affecting the accuracy of the equipment situation awareness, therefore, the power equipment infrared image, especially the low-illumination image, needs to be enhanced.

[0004] Traditional low-illumination image enhancement methods include histogram equalization method, gray scale transformation method, image defogging method, and Retinex theory-based method, etc. These methods are mainly aimed at the enhancement of low-illumination visible light images, which can improve the overall contrast of the image, and solve the problems of poor visibility and color deviation of low-illumination images, but due to the complex image background, the details of the image are not considered, and the enhanced image may have problems such as blurred edge features or lost target feature details, resulting in poor visual effect of the processed image.

[0005] With the development of artificial intelligence technology, low-light image enhancement based on machine learning technology has also achieved deep processing. For example, Huang Shuying et al. in the article "Image Dehazing Algorithm Based on Generative Adversarial Network[J]. Pattern Recognition and Artificial Intelligence, 2021, 34(11): 990-1003." combined with deep learning technology, aiming at the problem of incomplete dehazing and color distortion in complex scenes, an image dehazing algorithm based on generative adversarial network was proposed, which fully extracted image feature information, and the final dehazing image was closer to the corresponding real haze-free image. Xu Shaoping et al. in the article "Low-light image enhancement algorithm based on hybrid implementation strategy of deep learning and image fusion[J]. Electronic Journal, 2021, 49(01): 72-76." based on deep learning, local structured fusion and chroma weighted fusion technology were used to fuse multiple images to obtain the final enhanced image. This method has good edge preservation and color fidelity effect on local image structure details. Jiang Zetao et al. in the article "A Low-light Image Enhancement Method Based on U-Net Generative Adversarial Network[J]. Electronic Journal, 2020, 48(02): 258-264." to improve the quality of low-light images, U-Net generative adversarial network was used to extract low-light image features and map them to normal illumination images. The part with large difference was selected for emphasis enhancement. This method can effectively improve the brightness and contrast of the image and handle local details well. Oujiamin et al. in the article "Improved Retinex-Net Low-light Image Enhancement Algorithm[J]. Pattern Recognition and Artificial Intelligence, 2021, 34(01): 77-86." introduced denoising loss and attention mechanism module based on Retinex-Net, which enhanced the brightness of the image while reducing color distortion and effectively reduced the noise of the enhanced image.

[0006] Analysis of the above methods shows that the low-light image enhancement method is mainly used for visible light image recognition, and further research is needed for low-light infrared image enhancement of power equipment. On the other hand, these methods do not consider the case of uneven illumination, and cannot adaptively enhance different brightness areas, resulting in poor image contrast, low brightness and loss of target feature details, reducing the overall quality of the image and increasing the difficulty of image target detection, affecting the accurate judgment of power equipment failure. SUMMARY

[0007] The present application provides a power equipment infrared image enhancement method based on improved Retinex-Net, which adjusts the image brightness component separately by changing the image color space, avoiding color distortion; improves the Retinex-Net network to solve the problems of image blur, target detail loss and uneven illumination; finally, through superpixel segmentation and reconstruction, the purpose of low-light infrared image enhancement is achieved.

[0008] The technical scheme of the present application is: an improved Retinex-Net-based power equipment infrared image enhancement method, comprising:

[0009] S1, HSV color space conversion is performed on the power equipment infrared image to obtain H, S and V three-channel components;

[0010] S2, based on the improved Retinex-Net network, the H, S and V three-channel components are obtained to obtain a preliminary enhanced infrared image;

[0011] S3, superpixel segmentation and reconstruction are performed on the preliminary enhanced infrared image to obtain a final enhanced infrared image.

[0012] The S2 comprises:

[0013] S2.1, based on the traditional Retinex-Net network, a fast guided filter smoothing loss function is used to replace the smoothing loss function of the original network to obtain a preliminary enhanced V component;

[0014] S2.2, the preliminary enhanced V component is corrected and fused to obtain an enhanced V component;

[0015] S2.3, the enhanced V component and the original H and S components are merged to obtain a preliminary enhanced infrared image.

[0016] It also comprises a verification step S4, which verifies and analyzes the improved Retinex-Net network through three modules: ① a network only improving the loss function, denoted as FR-Net; ② a network improving the loss function and adding correction and adjustment, denoted as FGR-Net; and ③ an improved Retinex-Net adding a fusion module based on FGR-Net, which verifies the enhancement effect of the improved Retinex-Net through quantitative evaluation indexes; and then, for the final enhancement method, YOLO v3 and SSD target detection models are used to verify the detection effect.

[0017] The S2.2 comprises: adaptive Gamma correction and VSM+WLS fusion processing are performed on the preliminary enhanced V component to obtain an enhanced V component.

[0018] In the adaptive Gamma correction, the correction constant expression is:

[0019]

[0020] In the formula: γ is the correction constant; I(x, y) represents the gray value of a certain point of the image; μ and σ respectively represent the mean and standard deviation of the image gray value; and ε is the brightness threshold value.

[0021] Firstly, the illumination coefficient of each preliminary enhanced V component image is calculated by using the gray mean value of the image, and the illumination coefficient distribution of all the preliminary enhanced V components is counted; then, the average value of the illumination coefficients of all the preliminary enhanced V components is calculated, and the pixel mean value with the highest appearance probability in the illumination component is obtained according to the average value of the illumination coefficients, as the brightness threshold ε.

[0022] The VSM+WLS fusion processing is specifically: firstly, the original V component image is taken as a supplementary image of the V component after adaptive Gamma correction; then, the base layer and the detail layer of the corrected V component image are estimated by using a Gaussian filter and an iterative filter in turn, the base layer of the corrected V component image is weighted and superimposed by using VSM to calculate the weight, and each layer of the detail layer of the corrected V component image is superimposed by using WLS; finally, the processed base layer and the detail layer are fused and reconstructed to obtain the enhanced V component image.

[0023] The S3 comprises the following steps: firstly, the MSRM and SEEDS superpixel segmentation method is adopted to segment, and the finally segmented target image is obtained; then, the multi-scale hybrid guided filter algorithm is used for reconstruction to obtain the finally enhanced image.

[0024] The target device is segmented by using the MSRM and SEEDS superpixel segmentation method, and the specific process is as follows: firstly, the superpixel pre-segmentation is performed by using SEEDS, then the superpixel blocks segmented are marked and divided according to the marking rules, the MSRM superpixel merging is realized, finally, the finally segmented target image is obtained by using the morphological method to process the merged superpixels.

[0025] The multi-scale hybrid guided filter algorithm is used for reconstruction to obtain the finally enhanced image, and the specific process is as follows: according to the preliminary enhanced infrared image and the finally segmented target image, firstly, the image detail layer and the base layer are decomposed by using the multi-scale RGF, and the saliency map and the weight map are obtained by using filtering, then the weight map, the base layer and the detail layer are fused by using the weighted fusion method to obtain the fused base layer S B and the fused detail layer S D , finally, the global detail enhancement is performed on S D to obtain the enhanced S D ; according to the fused base layer S B and the enhanced S D , the final fusion result S is obtained as the finally enhanced infrared image.

[0026] The beneficial effects of the present application are as follows:

[0027] (1) The mutual independence of each channel of the HSV color space is utilized to convert the low-illumination infrared image to the HSV space, so that the image brightness can be enhanced without causing obvious color deviation and distortion.

[0028] (2) The fast guided filter smoothing loss function has real-time performance and effectively maintains the image edge; the adaptive Gamma correction function is constructed to solve the problem of poor enhancement effect in different brightness regions, and the VSM and WLS fusion algorithm are used to fuse the image brightness component, so that the problems of loss of target details and image blur in the correction process are avoided, and the quality of the V component image is improved.

[0029] (3) The superpixel segmentation based on MSRM and SEEDS and the multi-scale hybrid guided filter reconstruction method can further improve the contrast between the image target and the background without color distortion, and the image feature details are clearly and completely retained.

[0030] In summary, by the method of the present application, color distortion and image blur can be avoided, the image edge details can be well maintained, the influence of uneven illumination can be reduced, and the contrast between the image target and the background can be improved, so that the overall quality of the infrared image is effectively improved, the target detection model precision is improved, and the power equipment situation awareness is provided with favorable conditions, which lays an important foundation for subsequent power equipment fault diagnosis, and has good application value for realizing intelligent operation and maintenance of the substation. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is the overall flowchart of the present application;

[0032] Figure 2 is the original Retinex-Net network structure of the present application;

[0033] Figure 3 is the overall framework diagram of the improved Retinex-Net network of the present application;

[0034] Figure 4 is the superpixel segmentation and reconstruction flowchart of the present application. DETAILED DESCRIPTION

[0035] The application will be further described below in combination with the drawings and examples, but the content of the application is not limited to the scope described.

[0036] Example 1: As shown in the drawings, according to an aspect of an embodiment of the present application, a power equipment infrared image enhancement method based on an improved Retinex-Net is provided, which comprises: Figures 1-4

[0037] ​S1, converting an infrared image of a power device into an HSV color space to obtain H, S and V three channel components;

[0038] S2, based on an improved Retinex-Net network, obtaining a preliminary enhanced infrared image for the H, S and V three channel components;

[0039] S3, performing superpixel segmentation and reconstruction on the preliminary enhanced infrared image to obtain a final enhanced infrared image.

[0040] Further, the S2 comprises:

[0041] S2.1, based on a traditional Retinex-Net network, using a fast guided filter smoothing loss function to replace the original network smoothing loss function to obtain a preliminary enhanced V component; wherein the traditional Retinex-Net network is as shown in Figure 2

[0042] S2.2, correcting and fusing the preliminary enhanced V component to obtain an enhanced V component;

[0043] S2.3, merging the enhanced V component with the original H and S components to obtain a preliminary enhanced infrared image.

[0044] Further, it further comprises a verification step S4, for the improved Retinex-Net network, three modules are set up for verification analysis, and experiments are designed based on the traditional Retinex-Net network: ① the network only improving the loss function is recorded as FR-Net; ② the network both improving the loss function and adding correction and adjustment is recorded as FGR-Net; ③ on the basis of FGR-Net, a fusion module is added, that is, the improved Retinex-Net, through quantitative evaluation index, the enhancement effect of the improved Retinex-Net is verified; then for the final enhancement method, the YOLO v3 and SSD target detection model are used to verify the detection effect.

[0045] Further, the S2.2 comprises: performing adaptive Gamma correction on the preliminary enhanced V component and obtaining an enhanced V component based on VSM+WLS fusion processing.

[0046] In the adaptive Gamma correction, the correction constant expression is:

[0047]

[0048] In the formula: γ is the correction constant; I(x, y) represents the gray value of a certain point of the image; μ and σ respectively represent the mean and standard deviation of the image gray value; ε is the brightness threshold.

[0049] ​Furthermore, firstly, the illumination coefficient of each pre-enhanced V component image is calculated using the grayscale mean, and the illumination coefficient distribution of all pre-enhanced V components is statistically analyzed; then, the average illumination coefficient of all pre-enhanced V components is calculated, and the average value of the pixel with the highest probability of occurrence in the illumination component is obtained based on the average illumination coefficient, which is used as the brightness threshold ε.

[0050] Furthermore, the VSM+WLS fusion processing specifically involves: first, using the original V component image as a supplementary image to the adaptively Gamma-corrected V component; then, sequentially estimating the base layer and detail layer of the corrected V component image using Gaussian filtering and iterative filtering; weighting the base layer of the corrected V component image using VSM calculations; then, stacking each detail layer of the corrected V component image using WLS; and finally, fusing and reconstructing the processed base layer and detail layer to obtain the enhanced V component image.

[0051] like Figure 3 As shown, in the model improvement section, the loss function of the model is improved, and adaptive Gamma encoding and V component fusion modules are added to the model. First, the infrared image is converted to the HSV color space by utilizing the independence of each channel in the HSV space, which facilitates the adjustment of the V component of the image. Then, a fast guided filtering smoothing loss function is used to replace the original smoothing loss function of the network. At the same time, a no-flash / flash denoising method is used to effectively remove noise interference while preserving the image edges. An adaptive Gamma function is introduced to adjust the brightness of the V component image in different regions to reduce the impact of uneven illumination. Simultaneously, the V component of the image is fused based on the VSM and WLS fusion methods to obtain a preliminarily enhanced infrared image I, avoiding the loss of target details, improving image quality, and effectively suppressing color distortion.

[0052] Furthermore, such as Figure 4 As shown, S3 includes: firstly, using the superpixel segmentation method of MSRM and SEEDS to extract the target of interest P in the preliminarily enhanced low-light infrared image I, and then reconstructing the extracted target image using a multi-scale infrared image fusion method based on hybrid guided filtering to obtain the final enhanced image S.

[0053] Furthermore, the method of segmenting the target device using MSRM and SEEDS superpixel segmentation specifically involves: firstly, performing superpixel pre-segmentation using SEEDS; then, marking and dividing the segmented superpixel blocks according to the proposed marking rules to achieve MSRM superpixel merging; and finally, using morphological methods to process the merged superpixels to obtain the final segmented target image.

[0054] Further, the extracted target device is reconstructed by using a multi-scale hybrid guided filtering algorithm to obtain a final enhanced image, specifically: according to the preliminary enhanced infrared image and the finally segmented target image, first, the image detail layer and the base layer are decomposed by using a multi-scale RGF, and a saliency map and a weight map are obtained by using filtering, then the weight map, the base layer and the detail layer are fused by using a weighted fusion method to obtain a fused base layer S B and a fused detail layer S D , finally, S D is globally enhanced to obtain an enhanced S D ; according to the fused base layer S B and the enhanced S D , a final fusion result S is obtained as a final enhanced infrared image.

[0055] Further, one optional specific embodiment of the present application is described as follows:

[0056] Step 1: The collected infrared image data is normal illumination, and the local brightness of the image is randomly adjusted by using the curve-gamma principle to construct 2239 infrared images of low-illumination power equipment as an infrared image data set of power equipment; the infrared image data set of power equipment is divided into a training set and a test set according to a ratio of 8:2.

[0057] Step 2: The low-illumination infrared image data set of power equipment is converted to an HSV color space by using a python program, and H, S and V three-channel components are separated.

[0058] Step 3: For the V component, a fast guided filter smoothing loss function is used to replace the original network smoothing loss function based on the traditional Retinex-Net network to obtain a preliminary enhanced V component, which can maintain the image edge details;

[0059] The traditional Retinex-Net network smoothing loss function is L s , and the improved smoothing loss is Then the improved network decomposition model loss function L Decom includes a reconstruction loss L recon , a reflection component consistency loss L r and an incident component smoothing loss The enhanced model loss function L Enhance includes a reconstruction loss L recon and an incident component smoothing loss Then the improved network decomposition model loss function is:

[0060]

[0061]

[0062]

[0063]

[0064] L r =||R low -R normal ||1

[0065]

[0066]

[0067]

[0068] where i, j represent whether the image brightness is low or normal; R i represents the reflection component of different illumination images, I j represents the incident component of different illumination images, S j represents the source image of different illumination; ‖‖1 represents the 1-norm; λ r and λ s represent the coefficients of balancing the reflection consistency and the incident smoothness; is the gradient containing horizontal and vertical directions; λ g represents the smooth structure perception intensity coefficient; l and p respectively represent the guide image and the image to be filtered, which are the same image in this section; k is the window center, ω k represents the local rectangular filtering window with r as the radius; a k and b k are constant coefficients of linear functions; is the regularization parameter to avoid a k too large; σ k 2 is the variance of the guide image l within ω k ; is the mean value of the image to be filtered within ω k .

[0069] Step 4, performing adaptive Gamma correction on the preliminarily enhanced V component and obtaining the enhanced V component through VSM+WLS fusion processing;

[0070] In order to reduce the influence of random selection of correction parameter γ, an adaptive gamma correction function is constructed. Firstly, the illumination coefficient β of each preliminary enhanced V component image is calculated by using the mean gray value m, and the distribution of illumination coefficient of all preliminary enhanced V components is counted. Then, the mean value of illumination coefficient of all preliminary enhanced V components is calculated, and the pixel mean value ε with the highest occurrence probability in the illumination component is obtained according to the mean value of illumination coefficient (the mean value of illumination coefficient * 255 is taken as the pixel mean value ε). Therefore, the brightness threshold ε is set to distinguish the low / high contrast regions. Finally, the gamma correction parameter γ is designed for the low / high contrast regions respectively, and the gamma correction is performed on the preliminary enhanced V component of the target image, so that the adaptive correction of different contrast regions of the image is realized, and the overall brightness and contrast of the V component are improved. The adaptive gamma function is as follows:

[0071]

[0072]

[0073]

[0074] In the formula, V in and V out represent the input preliminary enhanced V component image and the output V component after gamma correction respectively, V in,max represents the maximum gray value of the input preliminary enhanced V component; γ is the correction parameter; I (x, y) represents the gray value of a certain point of the image; μ and σ represent the mean value and the standard deviation of the image gray value respectively.

[0075] Step 5, the enhanced V component is merged with the original H and S components to obtain a preliminary enhanced infrared image.

[0076] Firstly, the original V component image of the low-illumination infrared image itself is taken as the supplementary image of the V component after adaptive gamma correction. Then, the base layer and the detail layer of the corrected V component image are estimated by using the Gaussian filter and the iterative filter in turn, the base layer of the corrected V component image is weighted and superimposed by using the VSM to calculate the weight, and each layer of the detail layer of the corrected V component image is superimposed by using the WLS. Finally, the processed base layer and the detail layer are fused and reconstructed to obtain the enhanced V component image. The method can transfer more useful visual and detail information to the fused image, so that the image has a good overall appearance, and the noise can be well suppressed while the image edge is maintained. The calculation formula of the fusion result F, the base layer B F and the detail layer D is as follows:

[0077]

[0078] B F =W b B1+(1-W b B2

[0079]

[0080]

[0081]

[0082] In the formula, F represents the fusion result, i.e., the enhanced V component image; B1 and B2 represent the base layers of the two V component images, respectively; N represents the total number of pixels in image I; M j The detail layer represents the "maximum absolute" rule; λ represents the optimization parameter; W b Indicates the weight value; A j It is a diagonal matrix containing the weights of all pixels; d2 j V(p) represents the filter difference; V(p) represents the visual saliency mapping value of pixel p; I p I represents the brightness value of pixel p in image I. j M represents the brightness value of pixel j in image I; i This represents the number of pixels at pixel level i. Note that the expression V(p) should be normalized to the range [0,1]. V1 and V2 represent the visual saliency maps of the two V component images, respectively.

[0083] Step 6: Perform superpixel segmentation on the pre-enhanced infrared image: First, perform superpixel pre-segmentation using SEEDS, then label and divide the segmented superpixel blocks according to the proposed labeling rules to achieve MSRM superpixel merging, and finally use morphological methods to process the merged superpixels to obtain the final segmented target image.

[0084] When performing superpixel segmentation on preliminarily enhanced infrared images based on MSRM and SEEDS, reasonable labeling rules are formulated according to the characteristics of the input image to guide the merging of superpixel blocks, thereby improving the segmentation effect of the target device. The labeling rules are as follows:

[0085] Target block: Mark the superpixel block of power equipment at any location in the image with a relatively complete shape as the target.

[0086] Background blocks: ① Mark superpixel blocks in low-brightness areas of the image as background; ② Mark superpixel blocks in high-brightness non-device and device-independent areas of the image as background;

[0087] The specific segmentation steps are as follows:

[0088] 1) SEEDS superpixel initial segmentation. Set K grid to segment the input image, K is the number of segmented superpixel blocks;

[0089] 2) MSRM superpixel merging. First, the obtained superpixel blocks are divided into three categories, namely target blocks, background blocks and unmarked blocks; Then use the proposed marking rules to guide the superpixel merging. The whole merging rule is divided into two stages, the first stage merges the marked background blocks with their adjacent regions, and the second stage mainly merges the unmarked blocks left over from the first stage. After multiple cycles of merging in the two stages, if there is no new merging, it means that the superpixel block merging is complete.

[0090] (3) Morphological processing. The device segmentation image obtained in step (2) may appear over-segmentation and under-segmentation. Based on contour extraction, morphological processing is performed on the device segmentation image to obtain the final power device segmentation result.

[0091] Step 7, reconstructing the final segmented target image: according to the preliminary enhanced infrared image and the final segmented target image, first decompose the image detail layer and the base layer by multi-scale RGF, and obtain the saliency map and the weight map by filtering, then use the weighted fusion method to fuse the weight map, the base layer and the detail layer to obtain the fused base layer S B and the fused detail layer S D , finally do global detail enhancement on S D to obtain the enhanced S D ; according to the fused base layer S B and the enhanced S D , the final fusion result S is obtained.

[0092] Specifically, first, RGF with two different scales of parameters σ s is used to filter the image I, P respectively to obtain the base layer, and the image detail layer is obtained by additive principle to effectively eliminate small structure information and extract image features at different scales; Then generate image I, P saliency map and initial weight map through RGF filtering and LoG filtering, and select four different scale guide filters as joint filtering to optimize the initial weight map to obtain the final weight map; Finally, the weighted average fusion algorithm is used to obtain the fused base layer S B and the fused detail layer S D , and S D is globally enhanced to obtain the final fusion result, that is, the final enhancement result S of the low-illumination infrared image in this paper. The calculation and fusion expression of the weight map is:

[0093]

[0094] SW P(x, y) = 1 - SW I (x, y)

[0095]

[0096] S = S B + S D

[0097] In the formula, S I and S P respectively represent the saliency maps of images I, P; SW I and SW P respectively represent the initial weight maps of S I , S P at position (x, y); GF represents a filter function; m indicates images I, P; W m B , W m D respectively represent the maximum weight maps of the base layer and the detail layer; r, ε are the radius and the regularization parameter of the guided filtering, and the guided filtering radius selected by the present application is 3, 8, 15, 30 respectively.

[0098] The performance test of the enhanced method proposed in the present application is described as follows.

[0099] 1) Improve the performance of Retinex-Net network

[0100] Three modules improved on Retinex-Net are respectively analyzed by experiments, and experiments are designed based on the traditional Retinex-Net network: ①The network only improved the loss function is recorded as FR-Net; ②The network which improved the loss function and added adaptive Gamma adjustment is recorded as FGR-Net; ③The V component fusion module is added on the basis of FGR-Net, that is, the improved Retinex-Net, through experiments, it is known that compared with the traditional Retinex-Net network, the E and PSNR values of the FR-Net network are increased by 2.14% and 7.32% respectively, and the SSIM does not decrease obviously, which proves that the fast guided filter smoothing loss can well preserve the image detail information and structural features; compared with the FR-Net network, the SSIM of the FGR-Net network does not improve, and the image MSE and std values are increased by 19.81% and 17.55% respectively, which proves the effectiveness of adaptive Gamma coding; the improved Retinex-Net adds the V component fusion module on the basis of FGR-Net, the image brightness decreases slightly, but the E, PNSR and SSIM values are increased by 3.04%, 7.63% and 4.55% respectively, which proves the effectiveness of V component fusion. It can be known from the above that the overall brightness of the image enhanced by the improved Retinex-Net network is obviously improved, but compared with the image enhanced by the traditional Retinex-Net network, the contrast of the image is still low and the target structural features are not clear enough, so the preliminary enhanced infrared image needs to be further processed, segmented and reconstructed.

[0101] 2) The overall performance of the enhancement method

[0102] Through experiments, it is known that compared with the traditional Retinex-Net, the image information entropy, mean value and standard deviation of the enhancement method are increased by 13.48%, 6.42% and 14.09% respectively, the peak signal-to-noise ratio and structural similarity are increased by 16.08% and 3.20% respectively, which can improve the image brightness, clearly preserve the target feature details, avoid color distortion and effectively improve the comprehensive quality of the image. The mAP value of the YOLO v3 model is increased by 2.31%, and the mAP value of the SSD model is increased by 1.46%, which provides favorable conditions for realizing power equipment situation awareness.

[0103] According to another aspect of the embodiment of the application, an infrared image enhancement device for power equipment based on an improved Retinex-Net is also provided, comprising:

[0104] A first obtaining module is configured to perform HSV color space conversion on the infrared image of the power equipment to obtain H, S and V three-channel components;

[0105] The second obtaining module is configured to obtain a preliminary enhanced infrared image based on an improved Retinex-Net network for H, S, and V three-channel components.

[0106] The third obtaining module is configured to perform superpixel segmentation and reconstruction on the preliminary enhanced infrared image to obtain a final enhanced infrared image.

[0107] For parts not detailed in the above modules, please refer to the related description of the embodiments.

[0108] The specific embodiments of the present application are described in detail above with reference to the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.

Claims

1. A method for enhancing infrared images of power equipment based on an improved Retinex-Net, characterized in that, include: S1. Perform HSV color space conversion on the infrared image of the power equipment to obtain the H, S, and V three-channel components; S2. Based on the improved Retinex-Net network, preliminary enhanced infrared images are obtained for the H, S, and V channel components. S3. Perform superpixel segmentation and reconstruction on the preliminary enhanced infrared image to obtain the final enhanced infrared image; S2 includes: S2.1 For the V component, the smoothing loss function of the original network is replaced by the fast guided filter smoothing loss function based on the traditional Retinex-Net network to obtain the V component after preliminary enhancement. S2.2, Correct and fuse the initially enhanced V component to obtain the enhanced V component; S2.

3. Merge the enhanced V component with the original H and S components to obtain a preliminary enhanced infrared image; S2.2 includes: performing adaptive Gamma correction on the initially enhanced V component and obtaining the enhanced V component based on VSM+WLS fusion processing; The S3 includes: firstly, segmenting the target image using MSRM and SEEDS superpixel segmentation methods to obtain the final segmented image; then, reconstructing the image using a multi-scale hybrid guided filtering algorithm to obtain the final enhanced image.

2. The infrared image enhancement method for power equipment based on improved Retinex-Net according to claim 1, characterized in that, The process also includes a verification step S4, which verifies the improved Retinex-Net network by setting up three modules for separate analysis. Experiments are designed based on the traditional Retinex-Net network: ① The network that only improves the loss function is denoted as FR-Net; ② The network that improves both the loss function and adds correction adjustment is denoted as FGR-Net; ③ A fusion module is added to FGR-Net, which is the improved Retinex-Net. The enhancement effect of the improved Retinex-Net is verified by quantitative evaluation indicators. Then, for the final enhancement method, the detection effect is verified using YOLO v3 and SSD object detection models.

3. The infrared image enhancement method for power equipment based on improved Retinex-Net according to claim 1, characterized in that, In the adaptive Gamma correction, the expression for the correction constant is: ； In the formula: ε is the correction constant; I(x,y) represents the gray value of a point in the image; μ and σ represent the mean and standard deviation of the image gray values, respectively; ε is the brightness threshold.

4. The infrared image enhancement method for power equipment based on improved Retinex-Net according to claim 3, characterized in that, First, the illumination coefficient of each pre-enhanced V component image is calculated using the grayscale mean, and the illumination coefficient distribution of all pre-enhanced V components is statistically analyzed. Then, the average illumination coefficient of all pre-enhanced V components is calculated, and the average value of the pixel with the highest probability of occurrence in the illumination component is obtained based on the average illumination coefficient, which is used as the brightness threshold ε.

5. The infrared image enhancement method for power equipment based on improved Retinex-Net according to claim 1, characterized in that, The VSM+WLS fusion processing is as follows: First, the original V component image is used as a supplementary image to the V component after adaptive Gamma correction; then, Gaussian filtering and iterative filtering are used sequentially to estimate the base layer and detail layer of the corrected V component image; the base layer of the corrected V component image is weighted and superimposed using VSM to calculate weights; then, WLS is used to superimpose each detail layer of the corrected V component image; finally, the processed base layer and detail layer are fused and reconstructed to obtain the enhanced V component image.

6. The infrared image enhancement method for power equipment based on improved Retinex-Net according to claim 1, characterized in that, The target device is segmented using a superpixel segmentation method based on MSRM and SEEDS. Specifically, firstly, superpixel pre-segmentation is performed using SEEDS. Then, the segmented superpixel blocks are labeled and divided according to the proposed labeling rules to achieve MSRM superpixel merging. Finally, morphological methods are used to process the merged superpixels to obtain the final segmented target image.

7. The infrared image enhancement method for power equipment based on improved Retinex-Net according to claim 1, characterized in that, For the segmented device, a multi-scale hybrid guided filtering algorithm is used to reconstruct the extracted target device to obtain the final enhanced image. Specifically, based on the initially enhanced infrared image and the final segmented target image, the image detail layer and base layer are first decomposed using multi-scale RGF, and a saliency map and weight map are obtained using filtering. Then, a weighted fusion method is used to fuse the weight map, base layer, and detail layer to obtain the fused base layer S. B and the blended detail layer S D Finally, regarding S D Perform global detail enhancements to obtain the enhanced S. D Based on the fusion of the basic layer S B and the enhanced S D The final fusion result S is obtained, which serves as the final enhanced infrared image.

Images