GIS equipment gas leakage detection method and system based on image fusion

Abstract

The invention provides a GIS equipment gas leakage detection method and system based on image fusion, and relates to the technical field of GIS gas leakage detection. The method comprises the following steps: firstly, obtaining an infrared image and a visible light image, carrying out image preprocessing, then carrying out feature transformation and image registration, inputting the registered image into a fusion generation model, carrying out image fusion, and finally inputting the fused image into a leakage detection model to obtain prediction frame coordinate information of actual gas leakage. The GIS equipment gas leakage detection is realized by combining information of infrared and visible light images and utilizing a deep learning model, and the detection accuracy and efficiency are improved by utilizing advanced image processing and deep learning technologies.

Description

Technical Field

[0001] The present invention relates to the field of GIS gas leakage detection, and in particular to a GIS equipment gas leakage detection method and system based on image fusion. Background Art

[0002] In the prior art, GIS equipment was introduced in the 1960s. GIS equipment encloses all substation equipment or components except the transformer in a metal grounded casing filled with SF6 gas. Since SF6 has very strong insulation and arc extinguishing capabilities, this enables GIS equipment to insulate between phases and relative to the ground, improving the safety and reliability of substation equipment operation and the convenience of operation and maintenance. At the same time, because GIS equipment has large capacity and small footprint, it is widely used in substation construction in large cities and industrially intensive areas.

[0003] Although GIS equipment using SF6 gas as the insulating medium has many advantages, there are still safety hazards. After long-term use, GIS equipment may cause SF6 gas leakage due to its own manufacturing defects or the life of the sealing material. At present, the detection methods of SF6 gas leakage mainly include qualitative leak detector leakage detection method, ultrasonic detection method and infrared imaging method. The qualitative leak detector leakage detection method has the disadvantages of low detection accuracy and high risk factor; the ultrasonic detection method has the disadvantages of poor stability and lack of intuitiveness; the infrared imaging method has the problems of poor image quality and low image contrast, and its image visualization is poor. Summary of the invention

[0004] The main purpose of the present invention is to provide a GIS equipment gas leakage detection method and system based on image fusion, so as to solve the technical problem that the existing SF6 leakage identification has poor accuracy in harsh environments.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a GIS equipment gas leakage detection method based on image fusion, comprising the following steps:

[0006] Step S1: Obtain the infrared image and the corresponding visible light image of the target GIS device in the current environment, and perform image preprocessing;

[0007] Step S2: performing feature transformation on the infrared image after the image preprocessing and the visible light image, and then performing image registration to obtain the registered infrared image;

[0008] Step S3: Building a fusion generation model based on the VGG16 network, inputting the registered infrared image and the visible light image into, and obtaining the fused image; wherein, three convolutional layers and two fully connected layers are sequentially connected after the feature concatenation layer of the VGG16 network;

[0009] Step S4: Build a leakage detection model based on YOLOv5, input the fused image into the leakage detection model, and obtain the predicted frame coordinate information of the actual SF6 leakage area.

[0010] In the preferred embodiment, step S2 specifically includes the following steps:

[0011] S201: normalizing the infrared image and the visible light image, and performing edge detection on the two images using a Canny operator to extract the main contours of the images;

[0012] S202: using the SURF algorithm to detect and extract corner points in the main contour, and using the Hessian matrix to extract feature points of the image contour;

[0013] S203: using the Euclidean distance formula to match feature points between the two images;

[0014] S204: Fitting the matched feature points using the least square method to obtain the optimal registered infrared image.

[0015] In the preferred embodiment, step S3 specifically includes the following steps:

[0016] S301: inputting the infrared image and the corresponding visible light image into the fusion generation model;

[0017] S302: After performing convolution processing and maximum pooling respectively, feature splicing is performed;

[0018] S303: After feature concatenation, multiple convolution operations with different convolution kernels are performed, and then two different fully connected layers are concatenated;

[0019] S304: The fusion generates a model output, and obtains weight coefficients α and β corresponding to the infrared image and the visible light phase image;

[0020] S305: Using α and β, extract the pixel values ​​of the infrared image and the visible light image and perform linear weighting to obtain a fused image.

[0021] In the preferred solution, multiple convolution operations with different convolution kernels in the fusion generation model are performed, and the activation function of the convolution layer adopts ReLU;

[0022] Two different fully connected layers, with activation functions of ReLU or Sigmoid respectively;

[0023] The fusion generation model uses mean square error as the loss function and adopts Adam optimizer for model training.

[0024] In a preferred embodiment, the leakage detection model in step S4 is composed of a feature extraction network, a feature fusion network and a prediction network;

[0025] Feature extraction network, used to extract features of the input image from shallow to deep layers with multiple network depths;

[0026] The feature fusion network is a combination of FPN+PANet, which is used to fuse the feature maps extracted by the feature extraction network;

[0027] The prediction network is used to generate multiple feature maps of different scales based on the feature map after the input feature fusion, so as to detect objects of different sizes. The feature map contains the object position, category and confidence information.

[0028] In a preferred embodiment, the prediction network generates a plurality of feature maps of different scales, and further includes: performing post-processing operations of non-maximum suppression and confidence threshold filtering on the feature maps.

[0029] In the preferred solution, the data preprocessing comprises the following steps: based on the principle that the processing methods of the infrared image and the corresponding visible light image are consistent, randomly selecting a preset ratio of infrared images for horizontal flipping and vertical flipping; then randomly selecting a preset ratio of visible light images for translation and random rotation; and finally performing Mosaic data enhancement on the two;

[0030] The corresponding visible light image is processed in the same manner as the infrared image to maintain the corresponding relationship.

[0031] In a preferred embodiment, the leakage detection model further includes performing model training, including the following steps:

[0032] Obtain a dataset of infrared images and corresponding visible light images;

[0033] Dividing the data set into a training set and a validation set according to a preset ratio, and performing data preprocessing;

[0034] Set the number of iterations for training;

[0035] Inputting the training set after the data preprocessing into the leakage detection model for iterative training until the iterative training times are reached, and saving the optimal leakage detection model;

[0036] The iterative training includes forward prediction by the model, predicting the target detection box and category information, calculating the loss function, and back-propagating to optimize the leakage detection model;

[0037] The output of the leakage detection model includes: the position coordinates and confidence of the feature map detection box;

[0038] The loss function of the leakage detection model adopts the CIoU loss function.

[0039] A GIS equipment gas leakage detection system based on image fusion, including the following modules:

[0040] The image acquisition module is used to obtain the infrared image and the corresponding visible light image of the target GIS device in the current environment and perform image preprocessing;

[0041] An image registration module is used to register the infrared image after the image preprocessing and the visible light image acquisition module after image feature transformation to obtain the registered infrared image;

[0042] A fusion generation module is used to build a fusion generation model based on the VGG16 network, input the registered infrared image and the visible light image into, and obtain the fused image; wherein, three convolutional layers and two fully connected layers are sequentially connected after the feature concatenation layer of the VGG16 network;

[0043] The leakage detection module is used to build a leakage detection model based on YOLOv5, input the fused image into the leakage detection model, and obtain the predicted frame coordinate information of the actual SF6 leakage area.

[0044] An electronic device comprises: a processor, a memory and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, a gas leakage detection method for GIS equipment based on image fusion as described above is implemented.

[0045] A computer-readable storage medium comprises: a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for detecting gas leakage of GIS equipment based on image fusion as described above is implemented.

[0046] The present invention provides a GIS equipment gas leakage detection method and system based on image fusion. By acquiring an infrared image and a corresponding visible light image of a GIS equipment with SF6 leakage in the current environment, the input infrared image and the visible light image are registered to obtain an optimal registered infrared image. A fusion generation model is used to fuse the visible light image and the registered infrared image to obtain a high-quality fused image. The SF6 leakage detection model then detects effective information of the fused image, making full use of the image information of the two modes, realizing the positioning of the SF6 leakage area of ​​the GIS equipment, and ensuring that a relatively accurate positioning of the SF6 leakage area can be obtained even in harsh environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0048] Figure 1 It is a flow chart of the gas detection method of GIS equipment of the present invention;

[0049] Figure 2 It is a schematic diagram of the network structure of the fusion generation model of the present invention;

[0050] Figure 3 It is a network architecture diagram of the gas leakage detection model of GIS equipment of the present invention;

[0051] Figure 4 It is a structural diagram of the gas leakage detection system of GIS equipment of the present invention;

[0052] Figure 5 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, a first sample infrared image and a second sample infrared image are used to distinguish sample infrared images for training different models rather than to describe a specific order of the sample infrared images.

[0055] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0056] First, the technical terms involved in the embodiments of the present application are introduced.

[0057] Infrared imaging technology: Infrared imaging technology is a technology that uses infrared radiation emitted by objects themselves to generate images. It can capture infrared radiation generated by different objects or different parts of objects due to temperature differences, and convert it into a visual thermal image. The technology is suitable for imaging needs at night or in low-light environments.

[0058] Visible light imaging technology: Visible light imaging technology is a technology that collects and processes images based on light within the spectral range that the human eye can perceive (approximately 400 to 700 nanometers in wavelength). It captures visible light reflected or emitted by the scene through an optical lens and converts it into electronic signals to form a clear image.

[0059] Deep learning is a machine learning method that imitates the working principle of the human brain neural network. It automatically learns and extracts complex features from data by building a multi-level neural network model. The method is particularly good at processing data with highly nonlinear and abstract characteristics, such as images, sounds, and text. The key to deep learning lies in "depth", that is, the number of network layers is large, and each layer can learn features at different levels of abstraction from the input data. The hierarchical structure enables the deep learning model to automatically learn high-level abstract concepts from the raw data without the need for artificially designed features, and can achieve accurate positioning of the SF6 leakage area. Combined with traditional geometric methods, such as plane geometry and space geometry, it can effectively solve problems such as posture estimation and scale transformation in the positioning process.

[0060] Object detection is an important research direction in the field of computer vision. Its main task is to identify and locate one or more objects of interest in a given image or video frame, automatically detect the location of the object through an algorithm, and classify it. Object detection usually includes two core steps: one is positioning (bounding box), that is, determining the exact location of the object in the image; the other is classification, that is, identifying which category the object belongs to.

[0061] Visible light and infrared image fusion is a multimodal image processing technology that aims to combine the rich texture and color information of visible light images with the thermal radiation characteristics of infrared images to generate a new image that has the advantages of both images. Image fusion technology can enhance image details and improve scene understanding capabilities. By using different image fusion algorithms, such as wavelet transform-based methods, frequency domain fusion methods, or deep learning technologies, information from visible light and infrared sensors can be effectively integrated to achieve more accurate and robust target recognition and environmental perception.

[0062] Abbreviations used in this embodiment: VGG16, namely Visual Geometry Group 16-layer network architecture, is designed to solve image recognition problems; YOLOv5 is a novel convolutional neural network (CNN), which is a target detection algorithm; GIS is short for Geographic Information Science; SF6 is sulfur hexafluoride, an inorganic compound; SURF (Speeded Up Robust Features) is a robust image recognition and description algorithm; Canny operator is short for Canny edge detection operator, which is a multi-level edge detection algorithm developed by John F. Canny in 1986; FPN and PANet are two commonly used technologies in the field of object detection, representing Feature Pyramid Networks and Path Aggregation Network, respectively.

[0063] Next, the technical solutions provided in the embodiments of the present application are introduced.

[0064] Example 1

[0065] like Figure 1-5 As shown, a GIS equipment gas leakage detection method based on image fusion includes the following steps:

[0066] Step S1: Obtain the infrared image and the corresponding visible light image of the target GIS device in the current environment, and perform image preprocessing.

[0067] Step S2: After performing feature transformation on the infrared image and the visible light image after image preprocessing, image registration is performed to obtain a registered infrared image.

[0068] Step S3: Build a fusion generation model based on the VGG16 network, input the registered infrared image and the visible light image into, and obtain a fused image; wherein, three convolutional layers and two fully connected layers are sequentially connected after the feature concatenation layer of the VGG16 network.

[0069] Step S4: Build a leakage detection model based on YOLOv5, input the fused image into the leakage detection model, and obtain the predicted box coordinate information of the actual SF6 leakage area.

[0070] This embodiment first obtains infrared images and visible light images and performs image preprocessing, then performs feature transformation and image registration, inputs the registered images into a fusion generation model, performs image fusion, and finally inputs the fused images into a leakage detection model to obtain the predicted frame coordinate information of the actual gas leakage; by combining the information of infrared and visible light images, a deep learning model is used to realize the detection of gas leakage in GIS equipment, and advanced image processing and deep learning technologies are used to improve the accuracy and efficiency of detection.

[0071] In this embodiment, the detection is performed on SF6 leakage.

[0072] By obtaining the infrared image and the corresponding visible light image of the GIS equipment with SF6 leakage in the current environment, the input infrared image and the visible light image are registered to obtain the optimal registered infrared image. The fusion generation model is used to fuse the visible light image and the registered infrared image to obtain a high-quality fused image. The SF6 leakage detection model then detects the effective information of the fused image, making full use of the image information of the two modes to realize the positioning of the SF6 leakage area of ​​the GIS equipment, ensuring that a relatively accurate positioning of the SF6 leakage area can be obtained even in harsh environments.

[0073] Specifically, a dual-lens camera (visible light lens on the left, infrared lens on the right, two lenses with parallel optical axes and fixed positions) can be used to collect visible light images and corresponding infrared images of the target GIS equipment SF6 leakage in the current environment. Considering that the positions of infrared thermal imaging and visible light lens shooting are different, resulting in differences in field of view, it is necessary to first perform image registration through feature transformation before image fusion, and then apply the fusion generation model to combine the advantages of infrared image and visible light image to generate a high-definition fused image. On this basis, the SF6 leakage detection model can perform target detection on the fused image and obtain the location of the target SF6 leakage area.

[0074] Furthermore, in this embodiment, the sample registered infrared image and the sample visible light image, and the sample fusion image are respectively high-quality sample images used to train different models, and can be acquired by using a dual-lens camera and performing feature transformation.

[0075] In the preferred scheme, data preprocessing is performed, including the following steps: based on the principle of keeping the processing method of infrared images and corresponding visible light images consistent, randomly select infrared images of a preset ratio for horizontal flipping and vertical flipping; then randomly select visible light images of a preset ratio for translation and random rotation; finally, the two are subjected to Mosaic data enhancement.

[0076] The corresponding visible light image is processed in the same way as the infrared image to maintain the corresponding relationship.

[0077] Furthermore, the parameter information of SF6 leakage of GIS equipment is obtained by means of web crawlers and manual photography. In order to improve the robustness of the SF6 leakage detection model, the Labelimg tool is used to manually annotate the SF6 leakage targets, annotate the SF6 leakage detection frame of the infrared image, and generate label files respectively.

[0078] In the preferred embodiment, step S2 specifically includes the following steps:

[0079] S201: normalize the infrared image and the visible light image, and use the Canny operator to perform edge detection on the two images to extract the main contours of the images.

[0080] S202: Use the SURF algorithm to detect and extract corner points in the main contour, and use the Hessian matrix to extract feature points of the image contour.

[0081] S203: Use the Euclidean distance formula to match feature points between the two images.

[0082] S204: Fitting the matched feature points using the least square method to obtain the optimal registered infrared image.

[0083] The least squares method is used to fit the matched feature points to obtain the optimal registration transformation. This fitting usually involves finding a transformation matrix (such as affine transformation or perspective transformation) so that the feature points of the two images can correspond geometrically. The least squares method can effectively optimize the accuracy of the matching points, making the registered images more consistent in space.

[0084] The above steps S201-S204 realize the feature transformation and registration of the infrared image and the visible light image, ensuring that the subsequent leakage detection model based on the fused image can accurately and comprehensively utilize the image information of different bands.

[0085] In the preferred embodiment, step S3 specifically includes the following steps:

[0086] S301: Inputting the infrared image and the corresponding visible light image into a fusion generation model;

[0087] S302: After performing convolution processing and maximum pooling respectively, feature splicing is performed;

[0088] S303: After feature concatenation, multiple convolution operations with different convolution kernels are performed, and then two different fully connected layers are concatenated;

[0089] S304: Fusing the generated model output and obtaining weight coefficients α and β corresponding to the infrared image and the visible light phase image;

[0090] S305: Using α and β, extract the pixel values ​​of the infrared image and the visible light image and perform linear weighting to obtain a fused image.

[0091] In this embodiment, convolution operations are performed on the infrared image and the visible light image respectively to extract preliminary features, including information about the structure, edge and texture of the image.

[0092] The source of the data set in this embodiment, that is, the specific steps of obtaining the SF6 leakage image and image registration can be:

[0093] (1) Use a dual-lens camera to capture infrared images and corresponding visible light images during the day;

[0094] (2) Use feature transformation to register the infrared image and the corresponding visible light image;

[0095] (3) Use the Labelimg tool to manually annotate the GIS equipment with real SF6 leakage and create the corresponding data set.

[0096] like Figure 2 As shown in the figure, the convolutional feature map is input into the VGG16 network to further extract complex features to capture the deep information of the image; then the weight coefficients α and β of the infrared image and the visible light image are predicted; the pixel values ​​of the infrared image and the visible light image are linearly weighted using the predicted weight coefficients to generate a fused image. The formula is:

[0097] Fusion image = α*infrared image + β*visible light image.

[0098] In the preferred embodiment, multiple convolution operations with different convolution kernels are fused to generate the model, and the convolution layer activation function uses ReLU.

[0099] Two different fully connected layers, the activation functions are ReLU or Sigmoid respectively.

[0100] like Figure 2 As shown, a VGG16 network architecture is adopted in the fusion generation model of this embodiment, and infrared images and visible light images of preset sizes are input for feature extraction.

[0101] The specific operation examples are as follows:

[0102] A pair of 224x224x3 infrared and visible light images are input into the fusion generation model. The images are normalized and the image pixel values ​​are scaled from [0,255] to [0,1]. To adapt to the model input, the image dimensions are expanded from [224,224,3] to [1,224,224,3], where 1 represents the batch size. The pre-trained VGG16 model is loaded and the last fully connected layer is removed.

[0103] The infrared image and visible light image are input into the VGG16 model respectively to extract their feature representations.

[0104] First, the block5_pool output of VGG16 is used as the feature representation, and the dimension of the output feature map is [1, 7, 7, 512]. The feature maps of infrared and visible light are concatenated together to form a tensor of [1, 14, 7, 512].

[0105] Multiple convolutional layers are used to further process the concatenated feature maps. Convolutional layer 1: 64 3x3 convolution kernels, activation function is ReLU. Convolutional layer 2: 128 3x3 convolution kernels, activation function is ReLU. Convolutional layer 3: 64 3x3 convolution kernels, activation function is ReLU.

[0106] Then add two fully connected layers to predict α and β. Fully connected layer 1: 512 neurons, activation function is ReLU. Fully connected layer 2: 2 neurons, activation function is Sigmoid, ensure that the output is in the range of [0,1] and ensure that α+β=1.

[0107] In the preferred solution, the fusion generation model uses mean square error (MSE) as the loss function to calculate the difference between the predicted α and β and the true labeled α and β; and uses the Adam optimizer for model training.

[0108] Through the above steps, the fusion generation model uses the data set to obtain the best model after model training, inputs the visible light image to be tested and the infrared image and the best weight coefficients α and β, extracts the pixel values ​​of the infrared image and the visible light image for linear weighting, and fuses the infrared and visible light images to maximize the use of the information of the two images to obtain a higher quality fusion image, thereby providing a reliable foundation for subsequent leakage detection.

[0109] In the preferred embodiment, the leakage detection model in step S4 is composed of a feature extraction network, a feature fusion network and a prediction network.

[0110] The feature extraction network is used to extract features of the input image from shallow to deep layers with multiple network depths.

[0111] The feature fusion network is a combination of FPN+PANet, which is used to fuse the feature maps extracted by the feature extraction network.

[0112] The prediction network is used to generate multiple feature maps of different scales based on the feature map after the input feature fusion, which is used to detect targets of different sizes. The feature map contains the object location, category and confidence information.

[0113] In this embodiment, the fused image is input into the SF6 leakage detection model, which first uses a feature extraction network to extract feature information in the image and obtains outputs of three different scale features. The feature fusion network is used to capture information of different scales to enhance the receptive field of the model. Finally, the output of the feature fusion network is sent to the prediction network for target classification and bounding box regression. The prediction box with the highest confidence is retained through the non-maximum suppression (NMS) method to complete the entire target detection process and obtain the prediction box coordinate information of the SF6 leakage area.

[0114] In addition, the detection module uses three detection heads, each of which is responsible for detecting large, medium, and small objects, covering target objects of different sizes and ensuring the comprehensiveness of detection. The loss function is the CIoU loss function, which not only considers the intersection over union (IoU) between the predicted box and the true box, but also considers the distance between the center points of the two boxes and their aspect ratio, in order to optimize the prediction accuracy of the bounding box.

[0115] In the preferred embodiment, the prediction network generates multiple feature maps of different scales, and also includes: post-processing operations of non-maximum suppression and confidence threshold filtering on the feature maps.

[0116] It should be noted that the SF6 gas leakage infrared image obtained in the experiment is first input into the feature extraction network, and the features of the input image are extracted layer by layer. Due to the different functions of features at different levels, high-level features contain more semantic information, which is conducive to the distinction between SF6 gas and background, while low-level features contain richer spatial information, which is essential for the positioning of SF6 gas. Therefore, the feature extraction network retains high, medium and low features as outputs. Then, the three different levels of features are input into the feature fusion network, and the "FPN+PANet" structure is used to fuse the features at different levels. Finally, the fused features are output to the prediction network, and the prediction network generates three feature maps of different scales. These three feature maps correspond to different receptive field sizes and can be used to detect targets of different sizes. Specifically, these feature maps will generate outputs containing object location, category and confidence information through additional convolutional layers, which are used to determine the location and category of objects in the image. Finally, redundant detection frames are removed by methods such as non-maximum suppression, and only the most likely real object detection frames are retained. The detection module locates the SF6 leakage area based on the leakage characteristics. The detection module uses three detection heads, each of which is responsible for detecting large-sized, medium-sized and small-sized targets, covering target objects of different sizes and ensuring the comprehensiveness of the detection.

[0117] like Figure 3As shown, the YOLOv5 network architecture used in this embodiment not only reduces redundant information, but also effectively accelerates the convergence speed of the network. Experimental results show that the YOLOv5 architecture significantly improves the performance of infrared target detection and solves the problem of poor accuracy of existing SF6 leakage identification technology in harsh environments.

[0118] In this embodiment, the position coordinates of the detection frame include x and y coordinates indicating the center position of the frame, as well as the width and height; and the confidence level indicates the possibility that the detection frame contains the target SF6 leakage.

[0119] Based on any of the above embodiments, the loss function of the SF6 leakage detection model is a CIoU loss function, which not only considers the intersection over union (IoU) of the predicted box and the true box, but also considers the distance between the center points of the two boxes and the aspect ratio of the two boxes, so as to optimize the prediction accuracy of the bounding box.

[0120] In order to achieve higher detection accuracy, the leak detection model needs to be trained.

[0121] In a preferred embodiment, the leakage detection model further includes performing model training, including the following steps:

[0122] Get a dataset of infrared images and corresponding visible light images.

[0123] The data set is divided into training set and validation set according to the preset ratio, and data preprocessing is performed.

[0124] Set the number of training iterations.

[0125] The training set after data preprocessing is input into the leakage detection model for iterative training until the number of iterative training times is reached and the optimal leakage detection model is saved.

[0126] Among them, iterative training includes the model making forward predictions, predicting the target detection box and category information, then calculating the loss function, and backpropagating to optimize the leakage detection model.

[0127] The output of the leakage detection model includes: the location coordinates and confidence of the feature map detection box.

[0128] The loss function of the leakage detection model adopts the CIoU (Complete-IoU) loss function.

[0129] In this embodiment, the data set is divided into a preset ratio of 8:2, and infrared images and corresponding visible light images are collected as training data, and the infrared images and visible light images correspond to each other. The collected images are preprocessed, including image resizing, normalization, image registration, etc. The infrared images and visible light images are paired, and a data set is created so that each sample contains a pair of registered infrared images and visible light images.

[0130] The model is trained using the preprocessed training dataset. In each training iteration, the model accepts a pair of infrared and visible light images as input, calculates the loss function, and updates the model parameters through the back propagation algorithm. The training process continues until the model performance reaches a predetermined threshold. The input of the model is the registered infrared image, and the output is the object detection box and category information.

[0131] It should be noted that in order to achieve higher detection accuracy, CIoU is used in the YOLOv5 algorithm to optimize the regression method of the target frame. This method optimizes the "intersection-over-union ratio" in the detection results, that is, the ratio of the intersection and union of the predicted frame and the true frame in the detection results under a certain IoU threshold. CIoU takes into account factors such as the distance and overlap rate between the SF6 gas to be tested and the prior frame, determines the degree of overlap between the true frame and the predicted frame, and makes the target frame regression more stable, thereby avoiding problems such as non-convergence during training, and can obtain the precise position of the SF6 leak in the world coordinate system, thereby achieving high-precision SF6 leak positioning.

[0132] During training, the previously prepared labeled dataset is used to supervise the learning process and optimize the model parameters to minimize the gap between the predicted weights and the true labels.

[0133] Example 2

[0134] Further illustrate with reference to Example 1, Figure 4 As shown, a GIS equipment gas leakage detection system based on image fusion is provided, which includes the following modules:

[0135] The image acquisition module is used to obtain the infrared image and the corresponding visible light image of the target GIS device in the current environment and perform image preprocessing.

[0136] The image registration module is used to register the infrared image and the visible light image after image preprocessing, and obtain the registered infrared image after the image feature transformation.

[0137] The fusion generation module is used to build a fusion generation model based on the VGG16 network, and input the registered infrared image and the visible light image to obtain a fused image; wherein, three convolutional layers and two fully connected layers are sequentially connected after the feature concatenation layer of the VGG16 network.

[0138] The leakage detection module is used to build a leakage detection model based on YOLOv5, input the fused image into the leakage detection model, and obtain the predicted box coordinate information of the actual SF6 leakage area.

[0139] An electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, a gas leakage detection method for GIS equipment based on image fusion as in Example 1 is implemented.

[0140] A computer-readable storage medium includes: a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, a gas leakage detection method for GIS equipment based on image fusion as in Example 1 is implemented.

[0141] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A GIS equipment gas leakage detection method based on image fusion, characterized in that: The following steps are involved: Step S1: Obtain the infrared image and the corresponding visible light image of the target GIS device in the current environment, and perform image preprocessing; Step S2: performing feature transformation on the infrared image after the image preprocessing and the visible light image, and then performing image registration to obtain the registered infrared image; Step S3: Building a fusion generation model based on the VGG16 network, inputting the registered infrared image and the visible light image into, and obtaining the fused image; wherein, three convolutional layers and two fully connected layers are sequentially connected after the feature concatenation layer of the VGG16 network; Step S4: Build a leakage detection model based on YOLOv5, input the fused image into the leakage detection model, and obtain the predicted frame coordinate information of the actual SF6 leakage area.

2. The GIS equipment gas leakage detection method based on image fusion according to claim 1 is characterized in that: Step S2 specifically includes the following steps: S201: normalizing the infrared image and the visible light image, and performing edge detection on the two images using a Canny operator to extract the main contours of the images; S202: using the SURF algorithm to detect and extract corner points in the main contour, and using the Hessian matrix to extract feature points of the image contour; S203: using the Euclidean distance formula to match feature points between the two images; S204: Fitting the matched feature points using the least square method to obtain the optimal registered infrared image.

3. The GIS equipment gas leakage detection method based on image fusion according to claim 1 is characterized in that: Step S3 specifically includes the following steps: S301: inputting the infrared image and the corresponding visible light image into the fusion generation model; S302: After performing convolution processing and maximum pooling respectively, feature splicing is performed; S303: After feature concatenation, multiple convolution operations with different convolution kernels are performed, and then two different fully connected layers are concatenated; S304: The fusion generates a model output, and obtains weight coefficients α and β corresponding to the infrared image and the visible light phase image; S305: Using α and β, extract the pixel values ​​of the infrared image and the visible light image and perform linear weighting to obtain a fused image.

4. The GIS equipment gas leakage detection method based on image fusion according to claim 3 is characterized in that: The convolution operation of multiple different convolution kernels in the fusion generation model uses ReLU as the activation function of the convolution layer; Two different fully connected layers, with activation functions of ReLU or Sigmoid respectively; The fusion generation model uses mean square error as the loss function and adopts Adam optimizer for model training.

5. The GIS equipment gas leakage detection method based on image fusion according to claim 1 is characterized in that: The leakage detection model in step S4 is composed of a feature extraction network, a feature fusion network and a prediction network; Feature extraction network, used to extract features from the input image at multiple network depths from shallow to deep layers; The feature fusion network is a combination of FPN+PANet, which is used to fuse the feature maps extracted by the feature extraction network; The prediction network is used to generate multiple feature maps of different scales based on the feature map after the input feature fusion, so as to detect objects of different sizes. The feature map contains the object position, category and confidence information.

6. The GIS equipment gas leakage detection method based on image fusion according to claim 5 is characterized in that: The prediction network generates a plurality of feature maps of different scales, and also includes: performing post-processing operations of non-maximum suppression and confidence threshold filtering on the feature maps.

7. The GIS equipment gas leakage detection method based on image fusion according to claim 6 is characterized in that: The data preprocessing comprises the following steps: based on the principle that the processing methods of the infrared image and the corresponding visible light image are consistent, randomly selecting a preset proportion of the infrared image to perform horizontal flipping and vertical flipping; then randomly selecting a preset proportion of the visible light image to perform translation and random rotation; and finally performing Mosaic data enhancement on the two; The corresponding visible light image is processed in the same manner as the infrared image to maintain the corresponding relationship.

8. The GIS equipment gas leakage detection method based on image fusion according to claim 7 is characterized in that: The leakage detection model also includes model training, including the following steps: Obtain a dataset of infrared images and corresponding visible light images; Dividing the data set into a training set and a validation set according to a preset ratio, and performing data preprocessing; Set the number of iterations for training; Inputting the training set after the data preprocessing into the leakage detection model for iterative training until the iterative training times are reached, and saving the optimal leakage detection model; The iterative training includes forward prediction by the model, predicting the target detection box and category information, calculating the loss function, and back propagating to optimize the leakage detection model; The output of the leakage detection model includes: the position coordinates and confidence of the feature map detection box; The loss function of the leakage detection model adopts the CIoU loss function.

9. A GIS equipment gas leakage detection system based on image fusion, characterized in that: Includes the following modules: The image acquisition module is used to obtain the infrared image and the corresponding visible light image of the target GIS device in the current environment and perform image preprocessing; An image registration module is used to register the infrared image after the image preprocessing and the visible light image acquisition module after image feature transformation to obtain the registered infrared image; A fusion generation module is used to build a fusion generation model based on the VGG16 network, input the registered infrared image and the visible light image into, and obtain the fused image; wherein, three convolutional layers and two fully connected layers are sequentially connected after the feature concatenation layer of the VGG16 network; The leakage detection module is used to build a leakage detection model based on YOLOv5, input the fused image into the leakage detection model, and obtain the predicted frame coordinate information of the actual SF6 leakage area.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the GIS equipment gas leakage detection method based on image fusion as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: include: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting gas leakage of GIS equipment based on image fusion as described in any one of claims 1 to 8 is implemented.

Images