Cable anomaly detection method based on visible light and infrared image fusion

Abstract

The invention discloses a cable anomaly detection method based on visible light and infrared image fusion, and the method comprises the steps: obtaining a multi-modal image of a cable, and the multi-modal image comprises a visible light image and an infrared thermal image; fusing the multi-modal image based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image; training a multiple linear regression model according to the cable fusion image to predict the fault type and position of the cable; and acquiring multi-modal image data of the cable in real time, and inputting the multi-modal image data into the trained multiple linear regression model for cable fault prediction. According to the invention, the technical problem of insufficient cable fault type accuracy in the background technology is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of fault diagnosis, and in particular to a cable anomaly detection method based on the fusion of visible light and infrared images. Background Art

[0002] With the rapid development of my country's economy and the acceleration of urbanization, cable laying has gradually shifted from open-air to cable tunnels to improve the aesthetics of the city and the stability of the power system. However, due to environmental factors, aging, manufacturing defects and other reasons, cables may experience various abnormal conditions during use, such as local overheating and insulation damage. These problems not only affect the normal operation of the power system, but may also cause serious safety accidents. At present, for the health monitoring and maintenance of cables, commonly used detection methods include manual inspections and online monitoring systems. Among them, manual inspections rely on the experience and judgment of staff, which is inefficient and easy to miss potential hidden dangers. Most of the existing online monitoring systems only use a single type of sensor and cannot comprehensively and accurately evaluate the status of the cable. In recent years, with the development of imaging technology and data analysis methods, multimodal detection methods combining visible light images and infrared thermal images have gradually become a research hotspot. However, existing image fusion technologies often rely on traditional image processing methods, such as histogram matching and pixel-level weighted averaging. These methods have deficiencies in feature extraction and information fusion, resulting in poor final detection results and inability to accurately predict the type and location of cable faults. Summary of the invention

[0003] The main purpose of the present invention is to provide a cable anomaly detection method based on the fusion of visible light and infrared images, aiming to solve the technical problem of insufficient accuracy of cable fault detection in the background technology.

[0004] To achieve the above object, the present invention provides a first aspect of a cable anomaly detection method based on visible light and infrared image fusion, which is characterized by comprising the following steps:

[0005] Step S100: acquiring a multimodal image of the cable, wherein the multimodal image includes a visible light image and an infrared thermal image;

[0006] Step S200: fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image;

[0007] Step S300: training a multivariate linear regression model based on the cable fusion image to predict the cable fault type and location;

[0008] Step S400: Collect multimodal image data of the cable in real time and input the trained multivariate linear regression model to predict cable faults.

[0009] Optionally, in step S200, the step of fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image includes:

[0010] Step S210: fusing the multimodal images based on a wavelet analysis algorithm to obtain a first fused image, and fusing the multimodal images based on a deep learning algorithm to obtain a second fused image;

[0011] Step S220: performing secondary fusion on the first fused image and the second fused image to obtain the cable fused image.

[0012] Optionally, in step S210, the step of fusing the multimodal images based on a wavelet analysis algorithm to obtain a first fused image includes:

[0013] Performing wavelet decomposition on the multimodal image to extract low-frequency coefficients and high-frequency coefficients at different scales;

[0014] The low-frequency coefficients are fused using a fusion rule based on image edge information to generate low-frequency fusion coefficients;

[0015] The high frequency coefficients are fused using a fusion rule based on a combination of a window and a fractal to generate a high frequency fusion coefficient;

[0016] The low-frequency fusion coefficient and the high-frequency fusion coefficient are recombined through inverse wavelet transform to generate a first fused image.

[0017] Optionally, in step S210, the step of fusing the multimodal images based on a deep learning algorithm to obtain a second fused image includes:

[0018] Using a deep learning algorithm to perform multi-scale decomposition on the multimodal image to generate high-frequency sub-band information and low-frequency sub-band information;

[0019] The high frequency sub-band information is fused, and the fused high frequency sub-band information is determined by selecting the maximum feature in the neighborhood;

[0020] Fusing the low-frequency sub-band information, by dividing the low-frequency sub-band into small blocks and calculating an average value to determine the fused low-frequency sub-band information;

[0021] Through inverse transformation, an image is reconstructed according to the fused high-frequency sub-band information and the fused low-frequency sub-band information to generate a second fused image.

[0022] Optionally, in step S220, the step of performing secondary fusion on the first fused image and the second fused image to obtain the cable fused image includes:

[0023] Performing quality assessment on the first fused image and the second fused image respectively to obtain a first quality score and a second quality score;

[0024] Determine whether the first quality score is higher than the second quality score, if so, select the wavelet analysis algorithm as the secondary fusion algorithm, otherwise, select the deep learning algorithm as the secondary fusion algorithm;

[0025] The first fused image and the second fused image are fused for a second time using the secondary fusion algorithm to generate the cable fused image.

[0026] Optionally, in step S300, the step of establishing a multivariate linear regression model according to the cable fusion image includes:

[0027] Step S310: acquiring historical data of the cable fusion image and constructing an augmented matrix, and calculating a cross product matrix from the 1st to the Tth moment according to the augmented matrix;

[0028] Step S320: Calculate the eigenvalues ​​and eigenvectors from the 1st to the Tth time according to the cross product matrix from the 1st to the Tth time;

[0029] Step S330: Based on the eigenvalues ​​at the first to Tth moments, a time series analysis method is used to predict the eigenvalue at the T+1th moment, and based on the eigenvectors at the first to Tth moments, an orthogonal matrix prediction method is used to predict the eigenvector at the T+1th moment;

[0030] Step S340: Calculate the cross product matrix at the T+1th moment according to the eigenvalue and eigenvector at the T+1th moment, and calculate the estimated value of the regression parameter at the T+1th moment based on the cross product matrix at the T+1th moment;

[0031] Step S350: construct a multivariate linear regression model based on the estimated values ​​of the regression parameters at time T+1.

[0032] Optionally, in step S340, the method for calculating the estimated value of the regression parameter at the T+1th moment based on the cross product matrix at the T+1th moment is the least squares method.

[0033] Optionally, after step S100, a robust Gaussian filter is applied to perform denoising on the multimodal image.

[0034] A second aspect of the present invention provides a cable anomaly detection system based on visible light and infrared image fusion, comprising:

[0035] An acquisition module, used for acquiring a multimodal image of the cable, wherein the multimodal image includes a visible light image and an infrared thermal image;

[0036] A fusion module, used for fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image;

[0037] A training module, used for training a multivariate linear regression model according to the cable fusion image, so as to predict the type and location of the cable fault;

[0038] The prediction module is used to collect multimodal image data of cables in real time and input the trained multivariate linear regression model to predict cable faults.

[0039] The third aspect of the present invention provides a device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, the cable anomaly detection method based on visible light and infrared image fusion as described in the first aspect of the present invention is implemented.

[0040] The fourth aspect of the present invention provides a computer-readable storage medium, on which is stored a cable anomaly detection program based on the fusion of visible light and infrared images. When the cable anomaly detection program based on the fusion of visible light and infrared images is executed by a processor, the steps of the cable anomaly detection method based on the fusion of visible light and infrared images as described in the first aspect are implemented.

[0041] Compared with the prior art, the beneficial effects of the present invention include at least:

[0042] The present invention provides a cable anomaly detection method based on the fusion of visible light and infrared images. By combining visible light images and infrared thermal images and using wavelet analysis algorithms and deep learning algorithms to perform multimodal data fusion, the visual and thermal distribution information of the cable can be fully captured, and the accuracy of cable anomaly detection is significantly improved. The wavelet analysis algorithm can extract multi-scale features of the image, and the deep learning algorithm can learn complex image features. The combination of the two enables the fused image to better reflect the actual situation of the cable, improves the accuracy of anomaly detection, reduces data processing time, and enables cable anomaly detection to achieve real-time monitoring, and timely discover and warn of cable anomalies. By constructing a multivariate linear regression model and using historical data to train the model, the abnormal situation of the cable can be accurately predicted, the accuracy and reliability of fault prediction are improved, the problem of inaccurate fault prediction in traditional monitoring methods is solved, and the safe and stable operation of the power system is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without creative work. The purpose, functional features and advantages of the present invention will be further explained in combination with the embodiments and with reference to the drawings.

[0044] Figure 1 A flow chart of a cable anomaly detection method based on visible light and infrared image fusion provided by an embodiment of the present invention;

[0045] Figure 2 A flow chart of a method for fusing infrared images and visible light images based on a wavelet analysis algorithm and a deep learning algorithm provided in an embodiment of the present invention;

[0046] Figure 3 A flow chart of a method for fusing infrared images and visible light images based on a wavelet analysis algorithm provided in an embodiment of the present invention;

[0047] Figure 4 A flow chart of a method for fusing infrared images and visible light images based on a deep learning algorithm provided in an embodiment of the present invention;

[0048] Figure 5 A flow chart of a method for secondary fusion of a first fused image and a second fused image provided by an embodiment of the present invention;

[0049] Figure 6 A flow chart of a method for training a multivariate linear regression model provided by an embodiment of the present invention;

[0050] Figure 7 A schematic diagram of a cable visible light and infrared temperature measurement and detection system provided by an embodiment of the present invention;

[0051] Figure 8 A schematic diagram of a 360-degree panoramic image of a cable provided by an embodiment of the present invention;

[0052] Fig. 9 A schematic diagram of processing for fusing visible light images and infrared images provided by an embodiment of the present invention;

[0053] Fig.10 A flow chart of a wavelet decomposition method provided by an embodiment of the present invention;

[0054] Fig.11 A schematic diagram of a cable anomaly detection system based on visible light and infrared image fusion provided by an embodiment of the present invention;

[0055] Fig.12A schematic diagram of a cable anomaly detection device based on visible light and infrared image fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0058] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0059] In view of this, the main purpose of the present invention is to provide a cable anomaly detection method based on the fusion of visible light and infrared images, aiming to solve the technical problems of insufficient accuracy and real-time performance of cable fault detection in the background technology.

[0060] To achieve the above object, the present invention provides a first aspect of a cable anomaly detection method based on visible light and infrared image fusion, which is characterized by comprising the following steps:

[0061] Step S100: acquiring a multimodal image of the cable, wherein the multimodal image includes a visible light image and an infrared thermal image;

[0062] Step S200: fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image;

[0063] Step S300: training a multivariate linear regression model based on the cable fusion image to predict the cable fault type and location;

[0064] Step S400: Collect multimodal image data of the cable in real time and input the trained multivariate linear regression model to predict cable faults.

[0065] Specifically, refer to Figures 1 to 9 In one embodiment of the present invention, a high-resolution optical camera and an infrared imaging device are used to simultaneously photograph the target cable. The optical camera is used to capture the details of the cable surface, such as color, shape, and texture, while the infrared imaging device is responsible for recording the thermal radiation distribution of the cable under working conditions, especially those locations that may become potential failure points due to overheating. In this embodiment, see Figure 8 and Fig. 9,In order to prepare a dataset for training and evaluating the cable anomaly detection model based on the fusion of visible light and infrared images, we collected 2,000 360-degree panoramic partial images of underground cables from different cable wells and randomly divided them into training and test sets at a ratio of 0.95:0.05. In order to increase the diversity and anti-overfitting ability of the dataset, the images were enhanced, including rotation, flipping, brightness and contrast adjustment. All images were resized to a uniform size to ensure the consistency of model processing. Since cable anomalies are relatively rare, we used the method of oversampling the minority class and undersampling the majority class to balance the dataset. For infrared images, grayscale processing was performed, and threshold segmentation was used to remove background information, highlight the main body of the cable, and reduce the interference of background noise. In addition, methods such as median filtering and Gaussian filtering were used to remove noise in the image and improve the image quality. Key features in the image were enhanced through edge detection and histogram equalization techniques to help the model better identify cable anomalies. We also cleaned the dataset, handled missing values ​​and outliers, and ensured that there were no duplicate records. Each image is annotated by professional technicians to indicate the status of the cable (normal, hot, damaged, etc.), and these labels will be used as the target output for model training. Finally, the processed image data and corresponding label information are saved to the file system, forming a standard data set format, providing high-quality data support for subsequent model training and evaluation. Then, the wavelet analysis algorithm is used to perform multi-scale decomposition on the visible light image and infrared thermal image to extract image features at different levels. For each scale of the image, the appropriate fusion rule is selected according to the local characteristics of the image (such as edge strength), for example, the image with larger edge strength is selected as the representative of the area; secondly, the deep learning algorithm (such as convolutional neural network CNN) is used to further process the image, and the high-level features in the image are automatically extracted through the trained model, and the feature-level fusion is performed. The deep learning model can learn more complex image features, such as texture and shape, thereby enhancing the information content of the fused image; finally, the results of the wavelet analysis and deep learning algorithms are combined to generate the final cable fusion image. The results of the two methods can be combined by weighted averaging or other fusion strategies to ensure the high quality of the fused image. The wavelet analysis algorithm can effectively extract the multi-scale features of the image, while the deep learning algorithm can learn deeper image features. The combination of the two not only improves the quality of the fused image, but also enhances the expressiveness of the detail information in the image. This fusion method can better highlight the abnormal parts of the cable and provide a more accurate basis for subsequent fault detection. In order to extract key features that can be used for fault prediction from the fused image, we first defined a series of characteristic indicators related to the health status of the cable, such as temperature distribution, color change, etc. Then, based on these characteristic indicators, a multivariate linear regression model was constructed.The training data set of the model contains a large number of historical cable images, including cable images in normal state and cable images in various fault states (such as local overheating, insulation damage, etc.). Through the supervised learning method, the model parameters are optimized, so that the model can accurately distinguish normal cables from cables with potential faults. After completing the model training, the next step is to deploy the model in the actual operating environment to achieve real-time monitoring of cable status and fault prediction. First, the working status images of the cable are collected regularly by high-resolution optical cameras and infrared imaging equipment installed along the cable. The collected images undergo the same preprocessing steps as the training data, including standardization of image size, noise filtering and removal of background information to ensure the data quality and consistency of the input model. Then, the preprocessed visible light and infrared images are processed using the trained image fusion algorithm to generate a high-quality fused image. This fused image is then fed into the trained multivariate linear regression model, and the model outputs a cable status assessment report based on the feature indicators in the image, including whether there is a potential fault, the specific type of fault and its possible location. The entire process is highly automated and can quickly respond to changes in cable status, providing timely and effective technical support for the safe and stable operation of the power system. In addition, in order to improve the reliability and accuracy of the system, the model will be updated and optimized regularly, including but not limited to introducing new fault cases to enrich the training data set, adjusting model parameters to adapt to changes in the cable environment, etc. In this way, it can be ensured that the model is always in the best performance state and effectively improve the effect of cable anomaly detection. Specifically, the multivariate linear regression model calculates the influence weight of each feature on the health status of the cable, and then predicts whether the cable has a fault and its type based on the feature values ​​in the new cable fusion image. This method not only improves the accuracy of fault detection, but also provides a scientific basis for cable maintenance.

[0066] Optionally, in step S200, the step of fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image includes:

[0067] Step S210: fusing the multimodal images based on a wavelet analysis algorithm to obtain a first fused image, and fusing the multimodal images based on a deep learning algorithm to obtain a second fused image;

[0068] Step S220: performing secondary fusion on the first fused image and the second fused image to obtain the cable fused image.

[0069] Specifically, refer to Figure 2In one embodiment of the present invention, in step S210, a wavelet analysis algorithm is used to perform preliminary fusion of the infrared image and the visible light image. The wavelet analysis algorithm here can select discrete wavelet transform (DWT) to capture features at different scales through multi-level decomposition. Specifically, the two images can be firstly decomposed into three levels of DWT, and then the high-frequency coefficients are merged according to the maximum selection rule, that is, the coefficients with larger absolute values ​​are selected at the same position as the fused coefficients; for the low-frequency coefficients, the average value strategy is adopted. Finally, the first fused image is reconstructed using the inverse discrete wavelet transform (IDWT). In step S210, a convolutional neural network (CNN) can be used as the core part of the deep learning algorithm to extract and fuse image features under two modalities. First, a CNN model including multiple convolutional layers, pooling layers and fully connected layers is constructed, and a large number of annotated cable image data sets are used to train it. After the training is completed, the infrared image and visible light image to be processed are input into the model. After feature extraction, the feature maps of the two modalities are weighted summed at a specific fusion layer to obtain a second fused image. In step S220, in order to further improve the fusion quality, a weighted average method can be adopted to combine the first fused image and the second fused image. Specifically, a suitable weight ratio can be set according to actual needs, such as setting the weight of the first fused image to 0.6 and the weight of the second fused image to 0.4, and then performing pixel-level weighted average calculation on the two images according to this ratio, and finally generating a high-quality cable fusion image. Compared with a single technical solution, the dual-track fusion strategy proposed in the present invention not only improves the image processing efficiency, but also reduces the need for manual intervention, making the entire detection process more automated and intelligent.

[0070] Optionally, in step S210, the step of fusing the multimodal images based on a wavelet analysis algorithm to obtain a first fused image includes:

[0071] Performing wavelet decomposition on the multimodal image to extract low-frequency coefficients and high-frequency coefficients at different scales;

[0072] The low-frequency coefficients are fused using a fusion rule based on image edge information to generate low-frequency fusion coefficients;

[0073] The high frequency coefficients are fused using a fusion rule based on a combination of a window and a fractal to generate a high frequency fusion coefficient;

[0074] The low-frequency fusion coefficient and the high-frequency fusion coefficient are recombined through inverse wavelet transform to generate a first fused image.

[0075] Specifically, refer to Figure 3In one embodiment of the present invention, the fast wavelet decomposition and reconstruction algorithm proposed by Mallat is used in combination with the improved fusion rule to realize the effective fusion of infrared image and visible light image. Fig.11 , perform wavelet decomposition on the infrared image and the visible light image. Assuming the number of decomposition layers is J, according to the Mallat fast algorithm, each layer of decomposition can be expressed as:

[0076]

[0077] In the formula, C j represents the low-frequency coefficient of the Jth layer, C j-1 represents the low-frequency coefficients of the J-1th layer, represents the high frequency coefficients in the diagonal direction of the Jth layer, Represents the high-frequency coefficients in the vertical direction of the jth layer, Represents the high-frequency coefficients in the horizontal direction of the jth layer, H c Represents the low-pass filter in the horizontal direction, H r Represents the low-pass filter in the vertical direction, G c Represents the horizontal high-pass filter, G r Represents a high-pass filter in the vertical direction.

[0078] The low-frequency coefficients are fused using a fusion rule based on image edge information to generate low-frequency fusion coefficients, specifically: the infrared image and the visible light image are edge detected using a Sobel edge detection operator to obtain an infrared image edge map and a visible light image edge map. For a continuous function f(x, y), its gradient at the point (x, y) can be represented by a vector:

[0079]

[0080] Among them G x represents the gradient in the X direction, G y Represents the gradient in the Y direction. The gradient (amplitude) and direction angle of this vector are:

[0081]

[0082]

[0083] Based on the convolution method of the small area mask template of the directional derivative, the approximate operation is performed, and the gradient G in the X direction and the direction x , G y Corresponding to a mask template respectively, the template of the Sobel edge detection operator is:

[0084]

[0085] The fusion algorithm of low-frequency coefficients is as follows: if both images are edge points at the same position, the low-frequency coefficient with a larger absolute value is selected; if only one image is an edge point, the low-frequency coefficient of that image is selected; if both images are not edge points, they are fused using the weighted average method, with the weight determined by the local mean:

[0086]

[0087] In the formula, C(x, y) represents the low-frequency coefficient of the fused image at the point (x, y), A(x, y) represents the low-frequency coefficient of the first image at the point (x, y), B(x, y) represents the low-frequency coefficient of the second image at the point (x, y), and m A (x, y) represents the local mean of the first image at point (x, y), m B (x, y) represents the local mean of the second image at point (x, y); m A (x, y) and m B The calculation formula for (x, y) is:

[0088]

[0089] In the formula, m i (x, y) represents the local mean of the i-th image at point (x, y), f i (x, y) represents the grayscale value of the i-th image at point (x, y), M×N is the window size, usually 3×3 or 5×5.

[0090] The high-frequency coefficients are fused using a fusion rule based on a combination of windows and fractals to generate high-frequency fusion coefficients. Specifically, at the highest decomposition layer (the jth layer), the wavelet coefficient of the first image at point (x, y) is set to be A(x, y), and the wavelet coefficient of the second image at point (x, y) is set to be B(x, y). The two images with the smallest absolute values ​​of the coefficients in a local area of ​​3×3 centered at point (x, y) are respectively recorded as A1(x, y) and B1(x, y). The fused high-frequency coefficient at point (x, y) is recorded as C(x, y). The calculation formula of the fused high-frequency coefficient C(x, y) is as follows:

[0091]

[0092] Where a and b are weight coefficients, where 0 <a<1,0<b<1,a+b=1,a>b。

[0093] For other layers of the highest decomposition layer, the fractal method is used. First, the fractal dimension K1 corresponding to the high-frequency part of the first image and the fractal dimension K2 corresponding to the high-frequency part of the second image are calculated. The fractal dimension calculation method is: cover an M×M image G with a 1×1 square, and use g=1 / 2, 1 / 4, 1 / 8,... as the measurement unit to evenly divide the image into (1 / g)×(1 / g) small squares, then calculate the difference between the maximum grayscale value and the minimum grayscale value of each small square image, and sum them up. Finally, the fractal dimension is calculated using the least squares method.

[0094] Then, the image high-frequency fusion coefficient value F is obtained by the fractal dimension weighted average method. The calculation formula is as follows:

[0095]

[0096] Where F is the high-frequency fusion coefficient of the image, K1 is the fractal dimension corresponding to the high-frequency part of the first image, K2 is the fractal dimension corresponding to the high-frequency part of the second image, and A and B represent the pixel values ​​or feature values ​​of the two images at the point (x, y), respectively.

[0097] The low-frequency fusion coefficient and the high-frequency fusion coefficient are recombined by inverse wavelet transform to generate a first fused image. Specifically, the first fused image is reconstructed by inverse wavelet transform using the fused low-frequency and high-frequency coefficients. The inverse wavelet transform formula is:

[0098]

[0099] Among them, H* and G* are the conjugate transposed matrices of H and G respectively.

[0100] Optionally, in step S210, the step of fusing the multimodal images based on a deep learning algorithm to obtain a second fused image includes:

[0101] Using a deep learning algorithm to perform multi-scale decomposition on the multimodal image to generate high-frequency sub-band information and low-frequency sub-band information;

[0102] The high frequency sub-band information is fused, and the fused high frequency sub-band information is determined by selecting the maximum feature in the neighborhood;

[0103] Fusing the low-frequency sub-band information, by dividing the low-frequency sub-band into small blocks and calculating an average value to determine the fused low-frequency sub-band information;

[0104] Through inverse transformation, an image is reconstructed according to the fused high-frequency sub-band information and the fused low-frequency sub-band information to generate a second fused image.

[0105] Specifically, refer to Figure 4In one embodiment of the present invention, a deep learning algorithm is used to decompose the image. The image is regarded as a hierarchical structure. The registered infrared image and the visible light image are decomposed. The number of decomposition layers is Q. The decomposition results are Q multi-scale high-frequency sub-bands D of the infrared image. df and low frequency subband E x , Q high-frequency sub-bands D of the visible light image dk and low frequency subband E y , the coefficients in the subband are represented as the features of the source image. First, for each high-frequency subband, the local energy in the neighborhood is calculated. The calculation formula of the local energy is as follows:

[0106]

[0107] Where k(x, y) represents the local energy at the midpoint (x, y) of the high-frequency subband, x+s, y+u represents the neighborhood window, and α is the weight coefficient.

[0108] Then, the maximum feature in the neighborhood is selected as the fused feature, and the calculation formula for the fused high-frequency sub-band coefficient is:

[0109] m(x, y) = z aj b

[0110] In the formula, m(x, y) represents the fused high-frequency subband coefficient, z aj Represents the coefficients obtained after the b-th decomposition.

[0111] In step S223, the low-frequency sub-band is first divided into n×n sub-blocks, and the average value of each sub-block is calculated as the fused low-frequency sub-band information. The calculation formula of the fused low-frequency sub-band coefficient is:

[0112]

[0113] Where Q(x, y) represents the coefficient of the fused low-frequency subband at the point (x, y), and l(z, i) represents the coefficient of the Z-th layer low-frequency subband at the point (x, y).

[0114] Next, the hidden layer weights are obtained by fusing the image low-frequency subband coefficients and high-frequency subband coefficients obtained above. The calculation formula is as follows:

[0115] F(M,P)=d z (n)+z

[0116] Where: F(M, P) is the fused high-frequency sub-band and low-frequency sub-band coefficients, z is the image decomposition parameter, d z (n) is the coefficient of the high-frequency subband or low-frequency subband at pixel position n after decomposition at the Zth layer.

[0117] Finally, the second fused image is reconstructed through inverse transformation according to the fused low-frequency sub-band information and high-frequency sub-band information.

[0118] Optionally, in step S220, the step of performing secondary fusion on the first fused image and the second fused image to obtain the cable fused image includes:

[0119] Performing quality assessment on the first fused image and the second fused image respectively to obtain a first quality score and a second quality score;

[0120] Determine whether the first quality score is higher than the second quality score, if so, select the wavelet analysis algorithm as the secondary fusion algorithm, otherwise, select the deep learning algorithm as the secondary fusion algorithm;

[0121] The first fused image and the second fused image are fused for a second time using the secondary fusion algorithm to generate the cable fused image.

[0122] Specifically, refer to Figure 5 In one embodiment of the present invention, the first fused image and the second fused image are evaluated, and common image quality evaluation indicators such as peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) can be used to calculate the quality score Q1 of the first fused image and the quality score Q2 of the second fused image. Compare Q1 and Q2, and select the image with a higher quality score as the basis for secondary fusion; if Q1>Q2, select the wavelet analysis algorithm as the secondary fusion algorithm, otherwise, select the deep learning algorithm as the secondary fusion algorithm. Use the selected fusion algorithm to perform secondary fusion on the first fused image and the second fused image. If a secondary fusion algorithm based on wavelet analysis is selected, a method similar to that described in step S210 can be used, that is, by performing wavelet decomposition on the first fused image and the second fused image again, extracting low-frequency and high-frequency coefficients, and then fusing them according to certain rules (such as maximum selection rule, average value strategy, etc.), and finally reconstructing the final cable fusion image through inverse wavelet transform. If a secondary fusion algorithm based on deep learning is selected, a new convolutional neural network model can be designed, which is specifically used to process the first fused image and the second fused image. First, the model needs to be trained, and the data set used in the training process should contain a large number of high-quality cable image samples. After the training is completed, the first fused image and the second fused image are input into the model, and the model will automatically learn and select the most suitable features for fusion, thereby generating the final cable fusion image. It can be understood that this embodiment realizes the efficient secondary fusion of the first fused image and the second fused image, and the generated cable fusion image not only maintains a high image quality and rich information, but also effectively reduces noise interference, improves the contrast and clarity of the image, and significantly improves the accuracy and reliability of cable anomaly detection.

[0123] Optionally, in step S300, the training step of the multivariate linear regression model includes:

[0124] Step S310: acquiring historical data of the cable fusion image and constructing an augmented matrix, and calculating a cross product matrix from the 1st to the Tth moment according to the augmented matrix;

[0125] Step S320: Calculate the eigenvalues ​​and eigenvectors from the 1st to the Tth time according to the cross product matrix from the 1st to the Tth time;

[0126] Step S330: Based on the eigenvalues ​​at the first to Tth moments, a time series analysis method is used to predict the eigenvalue at the T+1th moment, and based on the eigenvectors at the first to Tth moments, an orthogonal matrix prediction method is used to predict the eigenvector at the T+1th moment;

[0127] Step S340: Calculate the cross product matrix at the T+1th moment according to the eigenvalue and eigenvector at the T+1th moment, and calculate the estimated value of the regression parameter at the T+1th moment based on the cross product matrix at the T+1th moment;

[0128] Step S350: construct a multivariate linear regression model based on the estimated values ​​of the regression parameters at time T+1.

[0129] Optional, reference Figure 6 In step S340, the method for calculating the estimated value of the regression parameter at the T+1th moment based on the cross product matrix at the T+1th moment is the least squares method.

[0130] Specifically, in one embodiment of the present invention, the training steps of the multivariate linear regression model specifically include: firstly, collecting the historical data of the cable fusion image, including the image features at multiple times, the sample point capacity is n, and the dependent variable at time t is y t , mainly for the predicted abnormal results, p independent variables are x j t (j=1,2,…,p), etc., then the overall linear regression model is in the form of y i t =β0+β1 t x i1 t +…+β p t x ip t +ε i t ,i=1,2,…,n, note

[0131]

[0132] Then the augmented matrix (x t ,yt )'s cross product v t for: Regression coefficient β t The least squares estimator of is It can also be proved that S at time t SSE and S SST The values ​​are as follows:

[0133]

[0134] in, Too The first component of , according to the above formula, the goodness of fit (adjusted multiple determination coefficient) at time t is At the same time, the estimated standard error at time t is The predictive modeling method of multiple linear regression is as follows:

[0135] 1) Calculate the augmented matrix (x) at time 1-T t ,y t )'s cross product matrix v t ;

[0136] 2) According to the v of 1-T in step 1 t Calculate v t The eigenvalue of and the corresponding orthonormal eigenvectors

[0137] 3) According to the v of 1-T in step 2 t The characteristic vector matrix of T+1 is predicted by using the orthogonal matrix prediction method.

[0138] 4) According to the v of 1-T in step 2 t The characteristic value of , using time series analysis methods (such as ARIMA model), predicts the characteristic value at time T + 1 respectively

[0139] 5) Based on the calculation results of the previous two steps, calculate the cross product matrix v at time T+1 T+1 for

[0140]

[0141] 6) According to the above formula, we can get the estimated value β of the regression parameter at time T+1 T+1 , construct a multiple linear regression model;

[0142] 7) According to the above formula, the goodness of fit of the regression model at time T+1 can be calculated and the estimated standard error

[0143] Through the above steps, the cable anomaly detection model based on the fusion of visible light and infrared images can be effectively trained and evaluated to ensure the accuracy and reliability of the model.

[0144] In order to further verify the performance of the model, the data in the test set can be used to verify the model. The specific steps are as follows:

[0145] Test set prediction: The cable fusion image features in the test set are input into the trained multivariate linear regression model to predict the anomaly probability of each image.

[0146] Performance evaluation: Calculate various performance indicators of the model on the test set, such as accuracy, recall, F1 score, etc. These indicators can help evaluate the performance of the model in practical applications.

[0147] Visualize results: Draw a comparison chart of the predicted results and the true labels to intuitively display the prediction effect of the model. For example, you can draw the ROC curve and confusion matrix to further analyze the classification performance of the model.

[0148] Optionally, after step S100, a robust Gaussian filter is applied to perform denoising processing on the visible light image and the infrared thermal image.

[0149] Specifically, in one embodiment of the present invention, the visible light image and the infrared thermal image are first read, a suitable filter size and a suitable standard deviation are selected, and a file Gaussian filter is applied to each image to obtain a denoised image. The Gaussian filter calculation formula is as follows:

[0150]

[0151] Among them, G(p) is the calculation method of the weight of a point p in the window centered on q during the Gaussian filtering process, σ is the standard deviation of the Gaussian filter, ∥pq∥ 2 is the square of the Euclidean distance between point p and point q.

[0152] However, in the edge area of ​​the image, due to the weight distribution of the Gaussian filter based on distance, the edge information will be lost, making the edge of the image blurred. When there are abnormal feature values ​​in the image, it is difficult for the traditional Gaussian filter to effectively eliminate the influence of these abnormal values ​​on the filtering effect. Therefore, the present invention proposes an improved Gaussian filtering algorithm, which improves the filtering effect by introducing the vertical weight function and Tukey double weight estimation, especially for the processing of abnormal values ​​and speckle noise. The robust Gaussian filtering expression is as follows:

[0153]

[0154] In the formula, Refers to a convolution operation of the surface profile. It refers to the value of the surface profile data when performing the convolution operation; q(α) refers to the low-frequency reference signal, α represents a pixel position in the image; γ(β) refers to the introduced vertical weight function, β represents the variable in the domain of the weight function: It refers to a convolution operation of Gaussian weight function. represents the variables α and The difference between them is used to define the parameters of the Gaussian weight function, which in turn affects the weight of each data point in the convolution operation. D(s) represents the frequency response function, which represents the response characteristics of the system to different frequency components. s represents the state variable of the system. α1 and α2 represent the positions of two pixel points in the image. w(α) represents the pixel value of the image at position α. ​​Through continuous iterative operations, we find the appropriate γ(β) to differentiate w(α) to calculate the centerline of the robust Gaussian filter.

[0155] The calculation formula of the robust Gaussian filter centerline is as follows:

[0156]

[0157] In the formula, Refers to a convolution operation of the surface profile, Represents a pixel position α in the image and surface contour data The deviation between them; η(f) refers to the frequency response function that determines the relative importance of each data point in the convolution process, which represents a convolution operation of the Gaussian weight function, f represents the frequency, and α1 and α2 represent the positions of two pixels in the image.

[0158] right Implement discrete calculations as follows:

[0159]

[0160] In the formula, Refers to a convolution operation of the surface profile, Represents a pixel position α in the image and surface contour data The deviation between them, in the context of Gaussian filtering, this difference is used to determine the offset of the center position of the Gaussian kernel relative to the current processing point α; γ(β) refers to the introduced vertical weight function, β represents the variable in the domain of the weight function: η(α) refers to a convolution operation of the Gaussian weight function, α represents a pixel position in the image: Each time DC(i) is iterated, DC(i) represents the value of the DC function at the i-th iteration, and the Gaussian filter is applied to implement the vertical weight function of γ(cv), respectively. cv in this context usually represents a continuous variable. The Gaussian kernel is based on the above The offset is used to calculate the weight, which in turn affects the influence of each data point in the convolution operation. α1 and α2 represent the positions of two pixels in the image, as shown in the following formula:

[0161] W E =5.6328×mid(|λ(α)-z(α)|)

[0162] In the above formula, mid refers to the median; λ(α) refers to the integral function, α represents a pixel position in the image, and z(α) represents the pixel value of the image at the pixel position α.

[0163] Tukey's bi-weighted estimation is used to identify and reduce the impact of outliers and to build a robust estimation model. The expression of Tukey's bi-weighted estimation is:

[0164]

[0165] In the formula, α represents a pixel position in the image, and U refers to the ratio of the parameter related to the deviation to the median of the parameter. The constructed robust estimation model is as follows:

[0166]

[0167] Robust Gaussian filtering is implemented through a robust estimation model to continuously iterate operations to obtain the optimal solution, and the interference of outliers, namely speckle noise, is eliminated through repeated iterative operations.

[0168] It can be understood that the robust Gaussian filtering model effectively reduces the noise in the image, improves the clarity and contrast of the image, helps the subsequent image fusion process, and improves the overall quality and reliability of the fused image.

[0169] Optionally, in one embodiment of the present invention, a statistical cycle can be preset (such as once a week or once a month, adjusted as needed), and a scheduled task scheduling tool can be used to regularly trigger the model prediction accuracy check. In each statistical cycle, the test sample set is input into the current model to obtain the model's prediction results, and the difference between the model's prediction results and the true label is calculated. Common indicators include mean square error (MSE), mean absolute error (MAE), etc. The calculated prediction accuracy value is saved in the database, and the time and accuracy value of each check are recorded. Each time the prediction accuracy of the model is checked, it is determined whether the model needs to be updated online based on preset conditions: minimum prediction accuracy value determination, a minimum prediction accuracy value is set according to actual needs and experience, such as MSE is less than 0.1, the latest prediction accuracy value is read from the database, and compared with the preset minimum prediction accuracy value. If the prediction accuracy value is lower than the preset minimum prediction accuracy value, the online update mechanism is triggered; update cycle threshold determination, the current time is recorded when the model is online, and the running time of the model is updated each time the model is checked, and a fixed update cycle threshold is set (for example, once every three months), and the model is calculated at each check. The current running time is compared with the preset update cycle threshold. If the running time of the model reaches the preset update cycle threshold, the online update mechanism is triggered. The prediction accuracy value continues to decline. The prediction accuracy values ​​of the previous N statistical periods including the current statistical period are extracted from the database, for example, N=3. The trend of these prediction accuracy values ​​can be calculated using simple linear regression or moving average methods. If the prediction accuracy values ​​of N consecutive statistical periods show a downward trend (for example, the prediction accuracy value of each period is lower than that of the previous period), it is considered that the performance of the model is gradually declining. If the prediction accuracy value continues to decline, the online update mechanism is triggered.

[0170] A second aspect of the present invention provides a cable anomaly detection system based on visible light and infrared image fusion, comprising:

[0171] An acquisition module, used for acquiring a multimodal image of the cable, wherein the multimodal image includes a visible light image and an infrared thermal image;

[0172] A fusion module, used for fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image;

[0173] A training module, used for training a multivariate linear regression model according to the cable fusion image, so as to predict the type and location of the cable fault;

[0174] The prediction module is used to collect multimodal image data of cables in real time and input the trained multivariate linear regression model to predict cable faults.

[0175] The third aspect of the present invention provides a device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the computer program, the cable anomaly detection method based on visible light and infrared image fusion as described in the first aspect of the present invention is implemented.

[0176] A fourth aspect of the present invention provides a computer-readable storage medium, on which is stored a cable anomaly detection program based on the fusion of visible light and infrared images. When the cable anomaly detection program based on the fusion of visible light and infrared images is executed by a processor, the steps of the cable anomaly detection method based on the fusion of visible light and infrared images as described in any of the above embodiments are implemented.

[0177] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the cable anomaly detection system based on visible light and infrared image fusion, and uses various interfaces and lines to connect the various parts of the entire cable anomaly detection system based on visible light and infrared image fusion to process the executable device.

[0178] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the cable anomaly detection system based on visible light and infrared image fusion by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0179] Compared with the prior art, the beneficial effects of the present invention include at least:

[0180] The present invention provides a cable anomaly detection method based on the fusion of visible light and infrared images. By combining visible light images and infrared thermal images and using wavelet analysis algorithms and deep learning algorithms to perform multimodal data fusion, the visual and thermal distribution information of the cable can be fully captured, and the accuracy of cable anomaly detection is significantly improved. The wavelet analysis algorithm can extract multi-scale features of the image, and the deep learning algorithm can learn complex image features. The combination of the two enables the fused image to better reflect the actual situation of the cable, improves the accuracy of anomaly detection, reduces data processing time, and enables cable anomaly detection to achieve real-time monitoring, and timely discover and warn of cable anomalies. By constructing a multivariate linear regression model and using historical data to train the model, the abnormal situation of the cable can be accurately predicted, the accuracy and reliability of fault prediction are improved, and the problems of cumbersome data processing, severe time lag and inaccurate fault prediction in traditional monitoring methods are solved, thereby ensuring the safe and stable operation of the power system.

[0181] Furthermore, in the image fusion process, fusion rules based on image edge information and fusion rules based on the combination of windows and fractals are adopted, which can better maintain the detailed information of the image, especially in the display of abnormal parts of the cable, and can provide a more accurate basis. The dual-track fusion strategy proposed in the present invention not only improves the efficiency of image processing, but also reduces the need for manual intervention, making the entire detection process more automated and intelligent. In addition, by regularly triggering the inspection and online update mechanism of the model prediction accuracy, the long-term effectiveness and adaptability of the model are guaranteed. After acquiring the image, Gaussian filtering is applied to denoise the image, which improves the clarity and contrast of the image, and is beneficial to the subsequent image fusion process. At the same time, the quality of the fused image is evaluated and the optimal fusion algorithm is selected to ensure that the generated cable fusion image has higher quality and better visual effects.

[0182] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and the equivalent structural transformations made by the contents of the present invention specification and drawings, or direct / indirect application in other related technical fields are all included in the scope of patent protection of the present invention. Therefore, the scope of protection of the patent of the present invention shall be subject to the attached claims.

Claims

1. A cable anomaly detection method based on visible light and infrared image fusion, characterized in that: The following steps are involved: Step S100: acquiring a multimodal image of the cable, wherein the multimodal image includes a visible light image and an infrared thermal image; Step S200: fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image; Step S300: training a multivariate linear regression model based on the cable fusion image to predict the cable fault type and location; Step S400: Collect multimodal image data of the cable in real time and input the trained multivariate linear regression model to predict cable faults.

2. The cable anomaly detection method based on visible light and infrared image fusion as claimed in claim 1 is characterized in that: In step S200, the step of fusing the multimodal images based on the wavelet analysis algorithm and the deep learning algorithm to obtain the cable fusion image includes: Step S210: fusing the multimodal images based on a wavelet analysis algorithm to obtain a first fused image, and fusing the multimodal images based on a deep learning algorithm to obtain a second fused image; Step S220: performing secondary fusion on the first fused image and the second fused image to obtain the cable fused image.

3. The cable anomaly detection method based on visible light and infrared image fusion as claimed in claim 2, characterized in that: In step S210, the step of fusing the multimodal images based on the wavelet analysis algorithm to obtain a first fused image includes: Performing wavelet decomposition on the multimodal image to extract low-frequency coefficients and high-frequency coefficients at different scales; The low-frequency coefficients are fused using a fusion rule based on image edge information to generate low-frequency fusion coefficients; The high frequency coefficients are fused using a fusion rule based on a combination of a window and a fractal to generate a high frequency fusion coefficient; The low-frequency fusion coefficient and the high-frequency fusion coefficient are recombined through inverse wavelet transform to generate a first fused image.

4. The cable anomaly detection method based on visible light and infrared image fusion as claimed in claim 3 is characterized in that: In step S210, the step of fusing the multimodal images based on a deep learning algorithm to obtain a second fused image includes: Using a deep learning algorithm to perform multi-scale decomposition on the multimodal image to generate high-frequency sub-band information and low-frequency sub-band information; The high frequency sub-band information is fused, and the fused high frequency sub-band information is determined by selecting the maximum feature in the neighborhood; Fusing the low-frequency sub-band information, by dividing the low-frequency sub-band into small blocks and calculating an average value to determine the fused low-frequency sub-band information; Through inverse transformation, an image is reconstructed according to the fused high-frequency sub-band information and the fused low-frequency sub-band information to generate a second fused image.

5. The cable anomaly detection method based on visible light and infrared image fusion as claimed in claim 4, characterized in that: In step S220, the step of performing secondary fusion on the first fused image and the second fused image to obtain the cable fused image includes: Performing quality assessment on the first fused image and the second fused image respectively to obtain a first quality score and a second quality score; Determine whether the first quality score is higher than the second quality score, if so, select the wavelet analysis algorithm as the secondary fusion algorithm, otherwise, select the deep learning algorithm as the secondary fusion algorithm; The first fused image and the second fused image are fused for a second time using the secondary fusion algorithm to generate the cable fused image.

6. The cable anomaly detection method based on visible light and infrared image fusion according to any one of claims 1 to 5, characterized in that: In step S300, the step of training a multivariate linear regression model according to the cable fusion image includes: Step S310: acquiring historical data of the cable fusion image and constructing an augmented matrix, and calculating a cross product matrix from the 1st to the Tth moment according to the augmented matrix; Step S320: Calculate the eigenvalues ​​and eigenvectors from the 1st to the Tth time according to the cross product matrix from the 1st to the Tth time; Step S330: Based on the eigenvalues ​​at the first to Tth moments, a time series analysis method is used to predict the eigenvalue at the T+1th moment, and based on the eigenvectors at the first to Tth moments, an orthogonal matrix prediction method is used to predict the eigenvector at the T+1th moment; Step S340: Calculate the cross product matrix at the T+1th moment according to the eigenvalue and eigenvector at the T+1th moment, and calculate the estimated value of the regression parameter at the T+1th moment based on the cross product matrix at the T+1th moment; Step S350: construct a multivariate linear regression model based on the estimated values ​​of the regression parameters at time T+1.

7. The cable anomaly detection method based on visible light and infrared image fusion according to any one of claim 6, characterized in that: In step S340, the method for calculating the estimated value of the regression parameter at the T+1th time based on the cross product matrix at the T+1th time is the least squares method.

8. The cable anomaly detection method based on visible light and infrared image fusion according to any one of claims 1 to 5, characterized in that: After step S100, a robust Gaussian filter is applied to perform denoising on the multimodal image.

9. A cable anomaly detection system based on visible light and infrared image fusion, characterized in that: include: An acquisition module, used for acquiring a multimodal image of the cable, wherein the multimodal image includes a visible light image and an infrared thermal image; A fusion module, used for fusing the multimodal images based on a wavelet analysis algorithm and a deep learning algorithm to obtain a cable fusion image; A training module, used for training a multivariate linear regression model according to the cable fusion image, so as to predict the type and location of the cable fault; The prediction module is used to collect multimodal image data of cables in real time and input the trained multivariate linear regression model to predict cable faults.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a cable anomaly detection program based on the fusion of visible light and infrared images. When the cable anomaly detection program based on the fusion of visible light and infrared images is executed by the processor, the steps of the cable anomaly detection method based on the fusion of visible light and infrared images as described in any one of claims 1 to 8 are implemented.

Images