Cable fault monitoring and health state assessment method based on multi-modal fusion

Abstract

The invention discloses a cable fault monitoring and health state assessment method based on multi-modal fusion, and the method comprises the following steps: 1), collecting cable multi-modal data, including electrical quantity data and cable image data; 2) preprocessing the electrical quantity data and the cable image data; 3) performing feature extraction and fusion on the preprocessed electrical quantity data and cable image data to obtain a multi-modal feature vector; 4) constructing a cable fault classification model; 5) inputting the multi-modal feature vector into a cable fault classification model to obtain a cable operation state type; and 6) calculating a cable health state index based on the cable operation state type and the multi-modal feature vector. According to the invention, through the large kernel convolution design of RepLKNet, the multi-modal feature fusion strategy, the feature learning optimization mechanism and the innovation of the data enhancement technology, the technical problems of insufficient robustness, low classification precision, poor real-time performance and the like of the traditional method in the cable monitoring process are successfully solved.

Description

Technical Field

[0001] The present invention relates to the field of cable fault monitoring, and in particular to a cable fault monitoring and health status assessment method based on multi-modal fusion. Background Art

[0002] Cables are a key component in power transmission and communication systems, and their health status is directly related to the safety and stability of the system. With the continuous development of power and communication networks, cable fault problems have gradually become prominent. How to achieve accurate cable fault monitoring and health assessment has become a technical problem that needs to be solved urgently. With the increase in society's demand for energy and information, the working environment of cables has become more complex, especially under extreme conditions such as high temperature, high humidity, pollution or mechanical shock. The risk of cable aging and damage has increased significantly. Therefore, timely monitoring of cable status and evaluating its health are of vital importance to ensuring the reliability of the power system and reducing the risk of accidents such as power outages and fires.

[0003] The inventors found that the existing cable fault monitoring and health status assessment technologies have at least the following shortcomings: ① The fault monitoring method based on electrical quantity data usually relies on a single signal such as current, voltage or temperature for fault identification, but in a complex operating environment, these signals are easily affected by noise and external interference, resulting in low diagnostic accuracy; ② Although the health status assessment method based on image data can reflect the external damage of the cable, the traditional image analysis method lacks effective automated feature extraction and analysis methods, and it is difficult to ensure efficient and stable fault diagnosis effects under different environments; ③ The existing multimodal data fusion method fails to effectively combine the advantages of electrical quantity data and image data, and often ignores the complementarity between the two, resulting in insufficient feature information after fusion, affecting the accuracy and real-time performance of fault prediction and health assessment. The existence of these problems mainly stems from the complexity and diversity of data involved in the cable fault monitoring and health assessment process, as well as the limitations of existing methods in feature extraction and multimodal information fusion.

[0004] In view of the above problems, there is an urgent need for a method that can fully integrate electrical quantity data and image data, and combine advanced deep learning and multimodal data fusion technology to improve the accuracy and real-time performance of cable fault monitoring and health status assessment. However, in the process of achieving this goal, how to effectively integrate the time series characteristics of electrical quantity data and the spatial characteristics of image data, construct an efficient feature representation suitable for multimodal data fusion, and maintain high diagnostic accuracy and robustness in complex environments are still technical difficulties and research focuses. Summary of the invention

[0005] The purpose of the present invention is to provide a cable fault monitoring and health status assessment method based on multi-modal fusion, comprising the following steps:

[0006] 1) Collect multimodal data of the cable, including electrical quantity data and cable image data;

[0007] 2) Preprocess the electrical quantity data and cable image data;

[0008] 3) Extract and fuse features from the preprocessed electrical quantity data and cable image data to obtain a multimodal feature vector;

[0009] 4) Build a cable fault classification model;

[0010] 5) Input the multimodal feature vector into the cable fault classification model to obtain the cable operation status type;

[0011] 6) Calculate the cable health status index based on the cable operation status type and the multimodal feature vector, and predict the remaining service life of the cable based on the cable health status index.

[0012] Furthermore, the electrical quantity data is periodically collected by voltage sensors and current sensors.

[0013] Furthermore, the cable image data includes visible light images of the cable surface and the surrounding environment, cable thermal imaging data, and cable three-dimensional point cloud data.

[0014] Furthermore, the visible light images of the cable surface and the surrounding environment are collected by a natural light camera;

[0015] The cable thermal imaging data is collected by an infrared camera;

[0016] The cable three-dimensional point cloud data is collected and reconstructed by a laser scanner.

[0017] Furthermore, the steps for preprocessing the electrical quantity data include low-pass filtering and filling missing data;

[0018] The low-pass filtering is implemented by a low-pass filter;

[0019] The transfer function H(f) of the low-pass filter is as follows:

[0020]

[0021] where f is the signal frequency, f c is the cut-off frequency of the filter; j is an imaginary number;

[0022] The filling of missing data is implemented by linear interpolation, that is:

[0023]

[0024] Among them, X1 and X2 are known data points, t1 and t2 are the corresponding time points, and t is the time point of the missing data. X missing is the filled missing data.

[0025] Furthermore, the steps of preprocessing the cable image data include noise smoothing, contrast stretching, edge detection, and geometric correction;

[0026] The image after noise smoothing is shown as follows:

[0027]

[0028] Among them, I out (x, y) is the filtered image, I(x, y) is the original image, N is the number of pixels in the neighborhood, and k is the radius of the filtering window;

[0029] The image after contrast stretching is shown as follows:

[0030]

[0031] Among them, f(x, y) and g(x, y) are the original image and the enhanced image respectively, max(f) and min(f) are the maximum and minimum gray values of the original image respectively, and max(g) and min(g) are the gray value ranges of the enhanced image.

[0032] Edge detection is implemented through the Sobel operator, that is:

[0033]

[0034] Among them, G x and G y represent the gradients of the image in the horizontal and vertical directions respectively.

[0035] Furthermore, in step 3), the steps of feature extraction and fusion of the preprocessed electrical quantity data and cable image data include:

[0036] 3.1) Perform data normalization processing on the preprocessed electrical quantity data;

[0037] 3.2) Use the electrical quantity data feature extraction model to map the electrical quantity data into an electrical quantity feature vector f with a fixed length elec ;

[0038] The electrical quantity data feature extraction model is a deep neural network; the electrical quantity data feature extraction model is trained by a historical sample data set with normalized electrical quantity data as the input and electrical quantity feature vector as the output;

[0039] 3.3) Use the image feature extraction model to extract the image feature vector f of the cable image dataimg ;

[0040] The image feature extraction model is RepLKNet, which includes an initial convolutional layer, a large kernel convolutional module, and a fully connected layer; the image feature extraction model is trained by a historical sample data set with cable image data as input and image feature vectors as output;

[0041] 3.4) Fuse the electrical quantity feature vector f elec and the image feature vector f img to obtain a multi-modal feature vector f concat , that is:

[0042] f concat = [f elec ; f img (6)

[0043] Furthermore, the cable fault classification model is a multi-modal fusion neural network based on an attention mechanism, and the fused features are mapped to the fault category space through a fully connected layer, and the output classification probability is:

[0044] P = Softmax(W a ·f concat + b a )(7)

[0045] In the formula, P is the classification probability; W a , b a are the weights and biases.

[0046] Furthermore, the cable operating state types include normal operation, local overheating, surface damage, or short circuit faults.

[0047] Furthermore, based on the fused data, calculate the cable health state index HI, that is:

[0048]

[0049] where R fault is the fault risk score, S feature is the multi-modal feature score, S historical is the historical score;

[0050] The remaining service life of the cable is as follows:

[0051]

[0052] In the formula, RUL represents the remaining service life of the cable, HI is the current health state index, H th is the health state threshold, is the decay rate of the health state index over time.

[0053] The technical effects of the present invention are beyond doubt. The present invention effectively solves technical problems such as insufficient monitoring accuracy and low robustness in complex cable environments, significantly improves the overall performance of cable condition assessment and fault monitoring, and in particular provides an efficient and reliable solution for the requirements of strong noise interference, multiple types of faults, and real-time processing. The core innovations include the application of RepLKNet in image feature extraction, the optimization of multi-modal feature fusion strategies, and the introduction of a dynamic loss balancing mechanism. These technical means provide important support for the efficient operation and intelligent operation and maintenance of cable monitoring systems.

[0054] First, the present invention uses RepLKNet as the feature extraction network for image data, and realizes a significant expansion of the global receptive field through its super-large kernel convolution design. Compared with traditional convolutional neural networks, RepLKNet has stronger global information capture ability when extracting complex cable surface features, and can effectively identify abnormalities such as cracks, wear, and damage to the surface insulation layer. In addition, through structural optimization, the large kernel convolution reduces the impact of high-dimensional calculations on network performance, improves processing efficiency, and significantly enhances the response speed of the system in real-time monitoring scenarios.

[0055] Second, in the process of fusing electrical quantity data and image data, the present invention proposes a fusion strategy based on multi-modal alignment and feature interaction. By encoding electrical quantity data through a deep neural network and combining the image features extracted by RepLKNet, the present invention precisely aligns and jointly models the features of the two modalities. First, the time series features of electrical quantity data are mapped to a unified feature space through a fully connected network, and image features are extracted to a high-dimensional space through multi-level convolution. The two features are initially integrated through a concatenation operation. Then, through a feature interaction module, collaborative optimization between modalities is achieved, enhancing its ability to identify multi-dimensional faults. The feature interaction module uses an attention mechanism to dynamically allocate feature weights, highlighting key features in the current task and avoiding the problem of insufficient single-modal feature information.

[0056] The present invention introduces a feature learning loss function and a dynamic weight adjustment mechanism based on multi-modal alignment in model optimization. The feature learning loss function reduces feature redundancy between modalities through joint optimization of electrical quantity and image features, improving the accuracy and consistency of feature representation. The dynamic weight adjustment mechanism realizes the balance of different optimization objectives by adjusting the optimization weights of classification loss and feature consistency loss in real-time during training, enabling the model to achieve better performance in both fault classification and environmental adaptability. This mechanism significantly improves the robustness of the system in high-noise and complex disturbance environments.

[0057] In addition, the present invention also introduces a multi-modal data augmentation technique. By adding independent augmentation and joint perturbation operations to electrical quantities and image data, the diversity of training samples is significantly enriched. This augmentation technique can simulate abnormal signals and background noise in the actual complex environment, thereby improving the generalization ability of the model and enabling it to maintain stable performance in unseen samples.

[0058] In summary, through the large kernel convolution design of RepLKNet, the multi-modal feature fusion strategy, the feature learning optimization mechanism, and the innovation of data augmentation technology, the present invention successfully solves the technical problems faced by traditional methods in the cable monitoring process, such as insufficient robustness, low classification accuracy, and poor real-time performance. The application of this technology in the power system can improve the intelligent level of cable monitoring and provide strong technical support for the safe operation and fault diagnosis of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a schematic diagram of the machine vision cable monitoring structure of the present invention;

[0060] Figure 2 is a schematic diagram of the multi-modal fusion cable fault monitoring and health status assessment proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] The present invention will be further described below in conjunction with embodiments, but it should not be understood that the above-mentioned subject matter scope of the present invention is limited to the following embodiments. Without departing from the above-mentioned technical ideas of the present invention, various substitutions and changes made according to ordinary technical knowledge and customary means in the art shall be included in the protection scope of the present invention.

[0062] Embodiment 1:

[0063] See Figures 1 to 2 , a method for cable fault monitoring and health status assessment based on multi-modal fusion, comprising the following steps:

[0064] 1) Collect cable multi-modal data, including electrical quantity data and cable image data;

[0065] 2) Preprocess the electrical quantity data and cable image data;

[0066] 3) Extract and fuse features from the preprocessed electrical quantity data and cable image data to obtain a multi-modal feature vector;

[0067] 4) Construct a cable fault classification model;

[0068] 5) Input the multi-modal feature vector into the cable fault classification model to obtain the cable operation status type;

[0069] 6) Calculate the cable health status index based on the cable operation status type and the multi-modal feature vector, and predict the remaining service life of the cable based on the cable health status index.

[0070] The electrical quantity data is periodically collected through voltage sensors and current sensors.

[0071] The cable image data includes visible light images of the cable surface and the surrounding environment, cable thermal imaging data, and cable three-dimensional point cloud data.

[0072] The visible light images of the cable surface and the surrounding environment are collected through a natural light camera;

[0073] The cable thermal imaging data is collected through an infrared camera;

[0074] The cable three-dimensional point cloud data is collected and reconstructed through a laser scanner.

[0075] The steps for preprocessing the electrical quantity data include low-pass filtering and missing data filling;

[0076] The low-pass filtering is implemented through a low-pass filter;

[0077] The transfer function H(f) of the low-pass filter is as follows:

[0078]

[0079] where f is the signal frequency, f c is the cut-off frequency of the filter; j is the imaginary number;

[0080] The missing data filling is implemented through linear interpolation, that is:

[0081]

[0082] where X1 and X2 are known data points, t1 and t2 are the corresponding time points, and t is the time point of the missing data. X missing is the filled missing data.

[0083] The steps for preprocessing the cable image data include noise smoothing, contrast stretching, edge detection, and geometric correction;

[0084] The image after noise smoothing is as follows:

[0085]

[0086] where I out (x,y) is the filtered image, I(x,y) is the original image, N is the number of pixels in the neighborhood, and k is the radius of the filtering window;

[0087] The image after contrast stretching is shown as follows:

[0088]

[0089] Among them, f(x, y) and g(x, y) are the original image and the enhanced image respectively, max(f) and min(f) are the maximum and minimum gray values of the original image respectively, and max(g) and min(g) are the gray value ranges after enhancement.

[0090] Edge detection is implemented through the Sobel operator, that is:

[0091]

[0092] Among them, G x and G y represent the gradients of the image in the horizontal and vertical directions respectively.

[0093] In step 3), the steps of feature extraction and fusion for the preprocessed electrical quantity data and cable image data include:

[0094] 3.1) Perform data normalization on the preprocessed electrical quantity data;

[0095] 3.2) Use the electrical quantity data feature extraction model to map the electrical quantity data into a fixed-length electrical quantity feature vector f elec ;

[0096] The electrical quantity data feature extraction model is a deep neural network; the electrical quantity data feature extraction model is trained by a historical sample data set with normalized electrical quantity data as the input and electrical quantity feature vector as the output;

[0097] 3.3) Use the image feature extraction model to extract the image feature vector f img ;

[0098] The image feature extraction model is RepLKNet, including an initial convolutional layer, a large kernel convolutional module, and a fully connected layer; the image feature extraction model is trained by a historical sample data set with cable image data as the input and image feature vector as the output;

[0099] 3.4) Fuse the electrical quantity feature vector f elec and the image feature vector f img to obtain a multi-modal feature vector f concat , that is:

[0100] f concat = [f elec ; f img (6)

[0101] The cable fault classification model is a multi-modal fusion neural network based on the attention mechanism. The fused features are mapped to the fault category space through a fully connected layer, and the output classification probability is:

[0102] P = Softmax(W a ·f concat +b a )(7)

[0103] where P is the classification probability; W a , b a are the weight and bias.

[0104] The types of cable operating states include normal operation, local overheating, surface damage, or short-circuit faults.

[0105] Based on the fused data, calculate the cable health index HI, i.e.:

[0106]

[0107] where R fault is the fault risk score, S feature is the multi-modal feature score, S historical is the historical score;

[0108] The remaining useful life of the cable is as follows:

[0109]

[0110] where RUL represents the remaining useful life of the cable, HI is the current health index, H th is the health state threshold, is the decay rate of the health index over time.

[0111] After predicting the remaining useful life of the cable, give risk warnings that may occur: low risk (slight aging or normal wear, etc.), medium risk (insulation deterioration, discharge phenomenon, overheating, joint abnormality), high risk (insulation breakdown, severe overheating, sheath damage, short-circuit risk).

[0112] Example 2:

[0113] A cable fault monitoring and health state evaluation method based on multi-modal fusion includes the following steps:

[0114] 1) Collect cable multi-modal data, including electrical quantity data and cable image data;

[0115] 2) Preprocess the electrical quantity data and cable image data;

[0116] 3) Extract and fuse the features of the preprocessed electrical quantity data and cable image data to obtain a multi-modal feature vector;

[0117] 4) Build a cable fault classification model;

[0118] 5) Input the multi-modal feature vector into the cable fault classification model to obtain the cable operation status type;

[0119] 6) Calculate the cable health status index based on the cable operation status type and the multi-modal feature vector, and predict the remaining service life of the cable based on the cable health status index.

[0120] Example 3:

[0121] A cable fault monitoring and health status evaluation method based on multi-modal fusion, the technical content is the same as that of Example 2. Further, the electrical quantity data is periodically collected by voltage sensors and current sensors.

[0122] Example 4:

[0123] A cable fault monitoring and health status evaluation method based on multi-modal fusion, the technical content is the same as any one of Examples 2-3. Further, the cable image data includes visible light images of the cable surface and the surrounding environment, cable thermal imaging data, and cable three-dimensional point cloud data.

[0124] Example 5:

[0125] A cable fault monitoring and health status evaluation method based on multi-modal fusion, the technical content is the same as any one of Examples 2-4. Further, the visible light image of the cable surface and the surrounding environment is collected by a natural light camera;

[0126] The cable thermal imaging data is collected by an infrared camera;

[0127] The cable three-dimensional point cloud data is collected and reconstructed by a laser scanner.

[0128] Example 6:

[0129] A cable fault monitoring and health status evaluation method based on multi-modal fusion, the technical content is the same as any one of Examples 2-5. Further, the steps for preprocessing the electrical quantity data include low-pass filtering and missing data filling;

[0130] The low-pass filtering is implemented by a low-pass filter;

[0131] The transfer function of the low-pass filter is as follows:

[0132]

[0133] Where f is the signal frequency, fc is the cut-off frequency of the filter;

[0134] The missing data filling is achieved by the linear interpolation method, i.e.:

[0135]

[0136] wherein, X1 and X2 are known data points, t1 and t2 are the corresponding time points, and t is the time point of the missing data.

[0137] Example 7:

[0138] A cable fault monitoring and health status evaluation method based on multimodal fusion, the technical content is the same as any one of Examples 2-6. Further, the steps of preprocessing the cable image data include noise smoothing, contrast stretching, edge detection, and geometric correction;

[0139] The image after noise smoothing is as follows:

[0140]

[0141] wherein, I out (x,y) is the filtered image, I(x,y) is the original image, N is the number of pixels in the neighborhood, and k is the radius of the filter window;

[0142] The image after contrast stretching is as follows:

[0143]

[0144] wherein, f(x,y) and g(x,y) are the original image and the enhanced image respectively, max(f) and min(f) are the maximum and minimum gray values of the original image respectively, and max(g) and min(g) are the gray value ranges after enhancement.

[0145] Edge detection is achieved by the Sobel operator, i.e.:

[0146]

[0147] wherein, G x and G y respectively represent the gradients of the image in the horizontal and vertical directions.

[0148] Example 8:

[0149] A cable fault monitoring and health status evaluation method based on multimodal fusion, the technical content is the same as any one of Examples 2-7. Further, in step 3), the steps of feature extraction and fusion of the preprocessed electrical quantity data and cable image data include:

[0150] 3.1) Perform data normalization on the preprocessed electrical quantity data;

[0151] 3.2) Use the electrical quantity data feature extraction model to map the electrical quantity data into a fixed-length electrical quantity feature vector f elec ;

[0152] The electrical quantity data feature extraction model is a deep neural network; the electrical quantity data feature extraction model is trained by a historical sample data set with normalized electrical quantity data as input and electrical quantity feature vector as output;

[0153] 3.3) Use the image feature extraction model to extract the image feature vector f of the cable image data img ;

[0154] The image feature extraction model is RepLKNet, including an initial convolutional layer, a large kernel convolutional module, and a fully connected layer; the image feature extraction model is trained by a historical sample data set with cable image data as input and image feature vector as output;

[0155] 3.4) Fuse the electrical quantity feature vector f elec and the image feature vector f img to obtain a multi-modal feature vector f concat , that is:

[0156] f concat = [f elec ; f img (6)

[0157] Example 9:

[0158] A cable fault monitoring and health status evaluation method based on multi-modal fusion, the technical content is the same as any one of Examples 2-8. Further, the cable fault classification model is a multi-modal fusion neural network based on the attention mechanism, and the fusion features are mapped to the fault category space through a fully connected layer, and the output classification probability is:

[0159] P = Softmax(W a · f concat + b a )(7)

[0160] Example 10:

[0161] A cable fault monitoring and health status evaluation method based on multi-modal fusion, the technical content is the same as any one of Examples 2-9. Further, the cable operation state types include normal operation, local overheating, surface damage, or short circuit fault.

[0162] Example 11:

[0163] A cable fault monitoring and health status assessment method based on multimodal fusion, the technical content is the same as any one of Embodiments 2-10. Further, based on the fused data, calculate the cable health status index, that is:

[0164]

[0165] where R fault is the fault risk score, S feature is the multimodal feature score, and S historical is the historical score.

[0166] Evaluate the change of the health status index through trend analysis and prediction model, and provide the prediction result of the remaining service life of the cable. Its calculation method is shown in formula (9). And give risk warnings that may occur: low risk (slight aging or normal wear, etc.), medium risk (insulation deterioration, discharge phenomenon, overheating, joint abnormality), high risk (insulation breakdown, severe overheating, sheath damage, short-circuit risk).

[0167]

[0168] where RUL represents the remaining service life of the cable, HI is the current health status index, and H th is the health status threshold, is the decay rate of the health status index over time, which can be obtained by fitting historical data.

[0169] Embodiment 12:

[0170] A cable fault monitoring and health status assessment method based on multimodal fusion, the training process is as attached Figure 2 shown, including the following steps:

[0171] Step 1: Multimodal data acquisition

[0172] (1) Electrical quantity data: Collect data according to the already installed voltage sensors, current sensors, etc., and transmit it to the data recording device or remote monitoring platform, and record the data at time intervals. During the process of collecting data from multiple sensors, perform clock synchronization through NTP technology, and attach a timestamp to each group of data to ensure the accurate timing of the data. The collected data will be stored and transmitted, saved through the local storage device, and can be transmitted to the central server for storage and processing through wireless communication or optical fiber, etc.

[0173] (2) Cable image data:

[0174] The natural light camera is used to acquire visible light images of the cable surface and the surrounding environment, and can effectively identify anomalies such as cracks, wear, and sheath damage on the cable surface; the infrared camera monitors potential faults such as local overheating or joint overheating of the cable by capturing the thermal imaging data of the cable, providing an intuitive representation of the temperature change; the laser scanner is used to generate high-precision three-dimensional point cloud data, and by scanning the cable tunnel, it detects problems such as structural deformation, bracket damage, and foreign object intrusion in the cable tunnel; the mobile detection unit (MPU), such as the rail robot and the autonomous mobile robot, can enter narrow and difficult-to-reach areas for supplementary monitoring.

[0175] All image acquisition devices have the functions of waterproof, dustproof, moisture-proof, and anti-electromagnetic interference, and can work stably in the complex environment of the underground cable channel. The devices also support the night vision function, enabling effective monitoring even in environments with insufficient light or complete darkness. The collected image data is transmitted to the server unit through the optical fiber network for subsequent processing.

[0176] Step Two: Data Preprocessing

[0177] (1) Electrical Quantity Data

[0178] The main purpose of preprocessing the electrical quantity data is to clean the noise, enhance the signal quality, and ensure the consistency and reliability of the data. A low-pass filter is used to remove high-frequency noise. The transfer function of the low-pass filter is:

[0179]

[0180] where f is the signal frequency, f c is the cut-off frequency of the filter. When the frequency of the signal is higher than the cut-off frequency, the signal will be attenuated, thus removing the noise.

[0181] The electrical quantity data may be missing due to reasons such as sensor failure and communication loss. To ensure the integrity of the data, a linear interpolation method is used to fill in the missing data, and its formula is:

[0182]

[0183] where X1 and X2 are known data points, t1 and t2 are the corresponding time points, and t is the time point of the missing data.

[0184] (2) Cable Image Data

[0185] Since the cable image data is itself susceptible to noise, light changes, motion blur, and environmental factors, a series of preprocessing steps must be carried out before subsequent analysis.

[0186] First, unnecessary noise is removed through mean filtering while retaining valid image information. The image is smoothed by calculating the mean of the neighborhood around each pixel, and the formula is:

[0187]

[0188] where, I out (x,y) is the filtered image, I(x,y) is the original image, N is the number of pixels in the neighborhood, and k is the radius of the filtering window.

[0189] Subsequently, to make the features of the cable image more obvious, the contrast stretching method is used to increase the contrast and brightness of the cable image in low-light situations, and the formula is:

[0190]

[0191] where, f(x,y) and g(x,y) are the original image and the enhanced image respectively, max(f) and min(f) are the maximum and minimum gray values of the original image respectively, and max(g) and min(g) are the gray value ranges after enhancement.

[0192] Edge detection calculations are performed on the cable image data to highlight features such as cracks and wear on the cable surface. The gradient calculation formula of the Sobel operator is:

[0193]

[0194] where, G x and G y represent the gradients of the image in the horizontal and vertical directions respectively. For regions with large gradient changes, the Sobel operator extracts the edge information of the cable image by calculating the image gradient.

[0195] Finally, the geometric correction method is used to eliminate the image distortion caused by the shooting angle, ensuring that images taken at different angles can be accurately aligned; and different perspective images are synthesized into a complete cable image through feature matching and image fusion techniques.

[0196] Step 3: Feature Extraction and Fusion

[0197] The electrical quantity data extracts features layer by layer through a deep neural network. First, the data is normalized to eliminate the dimension difference, and then it is gradually abstracted through multi-layer non-linear mapping to extract local patterns and complex trends, and finally a high-dimensional feature vector is output to provide support for fault diagnosis.

[0198] The extraction of cable image features is achieved using RepLKNet (Re-parameterized Large Kernel Network). The input image passes through the initial convolutional layer to extract basic low-level features such as edges and textures, and then enters the large kernel convolutional module. These modules use extremely large convolutional kernels (31×31), significantly expanding the receptive field, enabling the network to capture richer cable image information, and combining channel convolution to achieve the fusion of multi-channel features. After multi-level convolution and feature integration, RepLKNet generates high-dimensional feature maps, and its feature representation can cover local details (such as cracks and wear) and global structures (such as overall morphological changes) in the image, providing precise support for cable condition analysis.

[0199] Multi-modal feature fusion improves the detection accuracy and robustness of the model by integrating the features of electrical quantity data and image data. The electrical quantity data is mapped into a fixed-length feature vector f through a fully connected layer. elec While the image data generates an image feature vector f after convolution operations and fully connected layer processing. img The aligned features are combined into a multi-modal feature vector f through feature concatenation. concat This can directly retain the global information of both modalities, and its mathematical representation is:

[0200] f concat =[f elec ; f img (6)

[0201] A feature interaction module is used for deep fusion. During the interaction process, the attention mechanism is used to assign different weights to the features of the two modalities to highlight the modality features that are more important for the current task, enabling the model to effectively combine the feature information of the electrical quantity and image modalities.

[0202] Step Four: Cable Fault Monitoring

[0203] By analyzing the multi-modal fusion features, the operating status and potential faults of the cable are monitored in real time. The extracted and fused multi-modal features are input into a classification model, which is a multi-modal fusion neural network based on the attention mechanism. The fused features are mapped to the fault category space through a fully connected layer, and the output classification probability is:

[0204] P=Softmax(W a ·f concat +b a )(7)

[0205] It can handle complex high-dimensional features and classify cable faults. The classification model predicts the fault categories based on the input features, and the output results include multiple possible cable operating states, such as normal operation, local overheating, surface damage, or short-circuit faults, etc. Finally, the model generates alarm information or maintenance suggestions according to the prediction results to remind the operation and maintenance personnel to take measures in time.

[0206] Step Five: Cable Health Status Assessment

[0207] Based on long-term multi-modal monitoring data and historical trends, a comprehensive assessment model of cable health status is constructed. The model fuses electrical quantity features, image features, and the output results of the classification model to form a comprehensive time series data set. Then, based on the fused data, the cable health status index is calculated, that is:

[0208]

[0209] where, R fault is the fault risk score, S feature is the multi-modal feature score, S historical is the historical score.

[0210] It comprehensively reflects the operating conditions and potential risks of the cable. The health status index combines the time-series changes of electrical quantity features, the local damage manifestations of image features, and the risk scores of fault classification results, and comprehensively quantifies the health status of the cable from multiple dimensions. Through trend analysis and prediction models, the changes of the health status index are evaluated to provide the prediction results of the remaining service life of the cable, and its calculation method is shown in formula (9). And risk tips that may occur are given: low risk (slight aging or normal wear, etc.), medium risk (insulation deterioration, discharge phenomenon, overheating, joint abnormality), high risk (insulation breakdown, severe overheating, sheath damage, short-circuit risk).

[0211]

[0212] where, RUL represents the remaining service life of the cable, HI is the current health status index, H th is the health status threshold, is the decay rate of the health status index over time, which can be obtained by fitting historical data.

Claims

1. A cable fault monitoring and health status assessment method based on multimodal fusion, characterized in that It includes the following steps: 1) Collect cable multimodal data, including electrical quantity data and cable image data; 2) Preprocess the electrical quantity data and cable image data; 3) Extract and fuse features from the preprocessed electrical quantity data and cable image data to obtain a multimodal feature vector; 4) Build a cable fault classification model; 5) Input the multimodal feature vector into the cable fault classification model to obtain the cable operation status type; 6) Calculate the cable health status index based on the cable operation status type and the multimodal feature vector, and predict the remaining service life of the cable based on the cable health status index.

2. The cable fault monitoring and health status assessment method based on multi-modal fusion according to claim 1, wherein The electrical quantity data is periodically collected through voltage sensors and current sensors.

3. A method for cable fault monitoring and health status assessment based on multi-modal fusion according to claim 1, characterized in that, The cable image data includes visible light images of the cable surface and the surrounding environment, cable thermal imaging data, and cable three-dimensional point cloud data.

4. A cable fault monitoring and health status evaluation method based on multimodal fusion according to claim 3, characterized in that The visible light images of the cable surface and the surrounding environment are collected by a natural light camera; The cable thermal imaging data is collected by an infrared camera; The cable three-dimensional point cloud data is collected and reconstructed by a laser scanner.

5. A method for cable fault monitoring and health status assessment based on multi-modal fusion according to claim 1, characterized in that, The steps for preprocessing the electrical quantity data include low-pass filtering and missing data filling; The low-pass filtering is implemented by a low-pass filter; The transfer function H(f) of the low-pass filter is as follows: where f is the signal frequency, and f c is the cut-off frequency of the filter; j is the imaginary number; The missing data filling is implemented by a linear interpolation method, that is: Among them, X1 and X2 are known data points, t1 and t2 are corresponding time points, and t is the time point of the missing data; X missing is the filled missing data.

6. A method for cable fault monitoring and health status assessment based on multi-modal fusion according to claim 1, characterized in that, The steps for preprocessing the cable image data include noise smoothing, contrast stretching, edge detection, and geometric correction; The image after noise smoothing is as follows: where I out (x, y) is the filtered image, I(x, y) is the original image, N is the number of pixels in the neighborhood, and k is the radius of the filtering window; The image after contrast stretching is as follows: where f(x, y) and g(x, y) are the original image and the enhanced image respectively, max(f) and min(f) are the maximum and minimum gray values of the original image respectively, and max(g) and min(g) are the gray value ranges of the enhanced image. The edge detection is implemented by a Sobel operator, that is: Among them, G x and G y respectively represent the gradients of the image in the horizontal and vertical directions.

7. A method for cable fault monitoring and health status assessment based on multimodal fusion according to claim 1, characterized in that, In step 3), the steps for feature extraction and fusion of the preprocessed electrical quantity data and cable image data include: 3.1) Perform data normalization processing on the preprocessed electrical quantity data; 3.2) Map the electrical quantity data to the electrical quantity feature vector f with a fixed length by using the electrical quantity data feature extraction model elec ; The electrical quantity data feature extraction model is a deep neural network; the electrical quantity data feature extraction model is trained by a historical sample data set with normalized electrical quantity data as the input and electrical quantity feature vectors as the output; 3.3) Use the image feature extraction model to extract the image feature vector f of the cable image data img ; The image feature extraction model is RepLKNet, including an initial convolutional layer, a large kernel convolutional module, and a fully connected layer; the image feature extraction model is trained by a historical sample data set with cable image data as the input and image feature vectors as the output; 3.4) Fuse the electrical quantity feature vector f elec and the image feature vector f img to obtain the multi-modal feature vector f concat , that is: f concat = [f elec ; f img (6).

8. A method for cable fault monitoring and health status assessment based on multimodal fusion according to claim 1, characterized in that, The cable fault classification model is a multimodal fusion neural network based on an attention mechanism, and the fused features are mapped to the fault category space through a fully connected layer, and the output classification probability is: P = Softmax(W a · f concat + b a )(7) where P is the classification probability; W a , b a are the weights and biases.

9. A cable fault monitoring and health status assessment method based on multimodal fusion according to claim 1, characterized in that The cable operation status types include normal operation, local overheating, surface damage, or short circuit fault.

10. A method for cable fault monitoring and health status assessment based on multimodal fusion according to claim 1, characterized in that, Based on the fused data, calculate the cable health status index HI, that is: Among them, R fault is the failure risk score, S feature is the multimodal feature score, and S historical is the historical score; The remaining service life of the cable is as follows: where RUL represents the remaining useful life of the cable, HI is the current health state index, and H th is the health state threshold, is the decay rate of the health state index over time.

Images