Multi-modal sensor fusion inspection method and system

Abstract

The invention relates to the technical field of multi-modal data processing, and discloses a multi-modal sensor fusion inspection method and system, and the method comprises the steps: collecting the multi-modal original data of power equipment through a multi-modal sensor in an inspection robot, and constructing a feature vector set; performing adaptive weight calculation on the multi-modal sensor according to the feature vector set to obtain a sensor weight set; carrying out conflict identification and resolution on the multi-modal original data to obtain a fusion data set; performing abnormal feature extraction on the power equipment based on the fused data set to obtain an abnormal feature set; and carrying out routing inspection trajectory optimization based on the abnormal feature set to obtain a target routing inspection path sequence, and carrying out equipment state joint prediction in combination with historical equipment routing inspection data to obtain an equipment fault prediction result. And thus, more accurate equipment state joint prediction is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal data processing, and particularly to an inspection method and system for multimodal sensor fusion. Background Art

[0002] Traditional power equipment inspection mainly relies on manual regular inspections and single-sensor monitoring. This method has problems such as long inspection cycles, limited coverage, and large interference from human factors, making it difficult to detect potential equipment failures and abnormal states in a timely manner. With the development of robot technology and sensor technology, intelligent inspection robots based on multimodal sensors have gradually been applied to the field of power equipment monitoring, capable of carrying various types of sensors such as thermal imaging, infrared, ultrasonic, electromagnetic field, and vibration, to achieve all-round and multi-dimensional monitoring of power equipment.

[0003] However, multimodal sensors face technical challenges in data conflict and information fusion in practical applications. Due to differences in working principles, measurement accuracies, environmental adaptabilities, etc. among different types of sensors, the monitoring results of the same equipment at the same moment may be inconsistent or even contradictory. Traditional data fusion methods usually adopt simple weighted averaging or majority voting mechanisms, which cannot effectively handle dynamic conflict problems between sensors and easily lead to distorted or misjudged fusion results. In addition, existing inspection path planning mostly adopts fixed routes or experience-based path selection, lacking comprehensive consideration of the real-time state of equipment and failure risks, and it is difficult to achieve optimal allocation of inspection resources and accurate prediction of failures. Summary of the Invention

[0004] The present invention provides an inspection method and system for multimodal sensor fusion, which can accurately identify data conflicts between multimodal sensors, realize intelligent screening and fusion of conflict data, and further achieve more accurate joint prediction of equipment states.

[0005] In a first aspect, the present invention provides an inspection method for multimodal sensor fusion, and the inspection method for multimodal sensor fusion includes: Collect multimodal raw data of power equipment through multimodal sensors in the inspection robot and construct a feature vector set; Perform adaptive weight calculation on the multimodal sensors according to the feature vector set to obtain a sensor weight set; Based on the sensor weight set, perform conflict identification and resolution on the multimodal raw data to obtain a fusion data set; Based on the fusion data set, extract abnormal features of the power equipment to obtain an abnormal feature set; Optimize the inspection trajectory based on the abnormal feature set to obtain a target inspection path sequence, and combine historical equipment inspection data for joint prediction of equipment status to obtain equipment failure prediction results.

[0006] Combined with the first aspect, in the first implementation manner of the first aspect of the present invention, collecting multi-modal raw data of power equipment through multi-modal sensors in the inspection robot and constructing a feature vector set includes: Collect multi-modal raw data of power equipment through multi-modal sensors of the inspection robot, and the multi-modal sensors include a thermal imaging sensor, an electromagnetic field sensor, and a vibration sensor; Perform noise filtering, data normalization, and time synchronization processing on the multi-modal raw data to obtain a preprocessed standardized data set, and the preprocessed standardized data set includes thermal imaging sensor data, electromagnetic field sensor data, and vibration sensor data; Extract temperature distribution features from the thermal imaging sensor data to obtain a thermal feature parameter set including temperature peak, temperature mean, and temperature variance, and generate a thermal feature vector based on the thermal feature parameter set; Perform electromagnetic field strength spectrum analysis on the electromagnetic field sensor data to obtain an electromagnetic feature parameter set, and generate an electromagnetic feature vector based on the electromagnetic feature parameter set; Perform time-frequency domain transformation on the vibration sensor data to obtain a vibration feature parameter set, and generate a vibration feature vector according to the vibration feature parameter set; Use the thermal feature vector, the electromagnetic feature vector, and the vibration feature vector as the feature vector set.

[0007] Combined with the first aspect, in the second implementation manner of the first aspect of the present invention, calculating the adaptive weights of the multi-modal sensors according to the feature vector set to obtain a sensor weight set includes: Based on the feature vector set, calculate the output volatility of the multi-modal sensors under different temperature conditions to obtain a thermal stability index, analyze the historical measurement accuracy of the multi-modal sensors to obtain an accuracy historical index, and evaluate the signal-to-noise ratio of the multi-modal sensors to obtain a signal strength index; Set different thermal stability coefficients, accuracy historical coefficients, and signal strength coefficients according to the types of power equipment; By calculating the sum of the product of the thermal stability index and the thermal stability coefficient, the product of the accuracy historical index and the accuracy historical coefficient, and the product of the signal strength index and the signal strength coefficient, obtain the weight values of each sensor in the multi-modal sensors; Arrange the weight values of each sensor in the order of sensor identifiers and form a sensor weight set.

[0008] Combined with the first aspect, in the third implementation manner of the first aspect of the present invention, the conflict identification and resolution of the multi-modal raw data based on the sensor weight set to obtain a fusion data set includes: Calculating the data consistency between measurement values of different sensors based on the sensor weight set and the multi-modal raw data, and determining that there is a data conflict when the data consistency is lower than the consistency threshold to obtain a sensor conflict identification result; Performing feature correlation analysis on the conflicting sensors according to the sensor conflict identification result to obtain a feature correlation function value; Inputting the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm for data screening to obtain an initial conflict resolution result; Constructing a sensor trust network graph according to the initial conflict resolution result and applying the maximum weight spanning tree algorithm to identify the most credible sensor set, and at the same time performing time window analysis on periodic conflicts to obtain a fusion data set.

[0009] Combined with the first aspect, in the fourth implementation manner of the first aspect of the present invention, the inputting the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm for data screening to obtain an initial conflict resolution result includes: Inputting the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm to identify conflicting sensor data pairs and extract the corresponding weight values to obtain a conflict data input set including conflict sensor identifiers, measurement data, and weight values; Based on the conflict data input set, performing weighted processing on the measurement data of each conflicting sensor to obtain weighted sensor data values; According to the weighted sensor data values, calculating the ratio of the absolute difference between the data to the power equipment deviation threshold and subtracting this ratio from 1 to obtain a deviation correction factor; Performing a product operation on the maximum value in the weighted sensor data values and the deviation correction factor to obtain the most reliable sensor data after deviation correction, and generating a corresponding initial conflict resolution result based on the most reliable sensor data after deviation correction.

[0010] Combined with the first aspect, in the fifth implementation manner of the first aspect of the present invention, the abnormal feature extraction of the power equipment based on the fusion data set to obtain an abnormal feature set includes: Performing feature mapping on the fusion data set to obtain a feature space vector of the power equipment; Set the normal lower limit and normal upper limit of electrical characteristics, thermal characteristics, and mechanical characteristics according to the type of power equipment to obtain a set of equipment state boundaries including a minimum boundary vector and a maximum boundary vector; Calculate the minimum distance from the feature space vector to the set of equipment state boundaries and combine it with the equipment type sensitivity coefficient to obtain an abnormality index value; Perform abnormality determination and feature classification processing based on the abnormality index value to obtain a set of abnormal features.

[0011] Combined with the first aspect, in the sixth implementation manner of the first aspect of the present invention, the inspection trajectory is optimized based on the set of abnormal features to obtain a target inspection path sequence, and the equipment state is jointly predicted in combination with historical equipment inspection data to obtain an equipment failure prediction result, including: Execute failure probability density calculation based on the set of abnormal features to obtain the failure probability density value of the power equipment; Calculate the importance score of the power equipment based on the failure probability density value to obtain a set of equipment evaluation parameters; Input the failure probability density value, the set of equipment evaluation parameters, and the physical distance between equipment into the EIPO inspection optimization algorithm to perform multi-objective constrained inspection trajectory optimization to obtain a target inspection path sequence; Perform joint prediction of equipment state on the target inspection path sequence and historical equipment inspection data to obtain an equipment failure prediction result.

[0012] Combined with the first aspect, in the seventh implementation manner of the first aspect of the present invention, the input of the failure probability density value, the set of equipment evaluation parameters, and the physical distance between equipment into the EIPO inspection optimization algorithm to perform multi-objective constrained inspection trajectory optimization to obtain a target inspection path sequence includes: Input the failure probability density value, the set of equipment evaluation parameters, and the physical distance between equipment into the EIPO inspection optimization algorithm to construct a comprehensive optimization objective function including distance cost, failure risk, and equipment importance; Perform adaptive parameter setting on the distance weight coefficient, failure probability weight coefficient, and equipment importance weight coefficient in the comprehensive optimization objective function according to the inspection scenario type to obtain a set of optimized parameter configurations; Perform iterative solution based on the comprehensive optimization objective function and the set of optimized parameter configurations to obtain a candidate inspection path solution set; Perform convergence determination and optimal solution selection on the candidate inspection path solution set to obtain a target inspection path sequence.

[0013] Combined with the first aspect, in the eighth implementation manner of the first aspect of the present invention, the joint prediction of the device state for the target inspection path sequence and historical device inspection data to obtain a device failure prediction result includes: Construct a device association graph structure including device nodes, associated edges, and associated strength weights based on the target inspection path sequence and the historical device inspection data; Perform joint prediction of the power device state on the device association graph structure and the current device state sequence to obtain a predicted state data set; Based on the predicted state data set and the device association graph structure, simulate the propagation process of faults in the device network through the heat conduction equation and calculate the device-specific diffusion coefficient to obtain fault probability distribution data; Perform fault risk assessment and visualization processing according to the fault probability distribution data to obtain a device failure prediction result including potential fault propagation paths and device risk levels.

[0014] In a second aspect, the present invention provides an inspection system with multi-modal sensor fusion, and the inspection system with multi-modal sensor fusion includes: An acquisition module for collecting multi-modal raw data of power devices through multi-modal sensors in an inspection robot and constructing a feature vector set; A calculation module for adaptively calculating weights for the multi-modal sensors according to the feature vector set to obtain a sensor weight set; A conflict identification module for identifying and resolving conflicts in the multi-modal raw data based on the sensor weight set to obtain a fused data set; A feature extraction module for extracting abnormal features of the power devices based on the fused data set to obtain an abnormal feature set; A joint prediction module for optimizing the inspection trajectory based on the abnormal feature set to obtain a target inspection path sequence, and performing joint prediction of the device state in combination with historical device inspection data to obtain a device failure prediction result.

[0015] In the technical solution provided by the present invention, by constructing a sensor adaptive weight evaluation model based on thermal stability, accuracy history, and signal strength, the reliability index of each sensor can be dynamically quantified, effectively solving the problem that the traditional fixed weight method cannot cope with sensor aging, environmental interference, and equipment state changes, and ensuring the accuracy and robustness of multi-modal data fusion. Combining the dynamic confidence matrix and the feature correlation function can accurately identify data conflicts between multi-modal sensors and achieve intelligent screening and fusion of conflicting data. Compared with the traditional simple weighted average or majority voting mechanism, it has stronger conflict handling ability and data fusion accuracy. By establishing a three-dimensional feature space and equipment state boundary, real-time processing of conflicting data in milliseconds is achieved, avoiding the computational bottleneck of traditional deep learning methods on embedded platforms and meeting the real-time requirements of power equipment inspection. By constructing the power equipment failure probability density function as the core constraint, the inspection path planning of multi-objective optimization is realized, comprehensively considering the equipment state, importance, and spatial distribution. Compared with the traditional fixed path or simple greedy algorithm, it can achieve the optimal allocation of inspection resources and key equipment monitoring. The present invention can effectively capture the spatial correlation relationship between power equipment. By establishing an equipment association graph and analyzing the physical connection, electrical influence, and fault propagation relationship, it solves the limitation that the traditional time series model cannot handle spatial dependence and realizes more accurate joint prediction of equipment states. By simulating the fault propagation process in the equipment network through the heat conduction equation, potential fault propagation paths and cascading reaction risks can be identified. For different types of power equipment and inspection scenarios, the system can adaptively adjust algorithm parameters and optimization strategies, including sensor weight coefficients, deviation thresholds, inspection weight parameters, etc., ensuring universality and effectiveness in different application environments. Brief Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 It is a schematic diagram of the steps of the inspection method for multi-modal sensor fusion in the embodiments of the present invention; Figure 2 It is a schematic diagram of the structure of the inspection system for multi-modal sensor fusion in the embodiments of the present invention. Detailed Embodiments

[0018] An embodiment of the present invention provides a patrol inspection method and system for multi-modal sensor fusion. Terms such as "first", "second", "third", "fourth", etc. (if any) in the specification, claims and above-mentioned drawings of the present invention are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0019] For ease of understanding, the specific process of the embodiment of the present invention is described below. Please refer to Figure 1 , an embodiment of the patrol inspection method for multi-modal sensor fusion in the embodiment of the present invention includes: Step S1, collecting multi-modal raw data of power equipment through multi-modal sensors in the patrol inspection robot and constructing a feature vector set; It can be understood that the execution subject of the present invention can be a patrol inspection system for multi-modal sensor fusion, or a terminal or a server, and specific limitations are not made here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0020] Specifically, through the multi-modal sensing unit installed at the front end, top or rotatable platform of the inspection robot, the corresponding thermal, electromagnetic and mechanical response information is synchronously obtained at the power equipment operation site, and the original multi-modal data stream with continuous time series is obtained through the high-frequency sampling mechanism. Three standardization processing steps of noise filtering, data normalization and time synchronization are sequentially performed on the original data. Band-pass filtering algorithm is used for noise filtering, and environmental vibration and thermal drift interference are eliminated while retaining the effective signals in the range of 10 Hz to 500 Hz; the normalization process adopts the maximum-minimum normalization strategy to uniformly map different physical quantities to the interval [-1,1] to solve the problem of inconsistent dimensions of thermal, field and vibration signals; and time synchronization aligns the three data streams to the 1 ms level through the high-precision timestamp correction algorithm to construct a standardized multi-modal data set with a unified structure and consistent time series. Targeted feature extraction operations are respectively performed on the three types of data. Among them, the thermal imaging sensor data undergoes thermal image decoding and regional statistical analysis, and the temperature peak value of the key part in the thermal image, the average temperature of the whole image and the local temperature variance are extracted to construct a thermal feature parameter set reflecting the abnormal trend of thermal distribution, and based on this, a thermal feature vector is generated to depict the subtle change trend of the thermal state of the equipment surface. The electromagnetic field sensor data takes frequency spectrum analysis as the core, and the fast Fourier transform or wavelet packet transform is used to perform frequency domain analysis on the electromagnetic field intensity data to capture the amplitude change in the high-frequency and harmonic intervals. By identifying signals such as power frequency fundamental wave offset and harmonic energy anomaly, an electromagnetic feature parameter set including main frequency intensity, frequency band energy distribution and spectrum balance factor is constructed, and based on this, an electromagnetic feature vector is generated to depict the signs of electromagnetic leakage or insulation degradation of the equipment. For the vibration sensor data, through the combined time-frequency domain analysis method, the short-time Fourier transform and the Hilbert-Huang transform are combined to perform multi-scale feature extraction on the vibration waveform, forming a vibration feature parameter set including vibration amplitude peak value, main frequency component, energy concentration degree, impact response index, etc., and generating a vibration feature vector that can distinguish fault modes such as mechanical looseness and fatigue cracks. The thermal feature vector, electromagnetic feature vector and vibration feature vector are combined according to a unified structure to form a feature vector set.

[0021] Step S2: Calculate the adaptive weights of the multi-modal sensors according to the feature vector set to obtain a sensor weight set; Specifically, based on the feature vector set, analyze and quantitatively model three performance indicators of each sensor in the current inspection cycle, namely thermal stability, historical accuracy, and signal strength. The evaluation of the thermal stability indicator requires the output change data of each sensor under different environmental temperature conditions, and quantifies the response consistency under thermal disturbance conditions by statistically measuring its output volatility within a set temperature range, that is, calculates the standard deviation of thermal characteristic parameters such as temperature peak or mean during multiple sampling processes in the same equipment area, so as to evaluate the environmental adaptability of thermal imaging or electromagnetic induction sensors. At the same time, to reflect the measurement historical reliability of each sensor, analyze its compliance with the known equipment status in previous inspection tasks, calculate the deviation between the eigenvalue and the reference value generated in each acquisition cycle, and summarize it as the historical accuracy indicator to depict its long-term stability and reliability trend. The signal strength indicator is evaluated through the signal-to-noise ratio. The system measures the power of the signal collected by each type of sensor in the current operation cycle, and calculates the corresponding signal-to-noise power ratio in combination with the background noise level to obtain its effective data capture ability in a complex electromagnetic or vibration environment. Coefficient factors required for weight calculation are set for equipment such as transformers, high-voltage switches, and transmission lines, including thermal stability coefficients, accuracy historical coefficients, and signal strength coefficients. For example, when inspecting transformers, more attention is paid to thermal stability, so the weight of thermal stability is higher in the corresponding coefficient setting; when processing transmission line data, due to stronger electromagnetic interference, the proportion of the signal strength coefficient will be significantly increased. Based on this differential configuration, perform weighted calculation on each sensor, multiply the current thermal stability indicator of the sensor by the thermal stability coefficient under the corresponding equipment type, multiply the accuracy historical indicator by the corresponding accuracy coefficient, multiply the signal strength indicator by the signal strength coefficient, and sum up these three weighted results to obtain the total weight value of the current sensor in this inspection task. Sort the above calculation results according to the unique identifier of each sensor, arrange the weight values of all sensors in a fixed order in sequence, and construct a sensor weight set. This weight set participates in the decision-making process as a confidence basis in the subsequent data conflict identification and fusion stage, and is dynamically updated as the task environment changes to achieve real-time adaptation to factors such as sensor aging, environmental disturbance, and equipment status changes.

[0022] Step S3: Based on the sensor weight set, identify and resolve conflicts in the multimodal raw data to obtain a fused data set; Specifically, the sensor weight set and the multi-modal raw data collected by the inspection robot are jointly input into the data consistency calculation model. Based on time series, this model pairs and compares the measurement values of multiple sensors (such as thermal imaging, electromagnetic induction, and vibration sensors) for the same target area within the same time window, and calculates the consistency metric between their values accordingly. The consistency index is realized through a normalized relative error function based on the relative difference between the measurement values. During the comparison process, the difference between the measurement values of each pair of sensors is normalized with the sum of their total measurement values and a very small number to avoid numerical instability problems caused by extreme values. When the consistency index between the measurement values of two or more sensors is lower than a preset threshold (such as 0.85), the system determines that a data conflict has occurred, records the conflict time, conflict sensor number, and conflict type, and forms a preliminary sensor conflict identification result. Based on the identified conflict sensor combinations, feature correlation analysis is carried out to construct a correlation function for quantifying the similarity of measurement trends between the conflict sensors. This function comprehensively examines the response trends of the conflict sensors to key feature parameters (such as temperature peak, main frequency of electric field strength, or vibration energy density) over multiple past time steps. By calculating the Pearson correlation coefficient after normalizing the mean and variance of their historical feature vectors, the correlation value between each pair of conflict sensors is obtained, and further determine whether the data conflict is caused by short-term interference or due to sensor accuracy degradation or signal drift. The conflict identification result, the sensor weight set, the feature correlation function value, and the deviation threshold set for specific power equipment (such as the allowable temperature deviation of the transformer is 15°C, and the insulator surface is 25°C, etc.) are jointly input into the SyncDrop-E conflict resolution algorithm. Based on the weighted voting mechanism and combined with the principle of maximum reliability optimization, in the conflicting value pairs, the more stable data that is closer to the threshold range among the weighted measurement values is selected as the retained value, and the relative offset is used as the compensation correction coefficient to generate a preliminary conflict resolution result. The maximum weighted effective value is directly used in a single conflict pair, while in the complex scenario of multi-sensor conflicts, further fusion optimization depends on a structured graph model. To enhance the system's processing ability for complex scenarios, a sensor trust network graph is constructed based on the above preliminary results. All relevant sensors are regarded as nodes in the graph, and the edge weights between the nodes are the data consistency indicators at the corresponding moments. Through the maximum weight spanning tree algorithm in graph theory, a sensor set with the strongest global consistency and the best data quality is automatically selected to ensure that the final decision is based on a stable and reliable data subset. At the same time, considering the repetitive conflict phenomenon caused by external periodic interference or structural vibration in some parts of the power equipment, the system parallelly introduces a time window analysis mechanism to perform a sliding window analysis on the output trends of each suspected abnormal sensor in multiple consecutive sampling periods, and eliminates the misjudged data that shows periodic fluctuations but has no actual fault characteristics. A fused data set is obtained.

[0023] Input the sensor weight set, feature correlation function values, and power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm. This algorithm traverses the data pairs marked as abnormal in the conflict recognition results, retrieves the identifier of the corresponding sensor, the original measurement value of the current sampling period, and the weight value at that moment one by one. These three together form a structured conflict data input set, which encapsulates the conflict sensor number, measurement value, and its confidence level in the form of a tuple. Based on the conflict data input set, weighted processing is performed on the measurement data of each conflict sensor. For the measurement values of two or more sensors in the same group of conflicts, multiply them by their corresponding weight coefficients respectively, and fuse them into a weighted sensor data value using the weighted average strategy. Perform deviation test and correction processing on the weighted data, calculate the absolute difference between the weighted values of different sensors, perform a ratio operation on this difference and the deviation threshold set by the power equipment, and then obtain the deviation correction factor by subtracting this ratio from 1. This correction factor can be understood numerically as the proportional attenuation amount of confidence correction. The closer its value is to 1, the smaller the deviation between the data and the higher the credibility. Conversely, if the deviation value is close to or even exceeds the set threshold, this correction factor approaches 0, indicating that there is a large deviation in this set of measurement results. Multiply the maximum value in the weighted sensor data value by the deviation correction factor to obtain the most reliable sensor data after deviation correction. Structurally output the most reliable sensor data after deviation correction as the initial conflict resolution result, and let it flow into the subsequent trust network construction and multi-sensor fusion analysis process as high-confidence data.

[0024] Step S4: Extract abnormal features from the power equipment based on the fusion data set to obtain an abnormal feature set; Specifically, based on the high-confidence fusion dataset obtained after resolving multimodal sensor conflicts, the current operating state of power equipment is mapped and modeled in the feature space. The MFEB model is used to perform multi-dimensional feature mapping on the fusion dataset, and a unified multimodal feature vector is constructed through the mapping relationship between the three types of features: heat, electricity, and force. This vector not only reflects physical properties such as the surface temperature of the equipment, internal electromagnetic behavior, and housing vibration, but also realizes the co-space expression of cross-modal features, thereby converting the original physical parameters into a standardized feature space vector that is convenient for discriminant analysis. Each dimension in the vector structure corresponds to a predefined key index, such as temperature peak, electromagnetic harmonic intensity, vibration energy density, etc., so that the operating state of the equipment is represented as a point in the feature space. According to the type of power equipment currently being inspected (such as transformers, high-voltage circuit breakers, arresters, or disconnectors), the pre-set equipment state boundary set is searched. This set consists of a minimum boundary vector and a maximum boundary vector, representing the lower and upper limits of all feature dimensions of the equipment in the normal operating state respectively. The boundary parameters are obtained by statistical modeling of historical operating data and expert experience, ensuring that they can cover all state changes of the equipment within the normal fluctuation range. The system uses this boundary set as a reference framework, compares the feature space vector with the boundaries dimension by dimension, and uses the minimum distance calculation method to evaluate the degree to which the current state deviates from the normal operating range. This distance calculation considers the Euclidean or Manhattan distance between the vector and the boundary, and introduces the equipment type sensitivity coefficient as an adjustment factor to assign different response sensitivities to different types of equipment, so as to adapt to the different requirements such as high thermal stability but sensitive electromagnetic disturbances for transformers, and frequent vibration responses but tolerance for temperature changes for circuit breakers. After adjusting the sensitivity of the minimum distance between the current feature space point and the boundary, the anomaly index value is calculated. The closer this index value is to 1 numerically, the more likely it indicates that the current state has a significant deviation. On the contrary, if it is close to 0, it indicates that the equipment is within the safety boundary. The anomaly index value is compared with the preset threshold. If it exceeds the threshold, the anomaly determination mechanism is triggered and the feature classification processing flow is entered. In this process, classification labels are assigned according to the specific feature dimensions that exceed the boundaries in the anomaly vector. For example, if the heat feature exceeds the boundary, it is marked as "thermal anomaly", if the electric field strength deviates, it is marked as "electrical anomaly", and if the vibration parameter shows a sudden change, it is classified as "mechanical anomaly". If multiple indicators exceed the limit at the same time, it is further classified into the "composite anomaly" category. All equipment features determined to be abnormal are archived into the abnormal feature set, which stores information such as equipment number, abnormal type, anomaly value, and specific dimension exceeding the limit in a structured manner.

[0025] Step S5: Optimize the inspection trajectory based on the abnormal feature set to obtain the target inspection path sequence, and combine the historical equipment inspection data to perform joint prediction of the equipment state to obtain the equipment fault prediction result.

[0026] Specifically, based on the state offset data and abnormal determination results of various power equipment included in the abnormal feature set, a fault probability density calculation process is executed. This process takes the distance between the fused feature vector and the equipment state boundary as the core parameter, combines the reliability factor mapped by the sensor weight in the CWMM-Fusion algorithm, and non-linearly normalizes the current abnormal degree through the sigmoid function to form a fault probability density value, which is used to quantify the possibility of each device failing at the current moment. Based on the above fault probability density value, an importance score calculation is performed for each power equipment. This score considers the current abnormal degree of the equipment and introduces structural indicators and operating environment factors for comprehensive evaluation. The analytic hierarchy process is used to comprehensively calculate factors such as the position criticality, load importance level, and standby redundancy rate of the equipment in the power grid topology structure, and a set of equipment evaluation parameters is constructed through weighted superposition, so as to assign a comprehensive score to each equipment that reflects its priority in the operation and maintenance strategy. The fault probability density value, the set of equipment evaluation parameters, and the actual physical distance data between devices are input into the EIPO inspection optimization algorithm. This algorithm is a path optimization model under multi-objective constraints. Its objective function comprehensively considers the total length of the inspection path, the equipment risk level, and the importance score, maximizing the risk coverage and the access frequency of key nodes while ensuring the shortest path. This optimization model uses an improved genetic algorithm to implement path search. The device access order is encoded as a chromosome for iterative evolution, and sequence-preserving crossover and 2-opt local mutation operations are performed in each generation to ensure the diversity of the search space and the convergence speed. The optimization terminates when the maximum number of iterations is reached or there is no improvement for multiple consecutive generations, and the target inspection path sequence is output. This sequence is generated according to the logic of preferentially accessing high-risk and high-weight devices and has dynamic adaptive characteristics. Historical equipment inspection data and the state vectors of the equipment involved in the current path are introduced to jointly construct a device state joint prediction model. This model constructs a physical, electrical, and historical fault propagation relationship graph between devices based on the graph convolutional network. Each device in the optimized path is used as a graph node, and the fused historical state change sequence is used as the graph node feature input to the model. High-order correlation information between devices is captured through graph convolutional operations, and at the same time, the attention mechanism is used to strengthen the feature expression of key devices, and the state value and state change trend of the device in the next prediction period are output. The potential fault propagation path is predicted by means of diffusion simulation. The device fault prediction results generated by the model are presented in the form of a risk heat map, marking the expected risk level and propagation range of each key node.

[0027] Taking the fault probability density value output by the fusion analysis module, the set of equipment evaluation parameters obtained based on hierarchical analysis, and the actual physical layout distance between power equipment as the core input variables, a multi-objective optimization model of the EIPO inspection optimization algorithm is jointly constructed. The core of this model lies in constructing a controllable comprehensive optimization objective function, which jointly considers the inspection path length cost, the degree of fault risk of the current state of power equipment, and the importance level of the equipment in the operation of the power system, so as to form a path planning strategy with dual characteristics of global constraint and task-driven for the inspection sequence and scheduling method. Mathematically, the objective function consists of three terms: the physical distance measurement term between equipment, which reflects the inspection efficiency; the fault probability density weighted term, which emphasizes the priority detection of high-risk equipment; and the equipment importance scoring weighted term, which ensures that key nodes of the power system obtain more inspection resources. After the objective function is constructed, in order to adapt to different inspection task types and operation scenario characteristics, three types of weight parameters in the objective function, namely the distance weight coefficient, the fault probability weight coefficient, and the equipment importance weight coefficient, are adaptively set. This setting is dynamically adjusted according to the situation classification of inspection tasks. For example, in regular periodic inspections, more emphasis is placed on path efficiency, so a higher distance weight value is set; in emergency fault troubleshooting scenarios, the fault risk coefficient should be increased to prioritize the coverage of high-risk equipment; while in key equipment maintenance or seasonal power supply protection tasks, the equipment importance coefficient weight is placed first. The system presets multiple typical scenario parameter configuration schemes and supports the dynamic generation of an optimized parameter configuration set through expert strategies or historical task feedback, so as to enhance the adaptability of the optimization model to complex operating environments. Based on the constructed comprehensive optimization objective function and the corresponding optimized parameter configuration set, the EIPO algorithm performs an iterative solution process, and its solution strategy is implemented based on an improved genetic algorithm or a hybrid ant colony algorithm. The equipment list is encoded as a set of access sequences to be sorted at this stage, and each inspection path is a candidate solution individual. The algorithm gradually evolves the solution set through genetic operations such as crossover, mutation, and selection, and continuously approaches a better solution. During the evolution process, the algorithm scores each generation of candidate paths according to the optimization objective function, sorts and selects through the fitness function, and at the same time combines local search strategies (such as 2-opt or 3-opt) for path fine-tuning to ensure the balance between global convergence and local optimality, forming a solution set containing several high-quality inspection paths. Perform convergence determination and optimal solution screening on the candidate inspection path solution set. When there is no significant improvement in the optimal solution of the solution set for multiple consecutive generations or the maximum number of iterations is reached, trigger the convergence termination mechanism, and select the path plan with the lowest comprehensive cost, the highest risk coverage, and the best hit rate of important equipment from the final solution set as the target inspection path sequence.

[0028] Combine the target inspection path sequence output by the path optimization module with the historical equipment inspection data accumulated in the power system over the long term to construct a device association graph structure with topological structure and temporal information. In this structure, each inspected device corresponds to a graph node, and the node features are composed of its historical operation status sequence, anomaly marks, sensor feature vectors, etc.; while the physical connections, electrical interactions, and historical fault linkage records between devices form the edges in the graph. Assign an association strength weight to each edge, which is evaluated by combining the electrical dependence between devices, the physical distance influence coefficient, and the fault propagation probability, and construct a heterogeneous graph data structure that reflects the structural and functional dependence relationships between devices. Input the device association graph structure and the current device status sequence into the graph convolutional network model (GCN) for joint prediction of the power equipment status. GCN takes device nodes as the basic operation unit, extracts the high-order dependence relationships between nodes through multiple layers of graph convolution, and propagates the status information layer by layer. The model performs feature aggregation operations according to the adjacency matrix and node features at each layer, so that the update of each node's status is not only affected by its own historical status, but also reflects the linkage trend of the status of surrounding devices. At the same time, an attention mechanism is integrated inside the model to perform feature enhancement processing on the key nodes in the graph, giving priority to high-risk devices on the fault propagation path, and improving the accuracy and discriminability of the status prediction. The output of the model is a predicted status data set, which contains the status estimation values of each device at the next time step, and provides an incremental index of the status change trend for identifying exacerbating fault risks. Based on this predicted status data set and the device association graph structure, a heat conduction-like diffusion model is jointly constructed to simulate the propagation path and influence range of potential faults in the device network. By introducing a heat conduction-like equation, a diffusion relationship is established between the rate of change of the state and the state gradient of its adjacent nodes for each node, and a device-specific diffusion coefficient is calculated for each node, which is obtained based on the performance of this device as a fault propagation source or receptor in history, reflecting the ability of the device to act as a "fault conduction channel" in the network. The model simulates the propagation process of fault signals in the graph through multi-step time evolution, dynamically deduces the probability of each node's state deterioration in several future time steps, and forms a fault probability distribution data. Input the fault probability distribution data into the fault risk assessment and visualization processing module. The risk assessment logic comprehensively considers the fault probability value, diffusion rate, path coverage, and critical device hit rate, classifies the risk levels of all nodes, and identifies the main propagation path of the fault. This path is presented as a node sequence, accompanied by the propagation probability and time delay estimation on each edge, forming a complete potential fault diffusion chain. The visualization module then displays the prediction results in the form of a graph heat map, with different nodes marked with their fault risk levels by color intensity, and the propagation path highlighted in bold and colored edges to obtain the device fault prediction results.

[0029] In the embodiments of the present invention, by constructing a sensor adaptive weight evaluation model based on thermal stability, accuracy history, and signal strength, the reliability indicators of each sensor can be dynamically quantified, effectively solving the problem that the traditional fixed weight method cannot cope with sensor aging, environmental interference, and equipment state changes, and ensuring the accuracy and robustness of multimodal data fusion. Combining the dynamic confidence matrix and the feature correlation function can accurately identify data conflicts between multimodal sensors, and realize the intelligent screening and fusion of conflict data. Compared with the traditional simple weighted average or majority voting mechanism, it has stronger conflict handling ability and data fusion accuracy. By establishing a three-dimensional feature space and equipment state boundaries, real-time processing of conflict data in milliseconds is achieved, avoiding the computational bottleneck of traditional deep learning methods on embedded platforms and meeting the real-time requirements of power equipment inspection. By constructing a power equipment failure probability density function as the core constraint, an inspection path planning for multi-objective optimization is realized, comprehensively considering equipment status, importance, and spatial distribution. Compared with the traditional fixed path or simple greedy algorithm, it can achieve the optimal allocation of inspection resources and key equipment monitoring. The present invention can effectively capture the spatial correlation relationship between power equipment. By establishing an equipment association graph and analyzing physical connections, electrical influences, and fault propagation relationships, the limitation that the traditional time series model cannot handle spatial dependence is solved, and more accurate joint prediction of equipment status is realized. By simulating the propagation process of faults in the equipment network through the heat conduction equation, potential fault propagation paths and the risk of chain reactions can be identified. For different types of power equipment and inspection scenarios, the system can adaptively adjust algorithm parameters and optimization strategies, including sensor weight coefficients, deviation thresholds, inspection weight parameters, etc., ensuring generality and effectiveness in different application environments.

[0030] In a specific embodiment, the process of executing step S1 may specifically include the following steps: Collect multimodal raw data of power equipment through the multimodal sensors of the inspection robot. The multimodal sensors include thermal imaging sensors, electromagnetic field sensors, and vibration sensors; Perform noise filtering, data normalization, and time synchronization processing on the multimodal raw data to obtain a preprocessed standardized data set. The preprocessed standardized data set includes thermal imaging sensor data, electromagnetic field sensor data, and vibration sensor data; Extract temperature distribution features from the thermal imaging sensor data to obtain a thermal feature parameter set including temperature peak, temperature mean, and temperature variance, and generate a thermal feature vector based on the thermal feature parameter set; Perform electromagnetic field intensity spectrum analysis on the electromagnetic field sensor data to obtain an electromagnetic feature parameter set, and generate an electromagnetic feature vector based on the electromagnetic feature parameter set; Perform time-frequency domain transformation on the vibration sensor data to obtain a vibration characteristic parameter set, and generate a vibration characteristic vector based on the vibration characteristic parameter set; Use the thermal characteristic vector, electromagnetic characteristic vector, and vibration characteristic vector as the characteristic vector set.

[0031] Specifically, during the operation process, the inspection robot arranges a multi-modal sensor array through a fixed or rotating pan-tilt structure, enabling it to collect real-time data on power equipment from multiple directions, covering multiple physical dimensions including the thermal distribution on the surface of the equipment housing, the electromagnetic leakage behavior during operation, and the operational stability of the mechanical structure, ensuring the diversification of the perception dimension and the comprehensiveness of data acquisition. To achieve high-frequency and low-latency data capture, a distributed sampling structure is introduced in the design, and the upper limit of the sampling frequency for different equipment types is set. For example, the sampling frequency configurations of 100Hz, 75Hz, and 50Hz are respectively adopted for transformers, high-voltage switches, and transmission lines. A unified preprocessing process is performed on the original multi-modal data, which mainly includes three key steps: noise filtering, data normalization, and time synchronization. Due to the complex acquisition environment, there is a large amount of background interference noise in the original data. Therefore, a band-pass filtering model is constructed for each type of sensor signal respectively, and specific filtering parameters are configured according to its working frequency band. For example, the effective signals are retained in the range of 10Hz to 500Hz, and the non-equipment signal interference introduced by external vibrations, environmental thermal fluctuations, or electromagnetic disturbances is filtered. The output units and dimensions of different types of sensors are different, and there is a problem of numerical imbalance in direct fusion analysis. Therefore, a normalization operation is uniformly performed on all data, and all data is uniformly projected onto the [-1,1] numerical interval by using the maximum-minimum value mapping method, so as to achieve the structural alignment and statistical scale unification of multi-source data. To ensure a strict time correspondence relationship between multi-modal data, a time synchronization mechanism based on high-precision timestamps is introduced, and various types of data are aligned at the microsecond level according to their acquisition times, so as to ensure a consistent time basis between multi-channel data. After the above processing, a standardized data set is formed, in which the thermal imaging data, electromagnetic field data, and vibration data are uniformly stored and encapsulated as structured time-series data blocks. Targeted feature extraction operations are respectively performed on the three types of sensor data. First, for the thermal imaging sensor data, this type of data is presented in the form of infrared images or thermal matrices. By performing local and global statistics on the regional thermal distribution of the images, temperature peak values, full-frame temperature means, and temperature stability indicators constructed based on local area variances are extracted, thus forming a set of thermal feature parameters. This parameter set reflects whether there is local overheating on the equipment surface and depicts the fluctuation amplitude of the long-term thermal distribution trend. Then, this parameter set is encoded as a set of thermal feature vectors for subsequent model processing. For the electromagnetic field sensor data, a frequency-domain analysis method is used for processing. The fast Fourier transform is used to convert the time-domain electric field signal into a frequency spectrum distribution, and then feature indicators such as the amplitude of the main frequency component, harmonic energy density, and spectrum equilibrium coefficient are extracted. These indicators constitute a set of electromagnetic feature parameters, and the system uniformly encodes them as electromagnetic feature vectors to depict whether there are electromagnetic anomalies such as partial discharge, winding short circuit, or grounding fault during the operation of electrical components.For the vibration sensor data, multi-level processing is performed in the joint time-frequency domain transformation. By combining the short-time Fourier transform and the Hilbert-Huang transform, the instantaneous energy distribution, main frequency change, impact response index and other indicators of the equipment vibration signal are extracted to form a vibration feature parameter set, which is then transformed into a vibration feature vector to characterize potential problems such as bearing damage, structural looseness or external impact existing in the equipment operation. The thermal feature vector, electromagnetic feature vector and vibration feature vector are integrated in accordance with a unified format to construct a multi-dimensional feature vector set.

[0032] In a specific embodiment, the process of executing step S2 may specifically include the following steps: Based on the feature vector set, calculate the output volatility of the multi-modal sensor under different temperature conditions to obtain the thermal stability index, analyze the historical measurement accuracy of the multi-modal sensor to obtain the accuracy historical index, and evaluate the signal-to-noise ratio of the multi-modal sensor to obtain the signal strength index; Set differentiated thermal stability coefficients, accuracy historical coefficients and signal strength coefficients according to the types of power equipment; By calculating the sum of the product of the thermal stability index and the thermal stability coefficient, the product of the accuracy historical index and the accuracy historical coefficient, and the product of the signal strength index and the signal strength coefficient, obtain the weight values of each sensor in the multi-modal sensor; Arrange the weight values of each sensor in the order of the sensor identifier and form a sensor weight set.

[0033] Specifically, based on the feature vector set, the output performance of various sensors at different environmental temperatures is analyzed for stability, and the thermal stability index is calculated accordingly. This step is based on the temperature mean and temperature variance data in the thermal feature vector. By constructing a temperature interval sequence, the fluctuation degree of the output data of each sensor is analyzed within different temperature segments. That is, the standard deviation of the corresponding measurement value of the sensor is extracted within each set temperature segment, and the value reflecting this volatility in a normalized manner is used as the thermal stability index to characterize the ability of the sensor to maintain measurement stability under temperature change interference. The lower the value, the higher the stability. The measurement accuracy of each sensor is analyzed based on historical inspection data to construct an accuracy history index. This process traces back the output records of the sensor for specific equipment characteristic values (such as temperature peak, electromagnetic main frequency, or vibration amplitude) in multiple past inspection cycles, and compares them with the actual operating state of the corresponding equipment or the results of expert evaluation. The deviation ratio of each historical measurement is calculated, and by statistically analyzing the proportion of these deviation ratios that meet the set accuracy threshold, the historical measurement accuracy rate of the sensor is obtained. This proportion is introduced into the model as the accuracy history index to reflect the reliability trend of the sensor during long-term use. The higher the value, the more reliable its long-term performance. Evaluate the signal strength performance of the sensor in the current working environment, that is, calculate the signal-to-noise ratio to obtain the signal strength index. Perform energy analysis on the original data output by each sensor in the current cycle, calculate the average power values of the signal component and the background noise component respectively, and construct a signal-to-noise ratio expression based on this. To ensure the physical authenticity of the results, use band-pass filtering to extract the main signal channel, and then regard the filtered result as the input of the noise channel. By comparing the power ratio of the two, the signal-to-noise ratio value is obtained, and then logarithmic scaling is performed on it to form a signal strength index under a unified scale. The higher the value of this index, the clearer and more distinguishable the signal data provided by the sensor in the current environment, which is beneficial to improving the accuracy of fusion decision-making. After calculating the above three indexes, introduce the background information of the equipment type. According to the category of the equipment being inspected currently, such as transformers, high-voltage circuit breakers, lightning arresters, or cable branch boxes, etc., set the thermal stability coefficient, accuracy history coefficient, and signal strength coefficient respectively. These coefficients reflect the emphasis weights of different equipment in different fault-sensitive dimensions. For example, transformers are particularly sensitive to thermal drift, so the thermal stability coefficient is set higher; high-voltage circuit breakers are accompanied by electromagnetic agitation, and the importance of the accuracy of electromagnetic measurement is stronger, so the accuracy coefficient is increased; for a complex cable support system, the clarity of vibration signals is more important, so the signal-to-noise ratio accounts for a larger proportion. Automatically load the coefficient combination adapted to the equipment type according to the preset rule table to ensure that the evaluation model has the structural adaptability to the equipment environment. After completing the corresponding matching of the indexes and coefficients, multiply the three indexes of each sensor by their corresponding coefficients respectively, and then add these three products together to obtain the comprehensive weight value of the sensor under the current environment and task conditions.Arrange the weight values of all sensors in sequence according to their corresponding unique identifiers to uniformly form a structured sensor weight set. This weight set is cached in the system in vector form and dynamically updated at a fixed period (such as 200 ms).

[0034] In a specific embodiment, the process of executing step S3 may specifically include the following steps: Calculate the data consistency between measurement values of different sensors based on the sensor weight set and multi-modal raw data. When the data consistency is lower than the consistency threshold, it is determined that there is a data conflict, and a sensor conflict recognition result is obtained; Conduct a feature correlation analysis on the conflicting sensors according to the sensor conflict recognition result to obtain feature correlation function values; Input the sensor weight set, feature correlation function values, and power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm for data screening to obtain an initial conflict resolution result; Construct a sensor trust network graph according to the initial conflict resolution result and apply the maximum weight spanning tree algorithm to identify the most trusted sensor set. At the same time, perform a time window analysis on periodic conflicts to obtain a fused data set.

[0035] Specifically, the data consistency between different sensors is calculated based on the sensor weight set and multi-modal raw data. The consistency calculation is based on the current measurement value of each sensor and its confidence coefficient in the weight set, and is processed using a weighted normalization difference function. That is, after normalizing the difference ratio of the data in the same physical dimension collected by any two sensors at the same time, the sensor weight is introduced to smoothly regulate this difference, and a consistency score between 0 and 1 is obtained. The closer this value is to 1, the more consistent the data. If it is lower than the set consistency threshold (such as 0.85), it indicates that there is a significant deviation between the data of the two sensors, and the system determines it as a potential data conflict accordingly. When the system identifies a sensor combination with data conflicts, that is, a sensor conflict recognition result is formed, and the system starts the feature correlation analysis module to perform a timing feature comparison on each pair of conflicting sensors. In this analysis process, the historical feature vectors of each sensor in the recent K sampling periods are traced back, and the timing data is standardized by removing the mean and normalizing the variance. Subsequently, the Pearson correlation coefficient or mutual information index is used to calculate their correlation degree in the feature trend dimension, and a feature correlation function value reflecting the consistency of their timing changes is generated. Numerically, this function value reflects whether the observation results of the two sensors, although there are short-term differences, have trend consistency in the long term. If the correlation is strong, the system will assign a higher retention weight to it in subsequent conflict resolution. The sensor weight set, the feature correlation function value, and the power equipment deviation threshold are input into the SyncDrop-E conflict resolution algorithm for data screening. This algorithm takes the measured values of the conflicting sensors as input, combines their weights with the historical feature correlation of the paired sensors for confidence weighting calculation, then evaluates the deviation degree between the data of each sensor and the deviation threshold, calculates the correction factor and applies it to the raw data for deviation compensation. The algorithm selects the maximum credible value after multiplying the weight by the correction factor as the valid data output in the conflict pair, and marks its source sensor as the preliminary credible source. For the case where more than two sensors conflict simultaneously, SyncDrop-E retains the trust factors of each credible value. To improve the overall robustness of the multi-sensor system in high-dimensional data fusion scenarios, a sensor trust network graph is constructed based on the above preliminary results. All sensors are used as graph nodes, and edges are connected between nodes through their data consistency, with the consistency score as the edge weight. The maximum weight spanning tree algorithm is used to optimize this trust graph to construct a subgraph structure with the largest weight, which corresponds to the most reliable sensor subset. By pruning non-main nodes through the tree structure, low-confidence sensors are automatically removed, and a highly reliable sensor group with extensive consistent connection relationships is retained to ensure the comprehensiveness and credibility of the fused data source. In addition, the system synchronously executes a time window analysis mechanism to identify periodic conflict problems.This mechanism constructs a sliding window statistical model for sensors that repeatedly exhibit abnormal fluctuations in multiple consecutive sampling periods, analyzes the trend of outliers in their outputs. If it is found that a certain sensor shows continuous periodic anomalies rather than single-point bursts in multiple windows, the system determines it as "persistent pseudo-anomalies" caused by internal device offsets or environmental interferences, and thus gives a suppression weight during fusion to avoid such periodic distortions from interfering with the overall fusion result. After completing the trust network screening and time window anomaly suppression, all the remaining reliable sensor data are weighted and integrated to output a highly consistent and highly confident fusion data set.

[0036] In a specific embodiment, the process of inputting the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm for data screening to obtain the initial conflict resolution result may specifically include the following steps: Input the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm, identify the conflicting sensor data pairs and extract the corresponding weight values to obtain a conflict data input set containing conflict sensor identifiers, measurement data, and weight values; Based on the conflict data input set, perform weighted processing on the measurement data of each conflicting sensor to obtain the weighted sensor data values; According to the weighted sensor data values, calculate the ratio of the absolute difference between the data and the power equipment deviation threshold and subtract this ratio from 1 to obtain the deviation correction factor; Perform a multiplication operation on the maximum value in the weighted sensor data values and the deviation correction factor to obtain the most reliable sensor data after deviation correction, and generate the corresponding initial conflict resolution result based on the most reliable sensor data after deviation correction.

[0037] Specifically, the dynamic weight value corresponding to the sensor is extracted from the sensor weight set, and then combined with its respective real-time measurement data and the unique sensor identification code recorded by the system to construct a conflict data input set. This input set consists of several triples, each triple containing the sensor identification, the current measurement value, and its corresponding confidence weight, which is the basic information structure for judging the credibility and outputting the correction result in the SyncDrop-E algorithm. After the conflict data input set is constructed, the SyncDrop-E algorithm performs a weighting operation on each pair of conflicting sensors in the input set to calculate their weighted measurement values. The weighting method is based on the measurement values of each sensor, multiplies them by the corresponding weight coefficients, and normalizes the products to ensure that the final values reflect the relative contribution ratios that different sensors should occupy in the fusion determination. The weighted results form a new set of data points, which physically represent the "confidence estimate" of the system for the current conflicting measurement values. To determine the degree to which the current data deviates from the allowable operating fluctuation range of the device, a deviation threshold corresponding to the device type is introduced. The preset maximum deviation tolerance value is found according to the device type (such as transformers, circuit breakers, bus connectors, etc.), and then the absolute difference is calculated for each pair of weighted data, and the ratio of this difference to the deviation threshold is processed to obtain a normalized offset ratio, which numerically reflects the relative degree to which the current conflict data exceeds the tolerance range. To construct a correction coefficient, a first-order inversion is performed on this ratio, that is, a deviation correction factor is constructed by subtracting the ratio from 1. The closer this factor is to 1 numerically, the smaller the difference between the weighted measurement values, the more concentrated the data within the normal operating range, and the higher its confidence level; when this factor approaches 0, it indicates that the conflicting measurement has significantly deviated from the tolerance boundary and must be carefully processed or excluded. The SyncDrop-E algorithm enters the final correction stage, extracts the maximum value from the weighted data values, and multiplies this maximum value by the deviation correction factor calculated above to obtain a "most reliable sensor data" dynamically corrected by the deviation. This multiplication operation logic retains the strong signal contribution in the data and at the same time introduces tolerance regulation, making the final output value close to the dominant measurement value while having a steady-state suppression ability, thus achieving dual control of weight drive and offset correction in the conflict decision. This process dynamically adjusts the contribution ratio of the maximum weight data to the final output through the deviation correction factor to form a preliminary conflict resolution result with high response accuracy and robustness. This most reliable corrected data is marked as the initial conflict resolution result and saved to the fusion dataset through a structured mapping and time-series archiving mechanism. The system binds the corrected data to its source sensor number during this process to form a fusion identification vector for tracking its source path and trust level in the subsequent construction of the trust network graph and the maximum weight spanning tree algorithm.

[0038] It should be noted that the SyncDrop-E algorithm is specifically designed to handle data conflict problems among multi-modal sensors. The working mechanism of this algorithm includes the following key steps: The SyncDrop-E algorithm is based on a dual mechanism of weighted data selection and deviation correction. By comprehensively considering the reliability weights of sensors and the degree of data deviation, it screens the most reliable sensor data. The core calculation logic of the algorithm is to multiply the measurement data of conflicting sensors by their corresponding weight values to obtain weighted data, and then combine the deviation correction factor for the final screening. First, identify pairs of conflicting sensor data, extract the weight values corresponding to each conflicting sensor, and form a conflict data input set containing sensor identifiers, measurement data, and weight values. Then, perform weighted processing on the measurement data of each conflicting sensor. By multiplying the measurement data of each sensor by its corresponding weight value, a weighted sensor data value that can reflect the reliability of the sensor is obtained. The algorithm calculates the ratio of the absolute difference between the data of conflicting sensors to the deviation threshold of the power equipment, and then subtracts this ratio from 1 to obtain the deviation correction factor. The purpose of this mechanism is to reduce the credibility of sensor data when the data difference is large and maintain a high credibility when the data difference is small. Multiply the maximum value in the weighted sensor data values by the deviation correction factor. In this way, both the weight advantage of the sensor and the degree of data consistency are considered. Finally, select the most reliable sensor data after deviation correction as the conflict resolution result to ensure the accuracy and reliability of the fused data. For complex conflict scenarios involving more than three sensors, the SyncDrop-E algorithm can be combined with graph theory methods to construct a sensor trust network and identify the most credible sensor set through the maximum weight spanning tree algorithm. For periodic conflicts, the algorithm also supports time window analysis to judge abnormal sensors based on the data trends in consecutive sampling periods.

[0039] In a specific embodiment, the process of executing step S4 may specifically include the following steps: Perform feature mapping on the fused data set to obtain the feature space vector of the power equipment; Set the normal lower limit and normal upper limit of electrical features, thermal features, and mechanical features according to the type of power equipment to obtain a set of equipment state boundaries including a minimum boundary vector and a maximum boundary vector; Calculate the minimum distance from the feature space vector to the set of equipment state boundaries and combine it with the equipment type sensitivity coefficient to obtain the abnormality index value; Perform abnormality determination and feature classification processing based on the abnormality index value to obtain a set of abnormal features.

[0040] Specifically, the thermal, electromagnetic, and vibration features in the multimodal fusion dataset are numerically encoded and vectorized according to specific mapping rules. Using a pre-set feature mapping matrix, the various physical signal types are uniformly converted into a standardized three-dimensional feature space, forming feature components in the electrical, thermal, and mechanical subspaces, respectively. These three components are combined to form a multimodal feature space vector. This vector is a mathematically defined data structure with clear dimensional interpretation and physical meaning. Each dimension represents a measured parameter value for a specific operational dimension of the device, such as the voltage harmonic ratio, maximum surface temperature, and vibration main frequency energy density. It offers traceability and real-time update capabilities. The corresponding operational state boundary model is determined based on the device type. This process relies on matching the device's asset tag with its technical parameter profile. Based on the device classification, such as transformer, high-voltage circuit breaker, busbar connector, or cable branch box, the upper and lower limits of key physical quantities during operation are extracted from a standard parameter database to construct minimum and maximum boundary vectors. These vectors represent the minimum and maximum value ranges allowed for the device under normal conditions, respectively, and serve as reference boundaries for abnormality assessment. For example, for transformers, peak surface temperatures should be below 90°C and above 40°C, electromagnetic leakage should be within safety thresholds, and the vibration spectrum should be free of structural excitation frequencies. Circuit breakers, on the other hand, are more sensitive to short-duration arc electromagnetic peaks, and the boundary setting criteria are accordingly shifted to dimensions such as high-frequency voltage distortion and post-breakdown vibration recovery time. After obtaining the feature space vector and boundary set, the system enters the core anomaly calculation phase, quantifying the degree of deviation by calculating the minimum distance between the current feature vector and the normal boundary set. The distance function used is weighted Euclidean distance or Manhattan distance, with Mahalanobis distance employed in some scenarios to enhance adaptability to inter-dimensional covariance. The system performs a boundary comparison on each dimension's parameter value. If the current value falls between the maximum and minimum boundaries, the dimension is considered normal and the offset is set to zero. If it exceeds the upper and lower limits, the excess value is used as the offset indicator, and all offset values are weighted and accumulated to construct a total offset distance. This distance is then combined with the device type sensitivity coefficient to form the final anomaly indicator. The sensitivity coefficient acts as a regulator here. Different equipment has different tolerances for the same anomaly type. For example, a cable support system can tolerate a certain amount of structural vibration, while a transformer is more sensitive to sustained temperature rise. Therefore, different sensitivity levels are assigned to different feature dimensions based on the equipment's risk level and structural vulnerability, forming a numerical representation of the global anomaly degree A, constrained to the range [0, 1]. Values closer to 1 indicate more severe deviations and more unstable conditions. After obtaining the anomaly degree index value, it is logically compared with the threshold and combined with a rule base to determine anomalies and classify features. Multi-level anomaly response logic is established. For example, an anomaly degree A below 0.3 is considered normal, 0.3 to 0.6 is considered a mild anomaly, 0.6 to 0.8 is considered a significant deviation, and a value greater than 0.8 triggers a fault warning mechanism.Meanwhile, classify and label the dimensions in the feature space where out-of-bounds specifically occurs. If thermal parameters such as temperature peak and mean value are out-of-bounds, label them as "thermal anomaly"; if spectral indicators are out-of-bounds, it is "electrical anomaly"; if high-amplitude impact signals are prominent in the vibration subspace, classify them as "mechanical anomaly"; if multiple-dimensional parameters are out-of-bounds simultaneously, it is recognized as "compound anomaly", thus constituting an abnormal feature set.

[0041] In a specific embodiment, the process of executing step S5 may specifically include the following steps: Execute the calculation of the failure probability density based on the abnormal feature set to obtain the failure probability density value of the power equipment; Perform the calculation of the importance score for the power equipment according to the failure probability density value to obtain a set of equipment evaluation parameters; Input the failure probability density value, the set of equipment evaluation parameters, and the physical distance between equipment into the EIPO patrol optimization algorithm to execute the patrol trajectory optimization with multi-objective constraints, and obtain the target patrol path sequence; Perform the joint prediction of the equipment status for the target patrol path sequence and the historical equipment patrol data to obtain the equipment failure prediction result.

[0042] Specifically, taking the abnormal feature set as the input, this set contains multiple information such as the types of abnormal dimensions, abnormal values, severity levels, timestamps, and source sensors in the thermal, electrical, and mechanical multimodal features of each device. On this basis, by constructing a failure probability density function model, the probability of a device's possible failure is quantitatively predicted. The core calculation method of the failure probability density is based on the sigmoid activation function. After standardizing various abnormal feature indicators, they are used as the input of the feature function. Combining the sensitivity weights of each feature in historical failure cases, a multi-factor failure expression is constructed by proportional superposition, forming a failure probability inference model with strong adaptability and dynamic feedback ability. This model not only considers the single-dimensional abnormal intensity but also integrates the structural combined influence of abnormalities, enabling it to output the comprehensive probability density value of the device's failure under non-linear conditions. The value is limited between 0 and 1, where the closer it is to 1, the closer the current state of the device is to the failure threshold. To achieve the efficient scheduling of inspection tasks and the allocation of resource priorities, based on the above failure probability density value, the importance score of power equipment is calculated to form a set of device evaluation parameters. In this process, a structural evaluation model of the power system is introduced. This model numerically quantifies the network topology position of the device in the power grid (such as backbone nodes or edge nodes), load level (such as high-load circuit breakers or standby feeders), and alternative redundancy rate (i.e., the replaceability after the device fails) through the analytic hierarchy process or weighted index model, and calculates the weighted device score in combination with the failure probability density value. In the scoring function, the network position index is given the largest weight to ensure the stability of the core transmission path; the load index reflects the operating pressure; the redundancy rate is weighted inversely to enhance the overall resilience of the system. The set of device evaluation parameters generated by the system structurally marks the current failure risk level, importance weight, influence range, and inspection priority of each device. The failure probability density value, the set of device evaluation parameters, and the physical distance between devices are input into the EIPO inspection path optimization algorithm model to perform the optimal path solution under multi-objective constraint conditions. The EIPO algorithm takes three objective functions as the core, namely minimizing the total inspection path length, maximizing the coverage density of high-failure-probability devices, and maximizing the access priority of key devices. Its optimization objective function is composed of a distance cost function, a failure risk function, and a device importance function, and a weight coefficient is used for flexible adjustment to adapt to different application scenarios, such as focusing on path efficiency in regular inspections, risk response in emergency investigations, and device level in key guarantee scenarios. The system uses an improved genetic algorithm to implement path search. By encoding the device access order as a chromosome, combining mechanisms such as sequence-preserving crossover, 2-opt local mutation, and dynamic adjustment of the fitness function, the candidate path set is iteratively evolved in each generation, and the search is terminated when the maximum number of generations is reached or the optimal path has not improved for multiple consecutive generations, and the global optimal solution is output.The finally formed target inspection path sequence can meet the dual requirements of coverage rate and priority, and also realizes the optimal efficiency under resource constraints by adjusting the path density. After the path optimization, historical inspection data and the device status involved in the current inspection path are introduced to construct a joint device status prediction model. This model is designed based on the fusion of a graph convolutional network and a time series prediction model. The system uses the devices in the target inspection path as graph nodes, and takes the physical connections, electrical interconnections, and historical fault propagation paths between devices as the edge structure of the graph. At the same time, the multi-modal feature sequences in the historical inspection records are used as the attribute features of the graph nodes. In the graph neural network, the structural dependence relationship and state co-variation pattern between devices are mined through multi-layer graph convolutional operations, and then the attention mechanism is combined to highlight the high-order influence of key devices, so as to output the state prediction value and state change trend at the next moment at each node. The predicted state vector is input into an extended heat conduction diffusion model to simulate the process of potential faults spreading from high-risk nodes to adjacent devices, and to deduce the risk spatial distribution in the next few time steps. The results output by the joint prediction module are visually presented in the form of a fault risk distribution map and a propagation path map, where each device is marked with its predicted state level, risk change trend, and fault diffusion probability.

[0043] In a specific embodiment, the process of inputting the fault probability density value, the set of device evaluation parameters, and the physical distance between devices into the EIPO inspection optimization algorithm to perform multi-objective constrained inspection trajectory optimization to obtain the target inspection path sequence may specifically include the following steps: Input the fault probability density value, the set of device evaluation parameters, and the physical distance between devices into the EIPO inspection optimization algorithm to construct a comprehensive optimization objective function including distance cost, fault risk, and device importance; According to the inspection scenario type, perform adaptive parameter setting on the distance weight coefficient, fault probability weight coefficient, and device importance weight coefficient in the comprehensive optimization objective function to obtain an optimized parameter configuration set; Based on the comprehensive optimization objective function and the optimized parameter configuration set, perform iterative solution to obtain a candidate inspection path solution set; Perform convergence determination and optimal solution selection on the candidate inspection path solution set to obtain the target inspection path sequence.

[0044] Specifically, the probability density value of equipment failure is used as the main indicator to measure the current potential failure risk of each device. The structural importance score in the equipment evaluation parameter set is used as the key basis for measuring the network influence and inspection priority of the device. At the same time, the physical distance matrix is used as the spatial basis for measuring the path cost. These three types of data together constitute the input variable set of the EIPO algorithm. Based on this, the system constructs a multi-objective optimization function in the algorithm model that includes three core constraint objectives: distance cost, failure risk, and device importance. Structurally, this objective function consists of three parts. The first part is the path cost term, which represents the total physical length of the inspection path and is the cumulative value of the actual moving distances between all device nodes. This term is modeled using Euclidean distance or terrain-corrected distance to minimize energy consumption and inspection time. The second part is the failure risk term, which expresses the system's coverage ability for high-risk devices on the inspection path by weighted summing the probability density values of equipment failures in the path. The weights are dynamically adjusted according to device importance and failure level to ensure the priority of risk response. The third part is the device importance term. Its core is to assign higher weights to the key devices involved in the inspection path (such as main transformers, bus nodes, and high-load switches) according to the importance factors marked in the equipment evaluation parameter set, so that the system can prioritize ensuring the coverage frequency and timeliness of key nodes during path optimization. The objective function is defined as a weighted combination form of these three sub-functions. The comprehensive objective is to minimize the path cost and maximize the risk coverage and importance response ability. The system adjusts the proportion of each item according to different application scenarios. To ensure that this optimization model has good adaptability in different inspection task scenarios, an adaptive parameter configuration mechanism is introduced, that is, the three weight coefficients in the objective function are dynamically set according to the type, urgency of the inspection task, and the operating state of the power grid. During regular daily inspections, the system sets a relatively high distance weight coefficient (such as α = 0.6) to ensure path efficiency and resource conservation, while setting the failure risk weight and importance weight as secondary. If in a failure tracking state, the system increases the failure probability coefficient (such as β = 0.6) to prioritize covering high-risk devices. During critical guarantee periods such as peak summer, the importance weight coefficient is increased (such as γ = 0.6) to focus on the inspection tasks of highly critical devices. The system has multiple pre-defined scenario weight templates built-in and also allows fine-tuning of weights through expert parameters or historical task data to generate a complete set of optimized parameter configurations as the scheduling basis for the path solution module.After the construction of the objective function and the loading of weight parameters are completed, the system enters the iterative path solving stage. The EIPO algorithm uses an improved genetic optimization mechanism to encode the device access sequence as an individual chromosome. The initial population consists of several random or semi-heuristic paths, and in each generation, the path individuals are evolved multiple times through sequence-preserving crossover, inversion mutation (such as 2-opt or 3-opt), and local hill-climbing strategies. The fitness function is calculated based on the comprehensive objective function, and a weighted evaluation is performed on the total path cost, risk coverage quality, and hit rate of important nodes. During the population evolution process, multi-point selection and elitist retention strategies are introduced to avoid falling into local minima, and the crossover and mutation intensities are controlled by dynamic adaptive operators to ensure the exploration ability of the algorithm while maintaining the solution stability. The new path set generated in each generation constitutes the candidate inspection path solution set. The system continuously monitors the change trend of the path cost and the risk coverage level in the solution set to evaluate the search progress and the diversity of the solution space. When the candidate solution set accumulates to the specified number or no better path appears for several consecutive generations of the algorithm, the system enters the convergence determination and optimal solution selection stage. The system scores and ranks all paths in the solution set based on the comprehensive objective function, and checks their coverage breadth, risk concentration, and path efficiency stability on key performance indicators. The path with the best score is selected as the target inspection path sequence for the current task, and the task execution time window and device status mark of each segment of the path are added to provide a data interface for the subsequent path execution and task scheduling module. At the same time, this path sequence is also synchronously recorded in the historical trajectory library for subsequent task prediction and model adaptive training.

[0045] It should be noted that the EIPO inspection optimization algorithm is a multi-objective optimization algorithm specifically designed for optimizing the inspection trajectories of power equipment. It can comprehensively consider multiple factors such as equipment status, importance, and spatial distribution to achieve intelligent planning of inspection paths. The EIPO algorithm constructs a comprehensive optimization objective function that includes distance cost, failure risk, and equipment importance. The algorithm forms a unified optimization objective by performing weighted summation on the distance function, failure probability density function, and equipment importance scoring function. It automatically adjusts the optimization parameters according to different inspection scenario types. For regular inspection scenarios, the algorithm pays more attention to path efficiency and sets a higher distance weight; for fault tracking scenarios, the algorithm gives priority to the failure probability and increases the failure risk weight; for key equipment inspection scenarios, the algorithm highlights the equipment importance weight. This adaptive mechanism ensures the optimization effect of the algorithm in different application scenarios. The analytic hierarchy process is used to score the importance of power equipment, comprehensively considering factors such as the network position coefficient, load importance, and redundancy rate of the equipment in the power grid. Improve the genetic algorithm solution mechanism: The EIPO algorithm uses an improved genetic algorithm for optimization and solution, and uses the equipment access sequence as the chromosome coding method, which can intuitively represent the inspection path. The algorithm uses sequence-preserving crossover operations to ensure the validity of the chromosomes and uses 2-opt local search as the mutation operation to improve the quality of the solutions. Two termination conditions are set, including no improvement in the optimal solution for 50 consecutive generations or reaching the maximum number of generations of 300. This design not only ensures the convergence of the algorithm but also avoids waste of computing resources caused by excessive iteration. The algorithm continuously monitors the improvement of the population fitness during the iteration process and terminates the search in a timely manner when the quality of the solution is stable. It can dynamically adjust the inspection trajectory according to the real-time status of power equipment. When an equipment anomaly or a change in the failure probability is detected, the algorithm can recalculate the optimization objective function and update the inspection path. This real-time optimization ability enables the inspection robot to quickly respond to changes in equipment status and improve the pertinence and efficiency of inspections. Multiple constraint conditions are processed simultaneously during the optimization process, including equipment physical location constraints, inspection time window constraints, robot energy consumption constraints, etc. By introducing penalty terms into the objective function or using constraint handling techniques, it is ensured that the generated inspection trajectory reaches the optimal effect on the premise of meeting various actual constraints.

[0046] In a specific embodiment, the process of performing the step of jointly predicting the equipment status for the target inspection path sequence and historical equipment inspection data to obtain the equipment failure prediction result may specifically include the following steps: Construct an equipment association graph structure including equipment nodes, associated edges, and associated strength weights based on the target inspection path sequence and historical equipment inspection data; Perform joint prediction of the power equipment status on the equipment association graph structure and the current equipment status sequence to obtain a predicted status data set; Based on the predicted state data set and the device association graph structure, simulate the fault propagation process in the device network through the heat conduction equation and calculate the device-specific diffusion coefficient to obtain the fault probability distribution data; Conduct fault risk assessment and visualization processing based on the fault probability distribution data to obtain the device fault prediction results including potential fault propagation paths and device risk levels.

[0047] Specifically, a graph structure model is constructed based on the target inspection path sequence and historical equipment operation records. Each equipment to be inspected is used as a node entity in the graph. The node attributes include the latest state feature vector of the current equipment (such as temperature, electric field strength, vibration spectrum characteristics, etc.), its abnormal indicators, historical failure frequencies, the state change trajectory in the past inspection cycle, etc. For the connection relationship between nodes, that is, the edge structure of the graph, the edge weights are established based on factors such as the spatial physical connection, electrical connection relationship, historical resonance or co-failure records between equipment. The value of the edge weight is the "correlation intensity weight", which reflects the intensity of state transfer or influence between two equipment. The stronger the correlation, the more likely the state of one equipment is to affect another equipment. The equipment association graph structure, together with the equipment state sequence collected in the current cycle, is input into the state joint prediction model. This model is designed based on the graph convolutional network, and through multi-layer graph convolutional operations, feature aggregation and node embedding update are performed on the equipment graph to achieve joint modeling of equipment states in both spatial and temporal dimensions. During the calculation process of each layer of graph convolution, the state of a node not only depends on its own historical state, but also aggregates the state features of its adjacent nodes, integrating the high-order dependencies from the spatial structure. At the same time, by introducing the time series convolution or gated recurrent unit (such as GRU or LSTM) structure of historical equipment states, the time evolution process of node states is modeled, so that the final output state prediction result of each node includes both its own time trend and is affected by the state changes of upstream and downstream equipment in the topology, forming the predicted state of the next cycle driven by the current state, that is, the predicted state dataset. This dataset records the state index values and their growth trends of all equipment at the next moment in vector form. Based on the predicted state dataset and the aforementioned equipment association graph structure, the fault propagation modeling stage is entered. In this stage, the heat conduction equation is introduced as the mathematical basis for simulating fault diffusion, and the equipment state is regarded as the initial condition of the heat field, and a time-evolving diffusion model is constructed on the graph structure. The system regards the equipment with high-risk or significant abnormal indicators in the predicted state as the heat source node, initializes its state to a high value, and the other nodes to low or normal values, and then performs numerical solution of the heat conduction equation-like on the entire graph network. Through multiple rounds of time step deduction, the model gradually outputs the evolution trend of the state values of the equipment group in the next several cycles and the propagation path of the fault signal in the graph structure, forming the fault probability distribution data. This data is indexed by nodes and records the probability values of each equipment failing at future times t+1, t+2, t+3...Based on the fault probability distribution data, perform fault risk assessment and visualization processing operations. The risk assessment module sets the risk level classification criteria, such as a probability < 0.3 being normal, 0.3 - 0.6 being low risk, 0.6 - 0.8 being medium risk, and > 0.8 being high risk. Combining the density of key nodes in the fault propagation path, the importance score of the device itself, and the path connectivity, it constructs a comprehensive risk level scoring model to label the final risk level for each device. The visualization module converts the device association graph structure into an interactive heat map or a geographic information overlay map. Different colors represent different risk levels, and the color or thickness of the edges represents the intensity of the propagation probability. It also shows how the fault heat spreads from the core fault point to the peripheral nodes in an animated way, helping the operation and maintenance personnel to intuitively understand the evolution chain of potential faults in space.

[0048] The above describes the multi-modal sensor fusion inspection method in the embodiments of the present invention. Next, the multi-modal sensor fusion inspection system in the embodiments of the present invention will be described. Please refer to Figure 2 , an embodiment of the multi-modal sensor fusion inspection system in the embodiments of the present invention includes: An acquisition module, configured to collect multi-modal raw data of power equipment through multi-modal sensors in an inspection robot and construct a feature vector set; A calculation module, configured to perform adaptive weight calculation on the multi-modal sensors according to the feature vector set to obtain a sensor weight set; A conflict identification module, configured to perform conflict identification and resolution on the multi-modal raw data based on the sensor weight set to obtain a fusion data set; A feature extraction module, configured to perform abnormal feature extraction on the power equipment based on the fusion data set to obtain an abnormal feature set; A joint prediction module, configured to optimize the inspection trajectory based on the abnormal feature set to obtain a target inspection path sequence, and combine historical equipment inspection data to perform joint prediction of the equipment status to obtain an equipment fault prediction result.

[0049] Through the collaborative cooperation of the above-mentioned various components, by constructing a sensor adaptive weight evaluation model based on thermal stability, accuracy history, and signal strength, the reliability index of each sensor can be dynamically quantified, effectively solving the problem that the traditional fixed weight method cannot cope with sensor aging, environmental interference, and equipment state changes, and ensuring the accuracy and robustness of multimodal data fusion. Combining the dynamic confidence matrix and the feature correlation function can accurately identify data conflicts between multimodal sensors and achieve intelligent screening and fusion of conflicting data. Compared with the traditional simple weighted average or majority voting mechanism, it has stronger conflict handling ability and data fusion accuracy. By establishing a three-dimensional feature space and equipment state boundary, real-time processing of conflicting data in milliseconds is achieved, avoiding the computational bottleneck of traditional deep learning methods on embedded platforms and meeting the real-time requirements of power equipment inspection. By constructing the power equipment failure probability density function as the core constraint, the inspection path planning of multi-objective optimization is realized, comprehensively considering equipment state, importance, and spatial distribution. Compared with the traditional fixed path or simple greedy algorithm, it can achieve the optimal allocation of inspection resources and key monitoring of key equipment. The present invention can effectively capture the spatial correlation relationship between power equipment. By establishing an equipment association graph and analyzing physical connections, electrical influences, and fault propagation relationships, the limitation that the traditional time series model cannot handle spatial dependence is solved, and more accurate joint prediction of equipment states is realized. By simulating the propagation process of faults in the equipment network through the heat conduction equation, potential fault propagation paths and cascading reaction risks can be identified. For different types of power equipment and inspection scenarios, the system can adaptively adjust algorithm parameters and optimization strategies, including sensor weight coefficients, deviation thresholds, inspection weight parameters, etc., ensuring generality and effectiveness in different application environments.

[0050] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described system, system, and unit can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0051] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0052] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A multimodal sensor fusion inspection method, characterized in that: Including: Collecting multi-modal raw data of power equipment through multi-modal sensors in the inspection robot and constructing a feature vector set; Performing adaptive weight calculation on the multi-modal sensors according to the feature vector set to obtain a sensor weight set; Based on the sensor weight set, performing conflict identification and resolution on the multi-modal raw data to obtain a fusion data set; Based on the fusion data set, extracting abnormal features of the power equipment to obtain an abnormal feature set; Based on the abnormal feature set, optimizing the inspection trajectory to obtain a target inspection path sequence, and combining historical equipment inspection data to perform joint prediction of equipment status to obtain an equipment failure prediction result.

2. The inspection method for multi-modal sensor fusion according to claim 1, wherein The collecting multi-modal raw data of power equipment through multi-modal sensors in the inspection robot and constructing a feature vector set includes: Collecting multi-modal raw data of power equipment through multi-modal sensors of the inspection robot, where the multi-modal sensors include a thermal imaging sensor, an electromagnetic field sensor, and a vibration sensor; Performing noise filtering, data normalization, and time synchronization processing on the multi-modal raw data to obtain a pre-processed standardized data set, where the pre-processed standardized data set includes thermal imaging sensor data, electromagnetic field sensor data, and vibration sensor data; Extracting temperature distribution features from the thermal imaging sensor data to obtain a thermal feature parameter set including a temperature peak value, a temperature average value, and a temperature variance, and generating a thermal feature vector based on the thermal feature parameter set; Performing electromagnetic field strength spectrum analysis on the electromagnetic field sensor data to obtain an electromagnetic feature parameter set, and generating an electromagnetic feature vector based on the electromagnetic feature parameter set; Performing time-frequency domain transformation on the vibration sensor data to obtain a vibration feature parameter set, and generating a vibration feature vector according to the vibration feature parameter set; Taking the thermal feature vector, the electromagnetic feature vector, and the vibration feature vector as the feature vector set.

3. The inspection method of multimodal sensor fusion according to claim 1, characterized in that: The performing adaptive weight calculation on the multi-modal sensors according to the feature vector set to obtain a sensor weight set includes: Based on the feature vector set, calculating the output volatility of the multi-modal sensors under different temperature conditions to obtain a thermal stability index, analyzing the historical measurement accuracy of the multi-modal sensors to obtain an accuracy history index, and evaluating the signal-to-noise ratio of the multi-modal sensors to obtain a signal strength index; Setting differentiated thermal stability coefficients, accuracy history coefficients, and signal strength coefficients according to the type of power equipment; By calculating the sum of the product of the thermal stability index and the thermal stability coefficient, the product of the accuracy history index and the accuracy history coefficient, and the product of the signal strength index and the signal strength coefficient, obtaining the weight values of each sensor in the multi-modal sensors; Arranging the weight values of each sensor in the order of sensor identifiers and forming a sensor weight set.

4. The inspection method for multimodal sensor fusion according to claim 1, wherein The performing conflict identification and resolution on the multi-modal raw data based on the sensor weight set to obtain a fusion data set includes: Calculate the data consistency between measurement values of different sensors based on the sensor weight set and the multimodal raw data. When the data consistency is lower than the consistency threshold, it is determined that there is a data conflict, and a sensor conflict recognition result is obtained; Conduct a feature correlation analysis on the conflicting sensors according to the sensor conflict recognition result to obtain the feature correlation function value; Input the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm for data screening to obtain an initial conflict resolution result; Construct a sensor trust network graph according to the initial conflict resolution result and apply the maximum weight spanning tree algorithm to identify the most credible sensor set. At the same time, perform a time window analysis on periodic conflicts to obtain a fused data set.

5. The inspection method for multimodal sensor fusion according to claim 4, characterized in that, The step of inputting the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm for data screening to obtain an initial conflict resolution result includes: Input the sensor weight set, the feature correlation function value, and the power equipment deviation threshold into the SyncDrop-E conflict resolution algorithm to identify pairs of conflicting sensor data and extract the corresponding weight values, obtaining a conflict data input set containing conflicting sensor identifiers, measurement data, and weight values; Based on the conflict data input set, perform weighted processing on the measurement data of each conflicting sensor to obtain weighted sensor data values; According to the weighted sensor data values, calculate the ratio of the absolute difference between the data to the power equipment deviation threshold and subtract this ratio from 1 to obtain a deviation correction factor; Perform a multiplication operation on the maximum value in the weighted sensor data values and the deviation correction factor to obtain the most reliable sensor data after deviation correction, and generate a corresponding initial conflict resolution result based on the most reliable sensor data after deviation correction.

6. The inspection method for multimodal sensor fusion according to claim 1, wherein The step of extracting abnormal features of the power equipment based on the fused data set to obtain an abnormal feature set includes: Perform feature mapping on the fused data set to obtain the feature space vector of the power equipment; Set the normal lower limit and normal upper limit of electrical features, thermal features, and mechanical features according to the power equipment type to obtain a device state boundary set containing a minimum boundary vector and a maximum boundary vector; Calculate the minimum distance from the feature space vector to the device state boundary set and combine it with the device type sensitivity coefficient to obtain an abnormality index value; Conduct abnormality determination and feature classification processing according to the abnormality index value to obtain an abnormal feature set.

7. The inspection method for multi-modal sensor fusion according to claim 1, wherein The step of optimizing the inspection trajectory based on the abnormal feature set to obtain a target inspection path sequence and performing a joint prediction of the device state in combination with historical device inspection data to obtain a device failure prediction result includes: Perform a failure probability density calculation based on the abnormal feature set to obtain the failure probability density value of the power equipment; Calculate the importance score of the power equipment according to the failure probability density value to obtain a set of device evaluation parameters; Input the fault probability density value, the set of device evaluation parameters, and the physical distance between devices into the EIPO patrol inspection optimization algorithm to perform the patrol inspection trajectory optimization with multi-objective constraints, and obtain the target patrol inspection path sequence; Perform joint prediction of device status on the target patrol inspection path sequence and historical device patrol inspection data to obtain the device fault prediction result.

8. The inspection method for multimodal sensor fusion according to claim 7, wherein The step of inputting the fault probability density value, the set of device evaluation parameters, and the physical distance between devices into the EIPO patrol inspection optimization algorithm to perform the patrol inspection trajectory optimization with multi-objective constraints and obtain the target patrol inspection path sequence includes: Input the fault probability density value, the set of device evaluation parameters, and the physical distance between devices into the EIPO patrol inspection optimization algorithm to construct a comprehensive optimization objective function including distance cost, fault risk, and device importance; Perform adaptive parameter setting on the distance weight coefficient, fault probability weight coefficient, and device importance weight coefficient in the comprehensive optimization objective function according to the patrol inspection scenario type to obtain the set of optimized parameter configurations; Perform iterative solution based on the comprehensive optimization objective function and the set of optimized parameter configurations to obtain the candidate patrol inspection path solution set; Perform convergence determination and optimal solution selection on the candidate patrol inspection path solution set to obtain the target patrol inspection path sequence.

9. The inspection method of multimodal sensor fusion according to claim 7, characterized in that: The step of performing joint prediction of device status on the target patrol inspection path sequence and historical device patrol inspection data to obtain the device fault prediction result includes: Construct a device association graph structure including device nodes, associated edges, and associated strength weights based on the target patrol inspection path sequence and the historical device patrol inspection data; Perform joint prediction of power device status on the device association graph structure and the current device status sequence to obtain the predicted status data set; Based on the predicted status data set and the device association graph structure, simulate the propagation process of faults in the device network through the heat conduction equation and calculate the device-specific diffusion coefficient to obtain the fault probability distribution data; Perform fault risk assessment and visualization processing according to the fault probability distribution data to obtain the device fault prediction result including potential fault propagation paths and device risk levels.

10. A patrol inspection system for multi-modal sensor fusion, characterized in that, A patrol inspection system for performing the multi-modal sensor fusion patrol inspection method according to any one of claims 1-9, the multi-modal sensor fusion patrol inspection system includes: An acquisition module, configured to acquire multi-modal raw data of power devices through multi-modal sensors in a patrol inspection robot and construct a feature vector set; A calculation module, configured to perform adaptive weight calculation on the multi-modal sensors according to the feature vector set to obtain a sensor weight set; A conflict identification module, configured to perform conflict identification and resolution on the multi-modal raw data based on the sensor weight set to obtain a fusion data set; A feature extraction module, configured to perform abnormal feature extraction on the power devices based on the fusion data set to obtain an abnormal feature set; A joint prediction module, configured to perform patrol inspection trajectory optimization based on the abnormal feature set to obtain a target patrol inspection path sequence, and perform joint prediction of device status in combination with historical device patrol inspection data to obtain a device fault prediction result.

Images