Equipment self-healing method and device based on space-time correlation characteristics, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses an equipment self-healing method and device based on time-space correlation characteristics, equipment and a medium. The method comprises the steps of generating time-space correlation characteristics based on multi-modal sensing data, performing anomaly detection by utilizing the time-space correlation characteristics to obtain an anomaly score, quantifying multiple operation dimension health indexes of equipment based on the time-space correlation characteristics, fusing the multiple operation dimension health indexes to generate a comprehensive health index, identifying a fault mode based on the anomaly score and the comprehensive health index, and generating a self-healing instruction. And performing simulation verification on the self-healing instruction in the digital twin model, sending the self-healing instruction passing the verification to a control unit for execution, and generating a self-healing result. According to the method, by fusing multi-modal data and cross-modal feature analysis and combining simulation verification of the digital twin model, early recognition and self-healing intervention of equipment faults are realized, and the operation stability of the equipment is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a device self-healing method and device based on spatiotemporal correlation features, an apparatus, and a storage medium. BACKGROUND

[0002] In the field of medical and health services, the operation and maintenance of key devices such as MRI, CT, and respirators still mainly rely on regular manual inspection and passive alarm mechanisms based on single thresholds, lacking multi-dimensional and comprehensive fault early warning and health assessment methods. When multiple sources of data interact or early performance degradation occurs, the traditional monitoring system cannot effectively integrate data from different types of sensing devices, making it difficult to identify abnormal patterns in a timely manner, and fault discovery is significantly delayed. At the same time, existing systems are mostly based on static thresholds of a single physical quantity, which cannot accurately reflect the overall operation status of the device, and health management often lacks quantitative indicators, leading to a high false positive rate.

[0003] In the field of financial technology services, devices such as payment terminals, intelligent teller machines, and clearing hosts also rely on single-point monitoring indicators (such as temperature, voltage, and network latency) for risk management, lacking fusion analysis of multiple types of data and quantitative health status of the system. When the device operating environment is complex, potential faults are hidden, or have multi-factor coupling characteristics, traditional monitoring cannot timely detect abnormal trends, affecting business continuity and system stability. In addition, existing device maintenance methods usually only perform single repair operations after a fault occurs, lacking comprehensive judgment and dynamic optimization strategies for overall health status, and cannot effectively improve the device self-healing capability.

[0004] Overall, existing technologies have obvious shortcomings in multi-modal data fusion, cross-modal feature extraction, anomaly detection, health indicator quantification, fault pattern recognition, and self-healing instruction generation and execution, leading to delayed device fault early warning, lack of systematic health management, and lack of efficient closed-loop mechanisms in the self-healing process, making it difficult to meet the actual needs of high-value devices and key business systems in terms of safety, reliability, and intelligent operation and maintenance. SUMMARY

[0005] The main purpose of the present application is to provide a device self-healing method and device based on spatiotemporal correlation features, and a storage medium, aiming to solve the technical problem that existing technologies lack cross-modal spatiotemporal correlation analysis and predictive self-healing closed-loop mechanisms for multi-modal sensing data, making it difficult to early identify and actively intervene in device faults.

[0006] To achieve the above-mentioned purpose, the present application provides a device self-healing method based on spatiotemporal correlation features, comprising:

[0007] acquiring multi-modal sensing data collected by multiple types of sensing devices deployed on a target device;

[0008] generate spatio-temporal correlation features across modalities based on the multi-modal sensor data;

[0009] perform anomaly detection using the spatio-temporal correlation features to obtain an anomaly score;

[0010] quantify health indicators of multiple operating dimensions of the target device based on the spatio-temporal correlation features, and generate a comprehensive health index by fusing the health indicators of the multiple operating dimensions;

[0011] identify a failure mode of the target device based on the anomaly score and the comprehensive health index, and generate a self-healing instruction according to the failure mode;

[0012] simulate and verify the self-healing instruction in a digital twin model of the target device, and send the self-healing instruction that passes the simulation verification to a control unit of the target device for execution to generate a self-healing result.

[0013] Further, to achieve the above object, the present application provides a device self-healing apparatus based on spatio-temporal correlation features, comprising:

[0014] a multi-modal data acquisition module for acquiring multi-modal sensor data collected by multiple types of sensing devices deployed on a target device;

[0015] a cross-modal feature fusion module for generating spatio-temporal correlation features across modalities based on the multi-modal sensor data;

[0016] an anomaly discrimination module for performing anomaly detection using the spatio-temporal correlation features to obtain an anomaly score;

[0017] a health index evaluation module for quantifying health indicators of multiple operating dimensions of the target device based on the spatio-temporal correlation features, and generating a comprehensive health index by fusing the health indicators of the multiple operating dimensions;

[0018] a failure self-healing decision module for identifying a failure mode of the target device based on the anomaly score and the comprehensive health index, and generating a self-healing instruction according to the failure mode;

[0019] a digital simulation execution module for simulating and verifying the self-healing instruction in a digital twin model of the target device, and sending the self-healing instruction that passes the simulation verification to a control unit of the target device for execution to generate a self-healing result.

[0020] Further, to achieve the above object, the present application also provides a computer device, comprising a memory, a processor and a device self-healing program based on spatio-temporal correlation features stored in the memory and executable on the processor, which, when executed by the processor, implements the steps of the device self-healing method based on spatio-temporal correlation features as described above.

[0021] Further, to achieve the above object, the present application also provides a computer device, comprising a memory, a processor and a device self-healing program based on spatio-temporal correlation features stored in the memory and executable on the processor, which, when executed by the processor, implements the steps of the device self-healing method based on spatio-temporal correlation features as described above.

[0022] Beneficial effects: The present application relates to the field of artificial intelligence technology, which can be applied to business scenarios such as financial technology and medical health, and discloses a device self-healing method, device and medium based on spatio-temporal correlation features, comprising: acquiring multi-modal sensor data collected by multiple types of sensor devices deployed on a target device, generating spatio-temporal correlation features across modalities based on the multi-modal sensor data, performing anomaly detection using the spatio-temporal correlation features to obtain an anomaly score, quantifying health indicators of multiple running dimensions of the target device based on the spatio-temporal correlation features and fusing to generate a comprehensive health index, identifying a failure mode of the target device based on the anomaly score and the comprehensive health index and generating a self-healing instruction, simulating and verifying the self-healing instruction in a digital twin model of the target device, sending the self-healing instruction that passes the simulation verification to a control unit of the target device for execution, and generating a self-healing result. The present application constructs spatio-temporal correlation features across modalities of multi-modal sensor data, combines anomaly scores and multi-dimensional health indicators, accurately identifies the failure mode of the target device, relies on a digital twin model to complete simulation verification of the self-healing instruction, avoids the risk of false triggering, and ultimately improves the real-time performance and reliability of device self-healing intervention. BRIEF DESCRIPTION OF DRAWINGS

[0023] The present application will be further described below in conjunction with the drawings and embodiments, in which:

[0024] Figure 1 An application environment diagram of the device self-healing method based on spatio-temporal correlation features in an embodiment of the present application;

[0025] Figure 2 A flow diagram of an embodiment of the device self-healing method based on spatio-temporal correlation features of the present application;

[0026] Figure 3 A functional module diagram of a preferred embodiment of the device self-healing device based on spatio-temporal correlation features of the present application;

[0027] Figure 4A structural schematic diagram of a computer device in an embodiment of the present application;

[0028] Figure 5 Another structural schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0029] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0030] The device self-healing method based on spatiotemporal correlation features provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 , wherein a user end communicates with a service end through a network. The service end can obtain multi-modal sensor data collected by multiple types of sensor devices deployed on a target device through the user end, generate spatiotemporal correlation features across modalities based on the multi-modal sensor data, perform anomaly detection using the spatiotemporal correlation features to obtain an anomaly score, quantify health indicators of multiple running dimensions of the target device based on the spatiotemporal correlation features and fuse to generate a comprehensive health index, identify a failure mode of the target device based on the anomaly score and the comprehensive health index and generate a self-healing instruction, simulate and verify the self-healing instruction in a digital twin model of the target device, send the self-healing instruction that passes the simulation and verification to a control unit of the target device for execution, and generate a self-healing result. By constructing spatiotemporal correlation features across modalities of multi-modal sensor data and combining an anomaly score with multi-dimensional health indicators, the present application realizes accurate identification of a failure mode of a target device, relies on a digital twin model to complete simulation and verification of a self-healing instruction, avoids the risk of false triggering, and ultimately improves the real-time performance and reliability of device self-healing intervention. The user end can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices. The service end can be implemented by an independent server or a server cluster composed of multiple servers. The present application will be described in detail below through specific embodiments.

[0031] Please refer to Figure 2 , Figure 2 A flowchart of an embodiment of the device self-healing method based on spatiotemporal correlation features provided by the present application. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be performed in an order different from that shown herein.

[0032] As shown in Figure 2 , the device self-healing method based on spatiotemporal correlation features proposed by the present application includes the following steps:

[0033] S10, obtaining multi-modal sensor data collected by multiple types of sensor devices deployed on a target device;

[0034] In this embodiment, in the process of acquiring multi-modal sensor data collected by multiple types of sensing devices deployed on the target device, it is first necessary to determine the type and application environment of the target device. The target device refers to a physical entity equipped with various functional modules and performing specific tasks in actual operation. For example, equipment involving continuous operation and high stability requirements is commonly used in medical detection, industrial production, information processing, and financial operation scenarios. Multiple types of sensing devices refer to sensing units of different principles and different measurement objects, which are used to synchronously acquire state information of the target device in different physical dimensions. The types of sensing devices include but are not limited to vibration sensors for detecting mechanical vibration state, infrared thermal imaging devices for detecting temperature distribution, acoustic emission sensors for acquiring ultrasonic signals, current sensors for monitoring electrical parameters, inertial measurement units for collecting position information, and other devices with signal sensing and data output functions. These devices are usually connected to the data acquisition network through standard communication interfaces, and the transmission path can be wired or wireless. The data format is set according to actual needs to ensure the integrity and timeliness of data transmission.

[0035] Multi-modal sensor data refers to a collection of data obtained and integrated by multiple types of sensing devices, including different physical quantities or different information dimensions. Multi-modal data comes from different perception channels, including time domain, frequency domain, or spatial distribution information, and its combination provides more comprehensive expression of device operating state. To ensure the time sequence consistency and physical correlation of the data, the acquisition process generally coordinates the sampling time points of each sensing device through a unified clock synchronization mechanism or a distributed time protocol. Typical synchronization mechanisms include synchronization based on Network Time Protocol (NTP) or synchronization acquisition through hardware trigger signals. The acquisition of multi-modal sensor data not only includes real-time collection of original physical quantity signals, but also includes necessary preprocessing of the data, such as filtering and noise reduction, outlier removal, and data reconstruction, to improve the accuracy of subsequent data analysis and state judgment.

[0036] Data acquisition can be achieved by deploying a combination of multiple types of sensing devices in key locations of the target device. For vibration monitoring, high-sensitivity acceleration sensors can be placed at the rotating shaft, bearing seat, and structural connection points of the device to capture mechanical vibration signals in real time. For thermal distribution monitoring, infrared thermal imaging modules can be configured at locations such as the motor, power module, and heat dissipation system to obtain device surface temperature field data at set time intervals. If there is a need to monitor weak acoustic signals, wide-band acoustic emission sensors can be deployed on the device shell or internal structure to capture acoustic signals caused by material defects or abnormal operation. In the electrical system part, current sensors or voltage detection modules can be used to monitor the operation status of the circuit. The above-mentioned multiple types of sensing devices are connected to the data processing unit through wired network interfaces or wireless communication protocols, forming a multi-modal sensing data acquisition network. To ensure the time synchronization of different data types, a centralized hardware synchronization pulse can be used to control the sampling frequency of each sensing device, or a network-level time synchronization protocol can be configured to ensure that multi-modal data have a unified timestamp, facilitating subsequent joint analysis.

[0037] In different implementation environments, the type, number, and placement of sensing devices can be flexibly adjusted according to the structural characteristics and functional requirements of the device. For example, in complex large-scale devices, sensing devices can be dispersedly placed near different functional modules or key components, and multi-point data fusion can be achieved through a modular data acquisition architecture. In compact devices or mobile terminals, multi-functional sensing devices can be designed to reduce device size and system complexity, ensuring the efficiency and stability of multi-modal data acquisition.

[0038] Example: In the medical health business field, by placing vibration, temperature, and acoustic emission sensing devices on key components such as the cooling system, gradient coil structure, and liquid helium storage bin inside the MRI device, the multi-dimensional state changes of the device under high-intensity operation can be monitored in real time. Multi-modal data can help capture complex fault precursor information such as abnormal vibration of the cooling system, temperature rise of the gradient coil, and acoustic emission signals of structural micro-cracks, allowing early detection of potential risks and ensuring the continuity and safety of image diagnosis devices.

[0039] In the financial technology business field, multi-modal sensing data acquisition of data center server devices or ATM terminals also has practical application value. Temperature sensors and vibration sensors can be deployed inside server cabinets to monitor the internal environment and mechanical stability of the device. For ATM terminals, acoustic emission and vibration signal detection can be used to detect illegal intrusion or device abnormalities. The fusion and acquisition of multi-modal data improves the risk identification capability of key financial devices, effectively reduces the probability of failure and security incidents, and ensures the stable and efficient operation of financial businesses.

[0040] The embodiment can realize comprehensive perception of the running state of the target device by collecting multi-modal sensing data through various types of sensing devices. The collaborative monitoring of multi-dimensional physical quantities such as vibration signals, temperature data, and acoustic information enables the capture of abnormal phenomena that cannot be covered by a single data channel. Multi-modal data acquisition improves the sensitivity of early fault signs of the device, compensates for the problems of abnormal missed detection and response lag in traditional single-parameter monitoring methods, and improves the reliability of device state evaluation and risk prevention and control.

[0041] S20, generating a cross-modal spatio-temporal correlation feature based on the multi-modal sensing data;

[0042] In the embodiment, in the process of generating a cross-modal spatio-temporal correlation feature based on multi-modal sensing data, it is first necessary to clarify the composition and source of the multi-modal sensing data. Multi-modal sensing data refers to a comprehensive data set obtained through various types of sensing devices, reflecting the different physical states of the target device, and usually includes but is not limited to vibration signals, ultrasonic signals, temperature field distribution data, electrical parameter data, or other observation information closely related to the running state of the device. Different data types have different time scales, spatial resolutions, and physical meanings, so a unified data processing framework is needed for fusion and structure reconstruction to improve the completeness of data representation and the accuracy of device state recognition.

[0043] Generating a cross-modal spatio-temporal correlation feature means extracting comprehensive feature information from time, space, and multiple data sources based on the above multi-modal data. This process not only includes basic feature extraction for a single data channel, such as extracting time-domain statistical features from vibration signals, extracting frequency energy distribution from ultrasonic signals, and extracting spatial distribution features from temperature field data, but also includes structured fusion between multiple data sources and spatio-temporal relationship modeling. Spatio-temporal correlation means mapping multiple data sources to a unified analysis space through mathematical methods or model frameworks, and mining the internal coupling relationship between different data types in the time and space dimensions. For example, by constructing a sensor network graph structure, using the physical layout relationship and signal transmission characteristics between nodes, the observation data of different sensing devices can be converted into a network model with spatial topology.

[0044] Cross-modal features refer to high-dimensional comprehensive features mined from different types of data based on data fusion, with stronger representation ability and multi-information integration ability. The generation process can use advanced structures such as graph neural networks, graph convolution networks, and spatio-temporal fusion networks in deep learning frameworks to jointly calculate and aggregate the input multi-modal data and the constructed sensor network structure, forming a comprehensive representation result containing device state time series features, spatial layout features, and multi-data source collaborative features.

[0045] The spatio-temporal correlation feature generation across modalities can be achieved in the following manner. First, vibration signals are extracted from the multi-modal sensor data, and statistical quantities such as kurtosis coefficient, wave factor, skewness, peak value, etc. are calculated in the time domain range to represent the dynamic response characteristics of the equipment operation. Second, ultrasonic signals are extracted from the multi-modal sensor data, and based on the wavelet packet decomposition method, the signals are decomposed at different frequency bands in multiple levels, the energy proportion of each decomposition layer node is extracted, and the frequency energy distribution features are constructed to reflect the potential defects or small abnormal signals inside the equipment structure. Third, temperature field distribution data are extracted from the multi-modal sensor data, combined with the thermal imaging results and the spatial position mapping relationship, a spatial distribution matrix of the equipment surface temperature is formed to identify the local heating or heat dissipation abnormal areas of the equipment.

[0046] To realize the spatial fusion of different data types, the deployed vibration sensors, infrared thermal imaging devices, acoustic emission sensors, etc. can be used as nodes in the network structure, the edge weights between the nodes are defined through the cross-correlation function values, spatial distance weights or other physical correlation indicators between the nodes, and a sensor node network graph is constructed. Then the time domain statistical features, frequency energy distribution, temperature field distribution data extracted above are input into the graph convolution network together with the sensor network graph structure, and the graph structure information transmission mechanism and deep feature extraction capability inside the network are used to generate spatio-temporal correlation features across modalities. The final features not only contain independent information of each data source, but also comprehensively reflect the cooperative relationship of the equipment operation state under multiple parameters, multiple positions and multiple time scales.

[0047] According to different equipment structures and application environments, the selection type and sensor layout strategy of multi-modal data can be flexibly adjusted to optimize the expression ability of spatio-temporal correlation features. For example, in the equipment with complex spatial layout and dispersed components, the sensor node density is increased to improve the resolution of spatial feature capture; in the environment where data acquisition is limited and single signal dominates, the time series feature extraction and multi-scale information decomposition of single channel data are enhanced to ensure that the generated spatio-temporal correlation features have comprehensive state expression ability.

[0048] Example: In the medical health business field, for MRI equipment, multi-modal sensor data are obtained by deploying vibration sensors, temperature detection devices and acoustic emission equipment on the gradient magnet, cooling system and shell structure, and spatio-temporal correlation features across modalities are generated by using graph convolution network, which can realize comprehensive perception of internal defects, temperature abnormalities, structural looseness, etc. of the equipment, and reduce the medical image errors and safety risks caused by equipment failure.

[0049] In the field of financial technology business, for data center server equipment, through the cabinet internal vibration sensor, the computer room environment temperature monitoring device and the acoustic anomaly detection module, a multi-modal sensing data input is formed, through the construction of a sensor network graph and a cross-modal spatio-temporal feature generation process, the comprehensive recognition ability of the abnormal state of the server is improved, the risk of business interruption and data loss caused by hardware failure of the key financial information system is prevented, and the stable operation of the financial service system is ensured.

[0050] The embodiment can effectively break through the technical bottlenecks of limited information in a single data channel and lack of spatial correlation modeling in traditional data fusion methods by generating cross-modal spatio-temporal correlation features based on multi-modal sensing data. This method combines multiple data sources to construct a comprehensive expression result with time sequence dynamic characteristics, spatial structure characteristics and multi-source collaborative characteristics, improves the accuracy and detail capture ability of equipment abnormal state recognition, and enhances the early detection ability of potential risks of complex systems.

[0051] S30, performing anomaly detection using the spatio-temporal correlation features to obtain an anomaly score;

[0052] In the embodiment, the process of performing anomaly detection using the spatio-temporal correlation features to obtain an anomaly score is based on the spatio-temporal correlation features as input data sources. The spatio-temporal correlation features are generated by joint modeling and feature extraction of multi-modal sensing data, and have the ability of multi-source information fusion, spatial structure coupling and time dynamic change comprehensive expression. Anomaly detection refers to identifying abnormal patterns or potential failure signs in the equipment operating state based on the feature information. The anomaly score is a quantitative representation of the current abnormal risk level of the equipment by comparing the deviation degree of the normal state and the current observation state. The higher the value, the greater the abnormal risk.

[0053] In the specific implementation process, the input spatio-temporal correlation features need to be reduced in dimension and compressed in features first to reduce the data complexity and retain the key feature information. This process can be realized by an encoder containing a long short-term memory unit. The encoder structure combines the time series processing ability of the recurrent neural network and the history information retention advantage of the long short-term memory unit, and can effectively capture the time dynamic change pattern and spatial structure relationship in the spatio-temporal correlation features. Through multi-layer neural network transformation, the high-dimensional spatio-temporal correlation features are mapped to low-dimensional compressed feature vectors, retaining the most representative information expression of the equipment operating state.

[0054] Subsequently, the compressed feature vector is input into the deconvolution decoder to perform a reconstruction operation, reconstructing the dimension and structure of the feature consistent with the original spatiotemporal correlation feature, simulating the feature expression of the device in the normal state. By calculating the difference between the spatiotemporal correlation feature and the reconstructed feature, a weighted error value is determined. To enhance the detection capability of abnormal sensitive areas, a weighting mechanism is introduced during the difference calculation process, giving higher weights to signal mutation areas or high-risk feature areas, and improving the ability to capture small abnormal changes.

[0055] The weighted error value reflects the deviation of the current device state from the theoretical normal state. Based on historical error data, combined with device operating environment and historical working condition data, a dynamic abnormal threshold is determined. The dynamic abnormal threshold is adapted to the long-term degradation trend and short-term state fluctuation of the device through statistical analysis, sliding window calculation, and trend monitoring, avoiding false positives and false negatives. When the weighted error value exceeds the dynamic abnormal threshold, it is determined that the device has an abnormality, and the weighted error value is further mapped to an abnormality score to form a quantitative expression result, assisting subsequent fault pattern recognition and self-healing decision-making.

[0056] An anomaly detection model based on an autoencoder structure can be used. First, the spatiotemporal correlation feature is input into an encoder containing a long short-term memory unit, a multi-layer LSTM network is configured, the time window length and hidden layer parameters are set, and a low-dimensional compressed feature vector is generated. Then, the compressed feature vector is restored to the same dimension as the input feature using a deconvolution decoder, and a reconstructed feature is output. During the reconstruction process, the attention mechanism can be combined to enhance the reconstruction accuracy of key features.

[0057] When calculating the weighted error value, indicators such as mean square error and absolute error can be used, and weighting coefficients can be designed based on spatial position, time period, and data type dimensions to increase error weights in abnormal sensitive areas. The determination of the dynamic abnormal threshold can use methods such as moving average, exponentially weighted moving average, and trend monitoring to update the threshold level in real time and adapt to changes in device state. The generation of the abnormality score can convert the error value into a risk score of 0 to 1 through standardized mapping and probability conversion, facilitating subsequent system linkage and decision output.

[0058] The structure design of the encoder and decoder can also be adjusted according to different device characteristics and data types, such as increasing the depth of the LSTM network in high dynamic environments and optimizing network parameters in low-speed stable environments to improve the accuracy and real-time performance of anomaly detection.

[0059] The embodiment breaks through the limitations of traditional single data source and static threshold method by using spatio-temporal correlation features for anomaly detection, combining multi-modal data fusion, spatial structure modeling and time series dynamic analysis. By introducing the self-encoder structure and dynamic anomaly threshold mechanism, the accurate identification of small, early and implicit abnormal states in complex systems can be realized, the sensitivity and reliability of equipment anomaly detection are improved, the false negative rate and false positive rate are reduced, and the equipment operation safety and stability are enhanced.

[0060] S40, based on the spatio-temporal correlation features, quantifying the health indicators of multiple running dimensions of the target equipment, and fusing the health indicators of the multiple running dimensions to generate a comprehensive health index;

[0061] In the embodiment, based on the spatio-temporal correlation features, the health indicators of multiple running dimensions of the target equipment are quantified, and the health indicators of multiple running dimensions are fused to generate a comprehensive health index, involving multi-dimensional state perception and health state comprehensive evaluation. The spatio-temporal correlation features come from the fusion modeling results of multi-modal sensor data, have the information expression ability of reflecting the different physical structures, spatial distribution and time dynamic change of the equipment, and contain multi-dimensional parameters such as vibration signal, temperature field data, acoustic emission signal and current signal.

[0062] Quantifying the health indicators of multiple running dimensions means extracting key parameters related to the state of the equipment from the spatio-temporal correlation features, constructing multiple independent health indicator systems for different physical structures and functional units, and comprehensively reflecting the running state and potential risk level of the equipment. The multiple running dimensions include but are not limited to mechanical wear, electrical stability, thermal stress and performance degradation. The mechanical wear indicator analyzes the envelope spectrum characteristics of the vibration signal to capture the wear degree of the mechanical structure such as bearing and gear. The electrical stability indicator is based on the harmonic distortion characteristics of the current signal to reflect the stability and reliability of the electrical system. The thermal stress indicator evaluates the thermal stress concentration and thermal fatigue state through the temperature field distribution characteristics. The performance degradation indicator combines the device output parameter data, and tracks the long-term performance change trend of the device through the adaptive filtering algorithm.

[0063] Fusing the health indicators of multiple running dimensions to generate a comprehensive health index, a weighted fusion method is used to integrate the health information of different dimensions into a single quantitative indicator, and the overall health level of the equipment is comprehensively reflected. The weighting coefficients are dynamically adjusted according to the equipment type, structural characteristics, use environment and other factors to ensure that the contribution of each dimension index to the comprehensive health index meets the actual running characteristics of the equipment, improving the accuracy and pertinence of health evaluation.

[0064] The quantification and integration of health indicators can be achieved in the following ways. First, the envelope spectrum characteristics of the vibration signal are extracted from the spatiotemporal correlation characteristics, the frequency components related to mechanical wear are identified, and the mechanical wear index is calculated based on the characteristic amplitude. Secondly, the harmonic distribution of the current signal is extracted, and the electrical stability index is determined using the total harmonic distortion (THD) or harmonic energy ratio. Thirdly, the temperature field distribution is analyzed, high temperature gradient areas or thermal stress concentration areas are identified, and the thermodynamic stress index is calculated. Finally, combined with the changing trend of the equipment output parameters, an adaptive filtering algorithm is used to eliminate short-term fluctuation interference, evaluate the performance attenuation trend, and quantify the performance attenuation index.

[0065] During the weighted fusion process, equipment structure classification can be used, such as increasing the weight of the thermodynamic stress index for magnetically sensitive equipment and the mechanical wear index for equipment with a high proportion of mechanical components. Fusion methods can employ various strategies, such as weighted averaging, fuzzy comprehensive evaluation, and neural network integration, to generate a comprehensive health index and output standardized, easily monitored health status indicators.

[0066] The health indicator system and integration strategy can also be optimized according to different application scenarios. For example, in scenarios with high reliability requirements, the weight of sudden risk indicators can be increased; in long-term stability monitoring scenarios, the weight of chronic performance degradation indicators can be increased, thereby improving the applicability and accuracy of the health index.

[0067] Example: In the healthcare business, for magnetic resonance imaging equipment, the mechanical wear index, electrical stability index, and thermodynamic stress index are quantified. These are combined with performance parameters such as the output image signal-to-noise ratio and magnetic field uniformity to generate a comprehensive health index. This allows real-time monitoring of the equipment's operating status, and early detection of hidden dangers such as magnet system degradation, cooling system anomalies, and gradient system failure, ensuring the quality of medical images and the safe operation of the equipment.

[0068] In the financial technology business field, for the servers and network equipment in the data center, by quantifying the mechanical wear index of the vibration signal, the electrical stability index of the power supply system, and the thermodynamic stress index of the computer room environment, combined with performance indicators such as server response speed and business processing capabilities, a comprehensive health index is generated. This can monitor the overall health level of the data center in real time, reduce the risk of business interruption and data loss caused by equipment failure, and improve the stability of the financial system and service continuity.

[0069] The embodiment quantifies health indicators of multiple operation dimensions, comprehensively reflects state changes of the target device in aspects of mechanics, electricity, thermodynamics and overall performance, and makes up for the limitations of single monitoring indicators. The health indicators are fused to generate a comprehensive health index, forming a standardized and unified health evaluation result, improving the systematicness and accuracy of device state monitoring, facilitating risk early warning, fault diagnosis and maintenance decision-making, and enhancing the safety, stability and reliability of device operation.

[0070] S50, identifying a fault mode of the target device based on the anomaly score and the comprehensive health index, and generating a self-healing instruction according to the fault mode;

[0071] In the embodiment, the fault mode of the target device is identified based on the anomaly score and the comprehensive health index, and the self-healing instruction is generated according to the fault mode, which belongs to the fault type determination and active intervention decision-making process driven by multiple source state data. The anomaly score is derived from the anomaly detection result, reflecting the abnormality degree of the target device in the time and space correlation feature level, and having the ability to sensitively capture sudden abnormal signals. The comprehensive health index fuses health indicators of multiple operation dimensions, quantifies the overall health state change trend, and has the function of evaluating gradual performance degradation and structural hazards.

[0072] Identifying the fault mode means determining the specific fault type currently existing in the target device based on the dynamic changes of the anomaly score and the comprehensive health index. The fault mode can include a sudden fault mode and a gradual fault mode. The former usually shows that the device operation parameters deviate sharply from the normal range in a short time, and the latter shows that the comprehensive health index of the device continues to decline, reflecting the gradual deterioration process of the device structure, function or performance. In the determination process, the change trend of the anomaly score is monitored to identify whether the anomaly score exceeds the mutation threshold in a set time window. If there is a mutation, it indicates that a sudden fault may occur. The change slope of the comprehensive health index is monitored to identify the downward trend of the health index in the continuous operation process. If the slope exceeds the set threshold, it indicates that the device has a gradual fault risk.

[0073] According to the identified fault mode, a self-healing instruction is generated, which is used to guide the target device to perform self-recovery or external intervention operation to alleviate or eliminate the fault risk. The self-healing instruction includes a dynamic parameter adjustment instruction and a physical maintenance operation sequence. The dynamic parameter adjustment instruction reduces the device load and optimizes the operation state by adjusting the device control parameters, operation strategy, protection configuration and other means, realizing online self-repair. The physical maintenance operation sequence is used to guide the maintenance personnel or remote control system to perform operations such as structure replacement, component maintenance and system calibration, to solve the fault problems that cannot be repaired online.

[0074] The fault mode recognition and self-healing instruction generation can be achieved in the following manner. First, the trend of the change in the anomaly score is continuously monitored, the maximum change amplitude of the anomaly score is counted based on a sliding time window, and if the change amplitude of the anomaly score exceeds a preset mutation threshold within a set time period, it is determined that a sudden fault mode has occurred, such as a sudden event such as mechanical impact, electrical short circuit, component shedding, etc. Second, the trend of the change in the comprehensive health index is monitored, the change slope is calculated based on multiple health index sampling results, and if the change slope exceeds a set slope threshold and the trend persists, it is determined that a gradual fault mode has occurred, such as mechanical wear and tear, continuous temperature rise, gradual performance degradation, etc.

[0075] For sudden fault modes, dynamic parameter adjustment instructions are automatically generated, such as reducing output power, limiting operating frequency, enabling emergency cooling measures, quickly mitigating the impact of faults, and protecting the safe operation of the device. For gradual fault modes, physical maintenance operation sequences are generated, including maintenance steps, operation specifications, and replacement component information, to guide maintenance personnel to orderly carry out maintenance and improve maintenance efficiency and accuracy.

[0076] The fault mode determination threshold, parameter adjustment strategy, and maintenance operation content can also be flexibly configured according to different device types, fault characteristics, and operating environments to enhance the adaptability and effectiveness of the self-healing instructions. For example, in high-risk scenarios, the time window is shortened, the mutation threshold is reduced, and the response speed of sudden faults is improved. In long-period operation scenarios, the health index slope threshold is dynamically adjusted to optimize the identification accuracy of gradual faults.

[0077] Example: In the medical health business field, for magnetic resonance devices, the anomaly score is used to monitor sudden abnormalities in the gradient system and cooling system, and the comprehensive health index is used to track the long-term degradation trend of the magnet system and radio frequency system. Different fault modes are distinguished, self-healing instructions are quickly generated, scanning parameters are dynamically adjusted, or engineers are guided to carry out maintenance, ensuring medical image quality and device safety.

[0078] In the financial technology business field, for data center devices, the anomaly score is analyzed in real time to identify sudden faults in servers and switches, the comprehensive health index is monitored to assess the overall device stability and long-term performance degradation risk, self-healing instructions are generated, load distribution strategies are dynamically adjusted, or physical maintenance is arranged, effectively reducing the probability of system interruption and data loss, and ensuring the continuity of financial business and data security.

[0079] The embodiment can accurately distinguish between sudden and gradual failure modes through joint analysis based on the abnormal score and the comprehensive health index, avoid misjudgment and missed judgment, and improve the accuracy and timeliness of fault identification. According to different failure modes, targeted self-healing instructions are generated to realize online self-recovery and structural level intervention of the equipment, enhance the stability, reliability and safety of the equipment operation, and reduce economic losses and safety risks caused by faults.

[0080] S60, in the digital twin model of the target device, the self-healing instruction is simulated and verified, and the self-healing instruction passed by the simulation verification is sent to the control unit of the target device for execution, and a self-healing result is generated.

[0081] In the embodiment, the digital twin model refers to a virtual mapping model constructed in the digital space based on the structural information, functional logic, running state, environmental parameters and historical data of the physical entity equipment. The model has a high degree of synchronization with the data state and dynamic response capability of the physical equipment, and can reflect the structural characteristics, working process and running state changes of the target equipment in real time.

[0082] The digital twin model usually includes the following core contents:

[0083] First, the equipment structure data, including the three-dimensional geometric structure, component parts, material information and assembly relationship of the equipment, is derived from the design drawings, CAD model or physical survey data of the equipment, to ensure that the virtual model is consistent with the actual equipment in the physical layer.

[0084] Second, the running parameter mapping synchronizes the running state, performance index, control strategy and external environmental parameters of the equipment to the virtual model through sensor data, control logic and real-time monitoring information, forming a dynamic correlation.

[0085] Third, the environment and working condition modeling couples external factors such as temperature, pressure, vibration and electrical characteristics with the internal state of the equipment based on the actual application scenario, simulating the comprehensive performance of the equipment under different working conditions.

[0086] Fourth, the behavior and response mechanism combines data-driven methods, physical simulation technology and intelligent algorithms to give the digital twin model the ability of autonomous analysis, prediction and verification, realizing state monitoring, anomaly prediction and scheme verification in a virtual environment.

[0087] The digital twin model has the ability of dynamic synchronization, virtual-real mapping and simulation deduction, and can be widely used in equipment design optimization, fault prediction, state evaluation, self-healing verification and intelligent operation and maintenance, etc., to improve the efficiency and reliability of equipment management.

[0088] In the digital twin model of the target device, the self-healing instruction is simulated and verified, and the self-healing instruction passed by the simulation verification is sent to the control unit of the target device for execution, generating a self-healing result, involving device control strategy testing based on a virtual environment and a physical execution process. The digital twin model refers to a digital virtual mapping environment constructed based on the structural information, running data, physical parameters and environmental information of the target device. The model has a synchronous data state and running logic with the actual device, and can be used for functional verification and effect prediction without interfering with the actual device operation.

[0089] The self-healing instruction is generated for specific abnormal types of the device from the previous fault mode recognition link, and contains dynamic parameter adjustment operations and physical maintenance operation sequences, aiming to achieve functional recovery and risk elimination of the device through online adjustment or physical intervention means. Simulation verification refers to executing the self-healing instruction in a virtual environment constructed based on the digital twin model, evaluating the feasibility and effectiveness of the self-healing instruction through the response state and parameter changes of the virtual device, ensuring that the instruction will not have a negative impact on the normal operation of the device, and improving the safety and reliability of the instruction execution.

[0090] The self-healing instruction that has passed simulation verification refers to the instruction that has completed instruction effect testing in the digital twin model, confirmed that the instruction can effectively alleviate or solve the abnormal problem of the device, and will not cause new system risks, and is selected for the actual device execution process. The control unit refers to a hardware or software module in the target device that has control logic, parameter management and running adjustment capability, responsible for receiving the self-healing instruction and specifically executing the corresponding operation at the physical device level to complete the functional recovery and performance optimization process.

[0091] Generating a self-healing result refers to forming result data reflecting the self-healing effect after the execution of the self-healing instruction through the changes of the real-time running parameters and performance state of the device, reflecting the improvement of the self-healing operation on the running state of the device, and assisting subsequent state monitoring and strategy optimization.

[0092] The simulation verification and actual execution of the self-healing instruction can be implemented in the following way. First, in the digital twin model, load the detailed structural data, running parameters and historical state information of the target device to construct a virtual mapping environment highly synchronized with the actual device. Second, inject the fault mode parameters detected by the device to simulate the running process of the device in the actual fault state, ensuring that the simulation environment has real abnormal performance. Then, input the generated self-healing instruction into the digital twin model to virtually execute dynamic parameter adjustment operations or physical maintenance operation sequences, observe the changes of key parameters of the virtual device, and judge the influence of the self-healing instruction on the fault state and system stability.

[0093] During the monitoring of the virtual device operation, the recovery trend of the key parameters, the system stability index and the potential abnormal risk are analyzed to determine whether the self-healing instruction is verified by simulation. If it is confirmed that the self-healing instruction effectively solves the device fault problem in the virtual environment and does not introduce new system abnormalities, it is determined that the simulation verification is passed. The self-healing instruction that passes the verification is sent to the control unit of the target device. The control unit adjusts the operation parameters, optimizes the control strategy or initiates the maintenance operation at the physical level according to the instruction content, and executes the specific self-healing operation.

[0094] After the execution of the self-healing instruction is completed, the running state and the key performance parameters of the target device are continuously monitored to form self-healing result data reflecting the effect of the self-healing operation. The result data includes the recovery situation of the device function index, the change trend of the system stability and the elimination effect of the potential risk, which assists in evaluating the actual effect of the self-healing operation and optimizing the subsequent device management strategy.

[0095] The simulation accuracy, parameter range and data update frequency of the digital twin model can also be flexibly adjusted according to the needs of different types of devices and running scenes to improve the accuracy and applicability of simulation verification and enhance the reliability and safety of the execution of the self-healing instruction. For example, for high-risk devices, the model parameter update frequency is increased to enhance real-time synchronization effect, and for complex device structures, the simulation scene is refined to cover more abnormal working conditions to ensure the adaptability of the self-healing instruction.

[0096] The embodiment simulates and verifies the self-healing instruction in the digital twin model to evaluate the feasibility and effect of the instruction in advance, avoids the direct operation of unverified instructions on actual devices, reduces the risk of instruction execution, sends the self-healing instruction that passes the simulation verification to the control unit for execution, ensures the safety and effectiveness of the actual operation, and improves the success rate of the device self-healing process. The self-healing result is generated to form objective data basis for the improvement of the device state, support subsequent state monitoring and operation and maintenance strategy optimization, and improve the reliability of device operation and system management efficiency.

[0097] The application relates to the technical field of artificial intelligence, can be applied to business scenes such as financial technology and medical health, and discloses a device self-recovery method and device based on space-time correlation features, equipment and a medium, which comprises the following steps: acquiring multi-modal sensing data collected by multiple types of sensing devices deployed on a target device, generating cross-modal space-time correlation features based on the multi-modal sensing data, performing abnormality detection by using the space-time correlation features to obtain an abnormality score, quantifying health indexes of multiple running dimensions of the target device based on the space-time correlation features and fusing to generate a comprehensive health index, identifying a fault mode of the target device based on the abnormality score and the comprehensive health index and generating a self-recovery instruction, simulating and verifying the self-recovery instruction in a digital twin model of the target device, sending the self-recovery instruction that has passed the simulation verification to a control unit of the target device for execution, and generating a self-recovery result. By constructing cross-modal space-time correlation features of multi-modal sensing data, combining an abnormality score and multi-dimensional health indexes, accurate identification of a fault mode of a target device is realized, simulation verification of a self-recovery instruction is completed relying on a digital twin model, the risk of false triggering is avoided, and the real-time performance and reliability of device self-recovery intervention are finally improved.

[0098] In one embodiment, the above step S10 comprises:

[0099] S101, a vibration sensor is deployed at a rotating component bearing seat of the target device to collect vibration signals in a preset vibration frequency band range;

[0100] S102, an infrared thermal imaging device is deployed at a circuit area of the target device to collect temperature field distribution data at a fixed sampling rate;

[0101] S103, an acoustic emission sensor is deployed at a mechanical structure of the target device to collect ultrasonic signals in a preset ultrasonic frequency band range;

[0102] S104, time sequence data of the vibration signals, the temperature field distribution data and the ultrasonic signals are aligned by using a preset precision time protocol to generate multi-modal sensing data synchronized in time.

[0103] In the embodiment, the multiple types of sensing devices deployed on the target device include multiple heterogeneous sensing units configured for different position and different parameter acquisition requirements, wherein the vibration sensor is used to capture the slight vibration response of the rotating component during the working process, common deployment positions include key positions such as bearing seats, shaft couplings and shafts, and specific sensors can adopt charge type accelerometers or piezoelectric vibration meters, have wide frequency response characteristics, and can cover the preset vibration frequency band range in a typical mechanical system, which can be dynamically set according to the device structure and fault characteristics, for example, in the rolling bearing monitoring scene, the preset frequency band generally covers the range of 50Hz to 10kHz, so as to facilitate the capture of early weak impact features.

[0104] The infrared thermal imaging device is arranged in the circuit area, and mainly functions to obtain the temperature field distribution state of the internal and external target equipment, and reflect the thermodynamic changes of electrical components, heat dissipation structures and environmental factors. The device can adopt short-wave, medium-wave or long-wave infrared detectors, combined with a high-resolution focal plane array, to periodically capture images at a fixed sampling rate, ensure the time continuity and spatial integrity of the temperature field distribution data, and the sampling rate can be set according to the thermal response rate of the equipment, and a typical value is 1 frame / second to 30 frames / second.

[0105] The acoustic emission sensor is used for monitoring the ultrasonic signals inside or on the surface of the mechanical structure, and is suitable for non-contact detection of early crack propagation, impact friction and micro-damage process. The sensor can be arranged at the housing, frame and key stress connection position of the equipment, and adopts a broadband high-sensitivity sensing element, which can cover a preset ultrasonic frequency band of 100 kHz to 1 MHz, and effectively capture the abnormal signal of the microstructure.

[0106] The multi-modal sensing data includes the vibration signal, the temperature field distribution data and the ultrasonic signal. Since the sampling frequency, data format and communication delay of different sensors are different, in order to realize high-precision time alignment of multi-source information, a preset precise time protocol is used for synchronization. The protocol can combine the IEEE 1588 PTP high-precision clock synchronization technology or GNSS time reference to ensure that all kinds of sensing data are uniformly marked with time stamps within a microsecond time window, generate time-synchronized multi-modal sensing data, form a joint data set with time sequence consistency and data structure integrity, and support subsequent correlation analysis, feature extraction and multi-source fusion processing.

[0107] The embodiment cooperatively deploys various types of sensing devices to comprehensively obtain the state information of the target equipment in different physical dimensions such as vibration, heat and acoustics, and realizes high-precision synchronization of multi-modal data by combining the precise time protocol. This can effectively improve the time consistency and structural integrity of multi-source monitoring data, enhance the capture ability of early weak abnormal signals, improve the comprehensiveness and reliability of equipment state perception, reduce the risk of feature misjudgment caused by data time sequence offset, and meet the high-precision, multi-dimensional state monitoring and data fusion needs of complex equipment.

[0108] In one embodiment, the above step S20 includes:

[0109] S201, extracting a vibration signal from the multi-modal sensing data, and determining time-domain statistical features including kurtosis coefficient and waveform factor in the vibration signal;

[0110] S202, extracting an ultrasonic signal from the multi-modal sensing data, and performing wavelet packet decomposition on the ultrasonic signal to extract an energy proportion of a preset decomposition layer node as a frequency energy distribution;

[0111] S203, extracting temperature field distribution data from the multi-modal sensor data;

[0112] S204, constructing a sensor node network graph with vibration sensors, infrared thermal imaging devices, and acoustic emission sensors as nodes and the cross-correlation function value of the signals between nodes as edge weights;

[0113] S205, inputting the time-domain statistical features, the frequency-domain energy distribution, the temperature field distribution data, and the sensor node network graph into a graph convolution network to generate cross-modal spatio-temporal correlation features.

[0114] In this embodiment, multi-modal sensor data is used as an input information set, including vibration signals, ultrasonic signals, and temperature field distribution data, which are obtained by joint acquisition of various types of sensing devices in the previous steps. The vibration signal reflects the dynamic response of the mechanical structure during operation, usually in the form of a continuous waveform sequence that changes over time. In the process of extracting this signal, the time-domain data is segmented and statistically analyzed, including the calculation of statistical features such as kurtosis coefficient and waveform factor. The kurtosis coefficient measures the sharpness of the signal and is usually calculated as the ratio of the fourth central moment to the variance, which can reflect the sudden impact characteristics. The waveform factor is the ratio of the effective value to the mean square value of the signal, which reflects the stability and abnormality of the vibration. The combination of the two constitutes the time-domain statistical features, which are sensitive to early weak features of mechanical faults.

[0115] The ultrasonic signal is captured by the acoustic emission sensor and is represented as a pulse response signal at high frequency. Wavelet packet decomposition method is used for multi-scale decomposition analysis to decompose the energy distribution characteristics of the signal in different frequency bands layer by layer. The specific steps include constructing a multi-scale wavelet packet tree, selecting a preset decomposition layer subnode, calculating the signal energy proportion of each subnode, and extracting energy distribution data reflecting frequency domain characteristics. This data reveals high-frequency abnormal information such as mechanical micro-cracks, wear, and impact.

[0116] The temperature field distribution data is derived from the continuous image output of the infrared thermal imaging device, which records the temperature variation trend of the device surface and internal structure, reflecting the thermodynamic state of the system. This data is represented in the form of an image matrix, containing the correspondence between spatial coordinates and temperature values, facilitating subsequent fusion processing.

[0117] The vibration sensor, infrared thermal imaging device, and acoustic emission sensor are used as nodes, and the spatial deployment position and data source are combined to construct a sensor node network graph. The cross-correlation function value of the signals between nodes is used as the edge weight. The cross-correlation function measures the similarity and time delay relationship between two signals. By calculating the time correlation and structural coupling effect between different sensor data streams, a spatial topology structure is formed.

[0118] Finally, the time domain statistical features, frequency domain energy distribution, temperature field distribution data and sensor node network graph are jointly input into the graph convolution network. The graph convolution network has feature extraction capability based on graph structure. Through the multi-layer neural network structure, information is transmitted along the network graph topology, and different node and edge weight information is fused to generate a cross-modal spatio-temporal correlation feature with multi-modal data fusion, spatio-temporal structure expression and cross-physical quantity information correlation capability. The feature contains dynamic relationships and structural coupling characteristics among vibration, thermal and acoustic signals, supporting subsequent fault detection and health assessment.

[0119] The embodiment comprehensively mines the deep correlation characteristics among multi-modal sensor data through joint processing of vibration signals, ultrasonic signals and temperature field distribution data, combined with time domain statistical analysis, frequency domain energy extraction and spatial topology structure construction. It realizes spatio-temporal information fusion across physical quantities and across sensor channels relying on the graph convolution network, improves the overall expression capability of multi-dimensional data of complex equipment, enhances the accuracy and sensitivity of abnormal pattern recognition, reduces the misjudgment risk caused by isolated analysis of different sensor information, and meets the needs of multi-source data deep fusion and complex system dynamic state monitoring.

[0120] In one embodiment, the above step S30 comprises:

[0121] S301, inputting the spatio-temporal correlation feature into an encoder containing a long short-term memory unit to generate a compressed feature vector;

[0122] S302, inputting the compressed feature vector into a deconvolution decoder to output a reconstructed feature;

[0123] S303, determining a weighted error value of the spatio-temporal correlation feature and the reconstructed feature;

[0124] S304, determining a dynamic anomaly threshold based on historical error data;

[0125] S305, comparing the weighted error value with the dynamic anomaly threshold to generate an anomaly score.

[0126] In this embodiment, the spatio-temporal correlation feature is a feature information generated by a graph convolution network based on multi-modal sensor data in a pre-step, which embodies the spatial layout relationship and temporal dynamic characteristics between different sensor data. The feature sequence is input as input data into an encoder structure containing long short-term memory units. The long short-term memory unit has the ability to capture long-distance dependence and short-term fluctuations within a time series, and specifically includes input gate, forget gate and output gate mechanisms, which dynamically adjust the information retention and discard strategy through the gating structure, avoiding the gradient vanishing or explosion problem existing in the traditional recursive structure, thereby effectively extracting the time sequence change pattern of the input data. The encoder structure is stacked and nonlinearly mapped through multiple layers to compress the original high-dimensional spatio-temporal correlation feature into a low-dimensional feature vector, retaining key abnormal information and improving subsequent computing efficiency. The compressed feature vector has the joint expression ability of time dependence and spatial relationship.

[0127] Subsequently, the compressed feature vector is input into a deconvolution decoder. The deconvolution structure uses layer-by-layer upsampling and inverse transformation operations to gradually restore the data dimension and time sequence structure, generating reconstructed features with the same dimension and structure as the original spatio-temporal correlation features. This process aims to measure the expression ability of the model in capturing the internal pattern of the input data through an end-to-end auto-encoding reconstruction mechanism.

[0128] By comparing the difference between the reconstructed feature and the original spatio-temporal correlation feature, a weighted error value is calculated. The error value reflects the areas where the model reconstruction ability is insufficient, and is usually measured using indicators such as mean square error and absolute error. During the error calculation process, higher weights are dynamically assigned to signal mutation areas, i.e. short-term impact in vibration signals, high-amplitude pulses in ultrasonic signals, and local abnormal hotspots in temperature distribution, highlighting the contribution of abnormal information in the overall error and improving the sensitivity and response speed to weak fault signs.

[0129] Combined with the historical error data sequence, a dynamic abnormal threshold is constructed. This threshold is based on error statistical characteristics within a sliding time window, such as mean, variance, and fluctuation range, and is self-adaptive to device operating state changes through a dynamic adjustment strategy, avoiding the limitations of fixed threshold schemes that lack adaptability to environmental interference, device aging, and other factors, ensuring that the abnormal judgment standard is optimized in real time with the actual operating state.

[0130] Finally, the weighted error value calculated in real time is compared with the dynamic abnormal threshold. If the error value exceeds the threshold, it indicates that there is a potential anomaly in the system, and the corresponding abnormal score is output. The abnormal score can be set as a continuous numerical value or a probability index, reflecting the current abnormal risk level of the device and providing basic data support for subsequent health assessment and fault identification.

[0131] The embodiment realizes compression expression and structure restoration of the spatio-temporal correlation feature by an encoder structure based on a long short-term memory unit combined with a deconvolution decoding process, fully retains dynamic correlation information in the multi-modal data, objectively quantifies the degree of system anomaly through self-encoding reconstruction difference calculation, and avoids misjudgment risk caused by relying on a single data source or static indicators. A dynamic weighting mechanism for signal mutation area is introduced to enhance the recognition ability of early weak abnormal signals, and a historical error adaptive dynamic threshold design is combined to ensure real-time adaptability and environmental robustness of the abnormal detection standard, effectively improve the abnormal monitoring accuracy and early warning response efficiency of complex equipment, reduce the fault omission rate, and meet the stable operation and health management requirements of the equipment.

[0132] In one embodiment, the above step S40 comprises:

[0133] S401, determining a mechanical wear index based on an envelope spectrum feature of a vibration signal in the spatio-temporal correlation feature;

[0134] S402, determining an electrical stability index based on a current signal harmonic distortion feature in the spatio-temporal correlation feature;

[0135] S403, determining a thermodynamic stress index based on a temperature field distribution feature in the spatio-temporal correlation feature;

[0136] S404, determining a performance attenuation index through an adaptive filtering module based on the operation output parameter data of the target equipment;

[0137] S405, dynamically adjusting weight coefficients of the mechanical wear index, the electrical stability index, the thermodynamic stress index, and the performance attenuation index according to the target equipment type;

[0138] S406, generating a comprehensive health index by weighted fusion of the mechanical wear index, the electrical stability index, the thermodynamic stress index, and the performance attenuation index based on the weight coefficients.

[0139] In the embodiment, an envelope spectrum feature is extracted from a vibration signal in the spatio-temporal correlation feature. The envelope spectrum refers to obtaining frequency spectrum information based on envelope demodulation of the vibration signal, and is especially suitable for early detection of small mechanical faults. The envelope spectrum feature reflects energy distribution and fault characteristic frequency component changes of the equipment in a high frequency interval. Based on the feature, a mechanical wear index is calculated. The mechanical wear index can adopt a normalized energy index, a characteristic frequency amplitude index, or a spectral peak intensity index, and comprehensively reflects the wear state and structural defect development trend of bearings, gears, and other components.

[0140] Harmonic distortion is mainly caused by nonlinear loads, insulation aging, or partial discharge phenomena in electrical systems. Fourier transform or high-order spectrum analysis is used to quantify the total harmonic distortion rate, specific harmonic content, or phase distortion degree, and determine the electrical stability index. This index reflects the stability of the electrical circuit of the device, the quality of power supply, and potential electrical fault risks.

[0141] By extracting the temperature field distribution characteristics in the spatiotemporal correlation feature, the thermodynamic stress index is calculated. The temperature field distribution reflects the heat conduction state and abnormal hot spot area of each functional unit or structural node of the device. Through thermal image feature extraction, temperature gradient calculation, and finite element thermal stress inversion, the risk level of thermal deformation, fatigue damage, and thermal instability of the device structure is evaluated, forming a thermodynamic stress index that comprehensively reflects the thermal health status of the device.

[0142] Combined with the running output parameter data of the target device, the adaptive filtering module is used to dynamically track the performance change trend. The running output parameters include device power output, speed, efficiency, stability index, and other real-time data reflecting the overall performance level. The adaptive filtering module uses Kalman filtering, exponential weighted moving average, or other dynamic filtering methods to suppress environmental interference and short-term fluctuations, extract long-term performance degradation trends, generate a performance degradation index, and quantify device performance degradation and service life changes.

[0143] Considering the differences in structure, function, and working environment of different types of target devices, the weight coefficients of each operating dimension health index are dynamically adjusted. The weight coefficients can be set according to device manufacturing parameters, historical operation data, or expert experience, highlighting the influence weight of the main fault risk source of the device, and improving the applicability and accuracy of health assessment.

[0144] According to the adjusted weight coefficients, the mechanical wear index, electrical stability index, thermodynamic stress index, and performance degradation index are weighted and fused. The fusion method can use weighted average, fuzzy comprehensive evaluation, or neural network integration model to integrate multi-dimensional health information into a single comprehensive health index. The comprehensive health index is presented in the form of continuous numerical value or graded index, reflecting the current overall health level and potential risk level of the device.

[0145] The embodiment decomposes multi-dimensional health information in multi-modal data, combines four operation dimensions of machinery, thermotics, electricity and performance, comprehensively quantifies health states of equipment from different angles, and avoids one-sidedness and uncertainty of evaluation by a single data source. Envelope spectrum analysis, harmonic distortion detection, thermodynamic inversion and performance trend extraction are adopted to realize sensitive monitoring of early weak fault signals and comprehensive performance degradation of equipment, and to improve objectivity and accuracy of health state determination. A device type adaptive weight adjustment mechanism is introduced to enhance universality and pertinence of the evaluation system, and to meet health management needs in different structures and application scenarios. Based on weighted fusion of multi-dimensional health indicators, a comprehensive health index is formed to simplify complex health information expression, improve intuitiveness and decision efficiency of health evaluation, and help equipment operation state transparency and risk management intelligentization.

[0146] In one embodiment, the step S50 comprises:

[0147] S501, monitoring a change trend of the anomaly score, and determining whether the change trend exceeds a mutation threshold within a set time period;

[0148] S502, if the change trend of the anomaly score exceeds the mutation threshold, identifying a sudden failure mode;

[0149] S503, monitoring a change slope of the comprehensive health index, and determining whether the change slope exceeds a set slope threshold;

[0150] S504, if the change slope of the comprehensive health index exceeds the set slope threshold, identifying a gradual failure mode;

[0151] S505, when the failure mode is the sudden failure mode, generating a dynamic parameter adjustment instruction;

[0152] S506, when the failure mode is the gradual failure mode, generating a physical maintenance operation sequence.

[0153] In the embodiment, by monitoring the change trend of the anomaly score, dynamic changes of the anomaly score in a time sequence are obtained. The anomaly score reflects the abnormality degree of the equipment under multi-source data correlation, and the change trend includes indexes such as rising rate, fluctuation amplitude and mutation amplitude of the anomaly score. The trend is analyzed based on a time window of a set time period. The set time period is a fixed time interval determined in combination with equipment response characteristics and failure development rate. A second-level or minute-level time window can be set according to different equipment types to ensure that the trend determination is real-time and targeted.

[0154] In combination with the change trend, it is judged whether it exceeds the mutation threshold in the set period. The mutation threshold is an abnormal score mutation critical standard determined according to historical data statistics and expert experience, reflecting the boundary level of the rapid transition of the device state from normal to abnormal. If the abnormal score rises in the set period or the rate exceeds the mutation threshold, it means that the device has a rapid failure risk, and it is determined as a sudden failure mode. The sudden failure mode usually corresponds to hardware damage, system instability or safety risk appearing in a short time.

[0155] At the same time, the change slope of the comprehensive health index is monitored to extract the long-term degradation trend reflecting the overall health level of the device. The comprehensive health index integrates multiple health indicators, and the change slope represents the decline rate or amplitude of the index in unit time. The health degradation degree of the device is analyzed in combination with the change slope trend and fluctuation.

[0156] It is judged whether the change slope of the comprehensive health index exceeds the set slope threshold. The set slope threshold is a health index decline rate threshold set in combination with the design life, performance standard and historical degradation data of the device. If the change slope continuously exceeds the set threshold, it means that the device has a gradual structural aging, performance degradation or function weakening trend, and it is identified as a gradual failure mode. The gradual failure mode generally shows that the device is in a sub-health state for a long time, accompanied by slow performance decline and reliability weakening.

[0157] Differentiated self-healing instructions are developed for different failure modes. When it is identified as a sudden failure mode, a dynamic parameter adjustment instruction is generated. The dynamic parameter adjustment instruction includes system adaptive configuration adjustment, running parameter limiting, control strategy optimization and other measures to quickly suppress abnormal expansion, reduce the failure influence range and protect the safety of the device.

[0158] When it is identified as a gradual failure mode, a physical maintenance operation sequence is generated. The physical maintenance operation sequence includes specific repair, replacement, calibration, reinforcement and other operation steps developed based on the failure analysis results, guiding maintenance personnel to implement targeted intervention, repair potential hazards, delay the device degradation process and improve the operation reliability.

[0159] This embodiment considers the monitoring needs of short-period rapid abnormalities and long-period health degradation through joint analysis based on the abnormal score and the comprehensive health index, improves the accuracy and comprehensiveness of failure mode identification. The device state is dynamically evaluated through the change trend and the change slope, breaking through the lag and one-sidedness of single threshold alarm, identifying sudden and gradual failures in advance, and avoiding major device failures. Through the differentiated self-healing instruction strategy, rapid response to sudden abnormalities and precise intervention to gradual degradation are realized, improving the self-recovery ability and maintenance efficiency of the device, reducing downtime and operation and maintenance costs, and ensuring the safe, stable and efficient operation of the device.

[0160] In one embodiment, the step S60 comprises:

[0161] S601, constructing a three-dimensional virtual model of the target device in a digital twin model of the target device;

[0162] S602, injecting an equivalent fault parameter corresponding to the fault mode in the digital twin model;

[0163] S603, executing the self-healing instruction in the digital twin model;

[0164] S604, monitoring the recovery state of the key parameters in the digital twin model;

[0165] S605, when the recovery state of the key parameters reaches a preset standard, determining that the self-healing instruction is verified in the digital twin model;

[0166] S606, sending the verified self-healing instruction to a control unit of the target device for execution;

[0167] S607, generating a self-healing result based on the running parameters of the target device after executing the self-healing instruction.

[0168] In the embodiment, the digital twin model is used to virtually restore and dynamically simulate the actual running state of the target device. The three-dimensional virtual model is an important component of the digital twin model. By integrating device structure, physical parameters, functional logic and spatial layout, a digital space object corresponding to the real device is constructed. The three-dimensional virtual model is derived from computer-aided design data, sensing data and operation and maintenance historical information. The model not only has geometric appearance, but also has dynamic response characteristics, realizing high simulation reproduction of device state and behavior.

[0169] By injecting an equivalent fault parameter corresponding to the fault mode in the digital twin model, the running state of the device under a specific fault condition is simulated. The equivalent fault parameter includes data such as structural defects, performance deviations, control abnormalities reflecting the actual fault performance, which is used to accurately reproduce the state evolution process of the device after the fault occurs in the virtual environment.

[0170] The self-healing instruction is executed in the digital twin model. According to the self-healing instruction content identified in advance, the influence effect of the instruction on the device is simulated in combination with the dynamic feedback mechanism of the virtual model. The self-healing instruction covers dynamic parameter adjustment, strategy optimization or maintenance sequence execution, and the action process and effect of the instruction in the device fault state are verified by running the virtual model.

[0171] The key parameter recovery state is monitored in the digital twin model to obtain the change of the key operating parameters of the equipment. The key parameters include indexes reflecting the core function and health level of the equipment, such as temperature, pressure, vibration, output power, etc. The state improvement degree after the execution of the self-healing instruction is evaluated by monitoring the parameter trend and fluctuation.

[0172] When the key parameter recovery state reaches the preset standard, it is determined that the self-healing instruction is verified in the digital twin model. The preset standard is set according to the equipment technical specification, operation baseline and industry standard, which is an objective reference for determining whether the equipment recovery effect meets the requirements, ensuring that the self-healing instruction is only allowed to be applied to the actual equipment on the premise of sufficient verification and reliable effect.

[0173] The self-healing instruction that passes the verification is sent to the control unit of the target equipment for execution. The control unit is an intelligent control module of the equipment, which has functions such as instruction analysis, parameter configuration, action control, etc. It receives and executes the self-healing instruction, directly acts on each subsystem of the equipment, and realizes state adjustment and fault intervention.

[0174] Based on the operating parameters of the target equipment after executing the self-healing instruction, the self-healing result is generated. The operating parameters are real-time monitoring data of the equipment, reflecting the actual effect after the implementation of the self-healing measure. By analyzing the parameter change and performance index, the effectiveness and stability of the self-healing operation are comprehensively determined to form the final self-healing result, which serves as the basis for operation and maintenance decision and subsequent strategy optimization.

[0175] Example: In the medical health business field, multi-modal intelligent self-healing management is implemented for high-end magnetic resonance imaging equipment (MRI system). In the key structure of the MRI system, a vibration sensor is first deployed in the liquid helium cooling unit of the superconducting magnet to collect real-time weak vibration signals in the low-temperature environment. The vibration signals cover the full-band dynamic response generated by the liquid helium circulation pipeline and the magnet support structure. The temperature field distribution data of the gradient coil and the radio frequency power amplifier module are obtained by an infrared thermal imaging device, with a fixed sampling rate controlled at 1 Hz to ensure complete capture of temperature changes in the heat-sensitive area. At the same time, an acoustic emission sensor is installed in the internal cavity of the equipment to collect real-time abnormal acoustic signals such as micro-cracks, structure impact and early discharge in the ultrasonic frequency band. All sensing signals are synchronized and calibrated using high-precision time protocol to form a time-consistent multi-modal sensing data set.

[0176] Based on the acquired multimodal sensor data, the kurtosis coefficient and form factor of the liquid helium vibration signal are extracted to quantify the microvibration characteristics of the magnet in a cryogenic environment. Combined with the wavelet packet decomposition results of the ultrasonic signal, the energy contribution of nodes at a specified level is obtained, reflecting the early evolution trend of device microstructural defects. Temperature field distribution data is extracted to capture potential thermal runaway risks in the gradient coil and radio frequency system. A cross-correlation network graph is constructed using distributed sensor nodes as the graph structure. The edge weights between nodes are dynamically adjusted based on the signal coupling strength. This comprehensively characterizes the spatiotemporal correlations of multi-physics field data. This input is then fed into a graph convolutional network to generate cross-modal spatiotemporal correlation features.

[0177] The generated spatiotemporal correlation features are fed into a deep encoding network with a memory mechanism to extract compressed feature vectors from the MRI system's multi-source data. The high-dimensional data features are then reconstructed using a deconvolution decoder. The weighted error between the original spatiotemporal correlation features and the reconstructed features is calculated. The error weight is automatically increased for areas with sudden signal anomalies, highlighting the impact of minor anomalies. The anomaly discrimination threshold is dynamically set based on historical error data, and combined with the current weighted error, a quantitative anomaly score is generated to reflect the degree of deviation from the overall operating status of the MRI system.

[0178] The system further quantifies multidimensional health indicators through spatiotemporal correlations. The mechanical wear index is calculated based on the envelope spectrum of the liquid helium system's vibration signal to assess the stability of the magnet structure. The electrical stability index is determined by combining the harmonic distortion level of the current signal to reflect the status of the RF power amplifier and gradient power supply. The thermodynamic stress index is assessed through temperature field distribution data to monitor the thermal safety of key components. Based on the MRI system's output image quality and device self-test parameters, an adaptive filtering algorithm dynamically extracts the performance degradation index to quantify the changing trend of imaging capabilities. The weight coefficients of various health indicators are intelligently adjusted based on the MRI system type and application scenario, and a comprehensive health index is generated to comprehensively reflect the multidimensional operational health of the device.

[0179] The anomaly score and comprehensive health index are combined to monitor the changing trends of the MRI system status in real time. If the anomaly score increases rapidly within a short window and exceeds the mutation threshold, the system automatically identifies it as a sudden failure mode and generates dynamic parameter adjustment instructions in real time to reduce gradient output intensity, optimize RF power distribution, and suppress potential fault expansion. If the comprehensive health index shows a continuous downward trend, with the slope exceeding the preset threshold, the system identifies it as a progressive failure mode and initiates a physical maintenance operation sequence, including liquid helium replenishment, cooling system maintenance, and component replacement, to ensure long-term stable system operation.

[0180] For the generated self-healing instructions, first perform simulation verification in the digital twin model of the MRI system, fully reproduce the device structure, function and fault state based on the three-dimensional virtual model. Inject the equivalent parameters corresponding to the current fault mode, simulate typical fault scenarios such as magnet temperature control failure, gradient overload or structural loosening. Run the self-healing instructions in the virtual environment, dynamically monitor the recovery state of key operating parameters, including temperature stability, vibration level and imaging indicators. When the key parameter recovery effect reaches the set standard, the system determines that the self-healing instruction passes the virtual verification, and issues the instruction to the MRI control unit for actual execution. After execution is completed, combined with real-time monitoring data analysis, generate the final self-healing result, evaluate the device recovery effect and potential risks, improve the reliability and self-repairing ability of the MRI system in high-intensity clinical applications, and ensure the continuous availability of key medical resources.

[0181] In the field of financial technology business, multi-modal data-driven device self-healing management is implemented for the core server group of the data center. In the key servers of the financial data center, multiple types of sensing devices are deployed to collect multi-modal information. By deploying vibration sensors inside high-performance server cabinets, vibration signals generated by hard disk arrays and air cooling systems are monitored in real time, covering the dynamic response state of rack structures and storage devices. Infrared thermal imaging devices are used to obtain temperature field distribution information of mainboard chips, memory modules and power supply units, with a sampling frequency controlled within 1 Hz to ensure the capture of internal thermal distribution anomalies. Acoustic emission sensors are deployed in the cabinet structure and power management module to collect early structural fatigue, connection loosening and potential discharge acoustic signals in the ultrasonic frequency band. High-precision time protocol is used to synchronize the above multi-modal sensing data, forming a time-consistent multi-modal sensing data set to provide a data basis for subsequent analysis.

[0182] From the above multi-modal sensing data, various information is extracted to obtain the kurtosis coefficient and waveform factor of the server vibration signal, quantifying the dynamic stability of the storage array and structural support. Combined with the wavelet packet decomposition results of the ultrasonic signal, the energy proportion of the preset hierarchical node is calculated to reflect the micro-crack and fatigue evolution trend of the internal structure. Temperature field distribution data is extracted to monitor the thermal safety state of the power supply system and core computing module. Taking each sensing node as a network graph structure, a sensor node network graph is dynamically constructed based on the signal cross-correlation function value between nodes, fully reflecting the spatio-temporal correlation relationship of multi-source physical quantities in the financial data center. The above features are input into a graph convolution network to generate cross-modal spatio-temporal correlation features that integrate vibration, acoustic and temperature information.

[0183] Based on the spatio-temporal correlation feature, a deep encoder with a memory mechanism is input to generate a compressed feature vector, and a deconvolution decoder is used to reconstruct high-dimensional data. The weighted error value between the original spatio-temporal correlation feature and the reconstructed feature is calculated, and a higher weight is given to the signal mutation area to highlight the small but key abnormal signal. According to the historical error data, the abnormal judgment threshold is dynamically determined, and the quantitative abnormal score is generated to reflect the overall running deviation of the server in real time, which helps to find early hidden dangers.

[0184] Further, through the spatio-temporal correlation feature, the multi-dimensional running health index of the server is quantified, the mechanical wear index is calculated based on the envelope spectrum feature of the vibration signal, and the mechanical state of the hard disk array and the air cooling system is evaluated. Combined with the harmonic distortion level of the power supply system current signal, the electrical stability index is determined to reflect the quality of the power supply link. Through the temperature field data, the thermodynamic stress index is extracted to monitor the internal thermal environment safety of the server. Combined with the device output performance data, an adaptive filtering algorithm is used to dynamically calculate the performance attenuation index to quantify the trend of the change of the computing and storage performance. For different types of servers, the weights of each health index are dynamically adjusted to generate a comprehensive health index to comprehensively represent the running health status of the key servers in the financial data center.

[0185] Based on the abnormal score and the comprehensive health index, the fault mode in the financial data center is identified in real time. When the abnormal score rapidly rises in a short time and exceeds the mutation threshold, the system automatically identifies it as a sudden failure mode and generates dynamic parameter adjustment instructions, including reducing the CPU frequency, adjusting the air cooling wind speed, switching the redundant storage node, and other measures to prevent the expansion of the fault. When the comprehensive health index continuously decreases and the change slope exceeds the set threshold, the system identifies it as a gradual failure mode and generates a physical repair operation sequence to arrange the server to be offline, modularized for inspection and hardware replacement, and to ensure the continuity of financial business.

[0186] Before sending the self-healing instruction, the system performs simulation verification in the digital twin model of the server. Through the three-dimensional virtual model, the server structure, power supply system and storage layout are restored, the equivalent parameters corresponding to the current fault mode are injected, and the fault scenarios such as air cooling anomaly, unstable power supply or structural looseness are simulated. Execute the self-healing instruction in the digital twin environment, monitor the recovery status of the key parameters of the virtual system, such as cabinet vibration level, power supply stability and computing performance recovery index. When the key parameters recovery effect reaches the preset standard, it is determined that the self-healing instruction passes the virtual verification, and the system sends the instruction to the server control unit to execute parameter adjustment or repair operation in real time. Combined with the multi-modal sensing data after execution, the final self-healing result is generated to quantify the improvement degree of the server state, ensuring the high availability, low failure and stable operation of the key computing platform in the financial data center, and ensuring the continuous and stable development of financial transaction and data processing business.

[0187] By constructing a digital twin model highly consistent with the target device, the embodiment can verify the effectiveness and applicability of the self-healing instruction in advance without affecting the actual operation of the device, avoiding system risks caused by blind operation. Through equivalent fault parameter injection and dynamic simulation, the complex fault state is fully restored to ensure the pertinence and accuracy of the self-healing measures. Combined with key parameter recovery state monitoring and standardized judgment, the application time and effect of the self-healing instruction are strictly controlled to improve the self-recovery ability and operation and maintenance intelligence of the device, reduce maintenance cost and downtime risk, and ensure the safe, stable and efficient operation of the device under complex working conditions.

[0188] In an embodiment, a device self-healing apparatus based on spatiotemporal correlation features is provided, which corresponds to the device self-healing method based on spatiotemporal correlation features in the above embodiment. Referring to Figure 3 , Figure 3 A functional module schematic diagram of a preferred embodiment of the device self-healing apparatus based on spatiotemporal correlation features is shown. The functional modules include a multi-modal data acquisition module 10, a cross-modal feature fusion module 20, an anomaly discrimination module 30, a health index evaluation module 40, a fault self-healing decision module 50, and a digital simulation execution module 60. The detailed descriptions of the functional modules are as follows:

[0189] The multi-modal data acquisition module 10 is configured to acquire multi-modal sensing data collected by a plurality of types of sensing devices deployed on a target device.

[0190] The cross-modal feature fusion module 20 is configured to generate cross-modal spatiotemporal correlation features based on the multi-modal sensing data.

[0191] The anomaly discrimination module 30 is configured to perform anomaly detection using the spatiotemporal correlation features to obtain an anomaly score.

[0192] The health index evaluation module 40 is configured to quantify health indicators of a plurality of operating dimensions of the target device based on the spatiotemporal correlation features, and to fuse the health indicators of the plurality of operating dimensions to generate a comprehensive health index.

[0193] The fault self-healing decision module 50 is configured to identify a fault mode of the target device based on the anomaly score and the comprehensive health index, and to generate a self-healing instruction according to the fault mode.

[0194] The digital simulation execution module 60 is configured to perform simulation verification of the self-healing instruction in a digital twin model of the target device, and to send the self-healing instruction that passes the simulation verification to a control unit of the target device for execution to generate a self-healing result.

[0195] In an embodiment, the multi-modal data acquisition module 10 is specifically configured to:

[0196] deploying a vibration sensor on a rotating component bearing seat of the target device to collect vibration signals in a preset vibration frequency band range;

[0197] deploying an infrared thermal imaging device on a circuit area of the target device to collect temperature field distribution data at a fixed sampling rate;

[0198] deploying an acoustic emission sensor on a mechanical structure of the target device to collect ultrasonic signals in a preset ultrasonic frequency band range;

[0199] aligning time sequence data of the vibration signals, the temperature field distribution data and the ultrasonic signals using a preset precision time protocol to generate time-synchronized multi-modal sensing data.

[0200] In an embodiment, the cross-modal feature fusion module 20 is specifically configured to:

[0201] extract vibration signals from the multi-modal sensing data and determine time-domain statistical features including kurtosis coefficients and waveform factors in the vibration signals;

[0202] extract ultrasonic signals from the multi-modal sensing data and perform wavelet packet decomposition on the ultrasonic signals to extract energy proportions of preset decomposition layer nodes as frequency energy distributions;

[0203] extract temperature field distribution data from the multi-modal sensing data;

[0204] construct a sensor node network graph with vibration sensors, infrared thermal imaging devices and acoustic emission sensors as nodes and cross-correlation function values of signals between nodes as edge weights;

[0205] input the time-domain statistical features, the frequency energy distributions, the temperature field distribution data and the sensor node network graph into a graph convolution network to generate cross-modal spatio-temporal correlation features.

[0206] In an embodiment, the anomaly discrimination module 30 is specifically configured to:

[0207] input the spatio-temporal correlation features into an encoder containing long short-term memory units to generate compressed feature vectors;

[0208] input the compressed feature vectors into a deconvolution decoder to output reconstructed features;

[0209] determine a weighted error value of the spatio-temporal correlation features and the reconstructed features;

[0210] determine a dynamic anomaly threshold based on historical error data;

[0211] compare the weighted error value with the dynamic anomaly threshold to generate an anomaly score.

[0212] In an embodiment, the health index evaluation module 40 is specifically configured to:

[0213] determine a mechanical wear index based on the vibration signal envelope spectrum feature in the spatiotemporal correlation feature;

[0214] determine an electrical stability index based on the current signal harmonic distortion feature in the spatiotemporal correlation feature;

[0215] determine a thermodynamic stress index based on the temperature field distribution feature in the spatiotemporal correlation feature;

[0216] determine a performance degradation index by the adaptive filtering module based on the operation output parameter data of the target device;

[0217] dynamically adjust the weight coefficients of the mechanical wear index, the electrical stability index, the thermodynamic stress index, and the performance degradation index according to the target device type;

[0218] weight the mechanical wear index, the electrical stability index, the thermodynamic stress index, and the performance degradation index based on the weight coefficients to generate a comprehensive health index.

[0219] In an embodiment, the fault self-recovery decision module 50 is specifically configured to:

[0220] monitor the change trend of the abnormal score and determine whether the change trend exceeds a mutation threshold within a set period;

[0221] if the change trend of the abnormal score exceeds the mutation threshold, identify a sudden failure mode;

[0222] monitor the change slope of the comprehensive health index and determine whether the change slope exceeds a set slope threshold;

[0223] if the change slope of the comprehensive health index exceeds the set slope threshold, identify a gradual failure mode;

[0224] when the failure mode is a sudden failure mode, generate a dynamic parameter adjustment instruction;

[0225] when the failure mode is a gradual failure mode, generate a physical maintenance operation sequence.

[0226] In an embodiment, the digital simulation execution module 60 is specifically configured to:

[0227] construct a three-dimensional virtual model of the target device in the digital twin model of the target device;

[0228] inject an equivalent fault parameter corresponding to the failure mode in the digital twin model;

[0229] executing the self-healing instruction in the digital twin model;

[0230] monitoring a key parameter recovery state in the digital twin model;

[0231] when the key parameter recovery state reaches a preset standard, determining that the self-healing instruction is verified in the digital twin model;

[0232] sending the verified self-healing instruction to a control unit of the target device for execution;

[0233] generating a self-healing result based on a running parameter of the target device after executing the self-healing instruction.

[0234] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with an external user terminal through a network connection. The computer program is executed by the processor to implement functions or steps of a server side of a device self-healing method based on spatiotemporal correlation features.

[0235] In one embodiment, a computer device is provided, which can be a user terminal, and an internal structure diagram thereof can be as shown in Figure 5 The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide determination and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with an external server through a network connection. The computer program is executed by the processor to implement functions or steps of a user terminal side of a device self-healing method based on spatiotemporal correlation features

[0236] In one embodiment, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the following steps:

[0237] acquiring multi-modal sensor data collected by multiple types of sensor devices deployed on a target device;

[0238] generating spatio-temporal correlation features across modalities based on the multi-modal sensor data;

[0239] performing anomaly detection using the spatio-temporal correlation features to obtain an anomaly score;

[0240] quantifying health indicators of multiple operating dimensions of the target device based on the spatio-temporal correlation features, and fusing the health indicators of the multiple operating dimensions to generate a comprehensive health index;

[0241] identifying a failure mode of the target device based on the anomaly score and the comprehensive health index, and generating a self-healing instruction according to the failure mode;

[0242] simulating and verifying the self-healing instruction in a digital twin model of the target device, and sending the self-healing instruction that passes the simulation and verification to a control unit of the target device for execution to generate a self-healing result.

[0243] In one embodiment, a computer-readable storage medium is provided, which stores a computer program that is executed by a processor to implement the following steps:

[0244] acquiring multi-modal sensor data collected by multiple types of sensor devices deployed on a target device;

[0245] generating spatio-temporal correlation features across modalities based on the multi-modal sensor data;

[0246] performing anomaly detection using the spatio-temporal correlation features to obtain an anomaly score;

[0247] quantifying health indicators of multiple operating dimensions of the target device based on the spatio-temporal correlation features, and fusing the health indicators of the multiple operating dimensions to generate a comprehensive health index;

[0248] identifying a failure mode of the target device based on the anomaly score and the comprehensive health index, and generating a self-healing instruction according to the failure mode;

[0249] simulating and verifying the self-healing instruction in a digital twin model of the target device, and sending the self-healing instruction that passes the simulation and verification to a control unit of the target device for execution to generate a self-healing result.

[0250] It should be noted that the functions or steps that the above computer-readable storage medium or computer device can implement can be referred to the related descriptions of the server side and the user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0251] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0252] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0253] It should be noted that if non-company software tools or components appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A device self-healing method based on spatiotemporal correlation characteristics, characterized in that: The following steps are involved: Acquire multimodal sensor data collected by various types of sensor devices deployed on the target device; generating cross-modal spatiotemporal correlation features based on the multimodal sensor data; Perform anomaly detection using the spatiotemporal correlation features to obtain an anomaly score; quantifying health indicators of multiple operating dimensions of the target device based on the spatiotemporal correlation features, and fusing the health indicators of the multiple operating dimensions to generate a comprehensive health index; identifying a failure mode of the target device based on the anomaly score and the comprehensive health index, and generating a self-healing instruction according to the failure mode; In the digital twin model of the target device, the self-healing instruction is simulated and verified, and the self-healing instruction that passes the simulation verification is sent to the control unit of the target device for execution to generate a self-healing result.

2. The device self-healing method based on spatiotemporal correlation characteristics according to claim 1, characterized in that: Acquire multimodal sensor data collected by various types of sensor devices deployed on the target device, including: Deploy vibration sensors on the bearing seats of rotating components of target equipment to collect vibration signals within a preset vibration frequency band. Deploy an infrared thermal imaging device in the circuit area of ​​the target device to collect temperature field distribution data at a fixed sampling rate; Deploy acoustic emission sensors on the mechanical structure of the target device to collect ultrasonic signals within a preset ultrasonic frequency band; A preset precision time protocol is used to align the timing data of the vibration signal, the temperature field distribution data, and the ultrasonic signal to generate time-synchronized multimodal sensing data.

3. The device self-healing method based on spatiotemporal correlation characteristics according to claim 1, characterized in that: Generating cross-modal spatiotemporal correlation features based on the multimodal sensing data, including: Extracting a vibration signal from the multimodal sensing data and determining time-domain statistical features of the vibration signal including a kurtosis coefficient and a form factor; Extracting ultrasonic signals from the multimodal sensing data, performing wavelet packet decomposition on the ultrasonic signals, and extracting energy proportions of nodes in a preset decomposition layer as frequency domain energy distribution; extracting temperature field distribution data from the multimodal sensing data; The sensor node network graph is constructed by taking vibration sensors, infrared thermal imaging devices and acoustic emission sensors as nodes and the cross-correlation function values ​​of the signals between nodes as edge weights. The time domain statistical features, the frequency domain energy distribution, the temperature field distribution data and the sensor node network graph are input into a graph convolutional network to generate cross-modal spatiotemporal correlation features.

4. The device self-healing method based on spatiotemporal correlation characteristics according to claim 1, characterized in that: Anomaly detection is performed using the spatiotemporal correlation features to obtain anomaly scores, including: Inputting the spatiotemporal correlation features into an encoder comprising a long short-term memory unit to generate a compressed feature vector; Inputting the compressed feature vector into a deconvolution decoder and outputting a reconstructed feature; Determining a weighted error value between the spatiotemporal correlation feature and the reconstruction feature; Determine dynamic anomaly thresholds based on historical error data; The weighted error value is compared with the dynamic anomaly threshold to generate an anomaly score.

5. The device self-healing method based on spatiotemporal correlation characteristics according to claim 1, characterized in that: Based on the spatiotemporal correlation features, quantifying the health indicators of multiple operating dimensions of the target device, and fusing the health indicators of the multiple operating dimensions to generate a comprehensive health index, including: Determining a mechanical wear index based on a vibration signal envelope spectrum feature in the spatiotemporal correlation feature; determining an electrical stability index based on a harmonic distortion feature of the current signal in the spatiotemporal correlation feature; determining a thermodynamic stress index based on the temperature field distribution characteristics in the spatiotemporal correlation characteristics; Determining a performance degradation index through an adaptive filtering module based on the operating output parameter data of the target device; Dynamically adjust the weight coefficients of the mechanical wear index, electrical stability index, thermodynamic stress index, and performance degradation index according to the target device type; Based on the weight coefficient, the mechanical wear index, electrical stability index, thermodynamic stress index and performance degradation index are weighted and integrated to generate a comprehensive health index.

6. The device self-healing method based on spatiotemporal correlation characteristics according to claim 1, characterized in that: Identifying a failure mode of the target device based on the anomaly score and the comprehensive health index, and generating a self-healing instruction according to the failure mode, including: Monitor the changing trend of the anomaly score and determine whether the changing trend exceeds a mutation threshold within a set time period; If the change trend of the anomaly score exceeds the mutation threshold, it is identified as a sudden failure mode; monitoring a change slope of the comprehensive health index and determining whether the change slope exceeds a set slope threshold; If the change slope of the comprehensive health index exceeds the set slope threshold, it is identified as a progressive failure mode; When the failure mode is a sudden failure mode, generating a dynamic parameter adjustment instruction; When the failure mode is a progressive failure mode, a physical repair operation sequence is generated.

7. The device self-healing method based on spatiotemporal correlation characteristics according to claim 1, characterized in that: In the digital twin model of the target device, the self-healing instruction is simulated and verified, and the self-healing instruction that passes the simulation verification is sent to the control unit of the target device for execution to generate a self-healing result, including: Constructing a three-dimensional virtual model of the target device in the digital twin model of the target device; Injecting equivalent fault parameters corresponding to the fault mode into the digital twin model; executing the self-healing instruction in the digital twin model; monitoring key parameter recovery states in the digital twin model; When the recovery state of the key parameter reaches a preset standard, it is determined that the self-healing instruction has been verified in the digital twin model; Sending the verified self-healing instruction to the control unit of the target device for execution; A self-healing result is generated based on the operating parameters of the target device after executing the self-healing instruction.

8. A device self-healing device based on spatiotemporal correlation characteristics, characterized in that: The device self-healing device based on time-space correlation characteristics includes: A multimodal data acquisition module, used to acquire multimodal sensor data collected by various types of sensor devices deployed on the target device; A cross-modal feature fusion module, configured to generate cross-modal spatiotemporal correlation features based on the multimodal sensing data; An anomaly discrimination module, configured to perform anomaly detection using the spatiotemporal correlation features to obtain an anomaly score; a health index evaluation module, configured to quantify health indicators of multiple operating dimensions of the target device based on the spatiotemporal correlation features, and to fuse the health indicators of the multiple operating dimensions to generate a comprehensive health index; a fault self-healing decision module, configured to identify a fault mode of the target device based on the anomaly score and the comprehensive health index, and generate a self-healing instruction according to the fault mode; A digital simulation execution module is used to simulate and verify the self-healing instruction in the digital twin model of the target device, and send the self-healing instruction that passes the simulation verification to the control unit of the target device for execution to generate a self-healing result.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a device self-healing program based on spatiotemporal correlation features that is stored in the memory and can be run on the processor. When the device self-healing program based on spatiotemporal correlation features is executed by the processor, the steps of the device self-healing method based on spatiotemporal correlation features as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The storage medium stores a device self-healing program based on time-space correlation features, which, when executed by a processor, implements the steps of the device self-healing method based on time-space correlation features according to any one of claims 1 to 7.

Images