Fault diagnosis and recovery verification method and device, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses a fault diagnosis and recovery verification method and device, equipment and a medium. Inputting the candidate event and the context state into an intelligent diagnosis model to obtain a diagnosis result and a fault type, selecting a recovery path in a recovery strategy library based on the diagnosis result, generating a recovery execution instruction, creating a verification experiment based on the instruction, and injecting drill data to obtain verification data, and forming a verification conclusion according to the verification data, associating a recovery execution instruction, and issuing the instruction in the execution environment to complete recovery operation. According to the method, through multi-source monitoring data processing, intelligent diagnosis model reasoning, strategy library path selection, verification experiment verification and execution environment automatic recovery, a full-link automatic process from detection, diagnosis, recovery to verification is constructed, and accurate diagnosis and dynamic recovery in a complex environment are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a fault diagnosis and recovery verification method, device, equipment and storage medium. BACKGROUND

[0002] In the existing automated operation system, common tools such as Ansible, Chef, etc. mainly rely on preset scripts to complete fault recovery. This kind of way can only deal with pre-defined fixed scenarios, and once complex or cross-system exceptions occur, manual intervention is required, resulting in delayed response. In the cloud computing environment, although platforms such as AWS Auto Scaling and Kubernetes have certain self-healing capabilities, their recovery capabilities are limited to faults at the specific environment or infrastructure level, and it is still difficult to achieve automated processing of the whole link for complex exceptions at the application layer, data layer and business logic layer. At the same time, widely used monitoring and alarm tools such as Prometheus and Zabbix rely more on manual judgment and simple script to trigger recovery actions, and it is difficult to achieve intelligent diagnosis and real-time recovery under the condition of multi-source and multi-dimensional data.

[0003] In the medical health business field, the stability of medical institution information systems, medical image systems and remote diagnosis platforms is directly related to the continuity of diagnosis and treatment activities and the safety of patients. However, existing operation schemes are mostly limited to the infrastructure level or rely on manual intervention, and when encountering database exceptions, image file loss or cross-node network interruption, intelligent identification and rapid automatic recovery cannot be achieved. This easily leads to delayed transmission of diagnosis and treatment data, short-term unavailability of medical record systems, and thus affects the timeliness and accuracy of clinical decision-making.

[0004] In the field of financial technology business, transaction matching systems, payment clearing platforms and risk control systems have very high requirements for continuity and high availability. Existing technologies rely on fixed rules or manual troubleshooting to deal with transaction delays, account abnormalities or risk identification failures, and the response speed often cannot meet the requirements of high-frequency trading and real-time payment. At the same time, due to the lack of intelligent differentiation of complex fault types in existing recovery mechanisms, misjudgment is easy to occur, for example, network jitter is misidentified as a software defect, thus executing the wrong recovery path, causing transaction interruption or data inconsistency, directly bringing financial security risks. SUMMARY

[0005] The main purpose of the present application is to provide a fault diagnosis and recovery verification method, device, equipment and storage medium, aiming to solve the technical problems that the existing technology lacks a whole-link closed-loop mechanism capable of intelligently identifying fault types under the condition of complex multi-source monitoring data, and automatically generating, verifying and executing recovery paths based on the diagnosis results, resulting in delayed response, high misjudgment rate and unverifiable recovery.

[0006] To achieve the above object, the application provides a fault diagnosis and recovery verification method, comprising:

[0007] acquiring monitoring data and generating a fault candidate event based on the monitoring data;

[0008] inputting the fault candidate event and a context state into an intelligent diagnosis model to obtain an intelligent diagnosis result and a fault type;

[0009] selecting a recovery path in a recovery strategy library and generating a recovery execution instruction based on the fault type and the context state;

[0010] creating a verification experiment based on the recovery execution instruction, constructing a rehearsal environment and injecting rehearsal data to obtain verification data;

[0011] forming a verification conclusion according to the verification data and associating the verification conclusion with the recovery execution instruction;

[0012] issuing the recovery execution instruction in an execution environment and completing a recovery operation.

[0013] Further, to achieve the above object, the application provides a fault diagnosis and recovery verification device, comprising:

[0014] a monitoring acquisition module for acquiring monitoring data and generating a fault candidate event based on the monitoring data;

[0015] an intelligent diagnosis module for inputting the fault candidate event and a context state into an intelligent diagnosis model to obtain an intelligent diagnosis result and a fault type;

[0016] a strategy generation module for selecting a recovery path in a recovery strategy library and generating a recovery execution instruction based on the fault type and the context state;

[0017] a rehearsal verification module for creating a verification experiment based on the recovery execution instruction, constructing a rehearsal environment and injecting rehearsal data to obtain verification data;

[0018] a conclusion association module for forming a verification conclusion according to the verification data and associating the verification conclusion with the recovery execution instruction;

[0019] a recovery execution module for issuing the recovery execution instruction in an execution environment and completing a recovery operation.

[0020] Further, to achieve the above object, the present application also provides a computer device, which comprises a memory, a processor, and a fault diagnosis and recovery verification program stored in the memory and executable on the processor, and the fault diagnosis and recovery verification program realizes the steps of the fault diagnosis and recovery verification method when executed by the processor.

[0021] Further, to achieve the above object, the present application also provides a computer device, which comprises a memory, a processor, and a fault diagnosis and recovery verification program stored in the memory and executable on the processor, and the fault diagnosis and recovery verification program realizes the steps of the fault diagnosis and recovery verification method when executed by the processor.

[0022] Beneficial effects: The present application relates to the field of artificial intelligence technology, and can be applied to business scenarios such as financial technology and medical health, and discloses a fault diagnosis and recovery verification method, device, equipment and medium, which comprises the following steps: obtaining monitoring data and generating a fault candidate event, inputting the fault candidate event and a context state into an intelligent diagnosis model to obtain a diagnosis result and a fault type, selecting a recovery path in a recovery strategy library based on the fault type and the context state and generating a recovery execution instruction, creating a verification experiment based on the recovery execution instruction and injecting simulation data to obtain verification data, forming a verification conclusion according to the verification data and associating the recovery execution instruction, and issuing the recovery execution instruction in an execution environment and completing a recovery operation. The present application forms a full-link automatic process from fault detection, diagnosis, recovery to verification through the introduction of multi-source monitoring data processing, intelligent diagnosis model reasoning, strategy library path selection, verification experiment closed-loop verification and automatic recovery mechanism of the execution environment; can realize accurate identification of fault types and selection of dynamic recovery paths in a complex environment, ensure the reliability and consistency of the recovery operation, effectively reduce the delay and misjudgment caused by manual intervention, and improve the self-healing ability and operation stability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0023] The present application will be further described below in combination with the drawings and embodiments, and the drawings are as follows:

[0024] Figure 1 An application environment schematic diagram of the fault diagnosis and recovery verification method in an embodiment of the present application;

[0025] Figure 2 A flowchart of the fault diagnosis and recovery verification method in an embodiment of the present application;

[0026] Figure 3 A functional module schematic diagram of a preferred embodiment of the fault diagnosis and recovery verification device of the present application;

[0027] Figure 4 A structure schematic diagram of a computer device in an embodiment of the present application;

[0028] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0030] The fault diagnosis and recovery verification method provided in this invention can be applied to, for example... Figure 1 In this application environment, the user terminal communicates with the server via a network. The server can obtain monitoring data from the user terminal and generate candidate fault events. It inputs these candidate events and context state into an intelligent diagnostic model to obtain diagnostic results and fault types. Based on the fault type and context state, it selects a recovery path from the recovery strategy library and generates a recovery execution command. Based on the recovery execution command, it creates a verification experiment and injects exercise data to obtain verification data. Based on the verification data, it forms a verification conclusion and associates it with the recovery execution command. Finally, it issues the recovery execution command in the execution environment and completes the recovery operation. This invention introduces multi-source monitoring data processing, intelligent diagnostic model reasoning, strategy library path selection, closed-loop verification of verification experiments, and an automated recovery mechanism for the execution environment, forming a fully automated process from fault detection, diagnosis, recovery to verification. It can achieve accurate identification of fault types and dynamic selection of recovery paths in complex environments, ensuring the reliability and consistency of recovery operations, effectively reducing delays and misjudgments caused by manual intervention, and improving the system's self-healing ability and operational stability. The user terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster composed of multiple servers. The invention will be described in detail below through specific embodiments.

[0031] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the fault diagnosis and recovery verification method provided by the present invention. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0032] like Figure 2 As shown, the fault diagnosis and recovery verification method proposed in this invention includes the following steps:

[0033] S10, acquire monitoring data, and generate fault candidate events based on the monitoring data;

[0034] In this embodiment, obtaining monitoring data is the starting point of the whole process, and the sources of monitoring data are diverse, covering system operation logs, device operation indicators, network status information, and business interaction requests. The operation log records the running behavior of the operating system, application program, and middleware, and can be saved in text, binary, or structured form for subsequent analysis and modeling. Device operation indicators include CPU utilization, memory occupancy, disk I / O throughput, network bandwidth usage, etc. These data can be obtained through hardware sensors, driver interfaces, or virtualization platform management APIs. Network status information includes connection topology, link delay, packet loss rate, etc., which can be collected through traffic monitoring probes or controllers. Business interaction request data reflects the interaction between end users and the system, such as request success rate, response time, error code statistics, etc., which can be captured through service gateways and log collection agents. After collecting these data from different sources, they need to be timestamped and formatted to ensure that they can be fused in multiple dimensions later.

[0035] On the basis of data collection, pre-processing is needed to eliminate redundancy and noise, fill in missing values, and eliminate dimension differences between different indicators through normalization operations. Then introduce multi-source feature fusion mechanism, integrate abnormal keywords in operation log, critical point information in device health indicator, label features in historical fault record and connection relationship in network topology, form feature matrix containing spatial and time dual dimensions. Spatial dimension reflects the relationship between components, and time dimension reflects the trajectory of event evolution over time. Through the establishment of feature matrix, the correlation pattern between different fault signals can be better captured.

[0036] On the basis of the feature matrix, an index matching mechanism needs to be designed to compare the real-time calculated feature indicators with the pre-set threshold strategy. Threshold strategy can be defined by expert experience or trained from historical data. During the comparison process, if some indicators exceed the threshold range, they will be marked as potential abnormal points. These potential abnormal points need to be further analyzed for topological collision, i.e. checking whether multiple abnormal points are concentrated in the same subsystem or there is a causal transmission path in the network topology or dependency graph. If the conditions are met, it will be judged as a suspected fault aggregation area, thus forming an initial fault event set.

[0037] On the basis of the initial fault event set, event filtering and deduplication need to be performed. Different monitoring sources may trigger multiple times for the same problem, which will lead to event redundancy and misjudgment if not eliminated. Through event fusion algorithm, similar events can be merged according to time window and topological proximity. Finally, the result is a corrected and refined candidate event set, which has a high probability of fault occurrence and can be used as input for subsequent diagnosis model.

[0038] In specific implementations, various technical paths can be adopted. The running logs can be transmitted to the analysis engine in real time using a log collection agent combined with a distributed message queue; or lightweight monitoring probes can be deployed on the node side to directly collect CPU, memory, and I / O indicators and push them to the centralized platform. For topology data, connection information can be periodically exported using the network control plane, or the call chain relationship can be built through a distributed tracking framework. Feature fusion can be stored and queried through a graph database, or joint modeling of time series and spatial dependence can be achieved through a matrix calculation framework such as TensorFlow or PyTorch. The threshold strategy can be loaded through a static configuration file, or it can be dynamically adjusted through an online learning algorithm. Topology collision analysis can use algorithms based on graph traversal, or pattern recognition methods based on graph convolutional neural networks.

[0039] When adapting to different production environments, different data collection and processing methods can be selected. In high-concurrency financial transaction scenarios, low-latency in-memory data buses can be used to transmit monitoring indicators; in medical image processing scenarios, I / O performance collection of large-scale storage systems needs to be focused on. For resource-constrained edge environments, periodic sampling rather than full-quantity collection can be used to reduce monitoring load.

[0040] Example: In the medical health business field, by collecting the logs of medical record systems, device running status, and network connection data, a feature matrix can be formed, and compared with the threshold strategy to identify database access anomalies and I / O bottlenecks of image storage devices. When multiple anomalies are concentrated in the same department server group, they are aggregated into a fault candidate event to help quickly locate the problem.

[0041] In the financial technology business field, transaction gateway logs, host resource consumption, and cross-region network latency can be collected to build a spatio-temporal feature matrix. When detecting that the transaction delay indicator exceeds the threshold and overlaps with the network packet loss event in the topology, a candidate event is generated to prompt possible cross-region link anomalies. This mechanism can effectively shorten the time from anomaly detection to candidate event generation, providing protection for high real-time business.

[0042] This embodiment can avoid the bias caused by a single data source through multi-source collection of monitoring data, feature fusion, and anomaly detection, effectively improving the accuracy of fault candidate events. Real-time topology collision analysis can identify potential fault aggregation points in complex systems, thereby reducing the false positive rate and providing high-confidence input for subsequent diagnosis and recovery.

[0043] S20, inputting the fault candidate event and the context state into an intelligent diagnosis model to obtain an intelligent diagnosis result and a fault type;

[0044] In this embodiment, the fault candidate event is the result of fusion and screening of multi-source monitoring data, which is essentially structured information describing potential anomalies of the system. The candidate event usually contains trigger time, impact range, related resource nodes, and similarity label with historical faults. To further confirm the nature of the fault, the candidate event needs to be jointly analyzed with the context state. The context state includes the current configuration parameters of the system, the running tasks, external dependencies, and user behavior patterns, which can provide background conditions for diagnosis. For example, the same disk I / O surge event has different meanings in the data backup window and the non-backup window.

[0045] In the input stage, feature extraction and vectorization are needed for the fault candidate event and the context state. The key features of the candidate event can be obtained through semantic analysis of the event log, numericalization of the abnormal indicators, and graph structure coding of the topology nodes. The context state needs to extract relevant information from the knowledge graph or configuration database, such as process dependency, network call chain, and user load distribution, and convert it into a structured pattern set. These data need to be aligned and normalized before inputting into the intelligent diagnosis model to ensure that features from different sources can be calculated in the same dimensional space.

[0046] The intelligent diagnosis model can take many forms. Common methods include hybrid models based on rules and machine learning, where machine learning models are used to capture complex nonlinear relationships, and rule systems are used to ensure interpretability and stability of key logic. After inputting the candidate event vector and context pattern set into the model, the model first outputs an initial diagnosis probability distribution, representing the likelihood of different fault types. To avoid black-boxing, an attribution analysis module is usually introduced to decompose the model's prediction logic into evidence chains, showing which input features led to the current judgment. This evidence chain can assist manual verification and subsequent automated strategy optimization.

[0047] After generating the probability distribution, it needs to be compared with the preset confidence threshold. If the probability of a certain fault exceeds the threshold, that fault is marked as a high-confidence result. The high-confidence result set is integrated and combined with the evidence chain to form the final intelligent diagnosis result. The diagnosis result not only contains the fault type, but also the trigger cause, related resources, and prediction confidence, providing reliable input for subsequent recovery path selection.

[0048] In the implementation process, a natural language processing model can be used to perform word segmentation and entity recognition on the candidate event log to generate a semantic vector. A graph neural network-based method can also be used to embed the topological dependency relationship and capture the interaction patterns between nodes. The context state can be obtained by querying the running configuration in real time through an API or by obtaining the implicit causal chain through the reasoning module of the knowledge graph. In model selection, a deep neural network combined with an attention mechanism can be used to enhance the ability to capture complex features, or a Bayesian inference framework can be used to improve the uncertainty quantification capability. Attributive analysis can be achieved through explainable tools such as LIME and SHAP, or an explainable module can be pre-installed in the model structure, such as a decision tree nest.

[0049] For different production environments, input features and model structures can be adjusted. In a financial transaction system, context information such as transaction delay distribution and abnormal order features can be added to enhance the ability to diagnose performance bottlenecks and fraudulent behavior. In a medical imaging system, context information such as data inflow rate and image file integrity features can be added to better identify storage failures and data transmission abnormalities. In resource-constrained edge computing scenarios, lightweight models or distilled models can be used to ensure diagnostic speed and real-time performance.

[0050] Example: In the medical health business field, the electronic medical record system may experience disk I / O peak during night batch processing. The candidate event detects the anomaly and inputs it into the diagnostic model. If the context state indicates that an image archiving task is being performed at this time, the model will identify the anomaly as high load under normal operating conditions, rather than hardware failure, thereby avoiding false positives.

[0051] In the financial technology business field, candidate events in the transaction system may exhibit increased latency and rising failure rates. Combined with the network topology and market activity in the context state, the diagnostic model can distinguish between resource shortages caused by external market peaks and latency caused by link hardware abnormalities. The final output of the intelligent diagnostic result can provide clear evidence for the selection of recovery paths.

[0052] This embodiment can significantly reduce the misjudgment rate by jointly inputting candidate events and context states into an intelligent diagnostic model, enabling accurate classification of complex faults. The introduction of context states enables the model to understand the operating environment behind the events, thereby avoiding false alarms that rely solely on index anomalies. The intelligent diagnostic result not only provides the fault type but also generates an explainable evidence chain, laying the foundation for the automation and credibility of subsequent recovery operations.

[0053] S30, selecting a recovery path and generating a recovery execution instruction in a recovery strategy library based on the fault type and the context state;

[0054] In this embodiment, when the fault type and context state information output by the intelligent diagnosis model are obtained, retrieval and selection need to be performed in the recovery strategy library based on these inputs. The fault type represents a specific abnormal category, such as a database transaction fault, a storage device exception, or network congestion. The context state reflects the system running conditions at the time of the fault, including the current hardware configuration, service dependencies, and network topology. The combination of the two provides complete reference coordinates for strategy selection.

[0055] The recovery strategy library is a knowledge system that stores predefined recovery paths, each of which corresponds to an executable operation sequence. A recovery path usually contains multiple levels of strategy mapping, including hardware-oriented repair operations such as driver reloading or disk remapping, software-oriented operations such as transaction rollback or service restart, and business-level operations such as traffic switching or request degradation. In order to select the optimal path, the recovery strategy library needs to be queried first to generate a strategy set that matches the input conditions and form a hierarchical strategy mapping table.

[0056] After the hierarchical strategy mapping table is generated, each candidate strategy needs to be evaluated, and the evaluation indicators include resource consumption, recovery delay, and state consistency risk. Resource consumption involves CPU, memory, bandwidth, and other consumptions, recovery delay corresponds to the time for the system to recover to normal operation, and state consistency risk refers to the probability of introducing potential data inconsistency by different operations. By calculating these indicators and integrating them into a strategy execution cost matrix, the pros and cons of different paths can be quantified. The cost matrix provides a comparable numerical basis for selecting a recovery path.

[0057] Based on the cost matrix, the path with the minimum cost is selected and parsed into a multi-level degradation path. The multi-level degradation path embodies the idea of phased execution, such as performing a lightweight operation (such as restarting a service) first, and if it is ineffective, performing a higher-oversight operation (such as migrating an instance). The degradation path needs to be further decomposed into atomic operation steps, each of which is a minimum instruction unit that the system can execute independently, such as "close process PID = xxx" or "execute database rollback to LSN = yyy".

[0058] After generating the atomic operation sequence, compatibility verification is also required. This process detects the current system's hardware driver version, software dependencies, and configuration environment in real time, and compares them with the requirements in the operation sequence. If there is a mismatch, for example, if the recovery sequence depends on a non-existent driver, the sequence needs to be adjusted, and the incompatible steps need to be deleted or replaced with alternative operations to generate a compatibility-adapted sequence.

[0059] Finally, the compatibility adaptation sequence needs to be converted into an executable script format. The script is generated through templating to ensure consistent execution in different system environments. The template includes command syntax, execution order control, error capture and rollback mechanism. The encapsulated script is the recovery execution instruction, which is the recovery action carrier that can be directly used by the downstream execution module.

[0060] In the implementation process, different technical routes can be adopted to complete the recovery path selection and instruction generation. For example, heuristic algorithms such as A* search can be used to quickly find the optimal path in the strategy library, or reinforcement learning methods can be used to dynamically optimize path selection based on feedback from historical recovery results. Resource consumption and recovery delay evaluation can be modeled through historical execution data or simulated in real time.

[0061] In compatibility verification, some operations can be pre-executed in a container sandbox environment to verify whether the commands can be correctly executed; or static configuration comparison can be used to confirm compatibility at the parameter level before execution. When encapsulating scripts, Shell, Python or PowerShell scripts can be generated, depending on the target execution environment. For cross-platform scenarios, YAML or JSON can also be used as an intermediate format, which is then translated into commands supported by the corresponding system by the execution agent.

[0062] In the financial system, the evaluation of transaction consistency can be strengthened to ensure that the recovery path does not damage the accuracy of the fund flow; in the medical system, the verification of data integrity can be increased to avoid the loss of image data or medical records due to recovery operations. In resource-limited edge device scenarios, the strategy library can be pruned to retain only low-overhead recovery paths.

[0063] This embodiment can ensure that the recovery action is both accurate and efficient by selecting a recovery path based on fault type and context state in the recovery strategy library and generating a recovery execution instruction. The introduction of the cost matrix ensures the rationality of path selection, avoiding delays and resource waste caused by relying solely on human experience or fixed rules. Compatibility verification and script encapsulation ensure the executability and stability of recovery operations in different environments.

[0064] S40, creating a verification experiment based on the recovery execution instruction, constructing a rehearsal environment and injecting rehearsal data to obtain verification data;

[0065] In this embodiment, after receiving the recovery execution instruction, the parameters contained therein need to be parsed first. These parameters include the order of execution operations, isolation requirements for the environment, resource constraints, and data processing-related dependencies. Through parsing, a set of sandbox configuration parameters can be generated, which define the isolation boundary between the experimental environment and the production environment and explicitly require the allocation of storage, computing, network, and other resources.

[0066] After the sandbox configuration parameters are generated, the system will deploy an image of the production system according to the parameters, forming a copy consistent with the logical structure of the production system. This copy provides a controlled experimental environment that does not affect the production system. After the image is loaded, the environment needs to be initialized, and an initial state snapshot is captured at this stage as baseline data for subsequent comparison.

[0067] After the environment is prepared, the recovery execution instruction is further parsed into an injection strategy syntax, which describes the types of exercise data to be injected, the injection location, the injection intensity, and the duration, etc. Based on this syntax, the generated data injection template is constructed, and a signature digest is attached during the construction process to ensure the integrity and tamper-proofing properties of the template. Subsequently, the template is loaded into the sandbox copy, and the experimental data flow is distinguished from the actual business flow through traffic identification, thereby achieving the separation of exercise data and production data.

[0068] After the controlled environment is ready, the recovery execution instruction is executed, and the exercise data is injected, and the system captures the environment state snapshot again when the injection is completed. By comparing the pre-injection snapshot with the post-injection snapshot, performance differences and functional abnormal behaviors can be extracted, such as resource utilization changes, response delay deviations, and data consistency check results. Finally, these information is combined with traffic identification and signature digest to form structured verification data as input for the subsequent verification link.

[0069] In implementation, the construction of the sandbox environment can be based on virtual machine replication or container technology to quickly pull up a running instance consistent with the production environment. For resource isolation, cgroup or namespace mechanisms can be used to achieve fine-grained CPU, memory, and network isolation, and hardware virtualization support can be used to further reduce the impact on the production system. Snapshot capture can use the write-time copy mechanism at the file system level or use system call level monitoring interfaces to obtain memory state and process tables.

[0070] During the injection of drill data, predefined traffic replay files can be used to simulate historical business requests, or a synthetic data generator can dynamically generate data streams of specific patterns. The signature digest generation method can choose a hash algorithm, such as SHA-256, to ensure the integrity of the template during transmission and execution. When generating verification data, the types of recorded metrics can be expanded, such as adding system call exception frequency and database lock wait time, to enhance data coverage.

[0071] The injection strategy can also be adjusted for different application scenarios. For example, transaction conflict simulation can be added to financial business systems, and multi-user concurrent access scenarios can be added to medical information systems, thereby ensuring the representativeness and accuracy of the verification data.

[0072] Example Explanation: In the financial sector, a large amount of simulated transaction data can be injected into a sandbox replica. By replaying transaction requests, the system's latency and consistency after the execution of recovery instructions can be observed. For example, this can verify whether inconsistencies arise between accounting records and log records in a partial database recovery scenario. The generated verification data can be used to assess the impact of recovery operations on transaction integrity.

[0073] In the healthcare sector, patient records, prescriptions, and imaging data can be injected into isolated training environments. By simulating concurrent access scenarios across multiple terminals, the stability and consistency of data access can be checked after recovery operations. For example, it can be determined whether doctors' terminals can continuously read patient image files during the recovery of the imaging database. The generated validation data can be used to measure the degree to which the recovery operation ensures the continuity of clinical operations.

[0074] This embodiment builds a training environment and injects data based on the recovery execution instructions, enabling real-world verification of the recovery instructions' execution effect under conditions isolated from the production system, thus avoiding the risks associated with directly impacting the production system. Through dual snapshot comparison and traffic identification mechanisms, abnormal behavior can be efficiently extracted and structured verification data generated, providing a quantifiable basis for subsequent assessment of the recovery effect.

[0075] S50, a verification conclusion is formed based on the verification data, and the verification conclusion is associated with the recovery execution instruction;

[0076] In this embodiment, after obtaining the verification data, it is necessary to analyze the performance metrics and abnormal behaviors contained therein. Verification data typically includes a set of system performance anomaly metrics, environmental snapshot comparison results, traffic identifiers, and signature digests. During the analysis, the set of performance anomaly metrics can be extracted to quantify the impact of resuming execution instructions. These metrics may include multi-dimensional parameters such as CPU utilization deviations, abnormal memory usage, transaction failure rates, and increases in response latency.

[0077] After extracting the performance anomaly indicators, it is necessary to match them with a pre-defined library of fault patterns. The fault pattern library contains the mapping relationship between known fault scenarios and indicator characteristics, such as disk I / O anomalies corresponding to increased read-write delay, and memory leaks corresponding to a continuously growing memory usage curve. Through matching, the verification data can be assigned a corresponding fault pattern classification label, marking whether the specific fault pattern is triggered during this recovery execution.

[0078] To improve the reliability of the conclusion, the overall deviation of the abnormal indicators also needs to be calculated. Deviation reflects the difference between the current running state and the baseline state, which can be achieved through weighted average, clustering distance, or principal component analysis. Then, a time decay factor needs to be introduced to weight the historical verification conclusions to avoid outdated information having too much impact on the results. The final dynamic confidence score combines performance deviation and historical reliability weight to provide a comprehensive and reliable numerical result.

[0079] After forming the verification conclusion, it also needs to be bound with the recovery execution instruction. The recovery execution instruction contains the operation sequence, which can generate a unique identifier based on the sequence content, such as using a hash algorithm to generate a digital signature of the instruction sequence. By associating the verification conclusion with the unique identifier of the operation sequence, a versioned instruction identifier can be generated. This identifier not only distinguishes different recovery execution instructions, but also traces the source of each verification conclusion. Finally, the versioned instruction identifier is used to update the path mapping relationship in the recovery strategy library, so that the strategy library can refer to the verified results in future similar fault handling.

[0080] In implementation, the extraction of performance anomaly indicators can be collected through system monitoring interfaces, such as operating system performance counters, database transaction log analysis tools, or application layer API call statistics modules. Fault pattern matching can use a rule-based approach, such as using threshold conditions for comparison, or a machine learning-based classifier that inputs performance indicators into the model to automatically identify the most likely fault pattern.

[0081] In the calculation of dynamic confidence score, time-weighted average method can be used, that is, the weight of historical conclusions closer to the current time is larger, and the weight of too distant conclusions gradually decays. It can also use a sliding window mechanism to only keep the latest few verification results as a reference.

[0082] For the generation of the unique identifier of the recovery execution instruction, a hash function such as SHA-256 can be used, the content of the entire operation sequence is input into the hash algorithm, and a fixed-length identifier value is obtained. In this way, the uniqueness of the identifier can be guaranteed, and tampering can be avoided. The version management can use a database storage method to record the verification conclusion and the instruction identifier one by one, and call it when the policy library is updated.

[0083] In different scenarios, the index analysis method and the weight allocation method can be adjusted. For example, in a financial system, a higher weight needs to be given to the transaction consistency deviation index; in a medical information system, the data integrity and delay index need to be given priority to ensure business continuity.

[0084] Example: In the field of financial services, when the recovery execution instruction is executed on the payment system database, the verification data may include transaction transaction playback results and account balance consistency indicators. By extracting abnormal indicators, it can be found that the transaction submission delay exceeds the benchmark value and triggers the corresponding fault mode label. Combined with the historical verification conclusion, the dynamic confidence score is used to determine whether the recovery path is stable and reliable. The associated versioned instruction identifier ensures that the conclusion is bound to the specific recovery operation, which can be reused in the future payment system recovery.

[0085] In the field of medical health services, when the recovery execution instruction involves the medical record data system, the verification data may include image file reading delay and patient record consistency check results. By matching with the fault mode library, the system identifies the risk of storage I / O bottleneck. The dynamic confidence score combined with the historical verification conclusion shows that the risk level is high, so the path in the policy library will be marked as low priority. Through the binding with the unique identifier of the operation sequence, the medical data system can preferentially select a more reliable recovery path in future recovery operations, ensuring that the medical business does not be interrupted.

[0086] The embodiment forms a verification conclusion based on the verification data, and binds it with the recovery execution instruction, which can form a structured and traceable record after each recovery operation, ensuring the reliability of the verification result and providing a clear reference for subsequent policy library optimization. The introduction of the dynamic confidence score enables the verification conclusion to have time adaptive ability, which can avoid incorrect judgments caused by outdated information. The versioned instruction identifier ensures the consistency of the verification conclusion and the recovery operation, improving the controllability and traceability of the system in multiple iterations.

[0087] S60, issuing the recovery execution instruction in the execution environment and completing the recovery operation.

[0088] In this embodiment, when the recovery execution instruction has been generated and its rationality is confirmed by the verification conclusion, it is necessary to issue the instruction to the actual execution environment to complete the recovery operation. The execution environment can be a physical server cluster, a virtualization environment, a container orchestration platform, or a distributed microservice architecture. In order to ensure the correctness of the issuing process, the permission level and the interruption enable state need to be checked first to ensure that the current execution environment has legal execution permission and to avoid being affected by external interruptions during the operation process.

[0089] After confirming that the environment meets the conditions, it is necessary to build or configure a secure execution environment. The secure execution environment ensures that the recovery operation is not disturbed by isolating the running space, limiting the permission range, and shielding irrelevant interruption signals. Then the recovery execution instruction needs to be parsed to disassemble the complex instruction sequence into executable operation units and form a recovery job queue. Each task in the recovery job queue has operation context information to ensure that it can be completed in order and according to the dependency relationship.

[0090] The recovery job queue needs to be distributed to target components, including databases, file systems, network services, or application service modules. A unique asynchronous distribution identifier is generated during the distribution process to track the status of each task in subsequent monitoring. The execution of the recovery job adopts a phased strategy, and a snapshot is generated before each phase is executed and a rollback point is generated after the execution is completed. In this way, in the event of an exception, it can quickly fall back to the last stable state.

[0091] During the job execution process, resource consumption needs to be monitored in real time, including CPU, memory, disk IO, and network bandwidth, and a resource constraint report needs to be generated. If the resource consumption exceeds the set threshold, the task execution can be suspended or adjusted to avoid excessive impact of the recovery operation on the production environment. Finally, when the recovery operation is completed, a consistency check needs to be performed to check the logical consistency and physical consistency between the recovery result and the expected state, and a recovery completion marker needs to be generated. This marker is the final evidence that the recovery operation is truly completed and successful.

[0092] In different environments, the way of issuing the recovery execution instruction is different. In a containerized environment, the recovery job queue can be converted into a Pod deployment instruction through the control plane of Kubernetes, and the controller can be used for rolling recovery. In a virtualization environment, the instruction can be issued to the virtual machine manager through the API interface to trigger snapshot recovery and dependency reconstruction. In a physical server scenario, the recovery task can be issued and executed through a remote management interface (such as IPMI or Redfish) combined with an automation script.

[0093] The phased execution strategy can be divided by function module, such as first restoring the database, then restoring the application service, and finally restoring the network load balancing. It can also be divided by time criticality, first restoring the core transaction channel, and then restoring the peripheral non-critical module. The rollback point can be realized by file system snapshot, database consistency point or memory image, and different environments can choose appropriate means according to resource conditions.

[0094] In resource monitoring, CPU and memory data can be collected using system-level monitoring agents, I / O indicators can be obtained through storage system built-in monitoring, and bandwidth usage can be collected by combining network analysis tools. If resource overload is found, the recovery process can be ensured by adjusting the execution order of the job queue or reducing the parallelism.

[0095] Example: In the financial business field, when the payment clearing system encounters database damage, the recovery execution instruction is issued to first restore the database snapshot and verify transaction consistency, and then restore the payment gateway and risk control service. Through the phased snapshot and rollback mechanism, if an exception occurs during the recovery of the payment gateway, it can be rolled back to the state where the database recovery is completed, avoiding secondary data damage. The eventual consistency verification ensures that the account data and the historical snapshot are completely matched, thereby ensuring the accuracy of financial transactions.

[0096] In the medical health business field, after the electronic medical record system fails in the storage node, the recovery execution instruction is issued to the execution environment to first restore the storage volume, then restore the application service, and finally restore the medical terminal access interface. During the recovery process, resource consumption is monitored to avoid high disk I / O load affecting the normal query of the medical record system. After completion of the recovery, consistency verification is performed to ensure that the image file and the medical record are consistent, and a recovery completion flag is generated, thereby ensuring the continuity of the medical institution business and the reliability of the medical data.

[0097] This embodiment can realize automatic and controllable full-process recovery by issuing a recovery execution instruction in the execution environment and completing the recovery operation, avoiding the delay and risk caused by manual execution. The phased execution and rollback point setting ensure the fault tolerance of the operation process, the resource monitoring ensures the sustainability of the recovery operation on the running environment, and the consistency verification and recovery completion flag ensure the reliability and verifiability of the final result.

[0098] The application relates to the technical field of artificial intelligence, can be applied to business scenes such as financial technology and medical health, and discloses a fault diagnosis and recovery verification method, device, equipment and medium, which comprises the following steps: acquiring monitoring data and generating a fault candidate event, inputting the fault candidate event and a context state into an intelligent diagnosis model to obtain a diagnosis result and a fault type, selecting a recovery path in a recovery strategy library based on the fault type and the context state and generating a recovery execution instruction, creating a verification experiment based on the recovery execution instruction and injecting simulation data to obtain verification data, forming a verification conclusion according to the verification data and associating the recovery execution instruction, and issuing the recovery execution instruction in an execution environment and completing a recovery operation. The application forms a full-link automatic process from fault detection, diagnosis, recovery to verification through the introduction of multi-source monitoring data processing, intelligent diagnosis model reasoning, strategy library path selection, verification experiment closed-loop checking and an automatic recovery mechanism of an execution environment; the application can realize accurate identification of fault types and selection of dynamic recovery paths under complex environments, ensures the reliability and consistency of recovery operations, effectively reduces the delay and misjudgment caused by manual intervention, and improves the self-healing ability and operation stability of a system.

[0099] In one embodiment, the step S10 comprises:

[0100] S101, collecting real-time system running logs;

[0101] S102, monitoring device health indicators to generate device health state data;

[0102] S103, calling a historical fault record library;

[0103] S104, analyzing network topology connection relationships to generate a topology mapping relationship table;

[0104] S105, associating the real-time system running logs, the device health state data, the historical fault record library and the topology mapping relationship table to form a multi-source spatio-temporal feature matrix;

[0105] S106, extracting a set of spatio-temporal correlation feature indicators in the multi-source spatio-temporal feature matrix;

[0106] S107, loading a pre-trained threshold strategy library;

[0107] S108, matching the set of spatio-temporal correlation feature indicators with the pre-trained threshold strategy library to obtain an initial fault event set;

[0108] S109, performing topology collision detection on the initial fault event set to generate a fault candidate event.

[0109] In this embodiment, the monitoring data covers four types of information sources: operation logs, device health indicators, historical fault records, and network topology. The goal is to form a computable spatio-temporal data body with unified time base and component identification. Real-time system operation logs refer to the continuous output of text or structured event streams by business processes, system kernels, middleware, and security components, containing fields such as timestamp, level, source module, instance identification, and message body. Streaming aggregation is achieved through proxy or non-intrusive capture channels during collection, establishing field mapping and time unification. Key implementations include time zone normalization, millisecond-level alignment, host and container dual identification binding, and multi-source deduplication, ensuring that records of the same business operation on different nodes can be serialized and aligned. Device health indicators are derived from host and dedicated hardware, common dimensions include processor utilization, memory availability, disk and network throughput, temperature, and power stability, sampling frequency is determined by resource side pull or monitoring side push, device health status data generation requires unit normalization, sampling period resampling, and missing value imputation, while maintaining index metadata to record collection probes, precision level, and confidence interval. The historical fault record library saves the occurrence time, impact range, root cause type, disposal process, and recovery result of confirmed faults, after building a unified template it can be used as a reference sample and prior knowledge source, necessary fields include fault label, trigger condition, affected component set, and disposal work order index. Network topology connection relationships come from configuration management or automatic detection, including computing nodes, storage, network devices, and application dependencies, after parsing a topology mapping relationship table is generated, this table uses component unique identification, connection relationship, directionality, and weight fields, it supports both equivalent adjacency matrix representation and graph structure representation of edge table and attribute table, ensuring that subsequent path query and propagation analysis can be performed on a unified structure.

[0110] The log, health data, history record and topology mapping are spatio-temporally aligned and key-value associated to construct a multi-source spatio-temporal feature matrix. The matrix takes time slice and component as index axis, and the internal unit carries multi-dimensional statistics and event count aggregated in the same time window, and is attached with adjacency constraints or propagation weights from the topology. The time window division and water line control are implemented first, and then feature preprocessing is performed on each type of data, including text event vectorization or templating, index smoothing and outlier suppression, history record label alignment and confidence annotation, topology connected component identification and hierarchical annotation. Cross-source association uses a unified component namespace and instance landing point mapping to handle identification changes caused by instance drift, elastic scaling and failover, ensuring that each matrix slice in a time window has consistent dimensions and semantics. Based on the matrix, a set of spatio-temporal correlation feature indicators is extracted, focusing on capturing the mutation characteristics of the same component over time and the collaborative anomalies between adjacent components in space. The time side implementation includes sliding window statistics, change rate and stability measures, periodic residual and kurtosis deviation, etc. The space side implementation includes neighborhood consistency difference, path aggregation strength, upstream propagation delay and downstream back pressure signs, etc. To ensure generality, indicators are organized in the form of operator combinations, which can be enabled as needed in different business scenarios, and each indicator is bound to the source field, calculation window, semantic annotation and visualization scale, facilitating subsequent threshold matching and interpretation.

[0111] The pre-trained threshold strategy library gives the mapping of indicators to trigger conditions, derived from the statistical distribution of the historical fault record library, offline playback verification and operation and maintenance specification conventions. The strategy item uses a unified description structure, including applicable component type, indicator name, trigger threshold interval, duration requirement, co-occurrence condition and mutual exclusion condition. When loading the strategy library, version checking and dependency resolution are completed, and context correction factors are introduced according to the running domain, business peak period and maintenance window to realize the environmental adaptation of the threshold. The initial fault event set is obtained by matching the spatio-temporal correlation feature indicator set and the pre-trained threshold strategy library, and the core logic is to evaluate the strategy conditions on each time window and component. The combination that meets the trigger condition generates an initial event, which includes component identification, matched strategy and indicator value section, duration and confidence annotation. To prevent event generalization, the matching engine simultaneously performs co-occurrence verification and mutual exclusion resolution to retain strong evidence items that conflict with each other in the same time window, and concatenates multi-indicator common trigger conditions in the form of reason chain.

[0112] The initial fault event set generates fault candidate events through topology collision detection. The goal of topology collision detection is to eliminate repeated or dependent alarms caused by the same propagation chain and retain representative events closer to the potential root cause. In implementation, clustering of affected nodes and shortest propagation path search are performed on the topology mapping table to identify situations where upstream abnormalities are amplified downstream, and to determine whether they are source events or dependent events based on propagation direction and path weight. For initial events that are in the same connected subgraph and overlap in time, they are ranked according to path convergence degree, trigger strength and historical priori, and dependent events are deleted or down-weighted, weak events of adjacent nodes are merged to generate stronger candidates, and the output fault candidate events include coverage range, representative node, evidence indicator set and topology location. To ensure adaptation to complex dependencies, collision detection also handles multi-source convergence, multi-tenant isolation and cross-domain link situations, and avoids cross-domain mismerger through isolation labels and weight adjustment in edge attributes. The entire process runs in a pipeline, and the output of the previous stage directly enters the calculation of the next stage, and the structured results produced by any stage are preserved with version and time boundaries for traceability.

[0113] In this embodiment, the monitoring data becomes a multi-source spatiotemporal feature matrix that can be directly calculated after being aligned with unified time and identity. The fusion of logs, health indicators, historical records and topology makes event judgment no longer dependent on a single signal to produce false positives; the spatiotemporal correlation feature indicator set provides joint evidence of time mutation and space propagation, and the pre-trained threshold strategy library deposits experience knowledge into executable conditions, and the matching process quickly filters out high-correlation initial events; topology collision detection merges and disambiguates on the propagation chain, retains representative events closer to the root cause and compresses event volume, reducing the burden of subsequent diagnosis. Thus, the conversion from monitoring data to fault candidate events has real-time, interpretability and traceability, and the input quality is significantly improved, providing stable, low-noise and structured prerequisite data support for subsequent diagnosis, recovery path selection and verification.

[0114] In one embodiment, the above step S20 comprises:

[0115] S201, extracting key features of the fault candidate event to generate a spatiotemporal feature vector;

[0116] S202, querying a fault knowledge graph based on a context state to obtain a set of context-associated fault patterns;

[0117] S203, inputting the spatiotemporal feature vector and the set of context-associated fault patterns into an intelligent diagnosis model to obtain a preliminary diagnosis probability distribution;

[0118] S204, analyzing the decision logic of the intelligent diagnosis model through an attribution analysis module to generate an interpretable evidence chain;

[0119] S205, filtering out fault types higher than the preset confidence threshold from the preliminary diagnosis probability distribution to generate a high-confidence classification result;

[0120] S206, integrating the high-confidence classification result and the explainable evidence chain to generate an intelligent diagnosis result.

[0121] In this embodiment, after the fault candidate event enters the processing channel, the unified time axis and component identifier are verified for integrity and de-duplication to ensure that an event corresponds to only one set of computable inputs. Key feature extraction is based on event ontology, adjacent behavior and environmental clues, including timestamp, duration, source component, impact range, window collaborative event count, index mutation amplitude, log template fingerprint, historical repetition frequency, topology level and upstream and downstream propagation strength. For time correlation, multi-scale windows and difference quantities are constructed, and rhythm and drift measures are added. For spatial correlation, neighborhood aggregation strength and path congestion degree are introduced. All scalars are normalized by unit and stretched by distribution, discrete identifiers are converted to stable encodings, and text traces are converted to sparse or dense representations. Then the above elements are concatenated into a spatio-temporal feature vector according to the specified field order and mask rule. The vector retains source references and time levels to ensure traceability and recalculation.

[0122] Context state comes from environmental data such as running resources, configuration parameters, business windows, topology changes and security policies, which are managed by unified key space and time slice. When querying the fault knowledge graph based on the context state, the graph takes components, fault types, symptom patterns, trigger conditions, propagation relationships and recovery actions as entities and relationships; the relationships have direction, weight, time limit and environmental label. The query process first filters the effective subgraph according to the context state, then takes the component and symptom of the fault candidate event as the starting point, performs multi-hop path retrieval and pattern matching, extracts the causal fragments and co-occurrence clusters consistent with the current environment, and forms a set of context-related fault patterns. Each entry in the set carries prior confidence, applicable scenario and conflict exclusion information, avoiding bringing patterns that are only valid in different environments in history into the current judgment.

[0123] When the spatio-temporal feature vector and the set of context-related fault patterns are input into the intelligent diagnosis model, the input channel completes structure alignment and semantic fusion. Fusion can use concatenation, attention weighting or graph embedding aggregation of vector and pattern features, or can be merged after alignment in a shared representation space through a double-path structure, the goal is to constrain each other in a unified representation. The model outputs a preliminary diagnosis probability distribution covering a pre-set fault type label space, with probability components and types corresponding one-to-one, and the output containing uncertainty labels and temperature calibration coefficients to prevent overconfidence. To control the drift effect, the distribution is executed in the inference backend for light calibration and threshold clipping, while the original distribution is retained for explanation and traceability.

[0124] The attribution analysis module parses the decision logic of the intelligent diagnosis model to generate an explainable evidence chain. The parsing path starts from both ends. One end measures the marginal contribution of each dimension feature to the output component in the input space, and obtains the contribution ranking and direction by using mechanisms such as local perturbation, sensitivity measure or comparative score. The other end traces back in the knowledge graph subgraph, collects pattern fragments and constraint information along the high-weight path from the trigger condition to the fault type, and aligns and reweights the graph path weight and the model attention weight. The results of the two ends are merged in the time and component dimensions to form an explainable evidence chain containing feature contribution pairs, pattern path fragments, time sequence evidence and confidence notes. The evidence chain is checked for consistency and conflict resolution to ensure that the supporting information for decision-making is closed and free of self-contradiction.

[0125] According to the preliminary diagnosis probability distribution and the pre-set confidence threshold, screening is performed to output a high-confidence classification result. The threshold configuration supports hierarchical management according to business windows, component importance levels and historical calibration results, which can be in the form of single label or multiple labels. When all components are below the threshold, a low-confidence strategy is triggered, such as expanding the time window for re-evaluation, introducing more context features or requesting manual review. The candidate set of boundary samples and the second sequence selection are retained during the screening process to facilitate the subsequent recovery strategy in the multi-candidate scenario.

[0126] The high-confidence classification result and the explainable evidence chain are integrated to generate an intelligent diagnosis result. During integration, a structured record is constructed, covering the target fault type set, the corresponding probability value, the key feature contribution, the matched knowledge graph path, the time and space positioning, the impact range estimation and the uncertainty note. At the same time, the associated identifier is written to link the previous fault candidate events and the context state, and the output is accompanied by complete traceability line and version information, ensuring that the subsequent recovery path selection and verification link can be directly consumed without repeated reasoning.

[0127] This embodiment carries cross-source and cross-scale abnormal clues through the spatiotemporal feature vector, and provides environmental constraints and prior associations through the context-associated fault mode set. After the fusion of the two types of information in the intelligent diagnosis model, the judgment strength is expressed in the form of probability, avoiding false positives and false negatives caused by a single signal. The explainable evidence chain combines feature contributions and knowledge paths into verifiable supporting materials, making the classification result traceable and auditable. The hierarchical management of the pre-set confidence threshold and the boundary sample retention mechanism maintain high accuracy while ensuring coverage in complex scenarios. As a result, the diagnosis output is more robust, transparent and directly drives the subsequent recovery and verification process, reducing overall processing delay and misjudgment rate, and significantly reducing adaptation costs during cross-environment migration.

[0128] In one embodiment, the above step S30 comprises:

[0129] S301, obtaining a hierarchical strategy mapping table based on the fault type and the context state querying a recovery strategy library;

[0130] S302, analyzing resource consumption, recovery delay and state consistency risk of each strategy in the hierarchical strategy mapping table to generate a strategy execution cost matrix;

[0131] S303, selecting a minimum cost path according to the strategy execution cost matrix to generate a multi-level degradation path;

[0132] S304, parsing the multi-level degradation path into atomic operation steps to generate a recovery operation sequence;

[0133] S305, detecting hardware drivers and software dependencies in real time to verify the compatibility of the recovery operation sequence with the current system state to generate a compatibility check result;

[0134] S306, adding, deleting or replacing operation steps in the recovery operation sequence based on the compatibility check result to form a compatibility adapted operation sequence;

[0135] S307, converting and packaging the compatibility adapted operation sequence into an executable script template to generate a recovery execution instruction.

[0136] In this embodiment, when the fault type and the context state are obtained, a path that meets the conditions needs to be selected from the recovery strategy library and converted into an executable recovery execution instruction. First, the recovery strategy library needs to be queried based on the fault type and the context state. The recovery strategy library contains different categories of recovery schemes, which are hierarchically organized according to fault dimensions, dependency constraints and system environment. The querying process takes the fault type and the context state as retrieval conditions and outputs a hierarchical strategy mapping table. The mapping table structures and lists possible recovery strategies according to priority level, applicable conditions and target components, so as to be filtered subsequently.

[0137] Then, resource consumption, recovery delay and state consistency risk of each strategy in the hierarchical strategy mapping table are evaluated to form a multi-dimensional quantitative description. Resource consumption includes CPU, memory, storage and network consumption; recovery delay refers to the time interval from the start of the strategy to the completion of the fault recovery; and state consistency risk measures the probability of data inconsistency or dependency conflict caused by strategy execution. After analysis, these results are integrated into a strategy execution cost matrix. Each row of the matrix represents a strategy, and each column represents a measurement index, forming a unified framework that can be used for calculation and comparison.

[0138] Based on the strategy execution cost matrix, the path with the minimum cost can be selected through weighted calculation, constraint optimization or heuristic algorithm. If some paths are not desirable in terms of resource consumption and recovery delay, a suboptimal solution or a multi-level degradation path is selected. The multi-level degradation path reflects a hierarchical recovery mechanism, i.e., a low-risk and low-cost operation is preferred, and if it fails, the system gradually degrades to a more forced or higher consumption path, ensuring the robustness and fault tolerance of the recovery process.

[0139] After selecting the path, it needs to be parsed into more granular atomic operation steps, such as "restart a single service", "replace a driver module", "flush cache", "reload configuration file", etc. These atomic operation steps are combined into a recovery operation sequence, each step of which can be independently executed and verified. In order to ensure the executability of the operation sequence with the current running environment, it is necessary to detect the version of the hardware driver, the integrity of the software dependency and the resource state in the running environment in real time, so as to verify the compatibility of the recovery operation sequence with the system state and generate a compatibility verification result.

[0140] If the compatibility verification result indicates that there is a conflict or mismatch, the recovery operation sequence needs to be adjusted. The adjustment methods include deleting the unexecutable operation, replacing it with a more compatible alternative operation, or inserting necessary preparation steps. Through such addition, deletion and replacement operations, a compatibility adaptation operation sequence is formed, so that the entire sequence can run smoothly in the target execution environment.

[0141] Finally, the compatibility adaptation operation sequence needs to be converted into a directly executable script form. Through the executable script template, the syntax format is converted, and the operation steps are encapsulated in a sequential and parameterized manner, so as to generate a recovery execution instruction. The recovery execution instruction has portability and repeatability, and can be directly executed in the target environment to ensure the closed loop between the selection of the recovery path and the actual operation.

[0142] Through the above steps, the selection of the recovery path is no longer dependent on fixed rules, but is dynamically decided in combination with the fault type and the context state. The hierarchical strategy mapping table ensures the comprehensiveness of the strategy range, the strategy execution cost matrix provides a quantifiable basis for comparison, and the multi-level degradation path enhances the availability under complex conditions. The atomic recovery operation sequence ensures the flexibility of traceability and adjustment, and the compatibility verification mechanism ensures the consistency of the operation and the system environment, avoiding secondary failures caused by environment mismatch. Through the automatic encapsulation of the script template, the final recovery execution instruction can be directly implemented, shortening the time of fault recovery and reducing the risk and uncertainty of manual intervention, thereby improving the accuracy and efficiency of fault response.

[0143] In one embodiment, the above step S40 comprises:

[0144] S401, parse the recovery execution instruction, extract the environment isolation requirement and the resource constraint condition, and generate a sandbox configuration parameter;

[0145] S402, isolate the production environment resource and deploy a system image based on the sandbox configuration parameter, and create a sandbox copy;

[0146] S403, capture an initial state snapshot before injection after the initialization of the sandbox copy is completed;

[0147] S404, generate an injection strategy syntax according to the recovery execution instruction;

[0148] S405, construct a data injection template based on the injection strategy syntax, and add a signature digest;

[0149] S406, load the data injection template in the sandbox copy and mark a traffic identifier, and generate a controlled rehearsal environment;

[0150] S407, execute the recovery execution instruction in the controlled rehearsal environment, and capture an environment state snapshot after injection;

[0151] S408, compare the difference between the initial state snapshot before injection and the environment state snapshot after injection, and obtain a system performance abnormality index set;

[0152] S409, integrate the system performance abnormality index set, the traffic identifier, and the signature digest, and obtain verification data.

[0153] In the embodiment, the recovery execution instruction is used to drive a repeatable rehearsal and verification process. First, the recovery execution instruction is structurally parsed, the environment isolation requirement and the resource constraint condition are extracted, and are merged into a sandbox configuration parameter. The environment isolation requirement includes a network intercommunication range, a file system read-write permission, a device access whitelist, and a system call restriction set. The resource constraint condition includes a processor quota, a memory upper limit, a disk quota, a bandwidth upper limit, and a concurrency threshold. The parsing process standardizes text or binary fields into a unified configuration structure based on key-value segments, parameter bits, or embedded metadata in the instruction, to ensure that subsequent components can be directly consumed.

[0154] A sandbox copy is created by isolating production environment resources and deploying a system image on a different or same site computing node according to sandbox configuration parameters. The isolation actions include setting namespace boundaries, cgroup control group parameters, SELinux or AppArmor security policies, scheduling affinity and anti-affinity for containers or virtual machines. The system image selects an operating system version, kernel patch level, runtime dependencies and configuration file set consistent with production, and verifies consistency through image signature verification and hash reconciliation. After the sandbox copy is started, health probe checks and time synchronization corrections are performed to ensure that time sequence events and log timestamps can be compared.

[0155] An initial state snapshot before injection is captured after the sandbox copy is initialized. The snapshot content covers key observation surfaces from the kernel and process perspectives, including memory usage distribution, handle and file descriptor count, service port listening table, key configuration file hash set, system log location marker, and water level metering indicators. To reduce the performance disturbance introduced by observation, a hierarchical collection and asynchronous writing strategy is adopted, and the collection window and sampling granularity are recorded to unify the scale for difference calculation.

[0156] An injection strategy syntax is generated according to the recovery execution instruction. The injection strategy syntax gives the mapping relationship between the exercise data and the target interface, the trigger condition, the timing constraint and the conflict processing rule, adopts a readable and parseable structure expression form, supports field-level operation, record-level operation and traffic-level operation. Field-level operation defines target field, operation type and value domain, record-level operation defines data row splicing, deletion or replacement, and traffic-level operation defines rate, burst, random disturbance ratio and session order preservation constraint. The injection strategy syntax allows reference to context variables and time functions to reproduce the load pattern of the production site.

[0157] A data injection template is constructed based on the injection strategy syntax, and a signature digest is added. The data injection template is instantiated at the syntax level, including field mode, value generator, constraint verifier, serialization encoding method. The value generator supports rule value, distribution sampling value, related variable derivation value and playback binding value. The constraint verifier performs range check, regular matching, uniqueness and reference integrity check during template instantiation. The signature digest calculates an irreversible digest for the template body and key parameters, and adds a time label and a version label for integrity and traceability guarantee.

[0158] A controlled rehearsal environment is generated by loading the data injection template and marking the traffic identification in the sandbox copy. The loading process includes template rendering, dataset generation, payload fragmentation, and injection channel binding. The traffic identification is in the form of a marker key throughout the injection, processing, and collection link, written into the request header, message metadata, or context label, supporting subsequent playback, filtering, and merging. The controlled rehearsal environment maintains a consistent middleware stack and dependency topology with the production environment, while preventing cross-domain access to production domain names and addresses through network policies, routing isolation, and name resolution redirection.

[0159] The recovery execution instruction is executed in the controlled rehearsal environment, and the post-injection environment state snapshot is captured. Idempotent safeguards and phase rollback points are enabled during execution to avoid uncontrollable side effects. The post-injection environment state snapshot reuses the consistent observation surface, granularity, and encoding method as the pre-injection, ensuring difference alignment. By comparing the difference between the initial state snapshot before injection and the post-injection environment state snapshot, a set of system performance anomaly indicators is obtained. Difference calculation is performed in parallel in three channels: indicator layer, object layer, and event layer. The indicator layer handles continuous quantities such as response time quantile, error rate, throughput, and resource utilization, and outputs deviation and anomaly labels using window alignment and anomaly threshold mapping. The object layer handles the addition, deletion, and content changes of discrete objects such as files, configurations, ports, and processes, and outputs the object difference set. The event layer compares abnormal events and stack trace patterns within the log interval, and outputs event intensity and new event set. The outputs of the three channels are normalized into the system performance anomaly indicator set, retaining time coordinate, traffic identification alignment information, and collection window metadata.

[0160] The system performance anomaly indicator set, traffic identification, and signature digest are integrated to obtain verification data. The integration process performs source trace verification, payload and result fingerprint association check, time alignment, and packet loss fault tolerance repair, and outputs structured records for subsequent judgment, playback verification, and strategy iteration. Verification data contains machine-readable and auditable additional information, facilitating long-term archiving and cross-version comparison.

[0161] Computing stack isolation can be based on container virtualization, configuring namespaces, cgroups, and security policies to adapt to scenarios requiring fast startup and high-density concurrency. Hardware virtualization can also be used to obtain stronger isolation and kernel-level customization capabilities using lightweight virtual machines. Network isolation can use software-defined network policies to set ingress and egress rules and network segment whitelists, or use virtual switches and routers to implement three-layer isolation and mirror port collection. System image acquisition can verify and copy existing network images, integrate rehearsal-specific agents and probes, and ensure traceability through image repository versioning.

[0162] Snapshot collection can adopt a scheme combining kernel metrics and user space metrics. Kernel metrics obtain scheduling, memory, block device, and network stack observation values through system interfaces. User space metrics collect business perspective data through service probes and application in-line points. To reduce noise, the collection window and throttling parameters are associated with resource constraints in the recovery execution instructions to ensure that critical surfaces are prioritized when resources are scarce.

[0163] Injection policy syntax design can adopt a declarative structure to define resource, action, and constraint paragraphs, or a flow structure to define timing, conditions, and branches. Template rendering can use a hybrid approach of placeholder replacement and distributed sampling to support cross-field correlation and check failure fallback. Signature digest can be calculated based on the secure hash family or national cryptography algorithm family, and the digest object covers the template body and key variable name-value pairs, while recording the generation time and generation end identifier.

[0164] Traffic identification implantation can be carried out in HTTP headers, message queue metadata, or database session context, using a coexistence strategy of short identifiers and long fingerprints. Short identifiers facilitate real-time routing and filtering, while long fingerprints are used for tracing and reconciliation. Controlled exercise environment generation registers the identifier in the entry gateway and sidecar agent to achieve routing and mirror splitting based on the identifier. To ensure idempotency and rollback, recovery execution instructions are executed in segments, and a soft rollback point is generated at the end of each segment. If necessary, reverse operations or recovery snapshots are performed.

[0165] Difference calculation can use a sliding window and baseline model for the metric layer, outputting deviation and confidence threshold judgment results. The object layer uses a hash tree or directory snapshot to quickly locate file and configuration changes. The event layer uses pattern mining and clustering to merge similar anomalies into a category to reduce noise. When merging the results of the three layers, time alignment and identifier alignment are performed, and priority strategy is used to coordinate conflicts. Fingerprint verification is applied in the verification data generation link to ensure one-to-one correspondence between the injection template and the result, avoiding crosstalk.

[0166] In resource-constrained or latency-sensitive scenarios, a lightweight mode can be enabled to reduce snapshot dimensions and sampling frequency, retaining only the most important indicators and object sets that contribute to recovery effectiveness. In high-risk scenarios, an enhanced isolation mode can be enabled to prioritize virtual machine isolation and read-only root file systems, with additional system call whitelists and device access whitelists to improve security boundaries. For scenarios that require a large amount of playback, batch injection and parallel comparison can be enabled to support high throughput through identifier sharding and multi-channel collection.

[0167] The embodiment recovers the execution instruction and generates the sandbox configuration parameter by analysis, so that the environment isolation and resource constraint are converted from the text description into executable configuration. The consistency deployment of the sandbox copy and the system image ensures that the rehearsal environment and the production have comparability. The initial state snapshot before injection provides a reliable benchmark. The injection strategy syntax and the data injection template accurately map the expected load and disturbance to the target interface. The signature digest and the traffic identification form a traceable closed loop throughout the data and execution link. The controlled rehearsal environment avoids affecting the production. The environment state snapshot after injection and the difference calculation output the system performance anomaly index set. The integrated verification data has integrity and auditability.

[0168] In one embodiment, the step S50 comprises:

[0169] S501, analyzing the verification data and extracting the system performance anomaly index set;

[0170] S502, matching the system performance anomaly index set with the pre-defined fault mode library to generate a fault mode classification label;

[0171] S503, determining the comprehensive deviation degree of the system performance anomaly index set;

[0172] S504, weighting the historical verification conclusion based on a time decay factor to generate a historical confidence weighted value;

[0173] S505, fusing the comprehensive deviation degree and the historical confidence weighted value according to a pre-set strategy to generate a dynamic confidence score;

[0174] S506, integrating the fault mode classification label and the dynamic confidence score to form a verification conclusion;

[0175] S507, extracting the operation sequence of the recovery execution instruction to generate an operation sequence unique identifier;

[0176] S508, associating the verification conclusion with the operation sequence unique identifier to generate a versioned instruction identifier;

[0177] S509, updating the recovery path mapping relationship in the recovery policy library based on the versioned instruction identifier.

[0178] In this embodiment, after the verification data enters the analysis channel, first, the structure is unified and the time is aligned. The data may come from different collection surfaces, including index sequences, object change sets, and event logs, which need to be aligned on the same time reference to the unified sampling grid, and the missing fragments are repaired by interval interpolation or window backfilling. The analysis module extracts the system performance abnormal index set accordingly, covering throughput, delay quantile, error rate, resource utilization, queue length, thread saturation, and other continuous quantities, as well as port listening changes, configuration file hash changes, process survival status, and other discrete quantities. Each index is accompanied by a sampling window, a source identifier, and a confidence label, which facilitates subsequent weight allocation and conflict resolution. To reduce noise, the analysis stage applies detrending and robust scaling to indicators sensitive to jitter, maps indicators of different dimensions to dimensionless intervals, and ensures additivity and comparability when fusing across dimensions.

[0179] The abnormal index set is matched with the pre-defined fault mode library to obtain the fault mode classification label. The fault mode library is organized in a structured entry, each entry containing a mode feature signature, a trigger precondition, a mutual exclusion relationship, and an applicable context. The mode feature signature is composed of index offset direction, offset amplitude interval, mutual correlation relationship, and time sequence constraint, and the applicable context records hardware morphology, software version, and key configuration threshold. The matching process calculates the similarity score in the local time window, and if the precondition is met and the similarity threshold is crossed, the classification label is generated, while the score ranking and evidence reference of the candidate label are retained, forming a traceable category determination record.

[0180] The comprehensive deviation is used to quantify the severity and range of influence of the anomaly. When calculating, first generate the deviation component in the single-index dimension, determine the adaptive weight by combining the baseline variance, historical stability, and observation confidence of the index, and then perform aggregation at the index group level. When aggregating, consider the redundancy and coupling across indicators, avoid repeated scoring by correlation weighting, and enhance the sensitivity to bottleneck indicators by key path amplification strategy. The output comprehensive deviation takes into account the amplitude and coverage, and the higher the value, the more significant the anomaly.

[0181] To introduce historical experience but avoid over-reliance on outdated conclusions, a time decay factor is introduced to weight the historical verification conclusions, generating a historical confidence weighted value. The history library is keyed by scenario, including fault mode classification label, previous dynamic confidence, main anomaly evidence, and replay verification summary, and the time decay factor is a monotonically decreasing function that gradually reduces the influence of old conclusions over time. The historical entries are aggregated under the same context to obtain a historical confidence weighted value consistent with the current environment, while retaining the contribution decomposition for subsequent explanation and audit.

[0182] The dynamic confidence score is fused from the current comprehensive deviation and the historical confidence weighted value. The fusion rule is given by the preset strategy, and different decision surfaces are selected for different scenarios, such as increasing the inhibition weight of historical negative examples for safety-related scenarios, and increasing the amplification weight of the current deviation for capacity-related scenarios. Monotonicity and boundary stability are ensured during the fusion process to avoid oscillation near the threshold, and the confidence interval and sensitive factor are output to indicate the response strength of the score to the key input variables, facilitating subsequent threshold management.

[0183] The verification conclusion is generated by integrating the fault mode classification label and the dynamic confidence score. The integration unit combines the hierarchical structure of the label with the interval interpretation of the score, outputs the main label and the alternative label set, and attaches an evidence summary. The evidence comes from the contribution ranking of the index component, the key difference of the object change, and the causal fragment of the event sequence. The integration result follows the consistency constraint. If there is an exclusive relationship between the labels, the highest credible path is taken as the main one and the conflict note is retained to reduce misleading output.

[0184] The recovery execution instruction contains a replayable operation sequence, which needs to extract and generate an operation sequence unique identifier from the instruction body. To ensure consistent representation generated by different sources, first, normalize the operation sequence by removing irrelevant blanks and comments, determine the sorting of keys, and use stable encoding for parameters. Then, calculate the digest value as the unique identifier using the encryption digest algorithm. The identifier is recorded together with the sequence version, generation time, and consistency check information to ensure cross-environment and cross-time comparability and traceability.

[0185] The verification conclusion and the operation sequence unique identifier are associated to generate a versioned instruction identifier. The versioned identifier encapsulates the current unique identifier, sequence version, context hash, and policy source label, forming the ability to distinguish different contents of the same instruction. The association relationship adopts a bidirectional index structure, one end queries historical conclusions with instruction identifiers, and the other end locates reusable instruction versions with conclusion elements. The index maintains atomic updates and audit logs to avoid mapping disorder caused by concurrent writes.

[0186] Based on the versioned instruction identifier, the recovery path mapping relationship in the recovery strategy library is updated. The update process uses incremental merging, first locates the affected mapping entries, and then adjusts the path priority or state according to the label and dynamic confidence of the verification conclusion. For low-trust paths, mark them for degradation or freezing, and for high-trust paths, improve the recommended order or open the automatic execution mark. All updates are bound to versioned identifiers to form rollbackable change units, and consistency checks are performed to ensure that mapping changes do not introduce loops, orphan nodes, or policy conflicts. After completion, trigger the strategy snapshot and difference record to provide a basis for subsequent auditing and backtracking.

[0187] The embodiment drives the judgment chain by taking the verification data as the only data source, provides fine-grained metrics through the abnormal index set, comprehensively describes the current deviation intensity through the deviation degree, introduces historical experience and suppresses obsolete information through time decay weighting, obtains stable and sensitive dynamic confidence scores through the fusion of the two, generates verification conclusions with evidence support in combination with classification labels. The conclusion and the operation sequence are bound through unique identification and version identifier, ensuring that the conclusion of one verification only points to one determined instruction content, avoiding drift and misassociation. The policy library is updated for incremental merging with the binding as the anchor point, and the path priority and the enabled state are adjusted according to the latest conclusion, realizing continuous adaptive optimization of the recovery path.

[0188] In one embodiment, after the above step S60, further comprising:

[0189] S701, capturing the system state before the execution of the recovery operation to generate a pre-recovery baseline snapshot;

[0190] S702, capturing the system state after the completion of the recovery operation to generate a post-recovery state snapshot;

[0191] S703, comparing the file attributes of the pre-recovery baseline snapshot and the post-recovery state snapshot to generate a file consistency mark;

[0192] S704, replaying the transaction log to verify the data transaction integrity to generate a transaction consistency mark;

[0193] S705, simulating business requests to verify the availability of the function to generate a business function mark;

[0194] S706, integrating the file consistency mark, the transaction consistency mark and the business function mark to form a consistency verification mark set;

[0195] S707, associating the consistency verification mark set with the verification conclusion to generate a cross-verification matrix;

[0196] S708, encapsulating a policy update record based on the cross-verification matrix and the system timestamp;

[0197] S709, performing incremental update on the recovery path in the recovery policy library according to the performance indicators and failure association data in the policy update record;

[0198] S710, updating the parameters of the intelligent diagnosis model based on the failure case feature vector in the policy update record.

[0199] In this embodiment, the collection framework performs two state freezes on the execution environment to form comparable baselines after the recovery execution instruction completes landing. The pre-recovery baseline snapshot is obtained by double means of file system freeze and process consistency point, first applying an instant snapshot or copy-on-write to the key volumes, and then issuing a short log switch and buffer flush to the transactional services, so that the memory pages, file handles, permission bits, directory trees, binary image versions, service listening and session metadata are solidified at the same time baseline. The post-recovery state snapshot adopts the same freezing process as the baseline and reuses the same sampling granularity and path list to ensure consistency in dimensions. Each freezing output is accompanied by a timestamp, an environment fingerprint and a configuration fingerprint, the timestamp uses a monotonic clock source to avoid backtracking, the environment fingerprint includes the kernel version, driver signature and container layering digest, and the configuration fingerprint is derived from the stable ordering and hash digest of key configuration items.

[0200] File attribute comparison establishes a one-to-one mapping between the two snapshots. The scanning engine traverses the managed path set, calculates the name, size, permission, time tuple and content digest for each object, and constructs a multi-way hash index and a directory prefix tree in memory to speed up matching. Difference detection includes five types of events: addition, deletion, renaming, permission change and content change. Content change reduces the computational overhead of large files through block digest and sparse reading. The comparison results are filtered by the ignore list and tolerance set, system temporary files and rotating logs are exempted according to the strategy, and important binaries and configuration files are enabled for strong verification. File consistency labels are generated based on difference events and weight strategies, giving a positive label for complete consistency, a conditional consistent label with evidence reference for non-critical differences, and a negative label for critical differences and pointing to the affected object list.

[0201] Transaction consistency verification is implemented based on the log replay pipeline. For components with transaction semantics such as databases and message queues, the collection framework collects sequential log fragments within the recovery window, sorts and deduplicates them according to sequence number or site, and selects shadow instances or offline verification containers to perform logical replay. The replay process verifies commit and rollback sequences, constraint checks and foreign key dependencies, and triggers read-only degradation when encountering missing pages or missing external dependencies to ensure that the verification can be completed. After the replay is completed, the success rate, conflict rate and constraint violation count are calculated and compared with the transaction statistics of the recovery target to generate a transaction consistency label, which includes verification range, pass condition and problem details.

[0202] The business function verification is completed through controlled traffic playback. The traffic synthesizer constructs a set of representative requests according to the historical profile and interface contract, including mandatory paths, abnormal branches and boundary inputs, executes and records the response time distribution, response code proportion and business assertion result in the test account or isolated tenant. The verification engine adopts different playback rhythms and assertion sets for idempotent interfaces and non-idempotent interfaces, and isolates uncertainties through stub services or simulation endpoints for external dependencies. After the verification, a business function label is generated, with the pass rate of the assertion as the main indicator, and the delay and jitter as the evidence. All failed samples record the request reproduction materials for review.

[0203] The three types of labels are integrated into a consistent verification label set. The integration rule first aligns the verification range and time window, and then performs conflict arbitration in the order of component topology. If the file consistency is negative, but the transaction and business are positive, the arbitration engine drills down to the difference object and business coverage to determine whether it is a static resource replacement and does not affect the transaction and interface contract. In this case, the case can be disposed of with reduced weight. If the transaction consistency is negative, it has the highest priority, regardless of whether the file and business pass or not. The final set reflects the failure and carries the transaction evidence. The label set retains the original indicators and evidence references of each type of label in structure, avoiding loss of details.

[0204] The verification conclusion comes from the diagnostic link in the prologue, including the fault mode classification label and the dynamic confidence. The cross-verification matrix takes the fault mode as the row and the verification label as the column. The matrix unit records the pass rate of the corresponding label, the key evidence and the environmental constraints under this mode, and also carries the versioned identification of the recovery execution instruction. During the construction of the matrix, consistency check is performed. If the conclusion points to a storage fault, and the file consistency repeatedly fails and the transaction consistency also fails, the matrix unit is marked as a strongly coupled failure, and provides a high-weight input for policy update. If the conclusion points to network congestion, the business function passes, the transaction consistency is consistent, but the file has changes, this unit is marked as weakly related, and needs to avoid over-repair at the policy level.

[0205] The policy update record encapsulates the summary and evidence of the cross-verification matrix in a recovery cycle. The system timestamp and environmental fingerprint are written when encapsulating, the performance indicator set and fault association data are extracted, the performance indicators include recovery time, resource peak, playback time, file change size and interface availability, the fault association data includes mode label, confidence interval and key evidence chain. The record body is stably sorted and signed according to the field definition, forming an auditable version unit, and maintaining a bidirectional link with the versioned identification of the recovery execution instruction.

[0206] The recovery path incremental update is based on the policy update record. The update engine loads the affected path mapping entries, calculates the path priority adjustment amount based on the performance indicators and fault association data, up-regulates the weight of the path with high success rate and low resource consumption, down-regulates the weight of the path that has failed for more than twice and enters the frozen observation, and adds a restriction condition or a pre-check to the path that has side effects. When merging, optimistic locking and atomic commit are used to generate a new version of the mapping and trigger consistency verification to avoid ring dependency, hanging paths and links that do not reach the final state. All changes and associations with the original record are completely retained in the audit log, which can be rolled back to any historical version when needed.

[0207] The failure case feature vector is used to improve the parameters of the intelligent diagnosis model. The feature vector is composed of abnormal indicator components, evidence fragments, topological context and recovery path execution trajectory. After entering the data acceptance channel, de-identification, abnormal value trimming and class balancing are completed, and the weight is assigned according to time decay or sample value grading. The training plan can use small batch online update, or offline retraining and hot switching deployment. The former is used to quickly absorb the latest failure mode, and the latter is used to correct the cumulative bias. The new model performs consistency and regression testing in the shadow channel, and replaces the production reasoning configuration after passing, and the entire life cycle and versioning identification are synchronized with the policy update record.

[0208] Example: In a large medical health business environment, electronic medical records, image archives, tests, pharmacy and scheduling systems are interconnected through an integrated bus and service grid. Data paths span clinical workstations, department private clusters and central data lakes. To ensure continuity of diagnosis and treatment, first implement data-side perception: continuously extract running logs, time series indicators and network topology from clinical applications, interface engines and database layers, unify time axes according to three key objects: nodes, links and transactions, and map high-sensitive fields such as interface delays, queue depths, transaction locks and I / Os into multi-source spatio-temporal feature matrices. Historical fault records are supplemented with low-frequency patterns after deduplication and semantic alignment, and topological analysis obtains component dependency graphs and business call chains. Then, the pre-trained threshold strategy library is used to perform threshold matching on spatio-temporal correlation features, and the initial fault event is output. To suppress "same cause multiple reports", perform topological collision detection on the dependency graph: if multiple events are located in the same minimum cut or share an upstream bottleneck node, select the one with the largest impact domain as the representative to form a fault candidate event, and locate to "image retrieval service → database write pressure anomaly → storage write-back jitter" and other actionable objects.

[0209] Then, the identification and characterization are performed: the candidate fault events are combined with the context state. The candidate fault events are decomposed into key features, and are generated as a time-space feature vector according to a unified dimension. The content covers the near-window delay quantile, error code co-occurrence, lock waiting topology depth, disk queue length transition rate, and the like. The context state (scheduling peak and valley, department concurrency, recent changes) is used to search the fault knowledge graph to obtain a context-related fault mode set, such as a composite mode of “high concurrency reading + image compression task + single instance writing”. The feature vector and the mode set are jointly input into a diagnosis model to obtain an initial probability distribution, and an explainable evidence chain is extracted through attribution analysis to clearly show the causal support between “high write amplification + B stack driver version difference” and the high-probability diagnosis. According to a confidence threshold, a high-confidence classification result (such as “database deadlock triggers image retrieval speed reduction”) is screened, and then integrated with the evidence chain to form an intelligent diagnosis result, which not only gives the type but also retains the decision path, facilitating subsequent audit and review.

[0210] After diagnosis, the treatment scheme planning is performed: the fault type and the context state are combined to query the recovery strategy library to generate a hierarchical strategy mapping table, different tactics for the same target are labeled according to resource consumption, recovery delay, and state consistency risk to form a strategy execution cost matrix. The matrix solution gives the minimum cost path in the current scenario, which is represented as a multi-level sequence that can be degraded, such as “image query from main database to read database -> NPU inference to CPU -> cache time-out -> background batch compensation”. The path is parsed into atomic operations to generate a recovery operation sequence (switch connection string, adjust connection pool, driver reload, flow weighting, instance rolling, etc.), and then compatibility verification is performed to confirm that the hardware driver and software dependencies match the current environment. If inconsistencies are found, the sequence is added, deleted, or replaced according to the verification results to obtain a compatibility adapted operation sequence, which is converted and packaged into an executable script template to output a recovery execution instruction with idempotent identification, dependency constraints, and rollback anchor points.

[0211] To reduce the disturbance to production, the rehearsal verification is first performed in an isolated space. The recovery execution instruction is parsed to extract the isolation level and resource limitations to form sandbox configuration parameters, and a sandbox copy is created according to the parameters and a mirror image consistent with production is deployed. After initialization, a pre-injection snapshot is captured as a baseline. An injection strategy syntax is generated around the target link to build a signed injection template (covering HL7 messages, DICOM queries, SQL transaction samples), which is loaded into the sandbox and marked with traffic to ensure traceability. The instruction is executed in the controlled rehearsal environment to obtain a post-injection snapshot, and the differences between the pre- and post-snapshots are extracted to obtain a set of system performance anomaly indicators, covering P95 delay changes, error rate changes, lock depth changes, I / O saturation changes, and business-level success rate changes. The indicator set is integrated with the traffic identifier and signature digest to form verification data, including a traceable input, action, and result triple.

[0212] The verification data enters the review link: first, the system performance abnormal index set is obtained by analysis, matched with the pre-defined fault mode library to form a classification label, and the comprehensive deviation degree is calculated to measure the relative improvement or degradation after repair. The historical verification conclusion is weighted according to the time decay to obtain the historical confidence weighted value, and the current deviation degree is generated according to the preset fusion rule to generate a dynamic confidence score. The classification label and the dynamic confidence score are integrated into the verification conclusion, and the operation sequence of the recovery execution instruction is extracted to generate a unique identifier, and the two are associated to form a versioned instruction identifier, and the recovery path mapping relationship in the recovery strategy library is updated according to the above, realizing the traceable link from "conclusion-action" to "strategy-path".

[0213] After the completion of the exercise, the recovery execution instruction is issued in batches in the production environment and the disposal is completed, and the closed loop verification is performed. The production side captures the system state before and after the change to form the pre-recovery baseline snapshot and the post-recovery state snapshot, compares the file attributes to obtain the file consistency mark, replays the transaction log to obtain the transaction consistency mark, and simulates the clinical key business request to obtain the business function mark. The three types of marks are integrated into a consistency verification mark set, and are associated with the above-mentioned verification conclusion to generate a cross-verification matrix, and are encapsulated with a system timestamp as a strategy update record. The performance indicators and fault associated data in the record drive the incremental update of the strategy library: improve the priority of high success rate path, freeze high cost and low success path, and increase new trigger conditions when matching context. The feature vector of the failure case in the record is injected into the diagnosis model for parameter update, so that the model can distinguish between "storage jitter induced lock waiting" and "application side connection pool exhaustion" with similar symptoms in subsequent processes.

[0214] In the financial technology business environment, payment gateway, risk control engine, clearing compliance system and core account are interconnected through service grid and message bus. The transaction path spans the public network access layer, key management and hardware encryption module, authorization and clearing database, and cross-regional disaster recovery cluster. When continuously acquiring monitoring data, run logs and time series indicators are collected from the gateway, risk control, account and message queue, and historical fault record library and network topology information are synchronously pulled to build a multi-source spatio-temporal feature matrix containing interface delay quantile, authorization rejection rate, risk control hit ratio, database lock waiting, queue accumulation depth, link packet loss rate, etc. fields. Combined with the topology mapping table, high deviation points are aggregated to the component level and call chain level, and according to the pre-trained threshold strategy library, the initial fault event is matched out, and then the topology collision detection is performed on the dependency graph to eliminate the same cause multi-reporting, for example, when the payment gateway authorization timeout, risk control synchronous write-back delay and account entry lag appear at the same time and share the "encryption signature service→message queue serialization in the same partition→main database write amplification" upstream bottleneck, a fault candidate event is converged and the impact domain is defined. Input the fault candidate event and the context state into the intelligent diagnosis model, the event is disassembled into key features to generate a spatio-temporal feature vector (authorization P99, rejection rate transition, risk control rule hotspot, HSM signature throughput, Kafka ISR contraction, main database checkpoint interval, etc.), and the context state gives information such as active marketing, holiday peak, cross-border channel proportion, and history of local dual-active switching. Query the fault knowledge graph to return a set of context-related fault patterns, such as "high concurrency authorization + same partition hotspot + HSM queue congestion" and "account merge commit + playback pressure + storage jitter". The model outputs the preliminary diagnosis probability distribution, and the attribution analysis module gives the explainable evidence chain, establishing an explicit correspondence between "message queue partition hotspot + encryption signature queue + main database write amplification" and the high-probability conclusion; According to the pre-set confidence threshold, the high-confidence classification result is "payment authorization link queue congestion", and the evidence chain is integrated to form an intelligent diagnosis result, providing quantitative and explainable input for the disposal stage.A hierarchical strategy mapping table is generated around the fault type and context state retrieval recovery strategy library, listing resource overhead, recovery delay and state consistency risk of different paths to the target, forming a strategy execution cost matrix; matrix solving obtains multi-level degradation paths, such as "gateway card BIN and merchant label scattered partitioning → migrating signature tasks from the main HSM pool to the backup pool and enabling asynchronous batch signature → risk control synchronous write degradation to asynchronous write-back and setting compensation → using delay merging for account writing and enabling read library pressure bearing", which is then parsed into atomic operations and generates a recovery operation sequence (queue re-partitioning and replica rebalancing, HSM routing switching, risk control write strategy adjustment, connection pool and retry backoff parameter modification, read-write separation strategy publishing, compensation job registration), compatibility verification is performed on hardware drivers and software dependencies, if it is found that the host does not load the new HSM driver or the agent version is incompatible, the corresponding operation is added, deleted or replaced to form a compatibility adaptation operation sequence, and after conversion by an executable script template, a recovery execution instruction is generated, with an idempotent token, stage boundary and rollback anchor. To reduce production disturbance, a verification experiment is created based on the recovery execution instruction: parse the instruction to extract network isolation, CPU and memory upper limit, key access range, etc. to generate sandbox configuration parameters, isolate resources according to the parameters and deploy system images to create sandbox copies, and capture the initial state snapshot before injection after initialization; generate an injection strategy syntax according to the recovery execution instruction, build a data injection template and add a signature digest, load the template into the sandbox and mark it with traffic identification to form a controlled exercise environment; execute the recovery execution instruction in the controlled exercise environment and capture the post-injection environment state snapshot, compare the two snapshots to obtain a set of system performance anomaly indicators, which cover authorized P95 / P99 changes, rejection rate changes, queue accumulation peak changes, HSM queuing delay changes, main library lock waiting and WAL playback rate changes, business end-to-end success rate changes, etc., and then integrate the traffic identification and signature digest to obtain verification data. According to the verification data, a verification conclusion is formed and associated with the recovery execution instruction: parse the verification data to extract the set of system performance anomaly indicators, match them with the pre-defined fault mode library to generate a fault mode classification label, and calculate the comprehensive deviation degree to measure the relative improvement or degradation; weight the historical verification conclusions according to the time decay factor to obtain the historical confidence weighted value, and fuse the comprehensive deviation degree and the historical confidence weighted value according to the pre-set rule to generate a dynamic confidence score; integrate the fault mode classification label and the dynamic confidence score into the verification conclusion, extract the operation sequence from the recovery execution instruction and generate an operation sequence unique identifier, establish the association between the verification conclusion and the operation sequence unique identifier, obtain the versioned instruction identifier and use it to update the recovery path mapping relationship in the recovery strategy library, so that subsequent matching preferentially selects the verified high-confidence path under the same context.When entering the change execution phase, the recovery execution instruction is issued in stages under the execution environment, and the recovery operation is completed. The system state before and after the change is captured to generate the pre-recovery reference snapshot and the post-recovery state snapshot. The file attributes (configuration, binary, script checksum) are compared to generate the file consistency mark. The transaction log is played back to verify the transaction consistency of the account and the authorized flow to generate the transaction consistency mark. The business function availability is verified using the synthetic transaction and the low-value real transaction to generate the business function mark. The three types of marks are integrated into a consistent verification mark set and associated with the verification conclusion to form a cross-verification matrix. The system timestamp is encapsulated as a policy update record. Based on the performance indicators and failure association data in the policy update record, the recovery paths in the recovery policy library are incrementally updated: the priority of "queue re-partition + backup HSM + asynchronous write-back + compensation" is increased, and the applicable boundary in the context of "active peak + cross-border proportion increase + master library write amplification" is marked. The HSM switching path with increased failure probability under a specific firmware version is frozen. At the same time, the failure case feature vector (abnormal code sequence, queue lag distribution, signature time consumption distribution, lock waiting graph structure, etc.) in the policy update record is extracted to update the parameters of the intelligent diagnosis model, so that the model can more quickly distinguish between "queue congestion caused by partition hotspots" and "throughput decline caused by encryption module degradation" in subsequent peak periods. Through this series of actions, the authorization success rate and end-to-end delay of the payment authorization link under peak load are restored to the expected interval, the account consistency is verified at the file, transaction, and business levels, the path selection is automatically prioritized in the next round of decision-making, the recognition accuracy of the diagnosis model in the near mode is improved, and the false trigger degradation is reduced, meeting the requirements of compliance auditing for traceability and explainability.

[0215] In this embodiment, the differences before and after the recovery are quantified at the file, transaction, and business levels through the joint mechanism of double snapshot freezing and multi-dimensional verification. The cross-verification matrix aligns the evidence and diagnosis conclusions in the same structure, making the path advantages and scene adaptation relationship clear and calculable. The policy update record solidifies the performance and evidence of a recovery into an auditable version unit, and drives the incremental merging and priority adjustment of the recovery path, achieving continuous adaptive optimization of the policy library. The failure case feature vector feeds back to the diagnosis model, updates the classification boundary and confidence allocation as the environment evolves, and reduces false positives and delays.

[0216] In one embodiment, a fault diagnosis and recovery verification device is provided, which corresponds one-to-one with the fault diagnosis and recovery verification method in the above embodiments. For details, please refer to Figure 3 , Figure 3 The functional module schematic diagram of a preferred embodiment of the fault diagnosis and recovery verification device of the present application is shown in the figure. The monitoring and collection module 10, the intelligent diagnosis module 20, the strategy generation module 30, the exercise verification module 40, the conclusion association module 50, and the recovery execution module 60. The detailed description of each functional module is as follows:

[0217] a monitoring collection module 10, configured to acquire monitoring data and generate a fault candidate event based on the monitoring data;

[0218] an intelligent diagnosis module 20, configured to input the fault candidate event and a context state into an intelligent diagnosis model to obtain an intelligent diagnosis result and a fault type;

[0219] a strategy generation module 30, configured to select a recovery path in a recovery strategy library and generate a recovery execution instruction based on the fault type and the context state;

[0220] a rehearsal verification module 40, configured to create a verification experiment, build a rehearsal environment and inject rehearsal data based on the recovery execution instruction to obtain verification data;

[0221] a conclusion correlation module 50, configured to form a verification conclusion according to the verification data and correlate the verification conclusion with the recovery execution instruction;

[0222] a recovery execution module 60, configured to issue the recovery execution instruction in an execution environment and complete a recovery operation.

[0223] In an embodiment, the monitoring collection module 10 is specifically configured to:

[0224] collect a real-time system running log;

[0225] monitor a device health index to generate device health state data;

[0226] call a historical fault record library;

[0227] analyze a network topology connection relationship to generate a topology mapping relationship table;

[0228] correlate the real-time system running log, the device health state data, the historical fault record library and the topology mapping relationship table to form a multi-source spatio-temporal feature matrix;

[0229] extract a spatio-temporal correlation feature index set in the multi-source spatio-temporal feature matrix;

[0230] load a pre-trained threshold strategy library;

[0231] match the spatio-temporal correlation feature index set with the pre-trained threshold strategy library to obtain an initial fault event set;

[0232] perform topology collision detection on the initial fault event set to generate a fault candidate event.

[0233] In an embodiment, the intelligent diagnosis module 20 is specifically configured to:

[0234] extract key features of the fault candidate event to generate a spatio-temporal feature vector;

[0235] query the fault knowledge graph based on the context state, to obtain a context-associated fault mode set;

[0236] input the spatiotemporal feature vector and the context-associated fault mode set into an intelligent diagnosis model, to obtain a preliminary diagnosis probability distribution;

[0237] analyze the decision logic of the intelligent diagnosis model through an attribution analysis module, to generate an interpretable evidence chain;

[0238] filter, according to the preliminary diagnosis probability distribution and a preset confidence threshold, a fault type higher than the preset confidence threshold to generate a high-confidence classification result;

[0239] integrate the high-confidence classification result and the interpretable evidence chain, to generate an intelligent diagnosis result.

[0240] In an embodiment, the strategy generation module 30 is specifically configured to:

[0241] query a recovery strategy library based on the fault type and the context state, to obtain a hierarchical strategy mapping table;

[0242] analyze the resource overhead, recovery delay, and state consistency risk of each strategy in the hierarchical strategy mapping table, to generate a strategy execution cost matrix;

[0243] select a minimum cost path according to the strategy execution cost matrix, to generate a multi-level degradation path;

[0244] analyze the multi-level degradation path into atomic operation steps, to generate a recovery operation sequence;

[0245] real-time detect hardware drivers and software dependencies to verify the compatibility of the recovery operation sequence with the current system state, to generate a compatibility verification result;

[0246] add, delete, or replace operation steps in the recovery operation sequence based on the compatibility verification result, to form a compatibility adaptation operation sequence;

[0247] convert and encapsulate the compatibility adaptation operation sequence according to an executable script template, to generate a recovery execution instruction.

[0248] In an embodiment, the rehearsal verification module 40 is specifically configured to:

[0249] analyze the recovery execution instruction, to extract environment isolation requirements and resource constraint conditions, to generate sandbox configuration parameters;

[0250] isolate production environment resources and deploy a system image based on the sandbox configuration parameters, to create a sandbox copy;

[0251] capturing a pre-injection initial state snapshot after the sandbox copy is initialized;

[0252] generating an injection strategy grammar according to the recovery execution instruction;

[0253] constructing a data injection template based on the injection strategy grammar and adding a signature digest;

[0254] loading the data injection template in the sandbox copy and marking a traffic identifier, to generate a controlled rehearsal environment;

[0255] executing the recovery execution instruction in the controlled rehearsal environment and capturing a post-injection environment state snapshot;

[0256] comparing the pre-injection initial state snapshot and the post-injection environment state snapshot to obtain a system performance anomaly indicator set;

[0257] integrating the system performance anomaly indicator set, the traffic identifier and the signature digest to obtain verification data.

[0258] In an embodiment, the conclusion association module 50 is specifically configured to:

[0259] analyzing the verification data to extract the system performance anomaly indicator set;

[0260] matching the system performance anomaly indicator set with a predefined fault mode library to generate a fault mode classification label;

[0261] determining a comprehensive deviation degree of the system performance anomaly indicator set;

[0262] weighting historical verification conclusions based on a time decay factor to generate a historical confidence weighted value;

[0263] fusing the comprehensive deviation degree and the historical confidence weighted value according to a preset strategy to generate a dynamic confidence score;

[0264] integrating the fault mode classification label and the dynamic confidence score to form a verification conclusion;

[0265] extracting an operation sequence of the recovery execution instruction to generate an operation sequence unique identifier;

[0266] associating the verification conclusion with the operation sequence unique identifier to generate a versioned instruction identifier;

[0267] updating a recovery path mapping relationship in the recovery strategy library based on the versioned instruction identifier.

[0268] In an embodiment, the recovery execution module 60 is specifically configured to:

[0269] capturing a system state before execution of a recovery operation to generate a pre-recovery baseline snapshot;

[0270] capturing a system state after completion of the recovery operation to generate a post-recovery state snapshot;

[0271] comparing file attributes of the pre-recovery baseline snapshot and the post-recovery state snapshot to generate a file consistency flag;

[0272] verifying data transaction integrity by replaying the transaction log to generate a transaction consistency flag;

[0273] verifying functionality availability by simulating a business request to generate a business functionality flag;

[0274] integrating the file consistency flag, the transaction consistency flag and the business functionality flag to form a consistency verification flag set;

[0275] associating the consistency verification flag set with the verification conclusion to generate a cross-verification matrix;

[0276] based on the cross-verification matrix and a system timestamp, encapsulating a policy update record;

[0277] based on performance indicators and failure association data in the policy update record, performing incremental update on a recovery path in the recovery policy library;

[0278] based on a failure case feature vector in the policy update record, updating parameters of the intelligent diagnosis model.

[0279] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 4 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is configured to provide determination and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is configured to communicate with an external user terminal through a network connection. The computer program is executed by the processor to implement functions or steps of a fault diagnosis and recovery verification method on the server side.

[0280] In one embodiment, a computer device is provided, which can be a user terminal, and an internal structure diagram thereof can be as shown in Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide determination and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the user side of the fault diagnosis and recovery verification method.

[0281] In one embodiment, a computer device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the following steps:

[0282] Obtain monitoring data, and generate a fault candidate event based on the monitoring data;

[0283] Input the fault candidate event and the context state into an intelligent diagnosis model to obtain an intelligent diagnosis result and a fault type;

[0284] Select a recovery path in a recovery strategy library and generate a recovery execution instruction based on the fault type and the context state;

[0285] Create a verification experiment based on the recovery execution instruction, build a rehearsal environment and inject rehearsal data to obtain verification data;

[0286] Form a verification conclusion according to the verification data, and associate the verification conclusion with the recovery execution instruction;

[0287] Issue the recovery execution instruction in an execution environment and complete a recovery operation.

[0288] In one embodiment, a computer readable storage medium is provided, which stores a computer program, the computer program being executed by a processor to implement the following steps:

[0289] Obtain monitoring data, and generate a fault candidate event based on the monitoring data;

[0290] Input the fault candidate event and the context state into an intelligent diagnosis model to obtain an intelligent diagnosis result and a fault type;

[0291] Select a recovery path in a recovery strategy library and generate a recovery execution instruction based on the fault type and the context state;

[0292] Create a verification experiment based on the recovery execution instruction, build a rehearsal environment and inject rehearsal data to obtain verification data;

[0293] form a verification conclusion according to the verification data, and associate the verification conclusion with a recovery execution instruction;

[0294] issue the recovery execution instruction in an execution environment and complete a recovery operation.

[0295] It should be noted that the functions or steps described above in relation to the computer readable storage medium or the computer device can correspond to the relevant descriptions of the server side and the user side in the foregoing method embodiments, and will not be described again here to avoid repetition.

[0296] Those skilled in the art can understand that all or part of the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0297] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0298] It should be explained that if the software tools or components of other companies appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A fault diagnosis and recovery verification method, characterized in that, Includes the following steps: Acquire monitoring data and generate candidate fault events based on the monitoring data; The fault candidate events and context state are input into the intelligent diagnostic model to obtain the intelligent diagnostic results and fault types. Based on the fault type and context state, select a recovery path from the recovery strategy library and generate a recovery execution instruction; A verification experiment is created based on the recovery execution instruction, a training environment is constructed and training data is injected to obtain verification data; A verification conclusion is formed based on the verification data, and the verification conclusion is associated with the recovery execution instruction; The execution environment issues the recovery execution command and completes the recovery operation.

2. The fault diagnosis and recovery verification method as described in claim 1, characterized in that, Acquire monitoring data and generate candidate fault events based on the monitoring data, including: Collect real-time system operation logs; Monitor equipment health indicators and generate equipment health status data; Call the historical fault record database; Analyze the network topology connections and generate a topology mapping table; By associating the real-time system operation logs, device health status data, historical fault record database, and topology mapping table, a multi-source spatiotemporal feature matrix is ​​formed. Extract the spatiotemporal correlation feature index set from the multi-source spatiotemporal feature matrix; Load the pre-trained threshold policy library; The spatiotemporal correlation feature index set is matched with the pre-trained threshold strategy library to obtain the initial fault event set; Perform topology collision detection on the initial fault event set to generate fault candidate events.

3. The fault diagnosis and recovery verification method as described in claim 1, characterized in that, The candidate fault events and context state are input into the intelligent diagnostic model to obtain intelligent diagnostic results and fault types, including: Extract the key features of the candidate fault events and generate a spatiotemporal feature vector; Based on context state query, obtain the set of context-related fault modes from the fault knowledge graph; The spatiotemporal feature vectors and the context-related fault mode set are input into the intelligent diagnostic model to obtain a preliminary diagnostic probability distribution. The decision-making logic of the intelligent diagnostic model is analyzed by the attribution analysis module to generate an interpretable chain of evidence. Based on the preliminary diagnostic probability distribution and the preset confidence threshold, fault types that are higher than the preset confidence threshold are selected to generate high-confidence classification results; By integrating the high-confidence classification results with the interpretable chain of evidence, an intelligent diagnostic result is generated.

4. The fault diagnosis and recovery verification method as described in claim 1, characterized in that, Based on the fault type and context state, a recovery path is selected from the recovery strategy library and a recovery execution instruction is generated, including: Based on the fault type and context state, the recovery strategy library is queried to obtain the hierarchical strategy mapping table; Analyze the resource overhead, recovery latency, and state consistency risk of each strategy in the hierarchical strategy mapping table to generate a strategy execution cost matrix; Based on the strategy execution cost matrix, the path with the minimum cost is selected, and a multi-level degradation path is generated. The multi-level degradation path is analyzed into atomic operation steps to generate a recovery operation sequence; Real-time detection of hardware drivers and software dependencies is used to verify the compatibility of the recovery operation sequence with the current system state, and a compatibility verification result is generated. Based on the compatibility verification results, add, delete, or replace the operation steps in the recovery operation sequence to form a compatibility adaptation operation sequence. After converting the executable script template, the compatibility adaptation operation sequence is encapsulated to generate a recovery execution instruction.

5. The fault diagnosis and recovery verification method as described in claim 1, characterized in that, Based on the recovery execution instruction, a verification experiment is created, a training environment is constructed, and training data is injected to obtain verification data, including: The recovery execution command is parsed to extract the environmental isolation requirements and resource constraints, and sandbox configuration parameters are generated. Based on the sandbox configuration parameters, isolate production environment resources and deploy system images to create a sandbox copy; After the sandbox copy is initialized, capture a snapshot of the initial state before injection; Generate injection strategy syntax based on the resume execution instruction; A data injection template is constructed based on the injection strategy syntax, and a signature digest is added; The data injection template is loaded into the sandbox copy and traffic identifiers are marked to generate a controlled training environment; The recovery execution instruction is executed in the controlled training environment, and a snapshot of the environment state after injection is captured; By comparing the differences between the initial state snapshot before injection and the environmental state snapshot after injection, a set of system performance anomaly indicators is obtained; By integrating the system performance anomaly indicator set, traffic identifiers, and signature digests, verification data is obtained.

6. The fault diagnosis and recovery verification method as described in claim 1, characterized in that, A verification conclusion is formed based on the verification data, and the verification conclusion is associated with the recovery execution instruction, including: Parse the verification data and extract a set of system performance anomaly indicators; Match the system performance anomaly index set with a predefined fault mode library to generate fault mode classification labels; Determine the overall deviation of the system performance anomaly index set; Historical verification conclusions are weighted based on a time decay factor to generate a historical confidence weighted value. The overall deviation and historical confidence scores are combined according to a preset strategy to generate a dynamic confidence score; The failure mode classification labels and dynamic confidence scores are integrated to form a verification conclusion. Extract the operation sequence of the resumption execution instruction and generate a unique identifier for the operation sequence; Associate the verification conclusion with the unique identifier of the operation sequence to generate a versioned instruction identifier; Update the recovery path mapping relationship in the recovery strategy library based on the versioned instruction identifier.

7. The fault diagnosis and recovery verification method as described in claim 1, characterized in that, After issuing the recovery execution instruction in the execution environment and completing the recovery operation, the process also includes: Capture the system state before the recovery operation is executed and generate a baseline snapshot before recovery; Capture the system state after the recovery operation is completed and generate a post-recovery state snapshot; By comparing the file attributes of the baseline snapshot before recovery with the state snapshot after recovery, a file consistency marker is generated; Replay the transaction log to verify data transaction integrity and generate transaction consistency markers; Simulate business requests to verify the availability of the function and generate business function tags; Integrate the file consistency markers, transaction consistency markers, and business function markers to form a consistency verification marker set; By associating the consistency verification mark set with the verification conclusion, a cross-verification matrix is ​​generated; Based on the cross-verification matrix and the system timestamp, encapsulate the strategy update record; Based on the performance metrics and fault correlation data in the policy update record, perform incremental updates on the recovery paths in the recovery policy library; The parameters of the intelligent diagnostic model are updated based on the feature vectors of failed cases in the strategy update record.

8. A fault diagnosis and recovery verification device, characterized in that, The fault diagnosis and recovery verification device includes: The monitoring and acquisition module is used to acquire monitoring data and generate fault candidate events based on the monitoring data; The intelligent diagnosis module is used to input the candidate fault events and context state into the intelligent diagnosis model to obtain intelligent diagnosis results and fault types. The strategy generation module is used to select a recovery path from the recovery strategy library and generate a recovery execution instruction based on the fault type and context state. The exercise verification module is used to create a verification experiment based on the recovery execution instruction, construct an exercise environment and inject exercise data to obtain verification data; The conclusion association module is used to form a verification conclusion based on the verification data and associate the verification conclusion with the recovery execution instruction; The recovery execution module is used to issue the recovery execution command in the execution environment and complete the recovery operation.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a fault diagnosis and recovery verification program stored in the memory and executable on the processor. When executed by the processor, the fault diagnosis and recovery verification program implements the steps of the fault diagnosis and recovery verification method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a fault diagnosis and recovery verification program, which, when executed by a processor, implements the steps of the fault diagnosis and recovery verification method as described in any one of claims 1-7.

Images