Method and device for determining fault recovery strategy and electronic equipment

Abstract

The invention discloses a method and device for determining a fault recovery strategy and electronic equipment. The invention relates to computer technology. The method comprises the following steps: acquiring fault data of a target service system, obtaining system fault data, and determining a plurality of fault recovery actions based on a fault type corresponding to the system fault data; based on the system fault data, utilizing the target learning model to evaluate the recovery effect of each fault recovery action to obtain an evaluation result; obtaining a target detection rule, the target detection rule comprising a detection rule determined based on a service level agreement; based on the evaluation result and the target detection rule, a fault recovery strategy is determined, and the fault recovery strategy comprises a recovery strategy for recovering the fault of the target service system. According to the method and the device, the technical problem of poor fault recovery effect caused by taking the static plan as the fault recovery strategy of the civil aviation service system in the SLA management system of the civil aviation service system in the related technology is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a fault recovery strategy determination method and device and electronic equipment. BACKGROUND

[0002] The core system of civil aviation industry (such as ticket booking, flight status, security system) has extremely high requirements on high availability (99.99+), low latency (≤1 second), and strong consistency. For example, the relevant association stipulates that flight status update delay exceeding 1 second may cause airport scheduling confusion, etc.

[0003] Currently, the core system of civil aviation industry is mainly based on SLA management system. SLA management system refers to the management system of Service Level Agreement (SLA). Service Level Agreement refers to an agreement reached between a service provider and a customer, which specifies the service level and corresponding service level guarantee provided by the service provider. The purpose of SLA management system is to ensure that the service provider can provide high-quality services to the customer according to the agreed service level guarantee.

[0004] In related technologies, SLA management relies on manual monitoring and static pre-plan. For example, relevant personnel monitor CPU (Central Processing Unit) usage, and when CPU exceeds the threshold, the service is restarted using static pre-plan. However, this management method has many problems:

[0005] First, manual analysis of faults takes an average of 10+ minutes, which is too long to meet the real-time needs of civil aviation, resulting in problems of delayed response;

[0006] Second, static rules cannot handle complex dependencies. For example, flight status update failure may be caused by database, cache, and airline data source chain failure, resulting in incomplete coverage of recovery strategies;

[0007] Third, there is a lack of automated verification rules, and recovery actions may violate civil aviation regulations, such as flight delays that may cause compliance risks.

[0008] Therefore, manual monitoring and static pre-plan in traditional SLA management methods can cause low efficiency, poor accuracy, and compliance risks.

[0009] In view of the above problems, no effective solutions have been proposed so far. SUMMARY

[0010] Embodiments of the present application provide a method and device for determining a fault recovery strategy, and an electronic device, to at least solve the technical problem of poor fault recovery effect in the related art, in which a static plan is used as a fault recovery strategy for a civil aviation business system in a SLA management system of the civil aviation business system.

[0011] According to an aspect of an embodiment of the present application, a method for determining a fault recovery strategy is provided, including: obtaining fault data of a target business system to obtain system fault data, and determining a plurality of fault recovery actions based on a fault type corresponding to the system fault data, wherein the target business system includes a system for processing aviation business of an airline; based on the system fault data, using a target learning model to evaluate a recovery effect of each of the fault recovery actions to obtain an evaluation result, wherein a model type of the target learning model includes a reinforcement learning model based on proximal policy optimization; obtaining a target detection rule, wherein the target detection rule includes a detection rule determined based on a service level agreement; and determining a fault recovery strategy based on the evaluation result and the target detection rule, wherein the fault recovery strategy includes a recovery strategy for recovering a fault of the target business system.

[0012] Further, obtaining a target detection rule includes: obtaining a business system log of the target business system and a service level agreement standard; identifying a business indicator in the business system log to obtain a plurality of business indicators; and using a graph traversal strategy to query the plurality of business indicators in a knowledge graph to obtain an associated technical indicator chain, wherein the knowledge graph includes an architecture of the target business system, and the associated technical indicator chain includes an indicator chain composed of indicators having an association relationship with the plurality of business indicators; and determining the target detection rule based on the service level agreement standard and the associated technical indicator chain.

[0013] Further, determining the target detection rule based on the service level agreement standard and the associated technical indicator chain includes: parsing the service level agreement standard to obtain a plurality of performance indicators in the service level agreement; determining an association relationship between the plurality of performance indicators to obtain a target association relationship; and generating a detection rule for each business indicator in the associated technical indicator chain based on the target association relationship to obtain the target detection rule.

[0014] Further, the multiple business indicators are queried in the knowledge graph by using a graph traversal strategy to obtain an associated technology indicator chain, including: performing dependency analysis on the multiple business indicators to obtain a target structure, wherein the dependency analysis is used to identify an association structure between the multiple business indicators; performing standardization processing on the target structure to obtain structured business indicators; and querying the structured business indicators in the knowledge graph by using a graph traversal strategy to obtain the associated technology indicator chain.

[0015] Further, the evaluation result includes at least one recovery indicator associated with each of the fault recovery actions, and the recovery indicator is used to indicate the recovery effect of the fault recovery action. Based on the evaluation result and the target detection rule, a fault recovery strategy is determined, including: filtering the multiple fault recovery actions based on the recovery indicator associated with each of the fault recovery actions to obtain filtered multiple fault recovery actions; and determining multiple initial recovery strategies based on the filtered multiple fault recovery actions, wherein the initial recovery strategy includes at least one of the following: at least one of the fault recovery actions, description information of the initial recovery strategy, usage conditions of the initial recovery strategy, and expected results of the initial recovery strategy; testing each of the initial recovery strategies by using the target detection rule to obtain a test result of each of the initial recovery strategies, and determining the fault recovery strategy from the multiple initial recovery strategies based on the test result of each of the initial recovery strategies.

[0016] Further, the multiple fault recovery actions are filtered based on the recovery indicator associated with each of the fault recovery actions to obtain filtered multiple fault recovery actions, including: sorting the multiple fault recovery actions based on the recovery indicator associated with each of the fault recovery actions to obtain sorted multiple fault recovery actions; and filtering the sorted multiple fault recovery actions by using a preset compliance rule to obtain the filtered multiple fault recovery actions, wherein the preset compliance rule is used to identify whether each of the fault recovery actions meets a preset compliance requirement.

[0017] Further, based on the system fault data, a target learning model is used to evaluate the recovery effect of each of the fault recovery actions to obtain an evaluation result, including: performing feature extraction on the system fault data based on the knowledge graph to obtain fault features, wherein the fault features include a root cause node that causes the target business system to fail and an impact range of the system fault data on the target business system; and inputting the fault features into the target learning model to evaluate the recovery effect of each of the fault recovery actions to obtain the evaluation result.

[0018] Further, in a case where the model type of the target learning model is the reinforcement learning model, a state in the reinforcement learning model represents the fault feature, an action in the reinforcement learning model represents the fault recovery action, and a reward function in the reinforcement learning model includes a function determined based on a recovery degree of the fault, recovery time consumption, and a recovery risk evaluation value of the fault recovery action.

[0019] According to another aspect of the embodiments of the present application, a device for determining a fault recovery strategy is also provided, which includes: a processing unit configured to obtain fault data of a target service system, to obtain system fault data, and to determine a plurality of fault recovery actions based on a fault type corresponding to the system fault data, wherein the target service system includes a system for processing air services of an airline company; an evaluation unit configured to evaluate a recovery effect of each of the fault recovery actions based on the system fault data by using a target learning model, to obtain an evaluation result, wherein a model type of the target learning model includes a reinforcement learning model based on proximal policy optimization; an obtaining unit configured to obtain a target detection rule, wherein the target detection rule includes a detection rule determined based on a service level agreement; and a determining unit configured to determine a fault recovery strategy based on the evaluation result and the target detection rule, wherein the fault recovery strategy includes a recovery strategy for recovering a fault of the target service system.

[0020] Further, the obtaining unit includes: a first obtaining subunit configured to obtain a service system log of the target service system and a service level agreement standard; an identifying subunit configured to identify a service indicator in the service system log, to obtain a plurality of service indicators; a querying subunit configured to query the plurality of service indicators in a knowledge graph by using a graph traversal strategy, to obtain an associated technology indicator chain, wherein the knowledge graph includes an architecture of the target service system, and the associated technology indicator chain includes an indicator chain composed of indicators having an associated relationship with the plurality of service indicators; and a first determining subunit configured to determine the target detection rule based on the service level agreement standard and the associated technology indicator chain.

[0021] Further, the determining subunit includes: a parsing module configured to parse the service level agreement standard, to obtain a plurality of performance indicators in the service level agreement; a determining module configured to determine an associated relationship between the plurality of performance indicators, to obtain a target associated relationship; and a generating module configured to generate a detection rule for each service indicator in the associated technology indicator chain based on the target associated relationship, to obtain the target detection rule.

[0022] Further, the query subunit comprises: an analysis module, configured to perform dependency analysis on the plurality of business indicators to obtain a target structure, wherein the dependency analysis is configured to identify an association structure between the plurality of business indicators; a processing module, configured to perform standardization processing on the target structure to obtain a structured business indicator; and a query module, configured to query the structured business indicator in the knowledge graph by using a graph traversal strategy to obtain the associated technology indicator chain.

[0023] Further, the evaluation result comprises at least one recovery indicator associated with each of the fault recovery actions, the recovery indicator being configured to indicate a recovery effect of the fault recovery action, the determination unit comprises: a filtering subunit, configured to filter the plurality of fault recovery actions based on the recovery indicator associated with each of the fault recovery actions to obtain a plurality of filtered fault recovery actions; and a second determination subunit, configured to determine a plurality of initial recovery strategies based on the plurality of filtered fault recovery actions, wherein the initial recovery strategy comprises at least one of the following: at least one of the fault recovery actions, description information of the initial recovery strategy, a use condition of the initial recovery strategy, and an expected result of the initial recovery strategy; and a testing subunit, configured to test each of the initial recovery strategies by using the target detection rule to obtain a test result of each of the initial recovery strategies, and determine the fault recovery strategy from the plurality of initial recovery strategies based on the test result of each of the initial recovery strategies.

[0024] Further, the filtering subunit comprises: an ordering module, configured to order the plurality of fault recovery actions based on the recovery indicator associated with each of the fault recovery actions to obtain a plurality of ordered fault recovery actions; and a filtering module, configured to filter the plurality of ordered fault recovery actions by using a preset compliance rule to obtain the plurality of filtered fault recovery actions, wherein the preset compliance rule is configured to identify whether each of the fault recovery actions meets a preset compliance requirement.

[0025] Further, the evaluation unit comprises: an extraction subunit, configured to perform feature extraction on the system fault data based on the knowledge graph to obtain fault features, wherein the fault features comprise a root cause node causing the target business system to fail and an impact range of the system fault data on the target business system; and an evaluation subunit, configured to input the fault features into the target learning model to evaluate the recovery effect of each of the fault recovery actions to obtain the evaluation result.

[0026] Further, in a case where the model type of the target learning model is the reinforcement learning model, the state in the reinforcement learning model represents the fault feature, the action in the reinforcement learning model represents the fault recovery action, and the reward function in the reinforcement learning model includes a function determined based on a recovery degree of the fault, recovery time consumption, and a recovery risk evaluation value of the fault recovery action.

[0027] According to another aspect of the embodiments of the present application, an electronic device is also provided, which includes a processor and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the determination method of the fault recovery strategy of any one of the above via execution of the executable instructions.

[0028] According to another aspect of the embodiments of the present application, a computer readable storage medium is also provided, which stores a computer program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the determination method of the fault recovery strategy of any one of the above when the computer program is running.

[0029] In the present application, fault data of a target service system is acquired to obtain system fault data, and based on a fault type corresponding to the system fault data, a plurality of fault recovery actions are determined, wherein the target service system includes a system for processing air service of an airline; based on the system fault data, a target learning model is used to evaluate a recovery effect of each fault recovery action to obtain an evaluation result, wherein a model type of the target learning model includes a reinforcement learning model based on proximal policy optimization; target detection rules are acquired, wherein the target detection rules include detection rules determined based on a service level agreement; based on the evaluation result and the target detection rules, a fault recovery strategy is determined, wherein the fault recovery strategy includes a recovery strategy for recovering the fault of the target service system. Thus, the technical problem that in the related art, in the SLA management system of the civil aviation service system, a static plan is used as the fault recovery strategy of the civil aviation service system, resulting in poor fault recovery effect is solved. In the present application, the recovery effect of the fault recovery strategy is evaluated by the reinforcement learning model to obtain an evaluation result, and based on the evaluation result of the fault recovery strategy and the detection rules determined based on the service level agreement, a final fault recovery strategy is determined, which avoids the situation in the related art that when the use of the system is monitored by a human being and exceeds a corresponding threshold value, the service is restarted by using the static plan, which is low in efficiency and poor in accuracy, thereby achieving the technical effect of improving the fault recovery efficiency and accuracy of the civil aviation service system. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application. In the drawings:

[0031] Figure 1 is a flow chart of a method for determining an optional failure recovery strategy according to an embodiment of the present application;

[0032] Figure 2 is a flow chart of a method for determining an optional failure recovery strategy according to an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of a device for determining an optional failure recovery strategy according to an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0036] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.), collected information and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards in relevant regions, necessary security measures are taken, do not violate public order and good customs, and provide corresponding operation portal for user to choose authorization or refusal.

[0038] Embodiment one

[0039] According to an embodiment of the present application, an optional method for determining a fault recovery strategy is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0040] Figure 1 is a flowchart of an optional method for determining a fault recovery strategy according to an embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0041] In step S101, fault data of a target business system is obtained to obtain system fault data, and a plurality of fault recovery actions are determined based on the fault type corresponding to the system fault data, wherein the target business system comprises a system for processing aviation business of an airline.

[0042] The target business system described above can include a system for processing aviation business of an airline, such as a ticket booking system, a flight dynamic management system, a security check system, etc.

[0043] The fault data can be a record generated when the target business system encounters various abnormal situations during operation, and the abnormal situations can include but are not limited to system crash, performance degradation, service interruption, data consistency problem, etc. The fault data can be collected from multiple sources, such as system logs, alarm systems, performance monitoring tools, network monitoring systems, etc. For the system for processing aviation business of an airline, the fault data can be ticket booking response time too long, flight status update delay, database connection frequently disconnected, cache invalidation frequency high, etc.

[0044] Once the system detects an anomaly, a detailed fault record is generated. In the flight dynamic update system, if it is detected that the update delay exceeds the threshold value specified by the SLA, the system will record the specific value of this delay, the duration, the number of affected flights, and the preliminary judgment of the fault cause (such as database load too large, network congestion, etc.).

[0045] The fault types of the target business system are various, which can be hardware problems, software defects, network faults, external service unavailability, etc. According to the preset correspondence, different recovery actions can be corresponded to different fault types. For example, if the performance degradation is caused by the pressure of the cache layer, the recovery actions can include increasing the cache instance, adjusting the cache strategy, optimizing the cache data structure, etc.; if it is a database level problem, the recovery actions can include database read-write separation, data migration to standby server, etc.

[0046] In step S102, the recovery effect of each fault recovery action is evaluated based on system fault data by using a target learning model, and an evaluation result is obtained, wherein the model type of the target learning model includes a reinforcement learning model based on proximal policy optimization.

[0047] The reinforcement learning model can be a reinforcement learning model based on PPO (Proximal Policy Optimization). In this embodiment, a pre-trained reinforcement learning model (corresponding to the target learning model) can be used to simulate the effect of different recovery actions, and the expected success rate, recovery speed, resource consumption, etc. of each possible action (such as expanding cache, restarting service, throttling, switching instance, etc.) in the current scenario can be calculated to obtain the above-mentioned evaluation result.

[0048] In step S103, a target detection rule is obtained, wherein the target detection rule includes a detection rule determined based on a service level agreement.

[0049] The above-mentioned target detection rule can include a detection rule determined based on a service level agreement. For example, the civil aviation industry SLA (Service Level Agreement) standard and business system log input information can be obtained, and the input information is semantically parsed and index associated by using an SLA rule engine to obtain executable SLA detection rules (i.e., target detection rules).

[0050] In step S104, a fault recovery strategy is determined based on the evaluation result and the target detection rule, wherein the fault recovery strategy includes a recovery strategy for recovering the fault of the target business system.

[0051] The above-mentioned evaluation result can include the expected success rate, recovery speed, resource consumption, etc. of each fault recovery action (such as expanding cache, restarting service, throttling, switching instance, etc.). According to the evaluation result, a plurality of initial fault recovery strategies (referred to as initial recovery strategies) can be determined. Then, each initial recovery strategy can be automatically tested by using the target detection rule. The initial recovery strategy that passes the test can be determined as the recovery strategy for recovering the fault of the target business system. Then, the fault of the target business system can be recovered according to the fault recovery strategy.

[0052] Through the above steps, in this embodiment, the recovery effect of the fault recovery strategy is evaluated by the reinforcement learning model to obtain an evaluation result, and based on the evaluation result of the fault recovery strategy and the detection rule determined according to the service level agreement, a final fault recovery strategy is determined, thereby avoiding the situation in the related art that the service is restarted by using a static plan when the use of the system is monitored by a person and exceeds the corresponding threshold, and the efficiency is low and the accuracy is poor, thereby achieving the technical effect of improving the fault recovery efficiency and accuracy of the civil aviation business system. Further, the technical problem that the static plan is used as the fault recovery strategy of the civil aviation business system in the SLA management system of the civil aviation business system in the related art, resulting in poor fault recovery effect is solved.

[0053] Optionally, the target detection rule is obtained, including: obtaining a business system log of the target business system and a service level agreement standard; identifying a business index in the business system log to obtain a plurality of business indexes; querying the plurality of business indexes in a knowledge graph by using a graph traversal strategy to obtain an associated technical index chain, wherein the knowledge graph includes: an architecture of the target business system, and the associated technical index chain includes: an index chain composed of indexes having an association relationship with the plurality of business indexes; and determining the target detection rule based on the service level agreement standard and the associated technical index chain.

[0054] The business system log described above can include all behavior records of the target business system at runtime, for example, the business system log includes but is not limited to flight operation conditions, customer service quality records, running states of a background processing system, etc. For example, ticket booking API (Application Programming Interface) response time, database transaction delay, etc. In this embodiment, the business system log of the target business system and the service level agreement (SLA, Service Level Agreement) standard can be obtained. The knowledge graph described above can be a knowledge graph of the architecture of the target business system, and can include: inter-component dependency relationships, inter-service dependency relationships, resource topologies, and calling relationships, etc. The associated technical index chain described above: by using the graph traversal strategy, the system can find technical indexes associated with the business indexes in the knowledge graph to establish an index chain. For example, the flight query response time is associated with technical indexes such as Redis cache hit rate, database query delay, and network delay, etc., which constitute an associated technical index chain.

[0055] In this embodiment, the business system log in natural language form can be segmented and recognized using a business lexicon to obtain business indicator names; then the core structure is recognized through dependency analysis; and the recognized core structure is standardized and converted to obtain a JSON format structured rule. In the segmentation and recognition process, natural language processing and machine learning techniques can be used to automatically recognize business indicators from log text and convert them into structured data form, making it easier to analyze and process.

[0056] For business indicators (such as queue_time) in the structured rule, a graph traversal algorithm can be used to query the knowledge graph to output an associated technical indicator chain. Then, natural language processing techniques can be used to parse the SLA standard to identify key performance indicators (KPIs). Based on the associated technical indicator chain and the key performance indicators in the SLA standard, the target detection rule can be determined.

[0057] In this embodiment, the SLA-based detection rule system can ensure that the target business system can quickly identify the problem when encountering a fault or performance degradation and promptly initiate recovery strategies to ensure the continuity and quality of system services. In addition, the use of knowledge graphs further improves the accuracy of the rules and the efficiency of system recovery.

[0058] Optionally, based on the service level agreement standard and the associated technical indicator chain, the target detection rule is determined, including: parsing the service level agreement standard to obtain a plurality of performance indicators in the service level agreement; determining the association relationship between the plurality of performance indicators to obtain a target association relationship; and based on the target association relationship, generating a detection rule for each business indicator in the associated technical indicator chain to obtain the target detection rule.

[0059] In this embodiment, key performance indicators can be extracted from SLA documents (corresponding to service level agreement standards), which can be used to measure whether the target business system meets the agreement requirements. For example, the "ticket booking system response time is less than 500 milliseconds within 99.9% of the time." In the civil aviation IT architecture, different business indicators can be associated with multiple underlying technical indicators to form a technical indicator chain. For example, the business indicator "flight status update delay" may depend on database performance, cache efficiency, network status, and other technical indicators.

[0060] By analyzing the causal relationship or correlation between different performance indicators, the correlation between different performance indicators can be determined, which can involve in-depth understanding of business logic, system architecture and historical data to identify which changes in technical indicators will directly affect the achievement of service level agreement. The correlation between business indicators and underlying technical indicators is clarified, that is, the target correlation is established, which helps to locate the root cause of the problem, rather than just the surface phenomenon. Quantitative standards are set for each business indicator in the technical indicator chain, which can reflect the requirements of SLA, for example, if the SLA requires that the flight status update delay does not exceed 1 second, the detection rule of the business indicator "flight status update delay" should include quantitative checks on related technical indicators such as "database query delay" and "cache hit rate". Based on the target correlation, detection rules can be developed for each business indicator in the technical indicator chain, which can be converted into scripts or programs that can be automatically executed by IT systems.

[0061] For example, natural language processing techniques can be used to parse SLA standards, identify key performance indicators (KPIs) (corresponding to the above-mentioned multiple performance indicators), and convert the key performance indicators into machine-readable form; through statistical analysis and machine learning algorithms, the relationship between different KPIs is explored, and the key factors affecting the overall service quality and their interaction mode (such as the dependency relationship between different KPIs) are identified. For each associated technical indicator in the associated technical indicator chain, in combination with the key factors affecting the overall service quality and their interaction mode, executable SLA detection rules are generated for each associated technical indicator, for example, for each associated technical indicator in the associated technical indicator chain, target detection rules can be generated according to the preset rules and multiple performance indicators.

[0062] Optionally, the graph traversal strategy is used to query the multiple business indicators in the knowledge graph to obtain the associated technical indicator chain, including: performing dependency analysis on the multiple business indicators to obtain a target structure, wherein the dependency analysis is used to identify the association structure between the multiple business indicators; performing standardization processing on the target structure to obtain a structured business indicator; and using the graph traversal strategy to query the structured business indicator in the knowledge graph to obtain the associated technical indicator chain.

[0063] In this embodiment, the business word library can be used to perform word recognition on the business system log in natural language form to obtain the business indicator name (corresponding to the above-mentioned multiple business indicators), and then the core structure can be identified through dependency analysis; and the identified core structure is standardized and converted to obtain a JSON format structured rule (corresponding to the structured business indicator).

[0064] For example, assuming that the target business system log includes the value check-in queuing time < 10 minutes, first, the business lexicon can be used to perform word segmentation and recognition on the natural language form of the business system log to obtain the business indicator name: "check-in queuing time", the operator "<", and the time unit "10 minutes"; then the core structure is identified using dependency analysis, such as "queuing time < 10 minutes" → nsubj(time, queuing), nummod(minute, 10), acl(time, <); the identified core structure is then converted into a standardized form to obtain the structured rule in JSON format: queue_time < 600s.

[0065] For the business indicator (such as queue_time) in the structured rule, a graph traversal algorithm can be used to perform knowledge graph query to output the associated technical indicator chain.

[0066] For example, the business indicator queue_time can be input into the knowledge graph, and the graph traversal algorithm is executed to output the associated technical indicator chain: "flight query timeout" may be associated with Redis cache, database sharding, and airline data source, and the associated technical indicator chain can include: Redis cache hit rate → MySQL sharding delay → airline API QPS (Queries-per-second).

[0067] Optionally, the evaluation result includes at least one recovery indicator associated with each fault recovery action, the recovery indicator being used to indicate the recovery effect of the fault recovery action, and the fault recovery strategy is determined based on the evaluation result and the target detection rule, including: filtering the multiple fault recovery actions based on the recovery indicator associated with each fault recovery action to obtain filtered multiple fault recovery actions; determining multiple initial recovery strategies based on the filtered multiple fault recovery actions, wherein the initial recovery strategy includes at least one of the following: at least one fault recovery action, description information of the initial recovery strategy, usage conditions of the initial recovery strategy, and expected results of the initial recovery strategy; testing each initial recovery strategy using the target detection rule to obtain a test result of each initial recovery strategy, and determining the fault recovery strategy from the multiple initial recovery strategies based on the test result of each initial recovery strategy.

[0068] The evaluation result can include at least one recovery indicator associated with each fault recovery action, and the recovery indicator is used to indicate the recovery effect of the fault recovery action. For example, the recovery indicator can be: expected success rate, recovery speed, resource consumption, etc. Each fault recovery action (such as expanding cache, restarting service, throttling, switching instance, etc.) can include: expected success rate, recovery speed, resource consumption, etc. in the current scenario.

[0069] In this embodiment, the multiple fault recovery actions can be filtered based on the recovery indicators associated with each fault recovery action to obtain filtered multiple fault recovery actions to exclude operations that may trigger more serious consequences or violate regulatory rules. Based on the filtered multiple fault recovery actions, multiple initial recovery strategies can be determined, each including at least one of the following: at least one fault recovery action, description information of the initial recovery strategy, usage conditions of the initial recovery strategy, and expected results of the initial recovery strategy.

[0070] For example, each fault recovery action can be prioritized according to the recovery indicators associated with each fault recovery action to form a candidate strategy list. In combination with the compliance requirements of the civil aviation industry, the candidate strategy list can be filtered according to compliance rules to obtain filtered multiple fault recovery actions to exclude operations that may trigger more serious consequences or violate regulatory rules. Finally, the top-3 recovery strategies (corresponding to the multiple initial recovery strategies) can be output in the subsequent strategy list, and sorted according to success rate, risk, SLA recovery potential, etc. in multiple dimensions, with a brief description, applicable conditions and expected results of each recovery strategy.

[0071] According to the output top-3 recovery strategies and the original fault scenario, in combination with the SLA detection rules corresponding to each recovery strategy (corresponding to the target detection rules), an automated test is performed to output an automated test result and a compliance audit log.

[0072] The specific operation is as follows:

[0073] Input:

[0074] The recovery actions (such as "read-write separation + cache expansion") in the top-3 recovery strategies that have been simulated and executed.

[0075] The original fault scenario (such as 100,000 QPS flight query requests simulated by a performance test tool).

[0076] Processing process:

[0077] Test playback: automatically reconstruct the fault scenario and verify the recovery indicators (such as delay, error rate).

[0078] Compliance check: compare the repair results with the SLA threshold (such as "delay 0.8 seconds ≤ 1 second").

[0079] Output:

[0080] Automated test report (such as project and practical tracking tool bug single + performance comparison chart).

[0081] Compliance audit log (such as "reason for choosing strategy A: compliance with relevant provisions").

[0082] Afterwards, the fault recovery strategy can be determined from the automated test report among a plurality of initial recovery strategies.

[0083] Optionally, the plurality of fault recovery actions are filtered based on the recovery indicators associated with each fault recovery action to obtain filtered plurality of fault recovery actions, including: sorting the plurality of fault recovery actions based on the recovery indicators associated with each fault recovery action to obtain sorted plurality of fault recovery actions; filtering the sorted plurality of fault recovery actions using a preset compliance rule to obtain filtered plurality of fault recovery actions, wherein the preset compliance rule is used to identify whether each fault recovery action meets a preset compliance requirement.

[0084] For example, in this embodiment, for each fault recovery action, the plurality of recovery indicators associated with the fault recovery action can be weighted, and then the plurality of fault recovery actions can be sorted according to the weighted results of the fault recovery actions to obtain sorted plurality of fault recovery actions, which can constitute a candidate strategy list. Then, the candidate strategy list can be filtered according to the compliance rules of the civil aviation industry to exclude operations that may cause more serious consequences or violate regulatory rules to obtain filtered plurality of fault recovery actions.

[0085] Optionally, the recovery effect of each fault recovery action is evaluated using a target learning model based on system fault data to obtain an evaluation result, including: extracting features from the system fault data based on a knowledge graph to obtain fault features, wherein the fault features include: root cause nodes that cause the target business system to fail and the impact range of the system fault data on the target business system; inputting the fault features into the target learning model to evaluate the recovery effect of each fault recovery action to obtain the evaluation result.

[0086] With the help of the knowledge graph, the system fault data can be analyzed in depth, and the features that help understand the nature of the fault are extracted. The features extracted from the fault data mainly include the root cause node that leads to the failure of the target business system and the scope of the fault impact. The root cause node can be the most original and core reason in a series of fault events, such as database server overload, cache invalidation, network connection interruption, etc. The impact range can refer to the scope of the fault, including which business modules are affected. The fault features extracted from the system fault data are input into the target learning model, and the model can evaluate each possible fault recovery action based on previous learning experience. The key indicators of the evaluation include the success probability of the recovery action, the time required for recovery, resource consumption, and the impact on business stability. The evaluation result is the expected recovery effect of each fault recovery action in the current fault scenario. The system can recommend the recovery strategy that is most likely to effectively solve the fault to the operation and maintenance personnel or the automated system based on these evaluation results.

[0087] For example, in this embodiment, feature extraction can be performed on real-time fault data to obtain a feature vector. Specifically, for example, potential root cause nodes and impact ranges can be identified by tracing paths based on real-time fault data and historical similar cases in combination with a knowledge graph. If the real-time fault data is a chaos experiment injected fault, such as a flight query timeout. Correspondingly, the identified potential root cause nodes and impact ranges can be “Redis cache breakdown → database load surge → query interface slow down”, wherein the knowledge graph can be an existing IT architecture knowledge graph, which contains service interdependence, resource topology, and call relationship, etc.

[0088] In an optional example, the current fault features of the real-time fault data can be compared with historical fault patterns using the knowledge graph to determine whether they belong to the same fault type or whether there are new abnormal situations. If the current fault features are of a new type, they can be marked as learning samples to facilitate updating the knowledge graph.

[0089] In this embodiment, the real-time fault data is encoded into a feature vector that can be understood by the model, which is beneficial for subsequent input into the reinforcement learning model for strategy evaluation. Then, a pre-trained reinforcement learning model can be used to simulate the effects of different fault recovery actions. For each fault recovery action (such as expanding cache, restarting service, throttling, switching instance, etc.), the expected success rate, recovery speed, resource consumption, and other indicators in the current scenario can be calculated to obtain the above evaluation results.

[0090] Optionally, in the case where the model type of the target learning model is a reinforcement learning model, the state in the reinforcement learning model represents the fault feature, the action in the reinforcement learning model represents the fault recovery action, and the reward function in the reinforcement learning model includes a function determined based on the recovery degree of the fault by the fault recovery action, the recovery time consumption, and the recovery risk evaluation value.

[0091] For example, when the target learning model is specified as a reinforcement learning (RL) model, the model can learn to take optimal actions in a specific environment through trial and error to maximize cumulative rewards. For the state (S) in the reinforcement learning model, in reinforcement learning, the state can refer to the description of the current moment in the environment, providing the context for the model's understanding of the environment. In the context of fault recovery, the state S can represent the fault feature, that is, the information extracted from the system fault data that can reflect the essence of the fault, including the root cause node that leads to the fault of the business system and the scope of the fault impact. For example, the state can be "Redis cache overload, flight query service delay exceeds 1 second, affecting 1000 users".

[0092] For the action (A) in the reinforcement learning model, it can refer to the action that the model can take in the current state, with the purpose of changing the environment state to achieve a certain goal. In the fault recovery scenario, the action A can represent the fault recovery action, that is, the measures that the operation and maintenance personnel or the automated system can take to restore the system to a normal operating state. For example, the action can be "increase the number of Redis cache instances", "restart the database service", or "switch to the standby server".

[0093] The reward function (R) in the reinforcement learning model can define the feedback reward obtained by the model in a given state and action. The reward function guides the model to learn which actions are "good" (i.e., obtain higher rewards) and which are "bad" (i.e., obtain lower or negative rewards) in a specific state. In the context of fault recovery, the design of the reward function is particularly critical, as it can need to reflect the overall effect of the fault recovery action. In the reinforcement learning model, the reward function is usually determined by a combination of multiple indicators, and for fault recovery, these indicators can include: the recovery degree of the fault based on the fault recovery action: that is, how much the fault is recovered after the execution of the fault recovery action. For example, if the flight query service delay is recovered from 2 seconds to 1.2 seconds, the recovery degree is 40% (i.e., a 40% reduction in delay). Recovery time consumption: the time required to complete fault recovery. Fast recovery can reduce the impact of business interruption, so the less time-consuming, the higher the reward. Recovery risk evaluation value: the additional risk that may be brought by the execution of the fault recovery action, such as the impact on data consistency, the new faults that may be triggered, etc. The lower the risk, the higher the reward.

[0094] In this way, the reinforcement learning model can evaluate the expected effect and risk of each fault recovery action under the current fault state, so as to recommend the best recovery strategy.

[0095] For example, the reinforcement learning model can be a reinforcement learning model based on PPO (Proximal Policy Optimization), wherein the state S is defined as the current state vector of the system (including but not limited to the feature vector of real-time fault data), the action A is defined as a discrete set of executable fault recovery actions, and a composite reward function is designed to calculate the reward R.

[0096] The composite reward function is as follows:

[0097] R = a · S + b · e + g · (1 - R compliance risk -λT ) risk )

[0098] Wherein Scompliance∈[0,1] represents the SLA recovery degree of the current recovery action (for example, it can be represented by the reduction of the flight dynamic update delay time, assuming that when the delay time is reduced from 2 seconds to 0.5 seconds, Scompliance takes a value of 0.75), λ is the decay coefficient (λ = 0.02 can achieve nonlinear penalty), T is the recovery time consumption (seconds), Rrisk∈[0,1] is the operation risk evaluation value (such as restart service = 0.8, expansion = 0.1), and the rest are weight coefficients, for example, a = 0.6, b = 0.3, and g = 0.1.

[0099] In this embodiment, based on the civil aviation business SLA (such as "booking system 99.99% availability"), fault recovery actions are automatically generated and verified to ensure that fault handling meets industry compliance requirements. In the specific implementation process, first, the SLA rule engine can be used to define civil aviation key indicators and convert them into quantifiable thresholds; then intelligent strategy matching is performed, and when the chaos experiment triggers SLA violation, the plan library is automatically matched, and the AI model is used to recommend the optimal action (such as preferentially expanding rather than restarting, because historical data shows that the expansion success rate is higher); finally, the test platform is linked: the fault scenarios generated by the experiment are converted into automated test cases (such as project and practice tracking tools to generate bugs), and the R&D repair is promoted. Through the deep combination of chaos engineering and AI decision, the SLA governance has made a leap from "passive response" to "active defense", reducing the time consumption, improving the accuracy of strategy recommendation, and solving the problem of incomplete recovery strategy coverage. In this embodiment, compliance requirements are introduced throughout the entire process, which can maximize the avoidance of compliance risks.

[0100] Embodiment Two

[0101] Figure 2is a flowchart of an optional fault recovery strategy determination according to an embodiment of the present application, as shown, comprising: Figure 2

[0102] Step 1: Obtain civil aviation industry SLA (Service Level Agreement) standard and business system log and other input information, use SLA rule engine to perform semantic analysis and index association on the input information, and obtain executable SLA detection rules.

[0103] Specifically, the civil aviation industry SLA standard can include but is not limited to: formulated by industry associations or regulatory agencies, covering multiple aspects such as flight punctuality rate, customer service response time, etc.

[0104] Business system logs include but are not limited to flight operation conditions, customer service quality records, and background processing system running states, etc. For example, it includes ticket booking API (Application Programming Interface) response time, database transaction delay, etc.

[0105] The implementation process of this step is as follows:

[0106] Step 1.1 uses a business vocabulary to perform word segmentation and recognition on the natural language form of the business system log to obtain a business index name; then identifies the core structure through dependency analysis; and then converts the identified core structure to obtain a JSON format structured rule.

[0107] For example, assuming that the business system log includes check-in queuing time < 10 minutes, first use the business vocabulary to perform word segmentation and recognition on the natural language form of the business system log to obtain the business index name "check-in queuing time", the operator "<", and the time unit "10 minutes"; then use dependency analysis to identify the core structure, such as "queuing time < 10 minutes" → nsubj(time, queuing), nummod(minute, 10), acl(time, <); and then convert the identified core structure to obtain a JSON format structured rule: queue_time < 600s.

[0108] Step 1.2 uses a graph traversal algorithm to perform knowledge graph query on the business index (such as queue_time) in the structured rule, and outputs an associated technical index chain;

[0109] ​For example, input the business indicator queue_time into the knowledge graph, execute the graph traversal algorithm, and output the associated technology indicator chain: "flight query timeout" may be associated with Redis cache, database sharding, and airline data source, so the associated technology indicator chain can include: Redis cache hit rate → MySQL sharding delay → airline API QPS (Queries-per-second).

[0110] Step 1.3: Generate executable SLA detection rules for each associated technology indicator in the associated technology indicator chain.

[0111] Use natural language processing techniques to parse the civil aviation industry SLA standards in step 1.1, identify key performance indicators (KPIs), and convert them into machine-readable form; explore the relationships between different KPIs through statistical analysis and machine learning algorithms, and identify the key factors affecting overall service quality and their interaction patterns. For each associated technology indicator in the associated technology indicator chain, generate executable SLA detection rules for each associated technology indicator based on the key factors affecting overall service quality and their interaction patterns.

[0112] Step 2: Based on real-time fault data under the original fault scenario, use a reinforcement learning model to simulate the effects of different recovery actions, and obtain the Top-3 recovery strategies. In step 2, an end-to-end intelligent operation closed loop is realized from real-time fault perception, root cause identification, strategy automatic recommendation to execution effect prediction. By introducing AI models and knowledge graphs, fault response becomes more efficient and accurate, and the high availability and industry compliance requirements of the aviation system are taken into account.

[0113] The implementation process of this step is as follows:

[0114] Step 2.1: Feature extraction is performed on real-time fault data to obtain feature vectors;

[0115] In this step, real-time fault data and historical similar cases can be used to trace the path, identify potential root cause nodes and impact ranges, and other features in combination with the knowledge graph.

[0116] If the real-time fault data is a chaos experiment injected fault, such as flight query timeout. Correspondingly, the identified potential root cause nodes and impact ranges can be "Redis cache breakdown → database load surge → query interface slow".

[0117] The knowledge graph in step 2.1 can be an existing IT architecture knowledge graph, which includes service dependencies, resource topology, and call relationships.

[0118] In this step 2.1, the current fault characteristics of real-time fault data are compared with historical fault patterns using a knowledge graph to determine whether it belongs to the same type or whether there is a new abnormal situation. If it is a new type, it can be marked as a learning sample to facilitate the updating of the knowledge graph.

[0119] In step 2.1, real-time fault data is encoded into a feature vector that can be understood by the model, which is beneficial for subsequent input into the reinforcement learning model for policy evaluation.

[0120] Step 2.2 uses a pre-trained reinforcement learning model to simulate the effects of different recovery actions. For each possible action (such as expanding cache, restarting service, throttling, switching instance, etc.), the expected success rate, recovery speed, resource consumption, and other indicators are calculated in the current scenario. Then, each possible action is prioritized to form a candidate strategy list.

[0121] Step 2.3 combines the compliance requirements of the civil aviation industry to filter the candidate strategy list. This step aims to exclude operations that may cause more serious consequences or violate regulatory rules.

[0122] Step 2.4 outputs the top-3 recovery strategies, sorted by success rate, risk, SLA recovery potential, and other dimensions, along with a brief description of each recovery strategy, applicable conditions, and expected results.

[0123] In step 2.2, the reinforcement learning model can be a PPO (Proximal Policy Optimization) based reinforcement learning model, where the state S is defined as the current state vector of the system (including but not limited to the feature vector of real-time fault data), the action A is defined as a discrete set of executable recovery actions, and a composite reward function is designed to calculate the reward R.

[0124] The composite reward function is as follows:

[0125] R = a · S compliance + β · e -λT + γ · (1 - R risk )

[0126] Where Scompliance ∈ [0, 1] represents the SLA recovery degree of the current recovery action (for example, it can be represented by the reduction of flight dynamic update delay time, assuming that when it decreases from 2 seconds to 0.5 seconds, Scompliance takes a value of 0.75), λ is the decay coefficient (λ = 0.02 can achieve nonlinear punishment), T is the recovery time consumption (seconds), Rrisk ∈ [0, 1] is the operation risk assessment value (such as restarting service = 0.8, expansion = 0.1), and the rest are weight coefficients, for example, a = 0.6, β = 0.3, γ = 0.1.

[0127] Step 3: Based on the Top-3 recovery strategies output in Step 2 and the original fault scenario, combined with the SLA detection rules corresponding to each recovery strategy obtained in Step 1, perform automated testing, output automated testing results and compliance audit logs.

[0128] The specific operation is as follows:

[0129] Input:

[0130] The executed recovery action in the Top-3 recovery strategy (such as "read-write separation + cache expansion").

[0131] The original fault scenario (such as 100,000 QPS flight query requests simulated by performance testing tools).

[0132] Processing process:

[0133] Test playback: automatically reconstruct the fault scenario and verify the indicators after recovery (such as delay, error rate).

[0134] Compliance check: compare the repair results with the SLA threshold (such as "delay 0.8 seconds ≤ 1 second").

[0135] Output:

[0136] Automated testing report (such as project and practice tracking tool Bug single + performance comparison chart).

[0137] Compliance audit log (such as "the basis for choosing strategy A: compliance with relevant provisions").

[0138] In this embodiment, based on the civil aviation business SLA (such as "99.99% availability of ticketing system"), fault recovery actions are automatically generated and verified to ensure that fault handling complies with industry compliance requirements. In the specific implementation process, first, the SLA rule engine can be used: define key indicators for civil aviation and convert them into quantifiable thresholds; then intelligent strategy matching is performed, when the chaos experiment triggers SLA violation, the pre-plan library is automatically matched, and the optimal action is recommended by the AI model (such as preferential expansion rather than restart, because historical data shows that the expansion success rate is higher); finally, the test platform is linked: the fault scenarios generated by the experiment are converted into automated test cases (such as project and practice tracking tool Bug), and the R&D repair is promoted. Through the deep combination of chaos engineering and AI decision-making, the SLA governance has made a leap from "passive response" to "active defense", reducing the time-consuming, improving the accuracy of strategy recommendation, and solving the problem of incomplete recovery strategy coverage. And in this embodiment, compliance requirements are introduced throughout the processing process, which can maximize the avoidance of compliance risks.

[0139] Embodiment Three

[0140] The embodiment three of the present application provides a kind of optional fault recovery strategy determination device, each implementation unit in the determination device corresponds to each implementation step in the embodiment one.

[0141] Figure 3 It is according to the schematic diagram of an optional fault recovery strategy determination device of the embodiment of the present application, as shown in Figure 3 The determination device includes: processing unit 31, evaluation unit 32, acquisition unit 33, determination unit 34.

[0142] Wherein, processing unit 31, for obtaining the fault data of target business system, system fault data is obtained, and based on the fault type corresponding to system fault data, determine multiple fault recovery actions, wherein, target business system includes: the system of airline handling air business;

[0143] Evaluation unit 32 is used for evaluating the recovery effect of each fault recovery action based on system fault data using target learning model, to obtain evaluation result, wherein, the model type of target learning model includes: the reinforcement learning model based on near-end policy optimization;

[0144] Acquisition unit 33 is used for obtaining target detection rule, wherein, target detection rule includes: detection rule determined based on service level agreement;

[0145] Determination unit 34 is used for determining fault recovery strategy based on evaluation result and target detection rule, wherein, fault recovery strategy includes: the recovery strategy of the fault of target business system is recovered.

[0146] In the fault recovery strategy determination apparatus provided in the embodiment, the processing unit 31 can obtain fault data of a target business system to obtain system fault data, and determine a plurality of fault recovery actions based on a fault type corresponding to the system fault data. The target business system includes a system of an airline company processing air services. The evaluation unit 32 evaluates the recovery effect of each fault recovery action by using a target learning model based on the system fault data to obtain an evaluation result. The model type of the target learning model includes a reinforcement learning model based on a proximal policy optimization. The obtaining unit 33 obtains a target detection rule. The target detection rule includes a detection rule determined based on a service level agreement. The determination unit 34 determines a fault recovery strategy based on the evaluation result and the target detection rule. The fault recovery strategy includes a recovery strategy for recovering the fault of the target business system. Thus, the technical problem that in the related art, a static plan is used as the fault recovery strategy of the civil aviation business system in the SLA management system of the civil aviation business system, resulting in poor fault recovery effect is solved. In the embodiment, the recovery effect of the fault recovery strategy is evaluated by using the reinforcement learning model to obtain an evaluation result. The final fault recovery strategy is determined based on the evaluation result of the fault recovery strategy and the detection rule determined based on the service level agreement. The situation that in the related art, the service is restarted by using the static plan when the use of the system is monitored by the human and exceeds the corresponding threshold is avoided. The efficiency and accuracy of the fault recovery of the civil aviation business system are improved.

[0147] Optionally, in the fault recovery strategy determination apparatus provided in the embodiment, the obtaining unit includes a first obtaining subunit configured to obtain a business system log and a service level agreement standard of a target business system, an identifying subunit configured to identify a business index in the business system log to obtain a plurality of business indexes, a querying subunit configured to query the plurality of business indexes in a knowledge graph by using a graph traversal strategy to obtain an associated technology index chain, wherein the knowledge graph includes an architecture of the target business system, and the associated technology index chain includes an index chain composed of indexes having an association relationship with the plurality of business indexes, and a first determining subunit configured to determine a target detection rule based on the service level agreement standard and the associated technology index chain.

[0148] Optionally, in the fault recovery strategy determination apparatus provided in the embodiment, the determining subunit includes an analyzing module configured to analyze the service level agreement standard to obtain a plurality of performance indexes in the service level agreement, a determining module configured to determine an association relationship between the plurality of performance indexes to obtain a target association relationship, and a generating module configured to generate a detection rule for each business index in the associated technology index chain based on the target association relationship to obtain the target detection rule.

[0149] Optionally, in the determination apparatus for the fault recovery strategy provided in the embodiment, the querying subunit comprises: an analysis module, configured to perform dependency analysis on the plurality of service indicators to obtain a target structure, wherein the dependency analysis is configured to identify an association structure between the plurality of service indicators; a processing module, configured to perform standardization processing on the target structure to obtain structured service indicators; and a querying module, configured to query the structured service indicators in the knowledge graph by using a graph traversal strategy to obtain an associated technology indicator chain.

[0150] Optionally, in the determination apparatus for the fault recovery strategy provided in the embodiment, the evaluation result comprises at least one recovery indicator associated with each fault recovery action, the recovery indicator is configured to indicate a recovery effect of the fault recovery action, the determination unit comprises: a filtering subunit, configured to filter the plurality of fault recovery actions based on the recovery indicator associated with each fault recovery action to obtain filtered plurality of fault recovery actions; a second determination subunit, configured to determine a plurality of initial recovery strategies based on the filtered plurality of fault recovery actions, wherein the initial recovery strategy comprises at least one of the following: at least one fault recovery action, description information of the initial recovery strategy, a use condition of the initial recovery strategy, and an expected result of the initial recovery strategy; and a testing subunit, configured to test each initial recovery strategy by using a target detection rule to obtain a test result of each initial recovery strategy, and determine the fault recovery strategy from the plurality of initial recovery strategies based on the test result of each initial recovery strategy.

[0151] Optionally, in the determination apparatus for the fault recovery strategy provided in the embodiment, the filtering subunit comprises: an ordering module, configured to order the plurality of fault recovery actions based on the recovery indicator associated with each fault recovery action to obtain ordered plurality of fault recovery actions; and a filtering module, configured to filter the ordered plurality of fault recovery actions by using a preset compliance rule to obtain filtered plurality of fault recovery actions, wherein the preset compliance rule is configured to identify whether each fault recovery action meets a preset compliance requirement.

[0152] Optionally, in the determination apparatus for the fault recovery strategy provided in the embodiment, the evaluation unit comprises: an extraction subunit, configured to perform feature extraction on the system fault data based on the knowledge graph to obtain fault features, wherein the fault features comprise: a root cause node causing the target business system to fail and an impact range of the system fault data on the target business system; and an evaluation subunit, configured to input the fault features into a target learning model to evaluate the recovery effect of each fault recovery action to obtain an evaluation result.

[0153] Optionally, in the determination apparatus for the fault recovery strategy provided in the embodiment, in a case where the model type of the target learning model is a reinforcement learning model, a state in the reinforcement learning model represents a fault feature, an action in the reinforcement learning model represents a fault recovery action, and a reward function in the reinforcement learning model includes a function determined based on a recovery degree of the fault recovery action on the fault, a recovery time consumption, and a recovery risk evaluation value.

[0154] The determination apparatus for the fault recovery strategy can further include a processor and a memory, and the processing unit 31, the evaluation unit 32, the acquisition unit 33, the determination unit 34, and the like are stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0155] The processor includes a core, and the core calls the corresponding program units in the memory. The core can be one or more, and the recovery effect of the fault recovery strategy is evaluated by the reinforcement learning model to obtain an evaluation result. Based on the evaluation result of the fault recovery strategy and the detection rule determined according to the service level agreement, the final fault recovery strategy is determined, which avoids the situation in the related art that the service is restarted by using a static plan when the use of the system is monitored manually to exceed the corresponding threshold, and the efficiency is low and the accuracy is poor, thereby realizing the technical effect of improving the fault recovery efficiency and accuracy of the civil aviation business system.

[0156] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0157] According to another aspect of the embodiment of the present application, an electronic device is further provided, which includes a processor and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the determination method for the fault recovery strategy of any one of the above by executing the executable instructions.

[0158] According to another aspect of the embodiment of the present application, a computer readable storage medium is further provided, which stores a computer program, wherein the computer readable storage medium controls a device where the computer readable storage medium is located to execute the determination method for the fault recovery strategy of any one of the above when the computer program is running.

[0159] Figure 4 is a schematic diagram of an electronic device according to the embodiment of the present application, as Figure 4As shown, the embodiment of the present application provides an electronic device 40, which comprises a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the method for determining the fault recovery strategy of any one of the above embodiments when executing the program.

[0160] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0161] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0162] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0163] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed to multiple units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0164] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0165] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0166] The above is only the preferred embodiment of the present application, it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, can make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method for determining a fault recovery strategy, characterized in that, include: Obtain fault data from the target business system, obtain system fault data, and determine various fault recovery actions based on the fault type corresponding to the system fault data. The target business system includes: the airline's system for processing aviation business. Based on the system fault data, the recovery effect of each fault recovery action is evaluated using a target learning model to obtain the evaluation result. The model type of the target learning model includes: a reinforcement learning model based on proximal policy optimization. Obtain target detection rules, wherein the target detection rules include: detection rules determined based on the service level agreement; Based on the evaluation results and the target detection rules, a fault recovery strategy is determined, wherein the fault recovery strategy includes a recovery strategy for recovering from the faults of the target business system.

2. The determination method according to claim 1, characterized in that, Obtain object detection rules, including: Obtain the business system logs and service level agreement standards of the target business system; Identify business metrics in the business system logs to obtain multiple business metrics; A graph traversal strategy is used to query multiple business metrics in a knowledge graph to obtain a chain of related technical metrics. The knowledge graph includes the architecture of the target business system, and the chain of related technical metrics includes a chain of metrics that are related to multiple business metrics. The target detection rules are determined based on the service level agreement standard and the associated technical indicator chain.

3. The determination method according to claim 2, characterized in that, Based on the service level agreement standard and the associated technical indicator chain, the target detection rules are determined, including: The Service Level Agreement (SLA) standard is parsed to obtain multiple performance metrics within the SLA. Determine the correlation between multiple performance indicators to obtain the target correlation; Based on the target association, detection rules are generated for each business indicator in the associated technical indicator chain to obtain the target detection rules.

4. The determination method according to claim 2, characterized in that, A graph traversal strategy is used to query multiple business metrics in the knowledge graph to obtain a chain of related technical metrics, including: Dependency analysis is performed on multiple business metrics to obtain the target structure, wherein the dependency analysis is used to identify the association structure among the multiple business metrics; The target structure is standardized to obtain structured business metrics; The structured business metrics are queried in the knowledge graph using a graph traversal strategy to obtain the associated technical metric chain.

5. The determination method according to claim 1, characterized in that, The evaluation results include: at least one recovery indicator associated with each fault recovery action, the recovery indicator indicating the recovery effect of the fault recovery action; and based on the evaluation results and the target detection rules, a fault recovery strategy is determined, including: Based on the recovery indicators associated with each of the aforementioned fault recovery actions, the various fault recovery actions are filtered to obtain a variety of filtered fault recovery actions. Based on the filtered multiple fault recovery actions, multiple initial recovery strategies are determined, wherein the initial recovery strategy includes at least one of the following: at least one of the fault recovery actions, description information of the initial recovery strategy, usage conditions of the initial recovery strategy, and expected results of the initial recovery strategy; Each of the initial recovery strategies is tested using the target detection rules to obtain the test results of each initial recovery strategy. Based on the test results of each initial recovery strategy, the fault recovery strategy is determined among the multiple initial recovery strategies.

6. The determination method according to claim 5, characterized in that, Based on the recovery metrics associated with each of the aforementioned fault recovery actions, multiple fault recovery actions are filtered to obtain a variety of filtered fault recovery actions, including: Based on the recovery indicators associated with each of the aforementioned fault recovery actions, the various fault recovery actions are sorted to obtain a sorted set of fault recovery actions. The sorted fault recovery actions are filtered using preset compliance rules to obtain the filtered fault recovery actions. The preset compliance rules are used to identify whether each fault recovery action meets preset compliance requirements.

7. The determination method according to claim 2, characterized in that, Based on the system fault data, the recovery effect of each fault recovery action is evaluated using a target learning model, and the evaluation results are obtained, including: Based on the knowledge graph, feature extraction is performed on the system fault data to obtain fault features, wherein the fault features include: the root cause node that caused the target business system to fail and the scope of the impact of the system fault data on the target business system; The fault features are input into the target learning model, and the recovery effect of each fault recovery action is evaluated to obtain the evaluation result.

8. The determination method according to claim 7, characterized in that, When the model type of the target learning model is the reinforcement learning model, the state in the reinforcement learning model represents the fault feature, the action in the reinforcement learning model represents the fault recovery action, and the reward function in the reinforcement learning model includes: a function determined based on the degree of fault recovery, recovery time, and recovery risk assessment value of the fault recovery action.

9. A device for determining a fault recovery strategy, characterized in that, include: The processing unit is used to acquire fault data of the target business system, obtain system fault data, and determine various fault recovery actions based on the fault type corresponding to the system fault data. The target business system includes: the airline's system for processing aviation business. An evaluation unit is used to evaluate the recovery effect of each fault recovery action based on the system fault data using a target learning model, and obtain an evaluation result. The target learning model includes a reinforcement learning model based on proximal policy optimization. An acquisition unit is used to acquire target detection rules, wherein the target detection rules include: detection rules determined based on a service level agreement; The determining unit is configured to determine a fault recovery strategy based on the evaluation results and the target detection rules, wherein the fault recovery strategy includes a recovery strategy for recovering from the faults of the target business system.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for determining the fault recovery strategy according to any one of claims 1 to 7.

Images